Gram (Speakeasy) vs Manufact

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round to Manufact
   Gram (Speakeasy)✓fullprobed8/10

   Speakeasy directly hosts a working llms.txt (HTTP 200, confirmed by probe) plus a docs.md markdown-native page, meaning an agent can be pointed at agent-oriented docs formats; the docs also emphasize AI-native/agent workflows throughout (SKILL.md, platform MCP server, CLI). missing for 10: no evidence of llms.txt existing for Gram product docs specifically (only the root Speakeasy domain), and no independent/hands-on confirmation an agent actually consumed and acted on the llms.txt content.

   • [probe] “PROBE llms.txt: HTTP 200 at https://www.speakeasy.com/llms.txt # Speakeasy > Speakeasy is the AI control plane for the AI-native enterprise…”
   • [probe] “PROBE docs-md: HTTP 200 at https://www.speakeasy.com/docs/ai-control-plane.md # AI Control Plane Documentation {/* * Renders through the …”
   • [claimed-docs] “A skill is a versioned `SKILL.md` manifest: reusable instructions that agents load on demand.”
   • [claimed-docs] “An org admin working in Claude Code, Cursor, Codex, or another MCP client can find and add MCP servers, finish their setup, put them on a pl…”

   proof ↗⚑ flag

   Manufact✓fullprobed9/10

   A direct probe confirms llms.txt is live at https://docs.manufact.com/llms.txt (HTTP 200) with a structured index of docs, giving agents a machine-readable entry point, and this is corroborated by the docs site itself supporting agent-oriented navigation (mcp.md, cli.md, dashboard docs) that an agent could traverse. Missing for 10: no explicit vendor statement promoting llms.txt as a supported agent workflow, and no independent community confirmation of an agent successfully consuming it.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.manufact.com/llms.txt # Docs - Manufact Cloud: Deploy, monitor, and manage MCP servers on Manufact…”
   • [claimed-docs] “exposes the platform, organizations, servers, deployments, logs, and observability, as MCP tools, so you can manage everything from Cursor, …”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round to Manufact
   Gram (Speakeasy)~partialprobed6/10

   Gram ships an official CLI (verified runnable via runtime probe) that can build/publish MCP servers and toolsets, which supports CI/headless automation, and docs confirm a working command-line reference. However, there is no explicit CI/CD pipeline documentation, no GitHub Actions example, and no headless auth/service-account guidance for unattended runs. Missing for 10: documented CI examples (e.g., GitHub Actions/pipeline configs), non-interactive/service-account authentication flow for the CLI, and independent hands-on confirmation of full automation beyond version-check.

   • [claimed-docs] “install toolsets as MCP servers in AI agents, and keep the CLI itself up to date”
   • [probe] “official CLI documented at https://www.speakeasy.com/docs/ai-control-plane/reference/command-line”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/gram/): downloaded the official cli@0.16.0 release artifact (gram_dar…”

   proof ↗⚑ flag

   Manufact✓fullprobed7/10

   Manufact ships a CLI for local runs/deploys and API keys explicitly designed for 'REST API, CI pipelines, and scripts,' supporting headless automation of deployments and management outside the dashboard UI (manufact-docs-15, manufact-docs-16, manufact-docs-17, manufact-docs-5). Missing for 10: a concrete CI pipeline example/integration guide (e.g. GitHub Actions snippet), an OpenAPI spec (probe found 404s), and independent/hands-on corroboration of CI usage.

   • [claimed-docs] “Run your MCP server locally with hot reload + inspector”
   • [claimed-docs] “Deploys the current project to Manufact Cloud.”
   • [claimed-docs] “API keys authenticate programmatic access to your organization, from the REST API, CI pipelines, and scripts.”
   • [claimed-docs] “Trigger deployment: deploy a branch on demand.”
   • [probe] “official CLI documented at https://docs.manufact.com/cli”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.manufact.com/openapi.json, https://docs.manufact.com/swagger.json, https://docs.manufac…”

   proof ↗⚑ flag

3. ai-native userPlug MCP servers into this product so it can use their tools

   weight 3 · round drawn
   Gram (Speakeasy)✓fullclaimed7/10

   Gram documents fronting/governing third-party MCP servers ('put a governed endpoint in front of an MCP server someone else runs') and an in-dashboard harness to connect, authenticate, and chat with a model that calls tools from a connected server, plus a real-world case (MoonPay, 200+ MCP servers aggregated under the gateway). This directly satisfies plugging external MCP servers in so their tools become usable. Missing for 10: independent/hands-on verification of third-party server integration in production and more detail on tool-call fidelity across diverse upstream servers.

   • [claimed-docs] “Put a governed endpoint in front of an MCP server someone else runs, so access control, audit, and upstream credentials apply without changi…”
   • [claimed-docs] “An in-dashboard agent harness for testing MCP servers before connecting a real client: pick a server, authenticate, chat with a model that c…”
   • [claimed-docs] “MoonPay brought 200+ MCP servers under the MCP Gateway, going from proof of concept to a company-wide rollout in 30 days.”
   • [claimed-docs] “unsanctioned shadow MCP servers are blocked by default”

   proof ↗⚑ flag

   Manufact✓fullclaimed7/10

   Manufact lets users paste an MCP server URL and Manufact 'connects directly for Chat, testing, and the publish workflow,' meaning the platform itself invokes the pasted server's tools; testing docs also describe executing across different MCP clients/models. missing for 10: no independent/hands-on confirmation of actual tool invocation behavior, and no detail on tool-call mechanics or error handling.

   • [claimed-docs] “paste your MCP URL. No deploy required: Manufact connects directly for Chat, testing, and the publish workflow.”
   • [claimed-docs] “Manufact executes them across different MCP clients and LLM models”
   • [claimed-docs] “You get one-click installs or copy-paste configuration for”

   proof ↗⚑ flag

4. ai-native userConnect an agent via an official MCP server

   weight 3 · round to Gram (Speakeasy)
   Gram (Speakeasy)✓fullprobed9/10

   Gram documents and exposes an official 'Platform MCP' server that lets MCP clients (Claude Code, Cursor, Codex, etc.) connect directly, plus generates/hosts MCP servers as its core product function, and a CLI/runtime probe confirms the tooling actually installs and runs. missing for 10: independent (non-vendor) hands-on confirmation that an external agent successfully connects to and operates the Platform MCP server end-to-end.

   • [claimed-docs] “An org admin working in Claude Code, Cursor, Codex, or another MCP client can find and add MCP servers, finish their setup, put them on a pl…”
   • [probe] “official MCP server documented at https://www.speakeasy.com/docs/ai-control-plane/reference/platform-mcp”
   • [claimed-docs] “Built — the Control Plane generates the server itself from a first-party source: an OpenAPI document or a TypeScript Functions project.”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/gram/): downloaded the official cli@0.16.0 release artifact (gram_dar…”
   • [claimed-docs] “An in-dashboard agent harness for testing MCP servers before connecting a real client: pick a server, authenticate, chat with a model that c…”

   proof ↗⚑ flag

   Manufact✓fullprobed8/10

   Manufact ships an official MCP server that exposes the platform (orgs, servers, deployments, logs, observability) as MCP tools so agents like Cursor and Claude can connect and manage everything, with one-click installs/copy-paste config for clients and a dedicated docs page confirmed by probe (manufact-docs-14, manufact-docs-18, manufact-probe-3). Missing for 10: independent/hands-on confirmation that the MCP server connection works smoothly in practice (community evidence is thin/confused, e.g. manufact-comm-3) and clearer detail on auth/session robustness (manufact-comm-2 notes reauth issues in the broader MCP ecosystem).

   • [claimed-docs] “exposes the platform, organizations, servers, deployments, logs, and observability, as MCP tools, so you can manage everything from Cursor, …”
   • [claimed-docs] “You get one-click installs or copy-paste configuration for”
   • [probe] “official MCP server documented at https://docs.manufact.com/mcp”
   • [community] “Noted an issue with the offline_access OAuth scope causing frequent reauthentication requests from clients (citing a related Codex bug); man…”

   proof ↗⚑ flag

5. ai-native userUse an official CLI

   weight 2 · round drawn
   Gram (Speakeasy)✓fullprobed8/10

   Gram ships an official CLI documented at reference/command-line, used to install toolsets as MCP servers and keep itself updated, and a runtime probe confirms the binary actually downloads and runs (`gram --version`). missing for 10: independent/third-party review of CLI usage beyond the vendor docs and a single runtime probe, and more detail on the full breadth of CLI commands beyond install/update.

   • [claimed-docs] “install toolsets as MCP servers in AI agents, and keep the CLI itself up to date”
   • [probe] “official CLI documented at https://www.speakeasy.com/docs/ai-control-plane/reference/command-line”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/gram/): downloaded the official cli@0.16.0 release artifact (gram_dar…”

   proof ↗⚑ flag

   Manufact✓fullprobed8/10

   Manufact documents an official CLI with concrete commands (local dev with hot reload/inspector, deploy to Manufact Cloud) and a dedicated cli.md/cli docs page confirmed by probe, fitting the agentic/AI-native workflow of managing MCP servers from the terminal. Missing for 10: independent/hands-on community validation of the CLI itself and a fuller command reference beyond run/deploy.

   • [claimed-docs] “Run your MCP server locally with hot reload + inspector”
   • [claimed-docs] “Deploys the current project to Manufact Cloud.”
   • [probe] “official CLI documented at https://docs.manufact.com/cli”

   proof ↗⚑ flag

6. ai-native userDrive the product through a documented public API

   weight 3 · round to Gram (Speakeasy)
   Gram (Speakeasy)~partialprobed7/10

   Gram exposes a documented Platform MCP server and CLI that let an AI-native user/agent drive the product programmatically (find/add MCP servers, author skills, manage policies, read logs) without the dashboard, and the CLI was independently verified to install and run. However, a direct probe for a conventional public REST/OpenAPI API (openapi.json, swagger.json, etc.) returned 404s, so there is no evidence of a general-purpose documented HTTP API beyond the MCP/CLI surfaces. Missing for 10: a documented REST/OpenAPI public API endpoint, independent confirmation of the Platform MCP server's functional breadth beyond docs.

   • [claimed-docs] “An org admin working in Claude Code, Cursor, Codex, or another MCP client can find and add MCP servers, finish their setup, put them on a pl…”
   • [probe] “official MCP server documented at https://www.speakeasy.com/docs/ai-control-plane/reference/platform-mcp”
   • [claimed-docs] “install toolsets as MCP servers in AI agents, and keep the CLI itself up to date”
   • [probe] “official CLI documented at https://www.speakeasy.com/docs/ai-control-plane/reference/command-line”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/gram/): downloaded the official cli@0.16.0 release artifact (gram_dar…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://www.speakeasy.com/openapi.json, https://www.speakeasy.com/swagger.json, https://www.speakeas…”

   proof ↗⚑ flag

   Manufact~partialprobed5/10

   Manufact documents API-key based programmatic access to a REST API for CI/scripts and also exposes platform functionality as MCP tools usable from AI clients like Cursor/Claude, which supports AI-native driving of the product. However, a probe for actual API documentation (OpenAPI/swagger spec) returned 404s across all candidate paths, so there's no verifiable public API reference beyond a passing mention. Missing for 10: a discoverable OpenAPI/swagger spec or full REST API reference docs, and independent confirmation the REST API works as described.

   • [claimed-docs] “API keys authenticate programmatic access to your organization, from the REST API, CI pipelines, and scripts.”
   • [claimed-docs] “exposes the platform, organizations, servers, deployments, logs, and observability, as MCP tools, so you can manage everything from Cursor, …”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.manufact.com/openapi.json, https://docs.manufact.com/swagger.json, https://docs.manufac…”
   • [probe] “official MCP server documented at https://docs.manufact.com/mcp”

   proof ↗⚑ flag

7. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round to Gram (Speakeasy)
   Gram (Speakeasy)~partialclaimed6/10

   Gram supports OAuth 2.0/2.1 integration with external providers, RBAC enforced at team/server/tool level synced to enterprise IDPs, and governs upstream credentials for third-party MCP servers with shadow-server blocking — collectively enabling scoped access control for agent tool use. However, there's no explicit documentation of per-agent least-privilege token minting/vending or fine-grained scope selection at credential-issuance time; the emphasis is on gateway-level RBAC and OAuth pass-through rather than dynamically scoped credential issuance. Missing for 10: explicit per-agent credential scoping/token minting mechanism, documented least-privilege scope configuration UI, and independent verification of enforcement.

   • [claimed-docs] “you can use any OAuth 2.0 provider you have set up (such as Auth0, Okta, Keycloak, or your own OAuth server)”
   • [claimed-docs] “Put a governed endpoint in front of an MCP server someone else runs, so access control, audit, and upstream credentials apply without changi…”
   • [github] “Team, server, and tool level permissions enforced through RBAC and Oauth2.1. Synced to your enterprise IDP (Okta, Azure AD, Google Workspace…”
   • [claimed-docs] “unsanctioned shadow MCP servers are blocked by default”

   proof ↗⚑ flag

   Manufact—none0/10

   Docs confirm API keys exist for programmatic/CI access (manufact-docs-17), but there is no evidence of scoping, permission levels, or least-privilege configuration for these keys or for agent-specific credentials. Missing for 10: any mention of scoped/role-based API key permissions, per-agent credential restriction, or least-privilege access controls.

   • [claimed-docs] “API keys authenticate programmatic access to your organization, from the REST API, CI pipelines, and scripts.”

   proof ↗⚑ flag

8. ai-native userBuild against official SDKs

   weight 2 · round drawn
   Gram (Speakeasy)—none0/10

   The evidence pack documents Gram's CLI, MCP server distribution, OAuth, skills, and observability features, but nowhere mentions official client SDKs (e.g., Python/TypeScript/Go libraries) for building against Gram's platform itself — this is a fair question for a platform product but is simply unaddressed.

   ⚑ flag

   Manufact—none0/10

   Evidence shows an official CLI, REST API with API keys, and an MCP server for managing the platform, but no documented client SDKs (e.g., language libraries) for building against Manufact programmatically; the openapi spec probe also returned 404s, suggesting no formal API/SDK artifact is published.

   • [claimed-docs] “API keys authenticate programmatic access to your organization, from the REST API, CI pipelines, and scripts.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.manufact.com/openapi.json, https://docs.manufact.com/swagger.json, https://docs.manufac…”
   • [probe] “official CLI documented at https://docs.manufact.com/cli”

   proof ↗⚑ flag

9. ai-native userSubscribe to events via webhooks

   weight 2 · round drawn
   Gram (Speakeasy)—none0/10

   Gram's evidence covers MCP server generation, tool logs, CLI, and RBAC, but nothing describes webhook subscriptions or event-driven push notifications for AI agents; missing for 10: webhook subscription mechanism, event schema/documentation, or any mention of push-based event delivery.

   ⚑ flag

   Manufact—none0/10

   No evidence in the pack mentions webhooks or event subscription mechanisms; Manufact's docs cover deployment, observability, MCP tool exposure, and CLI, but nothing about outbound webhook subscriptions for events.

   ⚑ flag

Agentic features

1. ai-native userGet AI-generated insights and suggestions from my data inside the product

   weight 2 · round to Manufact
   Gram (Speakeasy)—none0/10

   Gram provides observability data (tool logs, usage/cost tracking, budgets) but there is no evidence that it generates AI-driven insights or suggestions from that data — it's raw logs and metrics for humans to review, not AI-generated analysis.

   • [claimed-docs] “the raw execution log for every tool call the platform observes across hosted MCP servers, tunneled and shadow MCP servers, skills, and loca…”
   • [github] “Track AI usage across teams and measure impact with either tokens or cost. Deep dive expensive sessions, create budgets and measure tool eff…”

   proof ↗⚑ flag

   Manufact~partialclaimed3/10

   The publish-checklist feature generates a 'fix prompt' and offers a built-in autofix flow based on automated audit results, which is the closest thing to AI-generated suggestions from data inside the product, but this is narrowly scoped to server compliance issues rather than broader data insights. Missing for 10: any evidence of AI-generated insights over usage/observability data, trends, or user-data analytics, and no independent corroboration that these suggestions are genuinely AI-driven versus rule-based checks.

   • [claimed-docs] “runs an automated audit of your server for protocol compliance and platform readiness”
   • [claimed-docs] “copy a fix prompt into your own coding agent, or run the built-in autofix flow”
   • [claimed-docs] “you can start a checklist automatically after each successful deployment”

   proof ↗⚑ flag

2. ai-native userSet up automations that run autonomously in the background

   weight 2 · round to Manufact
   Gram (Speakeasy)—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

   Manufact~partialclaimed4/10

   Manufact offers some automated background actions tied to deployment events—auto-running a publish checklist after each successful deployment and automated protocol-compliance audits—but these are CI/CD-style triggers, not general-purpose autonomous agent automations that a user configures to run independently in the background. Missing for 10: evidence of user-defined scheduled/triggered autonomous workflows beyond deployment-linked checks, no mention of persistent background agents or cron-like automation configuration, and no independent confirmation that these automations run reliably unattended.

   • [claimed-docs] “you can start a checklist automatically after each successful deployment”
   • [claimed-docs] “runs an automated audit of your server for protocol compliance and platform readiness”
   • [claimed-docs] “end-to-end checks that exercise your server live in ChatGPT and Claude”
   • [claimed-docs] “Trigger deployment: deploy a branch on demand.”

   proof ↗⚑ flag

3. ai-native userDelegate tasks to a built-in AI assistant inside the product

   weight 3 · round to Manufact
   Gram (Speakeasy)~partialclaimed3/10

   Gram includes an in-dashboard 'agent harness' where a user can chat with a model that calls tools, but this is explicitly framed as a way to test MCP servers before connecting a real client, not a general-purpose built-in assistant for delegating arbitrary tasks. Missing for 10: evidence of a general task-delegation assistant experience beyond the testing/playground harness, independent corroboration of its use, and any framing of it as a primary product feature rather than a dev-testing tool.

   • [claimed-docs] “An in-dashboard agent harness for testing MCP servers before connecting a real client: pick a server, authenticate, chat with a model that c…”

   proof ↗⚑ flag

   Manufact~partialclaimed4/10

   Manufact mentions a 'Chat' feature for testing MCP servers and a 'built-in autofix flow' that can automatically fix compliance issues, which are narrow forms of delegating tasks to an in-product AI capability, but there is no evidence of a general-purpose built-in AI assistant for broader task delegation across the platform. Missing for 10: a documented general AI assistant/agent persona, scope of tasks it can perform beyond autofix/chat testing, and independent corroboration of its effectiveness.

   • [claimed-docs] “paste your MCP URL. No deploy required: Manufact connects directly for Chat, testing, and the publish workflow.”
   • [claimed-docs] “end-to-end checks that exercise your server live in ChatGPT and Claude”
   • [claimed-docs] “copy a fix prompt into your own coding agent, or run the built-in autofix flow”

   proof ↗⚑ flag

4. ai-native userOperate the product with natural-language commands

   weight 2 · round to Manufact
   Gram (Speakeasy)✓fullprobed7/10

   Gram documents a platform MCP server that lets an org admin operate the product (find/add servers, manage risk policies, author skills, read observability) entirely from within an MCP client like Claude Code or Cursor using natural language, without opening the dashboard — this is direct agentic/NL control of the product itself. Missing for 10: independent/hands-on verification that these natural-language admin operations actually execute correctly (only the CLI binary itself was runtime-probed, not the platform-mcp NL workflow).

   • [claimed-docs] “An org admin working in Claude Code, Cursor, Codex, or another MCP client can find and add MCP servers, finish their setup, put them on a pl…”
   • [probe] “official MCP server documented at https://www.speakeasy.com/docs/ai-control-plane/reference/platform-mcp”
   • [claimed-docs] “An in-dashboard agent harness for testing MCP servers before connecting a real client: pick a server, authenticate, chat with a model that c…”

   proof ↗⚑ flag

   Manufact✓fullprobed8/10

   Manufact exposes its own platform (orgs, servers, deployments, logs, observability) as MCP tools so users can manage everything via natural language from clients like Cursor or Claude, and also supports copying fix prompts into a coding agent or running built-in autofix. This is a first-party, well-documented agentic control surface over the whole product. Missing for 10: independent hands-on verification of the MCP server actually working end-to-end, and broader third-party confirmation beyond docs/community chatter.

   • [claimed-docs] “exposes the platform, organizations, servers, deployments, logs, and observability, as MCP tools, so you can manage everything from Cursor, …”
   • [probe] “official MCP server documented at https://docs.manufact.com/mcp”
   • [claimed-docs] “copy a fix prompt into your own coding agent, or run the built-in autofix flow”
   • [claimed-docs] “You get one-click installs or copy-paste configuration for”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   Gram (Speakeasy)—none0/10

   Gram is positioned as an AI control plane/MCP gateway focused on server generation, governance, and observability; there is no evidence of an interactive API reference UI with runnable examples for exploring endpoints, nor documentation of such a feature.

   ⚑ flag

   Manufact—none0/10

   Manufact documents a REST API (via API keys) and a testing dashboard that runs test suites across MCP clients/models, but there is no evidence of an interactive API reference with runnable examples (e.g., a Swagger/OpenAPI explorer) — a direct probe for openapi.json/swagger.json returned 404s. Missing for 10: any interactive API reference UI, runnable code examples for the REST API, or OpenAPI spec exposure.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.manufact.com/openapi.json, https://docs.manufact.com/swagger.json, https://docs.manufac…”
   • [claimed-docs] “API keys authenticate programmatic access to your organization, from the REST API, CI pipelines, and scripts.”
   • [claimed-docs] “Manufact executes them across different MCP clients and LLM models”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   Gram (Speakeasy)—none0/10

   The evidence shows Gram consumes OpenAPI documents as input to build MCP servers, but there is no evidence Gram itself publishes a downloadable machine-readable spec of its own API — the direct probe for openapi.json/swagger.json/well-known paths all returned 404.

   • [probe] “PROBE openapi: all candidate paths 404 (https://www.speakeasy.com/openapi.json, https://www.speakeasy.com/swagger.json, https://www.speakeas…”

   proof ↗⚑ flag

   Manufact—none0/10

   Manufact provides a REST API and API keys (manufact-docs-17) but the probe explicitly found no OpenAPI/swagger spec at any standard location (manufact-probe-2), and no docs mention a downloadable machine-readable spec.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.manufact.com/openapi.json, https://docs.manufact.com/swagger.json, https://docs.manufac…”
   • [claimed-docs] “API keys authenticate programmatic access to your organization, from the REST API, CI pipelines, and scripts.”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round to Manufact
   Gram (Speakeasy)~partialclaimed4/10

   Gram provides an in-dashboard 'Connect' harness to test MCP servers with a model before wiring up a real client, which offers some sandbox-like isolation from production agent traffic, but there is no explicit mention of separate sandbox vs. production data/environments, test fixtures, or guarantees that tool calls in this harness avoid hitting live production data sources. missing for 10: explicit sandbox/staging environment concept, isolation guarantees from production data, mock or test data support during agent testing.

   • [claimed-docs] “An in-dashboard agent harness for testing MCP servers before connecting a real client: pick a server, authenticate, chat with a model that c…”

   proof ↗⚑ flag

   Manufact~partialclaimed6/10

   Docs show per-branch preview environments with distinct MCP URLs, on-demand deployment triggers, local hot-reload/inspector runs, and an explicit separate production-branch setting — all consistent with testing in isolation from production. However, there's no explicit statement about data isolation or guarantees that sandbox testing cannot touch production data/state. Missing for 10: explicit data-isolation guarantees between preview/local and production environments, and independent confirmation that preview deployments don't share production data stores.

   • [claimed-docs] “Preview environments are created per branch. Each gets its own MCP URL”
   • [claimed-docs] “Trigger deployment: deploy a branch on demand.”
   • [claimed-docs] “Run your MCP server locally with hot reload + inspector”
   • [claimed-docs] “Set which Git branch serves your canonical production URL from the Deployment paths & triggers section”
   • [claimed-docs] “Manufact executes them across different MCP clients and LLM models”

   proof ↗⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   Gram (Speakeasy)—none0/10

   The evidence pack covers OpenAPI-generated MCP servers, OAuth, governance, CLI, observability, and skills, but contains no documentation of API versioning practices or a deprecation policy for Gram's own APIs/servers.

   ⚑ flag

   Manufact—none0/10

   Manufact documents a REST API and API keys for programmatic access, but there is no evidence of API versioning or a documented deprecation policy; an openapi spec probe returned 404s across all candidate paths. Missing for 10: versioned API scheme, changelog/deprecation policy documentation, any mention of backward-compatibility guarantees.

   • [claimed-docs] “API keys authenticate programmatic access to your organization, from the REST API, CI pipelines, and scripts.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.manufact.com/openapi.json, https://docs.manufact.com/swagger.json, https://docs.manufac…”

   proof ↗⚑ flag

Multi tenant

1. platform-engineerMaintain separate connected accounts and scoped tokens per end user of my product, so each user's agent acts only on that user's credentials

   weight 3 · round drawn
   Gram (Speakeasy)—none0/10

   Gram's evidence describes OAuth 2.0 provider integration and team/server/tool-level RBAC synced to an enterprise IDP (gram-docs-2, gram-gh-2), but nothing documents per-end-user connected accounts or scoped tokens so that each of a platform-engineer's own product's end users gets isolated credentials — the RBAC described is org/team-level, not end-user-level multi-tenant credential isolation. Missing for 10: explicit documentation of per-end-user connected-account/token scoping, any multi-tenant credential isolation guide, and evidence of enforcement that one end user's agent cannot use another's credentials.

   • [claimed-docs] “you can use any OAuth 2.0 provider you have set up (such as Auth0, Okta, Keycloak, or your own OAuth server)”
   • [github] “Team, server, and tool level permissions enforced through RBAC and Oauth2.1. Synced to your enterprise IDP (Okta, Azure AD, Google Workspace…”
   • [claimed-docs] “Put a governed endpoint in front of an MCP server someone else runs, so access control, audit, and upstream credentials apply without changi…”

   proof ↗⚑ flag

   Manufact—none0/10

   Manufact's evidence covers deployment, observability, publish checks, API keys, and MCP client connections for the platform-engineer's own organization, but there is no mention of per-end-user account isolation, scoped tokens per end user, or mechanisms for agents to act only on individual end-user credentials. Missing for 10: any documentation of end-user-scoped OAuth/token issuance, multi-tenant credential isolation, or per-user connected account management.

   • [claimed-docs] “API keys authenticate programmatic access to your organization, from the REST API, CI pipelines, and scripts.”
   • [claimed-docs] “exposes the platform, organizations, servers, deployments, logs, and observability, as MCP tools, so you can manage everything from Cursor, …”
   • [claimed-docs] “URL-connected servers that use the Manufact proxy can now be downgraded back to a direct external connection from Settings.”

   proof ↗⚑ flag

Oauth

1. developerLet the platform run OAuth flows for downstream tools and vault the resulting credentials encrypted, with automatic token refresh

   weight 3 · round to Gram (Speakeasy)
   Gram (Speakeasy)~partialclaimed5/10

   Docs confirm Gram lets you configure external OAuth 2.0 providers (Auth0, Okta, Keycloak, custom) for downstream tool auth and enforces OAuth2.1/RBAC for platform access, implying it manages OAuth flows for connected servers, but there is no explicit documentation of encrypted credential vaulting or automatic token refresh mechanics. Missing for 10: explicit description of credential encryption at rest, automatic refresh-token handling, and any hands-on/independent verification of these security claims.

   • [claimed-docs] “you can use any OAuth 2.0 provider you have set up (such as Auth0, Okta, Keycloak, or your own OAuth server)”
   • [github] “Team, server, and tool level permissions enforced through RBAC and Oauth2.1. Synced to your enterprise IDP (Okta, Azure AD, Google Workspace…”
   • [claimed-docs] “Put a governed endpoint in front of an MCP server someone else runs, so access control, audit, and upstream credentials apply without changi…”

   proof ↗⚑ flag

   Manufact—none0/10

   Manufact is an MCP server deployment/observability platform; there is no evidence it runs OAuth flows for downstream tools or vaults credentials with automatic token refresh. The community note about offline_access OAuth scope refers to reauthentication bugs, not a credential-vaulting feature. This is an applicable axis for a platform managing MCP servers/tools, but no supporting evidence exists.

   • [community] “Noted an issue with the offline_access OAuth scope causing frequent reauthentication requests from clients (citing a related Codex bug); man…”

   proof ↗⚑ flag

White label

1. platform-engineerWhite-label the authorization flow with my own branding and my own OAuth app credentials

   weight 1 · round to Gram (Speakeasy)
   Gram (Speakeasy)~partialclaimed6/10

   Gram supports bringing your own OAuth 2.0 provider (Auth0, Okta, Keycloak, or a custom server) so the underlying credentials are fully controlled by the platform-engineer, and RBAC/OAuth2.1 syncs to enterprise IDPs. However, there is no documented evidence of UI-level 'white-labeling' — custom branding/logo/domain on the consent or login screen itself — only backend provider configuration. Missing for 10: evidence of custom branding/theming of the OAuth consent screen or hosted login page, and confirmation that the authorization UI itself (not just the provider) can be rebranded.

   • [claimed-docs] “you can use any OAuth 2.0 provider you have set up (such as Auth0, Okta, Keycloak, or your own OAuth server)”
   • [github] “Team, server, and tool level permissions enforced through RBAC and Oauth2.1. Synced to your enterprise IDP (Okta, Azure AD, Google Workspace…”

   proof ↗⚑ flag

   Manufact—none0/10

   No evidence in the pack addresses white-labeling the OAuth/authorization flow, custom branding, or bringing your own OAuth app credentials; the docs cover deployment, testing, observability, API keys for programmatic access, but nothing about customizable authorization UI or client credentials. Missing for 10: any mention of white-label branding, custom OAuth client ID/secret configuration, or customizable consent screens.

   ⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round drawn
   Gram (Speakeasy)—none0/10

   The evidence pack covers MCP server hosting, governance, observability, RBAC, and CLI/skills features, but nowhere describes any bulk or batch operation capability (e.g., bulk approving tools, bulk managing servers, or batch tool invocation across many items). This is a plausible axis for a control-plane product managing many servers/toolsets, but no evidence shows it delivers.

   ⚑ flag

   Manufact—none0/10

   Manufact's docs describe per-server actions (deploy, test, publish-check) and an MCP tool surface for managing servers/deployments/logs, but nothing describes performing an operation across many items/servers simultaneously (e.g., bulk deploy, bulk test, bulk delete). The axis is plausible for a management platform like this, but no evidence supports it.

   • [claimed-docs] “Trigger deployment: deploy a branch on demand.”
   • [claimed-docs] “Manufact executes them across different MCP clients and LLM models”
   • [claimed-docs] “exposes the platform, organizations, servers, deployments, logs, and observability, as MCP tools, so you can manage everything from Cursor, …”
   • [claimed-docs] “The history table shows every deployment with its MCP URL, source branch and commit, duration, creation time, and the latest completed publi…”

   proof ↗⚑ flag

2. ai-native userSchedule recurring jobs or workflows

   weight 2 · round drawn
   Gram (Speakeasy)—none0/10

   Gram is an MCP control plane focused on generating, governing, observing, and distributing MCP servers/toolsets/skills, but nothing in the evidence describes scheduling recurring jobs or workflows (e.g., cron-like triggers or automated recurring execution) — it only covers request-time tool calls, logging, and access control.

   ⚑ flag

   Manufact—none0/10

   Manufact is an MCP server deployment/observability platform; nothing in the evidence describes scheduling recurring jobs, cron-like triggers, or recurring workflow automation—deployments are triggered on push/on-demand, not on a schedule. missing for 10: any cron/scheduler feature, recurring job configuration, or workflow automation trigger based on time intervals.

   • [claimed-docs] “Trigger deployment: deploy a branch on demand.”
   • [claimed-docs] “you can start a checklist automatically after each successful deployment”
   • [claimed-docs] “Set which Git branch serves your canonical production URL from the Deployment paths & triggers section”

   proof ↗⚑ flag

3. ai-native userVersion, review, and roll back my automations

   weight 1 · round to Manufact
   Gram (Speakeasy)~partialclaimed4/10

   Gram documents that skills are versioned (`SKILL.md` manifest) and provides audit/observability logs of tool calls, giving some traceability, but there is no evidence of a review workflow or an explicit rollback mechanism for automations/toolsets. missing for 10: explicit review process, rollback/revert functionality, version history UI for toolsets or MCP server configs.

   • [claimed-docs] “A skill is a versioned `SKILL.md` manifest: reusable instructions that agents load on demand.”
   • [claimed-docs] “the raw execution log for every tool call the platform observes across hosted MCP servers, tunneled and shadow MCP servers, skills, and loca…”

   proof ↗⚑ flag

   Manufact~partialclaimed5/10

   Manufact deploys from GitHub with branch-based preview environments, deployment history showing commit/branch and publish checklist scores, and on-demand trigger deployment—giving a form of version tracking and deployment history, but there is no explicit rollback mechanism (e.g., 'revert to previous deployment') or automation-specific review/approval workflow described. missing for 10: explicit rollback/revert capability for deployments, versioned diff/review UI for automations themselves (not just deployments), and independent confirmation that history table supports reverting to prior versions.

   • [claimed-docs] “Preview environments are created per branch. Each gets its own MCP URL”
   • [claimed-docs] “Trigger deployment: deploy a branch on demand.”
   • [claimed-docs] “The history table shows every deployment with its MCP URL, source branch and commit, duration, creation time, and the latest completed publi…”
   • [claimed-docs] “Set which Git branch serves your canonical production URL from the Deployment paths & triggers section”

   proof ↗⚑ flag

Bridge

1. developerBridge a locally running MCP server into the platform as a managed remote connection without deploying it

   weight 1 · round to Manufact
   Gram (Speakeasy)~partialclaimed5/10

   Docs mention 'tunneled MCP servers' alongside hosted and shadow servers in the observability/tool-logs reference, implying a mechanism to bridge a locally running server into the platform without redeploying it, and the third-party governance guide covers fronting externally-run servers. However, no dedicated guide, CLI command, or hands-on walkthrough for setting up a local-to-remote tunnel is included in the evidence pack. Missing for 10: a dedicated tunnel setup guide/CLI reference, concrete steps for exposing a local dev MCP server, and independent confirmation the tunnel actually works.

   • [claimed-docs] “the raw execution log for every tool call the platform observes across hosted MCP servers, tunneled and shadow MCP servers, skills, and loca…”
   • [claimed-docs] “Put a governed endpoint in front of an MCP server someone else runs, so access control, audit, and upstream credentials apply without changi…”
   • [claimed-docs] “install toolsets as MCP servers in AI agents, and keep the CLI itself up to date”

   proof ↗⚑ flag

   Manufact✓fullclaimed8/10

   Docs explicitly state you can 'paste your MCP URL. No deploy required: Manufact connects directly for Chat, testing, and the publish workflow,' and a changelog entry confirms URL-connected servers via proxy can be downgraded to direct external connection, showing this is a real managed remote-connection mode distinct from GitHub deploys. Missing for 10: independent/hands-on confirmation beyond vendor docs and more detail on how 'local' servers specifically get bridged (vs. any external URL).

   • [claimed-docs] “paste your MCP URL. No deploy required: Manufact connects directly for Chat, testing, and the publish workflow.”
   • [claimed-docs] “URL-connected servers that use the Manufact proxy can now be downgraded back to a direct external connection from Settings.”
   • [claimed-docs] “Preview environments are created per branch. Each gets its own MCP URL”

   proof ↗⚑ flag

Access control

1. platform-engineerRestrict which servers and individual tools an agent may call through allowlists or per-tool access control

   weight 2 · round to Gram (Speakeasy)
   Gram (Speakeasy)✓fullclaimed8/10

   Gram documents team/server/tool-level RBAC enforced with OAuth2.1 synced to enterprise IDPs, and explicitly blocks unsanctioned shadow MCP servers by default, directly matching allowlist/per-tool access control for a platform-engineer persona. This is corroborated by third-party rollout evidence (MoonPay) and governance guides for wrapping third-party MCP servers with access control. Missing for 10: independent hands-on verification of the RBAC/permission enforcement mechanics beyond vendor docs and GitHub README claims.

   • [github] “Team, server, and tool level permissions enforced through RBAC and Oauth2.1. Synced to your enterprise IDP (Okta, Azure AD, Google Workspace…”
   • [claimed-docs] “unsanctioned shadow MCP servers are blocked by default”
   • [claimed-docs] “MoonPay brought 200+ MCP servers under the MCP Gateway, going from proof of concept to a company-wide rollout in 30 days.”
   • [claimed-docs] “Put a governed endpoint in front of an MCP server someone else runs, so access control, audit, and upstream credentials apply without changi…”

   proof ↗⚑ flag

   Manufact—none0/10

   Manufact's docs cover deployment, testing, observability, and MCP server management, but there is no mention of allowlisting servers/tools or per-tool access control for agents. Missing for 10: any documentation of tool/server allowlists, per-tool permission scoping, or gateway-level access control policies.

   • [claimed-docs] “shows individual events: every request flowing through the gateway, refreshed every few seconds”
   • [claimed-docs] “exposes the platform, organizations, servers, deployments, logs, and observability, as MCP tools, so you can manage everything from Cursor, …”

   proof ↗⚑ flag

Audit

1. platform-engineerSee a log of every tool call that passed through the platform, with enough detail to audit or replay what an agent did

   weight 2 · round to Gram (Speakeasy)
   Gram (Speakeasy)✓fullclaimed8/10

   Gram documents a raw execution log capturing every tool call across hosted, tunneled, shadow MCP servers, skills, and local tools ([gram-docs-5]), plus usage/cost tracking and session deep-dives ([gram-gh-1]) that support audit and replay-style analysis. RBAC/OAuth-scoped access adds context for auditability of who called what ([gram-gh-2]). Missing for 10: independent/hands-on verification of log completeness or replay functionality specifically, and detail on log retention/export format.

   • [claimed-docs] “the raw execution log for every tool call the platform observes across hosted MCP servers, tunneled and shadow MCP servers, skills, and loca…”
   • [github] “Track AI usage across teams and measure impact with either tokens or cost. Deep dive expensive sessions, create budgets and measure tool eff…”
   • [github] “Team, server, and tool level permissions enforced through RBAC and Oauth2.1. Synced to your enterprise IDP (Okta, Azure AD, Google Workspace…”
   • [claimed-docs] “unsanctioned shadow MCP servers are blocked by default”

   proof ↗⚑ flag

   Manufact~partialclaimed6/10

   Manufact's observability dashboard shows individual gateway events refreshed every few seconds, with optional full request/response payload capture, giving platform-engineers a log of tool calls suitable for auditing; the deployment history table also links MCP URLs, commits, and checklist scores. However, there's no explicit mention of replay functionality, retention/export controls, or fine-grained per-tool-call audit trail structure beyond the event list. Missing for 10: explicit replay capability, log retention/export details, and independent/hands-on confirmation of audit completeness.

   • [claimed-docs] “shows individual events: every request flowing through the gateway, refreshed every few seconds”
   • [claimed-docs] “Toggle Capture payloads to also store full request and response bodies for each event”
   • [claimed-docs] “The history table shows every deployment with its MCP URL, source branch and commit, duration, creation time, and the latest completed publi…”

   proof ↗⚑ flag

Connect

1. developerPoint my MCP-compatible client at a hosted remote server URL and reach downstream tools without deploying any infrastructure

   weight 3 · round drawn
   Gram (Speakeasy)✓fullprobed8/10

   Gram's Control Plane auto-generates hosted MCP servers from an OpenAPI doc or TypeScript project, so developers get a remote server URL to point MCP clients at without deploying infra themselves, and this is corroborated by the CLI runtime probe and a real customer rollout (MoonPay's 200+ servers under the MCP Gateway). missing for 10: independent hands-on confirmation of an end-to-end client connecting to a hosted URL (only the CLI tool itself was runtime-verified, not the hosted server reachability).

   • [claimed-docs] “Built — the Control Plane generates the server itself from a first-party source: an OpenAPI document or a TypeScript Functions project.”
   • [claimed-docs] “MoonPay brought 200+ MCP servers under the MCP Gateway, going from proof of concept to a company-wide rollout in 30 days.”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/gram/): downloaded the official cli@0.16.0 release artifact (gram_dar…”
   • [claimed-docs] “the raw execution log for every tool call the platform observes across hosted MCP servers, tunneled and shadow MCP servers, skills, and loca…”

   proof ↗⚑ flag

   Manufact✓fullclaimed8/10

   Manufact's docs describe hosted MCP servers reachable via a per-deployment MCP URL (docs-4, docs-6), with dashboard flows to paste an existing MCP URL with 'No deploy required' for connecting clients (docs-2), and one-click/copy-paste client configuration (docs-18) so any MCP-compatible client can point at the hosted URL without managing infrastructure. Manufact Cloud handles the deployment/hosting infra itself (docs-1, docs-3, docs-16), fulfilling the 'no infrastructure to deploy' promise for the client side. Missing for 10: independent/hands-on confirmation that a client actually connects and reaches downstream tools end-to-end, and more detail on latency/reliability of the hosted proxy path.

   • [claimed-docs] “paste your MCP URL. No deploy required: Manufact connects directly for Chat, testing, and the publish workflow.”
   • [claimed-docs] “Preview environments are created per branch. Each gets its own MCP URL”
   • [claimed-docs] “The history table shows every deployment with its MCP URL, source branch and commit, duration, creation time, and the latest completed publi…”
   • [claimed-docs] “You get one-click installs or copy-paste configuration for”
   • [claimed-docs] “deploy from GitHub in minutes”

   proof ↗⚑ flag

Generate

1. developerTurn an existing API definition (OpenAPI) or custom code into a hosted MCP server without writing protocol plumbing

   weight 2 · round to Gram (Speakeasy)
   Gram (Speakeasy)✓fullprobed8/10

   Gram's Control Plane generates a hosted MCP server directly from an OpenAPI document or a TypeScript Functions project, with no protocol plumbing required from the developer, and a CLI/runtime probe confirms the tooling actually installs and runs. missing for 10: independent (non-vendor) hands-on account of the OpenAPI-to-MCP generation flow itself, and more detail on custom-code/TypeScript function support beyond the docs mention.

   • [claimed-docs] “Built — the Control Plane generates the server itself from a first-party source: an OpenAPI document or a TypeScript Functions project.”
   • [probe] “official CLI documented at https://www.speakeasy.com/docs/ai-control-plane/reference/command-line”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/gram/): downloaded the official cli@0.16.0 release artifact (gram_dar…”

   proof ↗⚑ flag

   Manufact~partialprobed5/10

   Manufact clearly lets developers deploy existing code (via GitHub templates or CLI) into a hosted MCP server without handling protocol plumbing, with deployment history, publish checks, and observability handled by the platform (manufact-docs-3, manufact-docs-16, manufact-docs-7). However, there is no evidence of an OpenAPI-to-MCP conversion feature — the openapi.json probe returned 404s and no docs mention importing an OpenAPI spec to generate an MCP server. missing for 10: OpenAPI spec ingestion/conversion workflow, independent confirmation of code-to-MCP transformation beyond GitHub template deploy.

   • [claimed-docs] “Selecting a template creates a new repository from the template in your connected GitHub account or organization and deploys it”
   • [claimed-docs] “Deploys the current project to Manufact Cloud.”
   • [claimed-docs] “runs an automated audit of your server for protocol compliance and platform readiness”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.manufact.com/openapi.json, https://docs.manufact.com/swagger.json, https://docs.manufac…”

   proof ↗⚑ flag

Publish

1. developerPublish my own MCP server on the platform for distribution, discovery, and installation by others

   weight 2 · round to Gram (Speakeasy)
   Gram (Speakeasy)✓fullprobed8/10

   Gram lets developers generate/host MCP servers from OpenAPI or TypeScript sources on its Control Plane, with governance, RBAC, tool logs, and a CLI/dashboard to publish, discover, and install servers into MCP clients, and the CLI was independently verified to install and run. missing for 10: no independent (non-vendor) evidence of a public discovery/marketplace listing showing other developers actually finding and installing a third party's published server, and no detail on versioning/publishing workflow specifics beyond docs.

   • [claimed-docs] “Built — the Control Plane generates the server itself from a first-party source: an OpenAPI document or a TypeScript Functions project.”
   • [claimed-docs] “install toolsets as MCP servers in AI agents, and keep the CLI itself up to date”
   • [claimed-docs] “An org admin working in Claude Code, Cursor, Codex, or another MCP client can find and add MCP servers, finish their setup, put them on a pl…”
   • [claimed-docs] “MoonPay brought 200+ MCP servers under the MCP Gateway, going from proof of concept to a company-wide rollout in 30 days.”
   • [github] “Team, server, and tool level permissions enforced through RBAC and Oauth2.1. Synced to your enterprise IDP (Okta, Azure AD, Google Workspace…”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/gram/): downloaded the official cli@0.16.0 release artifact (gram_dar…”
   • [probe] “official MCP server documented at https://www.speakeasy.com/docs/ai-control-plane/reference/platform-mcp”

   proof ↗⚑ flag

   Manufact~partialclaimed6/10

   Manufact provides a full publish workflow (deploy, publish checklist audit, end-to-end checks in ChatGPT/Claude) and one-click install/copy-paste configs for connecting clients, which covers distribution and installation by others. However there is no explicit evidence of a public marketplace, registry, or discovery mechanism where other developers browse and find published servers. Missing for 10: public server marketplace/registry for discovery, evidence of third-party users finding/installing another developer's server organically, independent confirmation of the publish-to-discovery pipeline.

   • [claimed-docs] “paste your MCP URL. No deploy required: Manufact connects directly for Chat, testing, and the publish workflow.”
   • [claimed-docs] “runs an automated audit of your server for protocol compliance and platform readiness”
   • [claimed-docs] “end-to-end checks that exercise your server live in ChatGPT and Claude”
   • [claimed-docs] “You get one-click installs or copy-paste configuration for”
   • [claimed-docs] “copy a fix prompt into your own coding agent, or run the built-in autofix flow”

   proof ↗⚑ flag

Analytics

1. platform-engineerSee usage analytics for the servers and tools my org consumes or publishes — call volumes, users, errors, and cost

   weight 2 · round to Gram (Speakeasy)
   Gram (Speakeasy)✓fullclaimed7/10

   Gram documents tool-call logs across hosted/tunneled/shadow servers and local tools, and the GitHub repo description explicitly claims usage tracking across teams by tokens/cost, budgets, and tool effectiveness measurement, directly matching call volumes/cost/usage. Errors are implied via raw execution logs but not explicitly broken out as an error-rate metric, and there's no independent/hands-on corroboration of the analytics dashboard itself. Missing for 10: explicit error-rate reporting UI evidence, independent third-party validation of the analytics dashboard beyond vendor docs/GitHub description.

   • [github] “Track AI usage across teams and measure impact with either tokens or cost. Deep dive expensive sessions, create budgets and measure tool eff…”
   • [claimed-docs] “the raw execution log for every tool call the platform observes across hosted MCP servers, tunneled and shadow MCP servers, skills, and loca…”
   • [claimed-docs] “unsanctioned shadow MCP servers are blocked by default”

   proof ↗⚑ flag

   Manufact~partialcommunity5/10

   Manufact's observability dashboard shows per-event traffic through the gateway with optional payload capture, giving call volume and request-level visibility, and community feedback (manufact-comm-1) praises 'analytics, logs' as impressive. However there's no documented breakdown by user, explicit error-rate reporting, or per-server/tool cost analytics — billing shows only org-wide spend limits, not tool-level cost. Missing for 10: per-user attribution, explicit error/failure metrics, and per-server/tool cost breakdowns in the observability view.

   • [claimed-docs] “shows individual events: every request flowing through the gateway, refreshed every few seconds”
   • [claimed-docs] “Toggle Capture payloads to also store full request and response bodies for each event”
   • [claimed-docs] “Owners and admins can set a monthly spend limit for metered usage beyond included credits from the Billing page.”
   • [community] “Impressed with the demo video, particularly the analytics, logs, and test suite features, but pricing page credit system unclear and no visi…”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round drawn
   Gram (Speakeasy)~partialprobed6/10

   Gram exposes a Platform MCP server and CLI that let an org admin add/configure MCP servers, author skills, manage risk policies, and read observability data without opening the dashboard (gram-docs-4, gram-docs-7, gram-probe-5), and the CLI is confirmed to actually install and run (gram-probe-rt-1). However, no public REST/OpenAPI surface was found (gram-probe-3: openapi.json paths all 404), so full UI-parity via a general API is unconfirmed. Missing for 10: a documented general-purpose REST/OpenAPI API covering all dashboard actions, and independent confirmation that every UI feature (not just admin/config tasks) is reachable programmatically.

   • [claimed-docs] “install toolsets as MCP servers in AI agents, and keep the CLI itself up to date”
   • [claimed-docs] “An org admin working in Claude Code, Cursor, Codex, or another MCP client can find and add MCP servers, finish their setup, put them on a pl…”
   • [probe] “official CLI documented at https://www.speakeasy.com/docs/ai-control-plane/reference/command-line”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/gram/): downloaded the official cli@0.16.0 release artifact (gram_dar…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://www.speakeasy.com/openapi.json, https://www.speakeasy.com/swagger.json, https://www.speakeas…”

   proof ↗⚑ flag

   Manufact~partialprobed6/10

   Manufact ships an official MCP server exposing platform, orgs, servers, deployments, logs, and observability as tools, plus a CLI and REST API keys for programmatic access (manufact-docs-14,15,16,17), giving broad UI-parity for AI-native users. However, the probe found no discoverable OpenAPI/swagger spec (404s), so full REST API coverage (e.g. billing, connect-clients config, publish checklist) is unverified and some UI actions may lack documented programmatic equivalents. missing for 10: a published OpenAPI/REST API reference confirming full parity, and independent/hands-on confirmation that every UI action is API-reachable.

   • [claimed-docs] “exposes the platform, organizations, servers, deployments, logs, and observability, as MCP tools, so you can manage everything from Cursor, …”
   • [claimed-docs] “Run your MCP server locally with hot reload + inspector”
   • [claimed-docs] “Deploys the current project to Manufact Cloud.”
   • [claimed-docs] “API keys authenticate programmatic access to your organization, from the REST API, CI pipelines, and scripts.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.manufact.com/openapi.json, https://docs.manufact.com/swagger.json, https://docs.manufac…”
   • [probe] “official MCP server documented at https://docs.manufact.com/mcp”
   • [probe] “official CLI documented at https://docs.manufact.com/cli”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round drawn
   Gram (Speakeasy)—none0/10

   The evidence describes Gram's inputs (OpenAPI specs, SKILL.md manifests) and platform features like RBAC, logging, and CLI, but there is no documented data-export feature letting a user pull out their configured toolsets, logs, policies, or org data in open formats to migrate away. Axis applies to a platform like this but no evidence supports it.

   ⚑ flag

   Manufact—none0/10

   Evidence shows Manufact deploys MCP servers from user-owned GitHub repos and offers an API/CLI for programmatic access (manufact-docs-16, manufact-docs-17, manufact-probe-4), but there is no documentation of an explicit 'export all your data' feature covering logs, observability payloads, deployment history, or test results in open/portable formats, nor any account-closure data-export workflow.

   • [claimed-docs] “Selecting a template creates a new repository from the template in your connected GitHub account or organization and deploys it”
   • [claimed-docs] “API keys authenticate programmatic access to your organization, from the REST API, CI pipelines, and scripts.”
   • [claimed-docs] “shows individual events: every request flowing through the gateway, refreshed every few seconds”
   • [probe] “official CLI documented at https://docs.manufact.com/cli”

   proof ↗⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round drawn
   Gram (Speakeasy)—none0/10

   The evidence shows a public GitHub repository (github.com/speakeasy-api/gram) exists, but nothing in the pack states or documents an open-source license or that the full source is freely readable/licensed for reuse; docs focus on hosted product features (CLI, MCP servers, RBAC, observability) rather than source availability or licensing terms.

   • [github] “Track AI usage across teams and measure impact with either tokens or cost. Deep dive expensive sessions, create budgets and measure tool eff…”
   • [github] “Team, server, and tool level permissions enforced through RBAC and Oauth2.1. Synced to your enterprise IDP (Okta, Azure AD, Google Workspace…”

   proof ↗⚑ flag

   Manufact—none0/10

   No evidence anywhere in the pack indicates Manufact's own source code is open-licensed or publicly available; all references are to deploying/managing user MCP servers, not Manufact's own codebase. This is a fair question (SaaS platforms can be open-source) but no evidence supports it.

   ⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round drawn
   Gram (Speakeasy)—none0/10

   While Gram's code is hosted on GitHub (gram-gh-1, gram-gh-2), the evidence pack contains no documentation, guide, or deployment instructions describing a supported self-hosted deployment of the core Control Plane — all docs describe the hosted SaaS platform, dashboard, and CLI/skill features rather than self-hosting.

   • [github] “Track AI usage across teams and measure impact with either tokens or cost. Deep dive expensive sessions, create budgets and measure tool eff…”
   • [github] “Team, server, and tool level permissions enforced through RBAC and Oauth2.1. Synced to your enterprise IDP (Okta, Azure AD, Google Workspace…”

   proof ↗⚑ flag

   Manufact—none0/10

   All evidence describes deploying MCP servers to 'Manufact Cloud' (docs-1, docs-3, docs-16) and a CLI for local dev/testing (docs-15) plus REST/API keys for programmatic access (docs-17), but nothing indicates the Manufact platform itself (dashboard, gateway, observability, publish-checks) can be self-hosted outside their cloud.

   • [claimed-docs] “deploy from GitHub in minutes”
   • [claimed-docs] “Deploys the current project to Manufact Cloud.”
   • [claimed-docs] “API keys authenticate programmatic access to your organization, from the REST API, CI pipelines, and scripts.”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.manufact.com/llms.txt # Docs - Manufact Cloud: Deploy, monitor, and manage MCP servers on Manufact…”

   proof ↗⚑ flag

Pricing

1. developerPrototype my agent's tool connections on a meaningful free tier before paying anything

   weight 1 · round to Manufact
   Gram (Speakeasy)—none0/10

   The evidence pack contains no mention of pricing tiers, a free tier, or cost thresholds for Gram — nothing describes what is available without payment. This is a fair pricing-plans axis for any commercial dev tool, but there's simply no documentation or citation about free-tier limits or trial terms.

   ⚑ flag

   Manufact~partialcommunity3/10

   Docs mention 'included credits' and a spend-limit control for usage beyond them, implying some free/starter allotment, but there is no explicit description of a free tier's scope or limits for connecting tools, and community feedback explicitly calls the credit/pricing system unclear ahead of deployment. missing for 10: explicit free-tier definition, limits on tool connections/servers, and confirmation that the free tier is 'meaningful' for prototyping.

   • [claimed-docs] “Owners and admins can set a monthly spend limit for metered usage beyond included credits from the Billing page.”
   • [community] “Impressed with the demo video, particularly the analytics, logs, and test suite features, but pricing page credit system unclear and no visi…”

   proof ↗⚑ flag

2. developerSee transparent pricing per tool call or connection plus documented rate limits before committing to the platform

   weight 2 · round to Manufact
   Gram (Speakeasy)—none0/10

   No evidence pack item mentions pricing, cost per tool call/connection, or documented rate limits; all citations concern architecture, RBAC, skills, and CLI features. Absence of evidence for an applicable and standard pricing-transparency axis means this must be scored none.

   ⚑ flag

   Manufact!disputedcontradicted3/10

   Manufact documents a spend-limit/billing feature (monthly cap for metered usage beyond included credits) but no per-tool-call pricing or rate-limit documentation exists, and a community reviewer explicitly reports the credit/pricing system is unclear with no visibility into pay-as-you-go costs after plan limits, contradicting any claim of transparent pricing. missing for 10: per-tool-call/connection pricing breakdown, documented rate limits, resolution of the community-reported cost opacity.

   • [claimed-docs] “Owners and admins can set a monthly spend limit for metered usage beyond included credits from the Billing page.”
   • [community] “Impressed with the demo video, particularly the analytics, logs, and test suite features, but pricing page credit system unclear and no visi…”

   proof ↗⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round drawn
   Gram (Speakeasy)—none0/10

   No evidence in the pack mentions data residency, region selection, or geographic storage controls for Gram; the documentation focuses on governance, RBAC, tool logs, and MCP server hosting but never addresses where data is physically stored.

   ⚑ flag

   Manufact—none0/10

   No evidence anywhere in the pack mentions data region selection, residency options, or geographic deployment controls; Manufact Cloud deployment docs discuss branches, environments, and MCP URLs but never data locality. Missing for 10: any mention of region/residency selection, data center location options, or compliance certifications tied to geography.

   ⚑ flag

2. ai-native userControl data retention and deletion

   weight 2 · round drawn
   Gram (Speakeasy)—none0/10

   The evidence pack covers governance, RBAC, audit logs, OAuth, and shadow-server blocking, but no citation addresses data retention policies, deletion controls, or user data lifecycle management for the AI-native platform.

   ⚑ flag

   Manufact—none0/10

   Manufact is an MCP server deployment/observability platform; no evidence in the pack addresses data retention policies, deletion controls, or data lifecycle management for stored logs, payloads, or deployment data. Observability docs mention capturing payloads (manufact-docs-13) but nothing about retention periods or user-initiated deletion.

   ⚑ flag

3. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   Gram (Speakeasy)—none0/10

   No evidence in the pack addresses whether Gram's CLI, dashboard, or platform collects vendor-side telemetry/usage data or offers an opt-out; the only related items describe customer-facing usage analytics (tracking AI agent activity), not the product's own telemetry practices. Missing for 10: any mention of a telemetry setting, opt-out flag, or privacy policy statement about data collection from the CLI/platform itself.

   • [github] “Track AI usage across teams and measure impact with either tokens or cost. Deep dive expensive sessions, create budgets and measure tool eff…”

   proof ↗⚑ flag

   Manufact—none0/10

   No evidence pack item mentions telemetry, usage tracking, or an opt-out/privacy setting for such data collection; the docs cover deployment, observability of user's own MCP traffic, and billing, but nothing about Manufact's own telemetry practices.

   ⚑ flag

Api

1. ai agentQuery the platform's registry or catalog through a documented API to discover servers and tools programmatically

   weight 3 · round to Gram (Speakeasy)
   Gram (Speakeasy)~partialprobed6/10

   Gram documents an official 'platform-mcp' server that lets an agent working through an MCP client find and add MCP servers, manage toolsets, and read observability data programmatically (gram-docs-7, gram-probe-4), and the CLI can install toolsets as MCP servers (gram-docs-4/5). This covers the discovery story via the MCP protocol itself acting as the documented API, but there is no confirmed general-purpose REST/OpenAPI catalog endpoint (the openapi probe returned 404s) and no independent hands-on proof of the discovery flow itself. Missing for 10: a documented REST/OpenAPI registry endpoint, and third-party/hands-on verification that an agent can actually enumerate servers/tools via the platform MCP server.

   • [claimed-docs] “An org admin working in Claude Code, Cursor, Codex, or another MCP client can find and add MCP servers, finish their setup, put them on a pl…”
   • [claimed-docs] “install toolsets as MCP servers in AI agents, and keep the CLI itself up to date”
   • [probe] “official MCP server documented at https://www.speakeasy.com/docs/ai-control-plane/reference/platform-mcp”
   • [probe] “PROBE openapi: all candidate paths 404 (https://www.speakeasy.com/openapi.json, https://www.speakeasy.com/swagger.json, https://www.speakeas…”

   proof ↗⚑ flag

   Manufact~partialprobed5/10

   Manufact exposes an official MCP server (manufact-docs-14) that surfaces platform data—organizations, servers, deployments, logs—as MCP tools, letting an AI agent programmatically query the servers it has deployed from clients like Cursor or Claude. However, there is no documented public REST/OpenAPI catalog for external discovery (the openapi probe returned 404s), and the emphasis is on managing one's own deployed servers rather than browsing a broader tool/server catalog. Missing for 10: a documented REST/OpenAPI registry endpoint, evidence of cross-tenant or public catalog discovery, and independent confirmation of programmatic catalog querying beyond the MCP management tools.

   • [claimed-docs] “exposes the platform, organizations, servers, deployments, logs, and observability, as MCP tools, so you can manage everything from Cursor, …”
   • [claimed-docs] “API keys authenticate programmatic access to your organization, from the REST API, CI pipelines, and scripts.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.manufact.com/openapi.json, https://docs.manufact.com/swagger.json, https://docs.manufac…”
   • [probe] “official MCP server documented at https://docs.manufact.com/mcp”

   proof ↗⚑ flag

Metadata

1. developerInspect a listed server's tools, input schemas, transport, and hosting attributes before connecting to it

   weight 2 · round to Gram (Speakeasy)
   Gram (Speakeasy)~partialclaimed5/10

   Gram's in-dashboard 'Connect' harness lets a developer pick a listed server, authenticate, and chat with a model that calls its tools while inspecting logs, which implies pre-connection tool inspection, and servers are generated from OpenAPI/TypeScript sources implying schema definitions exist. However, no evidence explicitly documents a UI or API for viewing input schemas, transport type, or hosting attributes for a listed server. Missing for 10: explicit documentation of schema viewer, transport metadata display, hosting attribute display, and independent/hands-on confirmation of this inspection workflow.

   • [claimed-docs] “An in-dashboard agent harness for testing MCP servers before connecting a real client: pick a server, authenticate, chat with a model that c…”
   • [claimed-docs] “Built — the Control Plane generates the server itself from a first-party source: an OpenAPI document or a TypeScript Functions project.”

   proof ↗⚑ flag

   Manufact~partialclaimed4/10

   Manufact's dashboard exposes hosting/deployment attributes (MCP URL, branch, commit, publish-checklist score) and a testing flow that runs across MCP clients/models, which implies some pre-connection inspection, but there's no explicit documentation of viewing a server's tool list or input schemas before connecting. Missing for 10: explicit tool/input-schema inspection UI, transport-type display, and any independent/hands-on confirmation of this inspection workflow.

   • [claimed-docs] “paste your MCP URL. No deploy required: Manufact connects directly for Chat, testing, and the publish workflow.”
   • [claimed-docs] “The history table shows every deployment with its MCP URL, source branch and commit, duration, creation time, and the latest completed publi…”
   • [claimed-docs] “runs an automated audit of your server for protocol compliance and platform readiness”
   • [claimed-docs] “Manufact executes them across different MCP clients and LLM models”

   proof ↗⚑ flag

Playground

1. developerExercise an MCP server's tools from a browser-based inspector or playground before integrating it

   weight 2 · round to Gram (Speakeasy)
   Gram (Speakeasy)✓fullclaimed8/10

   Gram provides an in-dashboard agent harness (gram-docs-8) explicitly described as a way to pick a server, authenticate, chat with a model that invokes tools, and inspect logs before connecting a real client — directly matching the story of exercising tools from a browser-based playground pre-integration. This is first-party documentation with a clear feature name ('Connect') dedicated to this exact workflow. Missing for 10: independent/hands-on user reports confirming the playground works as described, and more detail on tool-level interaction granularity within the UI.

   • [claimed-docs] “An in-dashboard agent harness for testing MCP servers before connecting a real client: pick a server, authenticate, chat with a model that c…”

   proof ↗⚑ flag

   Manufact~partialclaimed6/10

   Manufact's dashboard lets developers paste an MCP URL for direct connection and 'Chat, testing' before publishing (manufact-docs-2), and testing runs across different MCP clients/models (manufact-docs-11), which supports exercising tools pre-integration; the CLI also offers a local inspector (manufact-docs-15). However, this is presented as an integrated dashboard/testing workflow rather than a dedicated browser-based 'inspector/playground' UI, and no screenshots, hands-on walkthrough, or independent corroboration of the playground UX are provided. Missing for 10: concrete UI evidence/screenshots of a dedicated tool-invocation playground, independent hands-on confirmation, and detail on how individual tool calls are exercised interactively.

   • [claimed-docs] “paste your MCP URL. No deploy required: Manufact connects directly for Chat, testing, and the publish workflow.”
   • [claimed-docs] “Manufact executes them across different MCP clients and LLM models”
   • [claimed-docs] “Run your MCP server locally with hot reload + inspector”

   proof ↗⚑ flag

Search

1. developerSearch and browse a large catalog of MCP servers and tools by capability before wiring anything into my agent

   weight 3 · round to Gram (Speakeasy)
   Gram (Speakeasy)~partialclaimed3/10

   Gram documents that an admin can 'find and add MCP servers' via the platform MCP client integration, implying some discovery capability, but there is no evidence of a searchable/browsable catalog of MCP servers by capability, catalog size, or a discovery UI — Gram is primarily framed as a control plane for generating/governing your own servers rather than a registry of third-party servers. missing for 10: a documented catalog/search UI, evidence of catalog scale or third-party server listings, and any capability-based filtering/browsing mechanism.

   • [claimed-docs] “An org admin working in Claude Code, Cursor, Codex, or another MCP client can find and add MCP servers, finish their setup, put them on a pl…”
   • [claimed-docs] “Put a governed endpoint in front of an MCP server someone else runs, so access control, audit, and upstream credentials apply without changi…”

   proof ↗⚑ flag

   Manufact—none0/10

   Manufact is a deploy/manage/observe platform for MCP servers you build and own — there is no evidence of a searchable public catalog/registry of third-party MCP servers or tools that a developer can browse by capability before wiring into an agent. All docs relate to deploying, testing, and monitoring your own server, not discovering existing ones.

   ⚑ flag

Cli

1. developerDrive the publish-and-deploy workflow from an official CLI — authenticate, stage or build a server, push it, and install it into agents

   weight 2 · round to Gram (Speakeasy)
   Gram (Speakeasy)~partialprobed7/10

   There's a documented, runtime-verified official CLI (gram-docs-4, gram-probe-5, gram-probe-rt-1) that supports installing toolsets as MCP servers into agents, plus docs on building servers from OpenAPI/TS and platform-mcp integration for org admins to add/configure servers from within MCP clients. However, the evidence doesn't clearly walk through explicit CLI subcommands for authenticate/stage/build/push in one coherent flow — most detail is on install and general CLI existence rather than a full documented publish pipeline. Missing for 10: explicit CLI command reference showing login/auth, build/stage, and push subcommands with example output, and independent (non-vendor) confirmation of the full workflow beyond version-check.

   • [claimed-docs] “install toolsets as MCP servers in AI agents, and keep the CLI itself up to date”
   • [claimed-docs] “Built — the Control Plane generates the server itself from a first-party source: an OpenAPI document or a TypeScript Functions project.”
   • [probe] “official CLI documented at https://www.speakeasy.com/docs/ai-control-plane/reference/command-line”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/gram/): downloaded the official cli@0.16.0 release artifact (gram_dar…”
   • [claimed-docs] “An org admin working in Claude Code, Cursor, Codex, or another MCP client can find and add MCP servers, finish their setup, put them on a pl…”

   proof ↗⚑ flag

   Manufact~partialprobed6/10

   Manufact has an official CLI documented with local run/hot-reload and a deploy command that pushes the project to Manufact Cloud (manufact-docs-15, manufact-docs-16, manufact-probe-4), plus API keys for programmatic/CI authentication (manufact-docs-17) and one-click/copy-paste install into agent clients (manufact-docs-18). However, the docs emphasize a GitHub/dashboard-driven deploy flow rather than a fully CLI-driven auth→build→push→install pipeline, and there's no explicit CLI login/auth command or CLI-based install-into-agent step shown. missing for 10: explicit CLI authenticate command, CLI-triggered install-into-agent step, and independent/hands-on confirmation that the full workflow (auth, build, push, install) works end-to-end via CLI alone.

   • [claimed-docs] “Run your MCP server locally with hot reload + inspector”
   • [claimed-docs] “Deploys the current project to Manufact Cloud.”
   • [claimed-docs] “API keys authenticate programmatic access to your organization, from the REST API, CI pipelines, and scripts.”
   • [claimed-docs] “You get one-click installs or copy-paste configuration for”
   • [probe] “official CLI documented at https://docs.manufact.com/cli”

   proof ↗⚑ flag

Frameworks

1. developerPlug the platform's tools into major agent frameworks and SDKs (Vercel AI SDK, LangChain, CrewAI, OpenAI or Anthropic SDKs) with documented adapters

   weight 2 · round drawn
   Gram (Speakeasy)—none0/10

   Gram documents MCP server generation, its own CLI, and MCP-client compatibility (Claude Code, Cursor, Codex), but there is no evidence of documented adapters or integration guides for Vercel AI SDK, LangChain, CrewAI, or OpenAI/Anthropic SDKs specifically.

   ⚑ flag

   Manufact—none0/10

   Manufact's docs describe connecting hosted MCP servers to MCP clients like Cursor, Claude, and ChatGPT (manufact-docs-14, manufact-docs-18), but there is no evidence of documented adapters or integration guides for agent frameworks/SDKs such as Vercel AI SDK, LangChain, CrewAI, or OpenAI/Anthropic SDKs. Missing for 10: any mention of LangChain, CrewAI, Vercel AI SDK, or OpenAI/Anthropic SDK integration guides or adapters.

   • [claimed-docs] “exposes the platform, organizations, servers, deployments, logs, and observability, as MCP tools, so you can manage everything from Cursor, …”
   • [claimed-docs] “You get one-click installs or copy-paste configuration for”

   proof ↗⚑ flag

Security

1. platform-engineerRely on the platform scanning or inspecting listed servers for malicious or vulnerable behavior before my agents use them

   weight 2 · round to Gram (Speakeasy)
   Gram (Speakeasy)~partialclaimed5/10

   Gram provides governance controls relevant to tool curation — RBAC/OAuth2.1 access control, full audit/tool-call logging across hosted, tunneled and shadow servers, 'risk policies' management via the platform MCP server, and blocking of unsanctioned shadow MCP servers by default — plus an in-dashboard harness to test a server's tool calls before connecting a real client. However none of this is documented as active scanning for malicious code or known vulnerabilities in listed servers; it's closer to access governance, logging and manual testing than automated security scanning. Missing for 10: explicit vulnerability/malware scanning of server code or behavior, third-party security audit integration, and independent evidence that risk policies actually catch malicious servers.

   • [claimed-docs] “unsanctioned shadow MCP servers are blocked by default”
   • [claimed-docs] “An org admin working in Claude Code, Cursor, Codex, or another MCP client can find and add MCP servers, finish their setup, put them on a pl…”
   • [claimed-docs] “the raw execution log for every tool call the platform observes across hosted MCP servers, tunneled and shadow MCP servers, skills, and loca…”
   • [claimed-docs] “An in-dashboard agent harness for testing MCP servers before connecting a real client: pick a server, authenticate, chat with a model that c…”
   • [github] “Team, server, and tool level permissions enforced through RBAC and Oauth2.1. Synced to your enterprise IDP (Okta, Azure AD, Google Workspace…”

   proof ↗⚑ flag

   Manufact—none0/10

   Manufact's publish-checklist audits cover protocol compliance, platform readiness, and end-to-end functional tests (manufact-docs-7, manufact-docs-8), but there is no evidence these checks screen for malicious behavior or security vulnerabilities in listed servers before agent use. Missing for 10: any documented security/malware scanning step, vulnerability detection, or a curated registry vetting process distinct from functional/protocol compliance testing.

   • [claimed-docs] “runs an automated audit of your server for protocol compliance and platform readiness”
   • [claimed-docs] “end-to-end checks that exercise your server live in ChatGPT and Claude”
   • [claimed-docs] “copy a fix prompt into your own coding agent, or run the built-in autofix flow”
   • [claimed-docs] “you can start a checklist automatically after each successful deployment”

   proof ↗⚑ flag

Signals

1. developerCompare servers using published quality signals — usage counts, health or liveness checks, scores, or verification badges

   weight 2 · round to Manufact
   Gram (Speakeasy)—none0/10

   Gram provides usage/cost tracking (gram-gh-1) and tool-call logs (gram-docs-5), plus governance controls like blocking shadow servers (gram-docs-10), but there is no evidence of published quality signals such as health/liveness checks, verification badges, or comparative quality scores that a developer could use to compare servers side-by-side. Missing for 10: liveness/health-check indicators, explicit quality/verification badges, and any server-comparison or ranking UI.

   • [github] “Track AI usage across teams and measure impact with either tokens or cost. Deep dive expensive sessions, create budgets and measure tool eff…”
   • [claimed-docs] “the raw execution log for every tool call the platform observes across hosted MCP servers, tunneled and shadow MCP servers, skills, and loca…”
   • [claimed-docs] “unsanctioned shadow MCP servers are blocked by default”

   proof ↗⚑ flag

   Manufact~partialclaimed3/10

   Manufact provides a 'publish checklist score' and automated audits for protocol compliance/readiness on servers you deploy (manufact-docs-6, manufact-docs-7), which is a quality signal, but this is scoped to your own deployed servers rather than a public catalog for comparing third-party MCP servers by usage counts, health/liveness, or verification badges. Missing for 10: cross-server marketplace/directory view, usage/popularity counts, verification badges, and independent evidence that these scores are used to compare multiple distinct servers rather than track one's own deployment quality.

   • [claimed-docs] “The history table shows every deployment with its MCP URL, source branch and commit, duration, creation time, and the latest completed publi…”
   • [claimed-docs] “runs an automated audit of your server for protocol compliance and platform readiness”
   • [claimed-docs] “you can start a checklist automatically after each successful deployment”

   proof ↗⚑ flag

Tool search

1. ai agentSearch and load tools at runtime through meta-tools instead of having thousands of tool definitions stuffed into my context

   weight 2 · round drawn
   Gram (Speakeasy)—none0/10

   Gram's evidence covers MCP server hosting, toolset governance, OAuth, RBAC, and 'skills' loaded on demand, but none of it describes a runtime meta-tool (e.g., search_tools/load_tool) mechanism that lets an agent dynamically discover and load individual tool definitions instead of receiving a large static toolset. The closest analog (SKILL.md manifests loaded on demand) is a different mechanism from runtime tool search across a large tool catalog.

   ⚑ flag

   Manufact—none0/10

   Manufact is a platform for deploying, monitoring, and managing MCP servers (deployments, observability, publish checks), not a runtime tool-search/meta-tool mechanism for agents to dynamically discover and load tools instead of loading full definitions. No evidence describes dynamic tool search/loading or meta-tool patterns to reduce context bloat.

   ⚑ flag