Smithery vs Manufact

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round to Manufact
   Smithery~partialprobed6/10

   llms.txt and docs.md are confirmed 404, so the specific llms.txt convention isn't supported, but Smithery does expose agent-oriented docs alternatives: docs served live over MCP (JSON-RPC initialize succeeds at /docs/mcp) and a machine-readable OpenAPI 3.1 spec at /docs/openapi.json that an agent could consume to understand the API. missing for 10: an actual llms.txt file, and any docs indicating agents are expected to consume these alternate machine-readable formats as a first-class 'llms.txt-style' workflow.

   • [probe] “PROBE llms.txt: HTTP 404 at https://smithery.ai/llms.txt”
   • [probe] “PROBE docs-md: HTTP 404 at https://smithery.ai/docs.md”
   • [probe] “PROBE openapi (2026-09-05): GET https://smithery.ai/docs/openapi.json returned HTTP 200 with an OpenAPI 3.1.0 description (35 documented pat…”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/smithery/): a JSON-RPC initialize POST to https://smithery.ai/docs/mc…”

   proof ↗⚑ flag

   Manufact✓fullprobed9/10

   A direct probe confirms llms.txt is live at https://docs.manufact.com/llms.txt (HTTP 200) with a structured index of docs, giving agents a machine-readable entry point, and this is corroborated by the docs site itself supporting agent-oriented navigation (mcp.md, cli.md, dashboard docs) that an agent could traverse. Missing for 10: no explicit vendor statement promoting llms.txt as a supported agent workflow, and no independent community confirmation of an agent successfully consuming it.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.manufact.com/llms.txt # Docs - Manufact Cloud: Deploy, monitor, and manage MCP servers on Manufact…”
   • [claimed-docs] “exposes the platform, organizations, servers, deployments, logs, and observability, as MCP tools, so you can manage everything from Cursor, …”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round drawn
   Smithery✓fullprobed7/10

   Smithery ships a CLI (search/add/call tools) and a documented REST API with an OpenAPI 3.1 spec, service tokens scoped for AI agents/automation, and a live registry API queryable without interactive auth — all of which support scripted/headless use in CI pipelines. Runtime probes confirm the CLI installs and runs (`smithery --version`) and the API/registry endpoints respond programmatically. missing for 10: explicit CI/CD pipeline examples or GitHub Actions integration docs, and independent reports of real-world CI usage.

   • [github] “smithery mcp search [term] # Search the Smithery registry smithery mcp add <url> # Add an MCP server connectio…”
   • [github] “smithery tool find [query] # Search tools by name or intent smithery tool call <connection> <tool> [args] # Call a tool”
   • [claimed-docs] “Smithery gives you a simple REST interface for connecting to MCP servers. Instead of implementing the MCP protocol directly, handling OAuth …”
   • [claimed-docs] “Service tokens let you safely expose Smithery to browsers, mobile apps, and AI agents without leaking your API key.”
   • [probe] “PROBE registry-api (2026-09-05): GET https://registry.smithery.ai/servers?pageSize=3 with NO API key returned HTTP 200 with a machine-readab…”
   • [probe] “PROBE openapi (2026-09-05): GET https://smithery.ai/docs/openapi.json returned HTTP 200 with an OpenAPI 3.1.0 description (35 documented pat…”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/smithery/): a JSON-RPC initialize POST to https://smithery.ai/docs/mc…”

   proof ↗⚑ flag

   Manufact✓fullprobed7/10

   Manufact ships a CLI for local runs/deploys and API keys explicitly designed for 'REST API, CI pipelines, and scripts,' supporting headless automation of deployments and management outside the dashboard UI (manufact-docs-15, manufact-docs-16, manufact-docs-17, manufact-docs-5). Missing for 10: a concrete CI pipeline example/integration guide (e.g. GitHub Actions snippet), an OpenAPI spec (probe found 404s), and independent/hands-on corroboration of CI usage.

   • [claimed-docs] “Run your MCP server locally with hot reload + inspector”
   • [claimed-docs] “Deploys the current project to Manufact Cloud.”
   • [claimed-docs] “API keys authenticate programmatic access to your organization, from the REST API, CI pipelines, and scripts.”
   • [claimed-docs] “Trigger deployment: deploy a branch on demand.”
   • [probe] “official CLI documented at https://docs.manufact.com/cli”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.manufact.com/openapi.json, https://docs.manufact.com/swagger.json, https://docs.manufac…”

   proof ↗⚑ flag

3. ai-native userPlug MCP servers into this product so it can use their tools

   weight 3 · round to Smithery
   Smithery✓fullprobed8/10

   Smithery's core purpose is connecting/aggregating MCP servers: docs and probes confirm a REST/registry interface, hosted MCP endpoints with OAuth, deep links, uplink for local servers, and CLI/SDK tooling for adding and calling tools across connections, with runtime probes corroborating a live registry and per-server endpoints. Community reports of tool-description injection risks are a security caveat, not evidence the connection/tool-use mechanism fails. missing for 10: independent hands-on confirmation that an AI agent (not just probes) successfully invokes tools end-to-end across multiple aggregated servers, and resolution of the tool-description-injection security concerns.

   • [claimed-docs] “Smithery gives you a simple REST interface for connecting to MCP servers. Instead of implementing the MCP protocol directly, handling OAuth …”
   • [claimed-docs] “Uplink exposes an MCP server running on any machine as a regular Smithery connection.”
   • [claimed-docs] “Deep links provide a seamless way to integrate Smithery MCPs into supported clients. When a user clicks a deep link from our server page, th…”
   • [claimed-docs] “Smithery handles OAuth, token refresh, and connection management automatically.”
   • [claimed-docs] “Connect to multiple servers and aggregate their tools”
   • [probe] “PROBE registry-api (2026-09-05): GET https://registry.smithery.ai/servers?pageSize=3 with NO API key returned HTTP 200 with a machine-readab…”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://server.smithery.ai/exa/mcp returned HTTP 401 with an OAuth challenge — the hoste…”
   • [github] “smithery mcp search [term] # Search the Smithery registry smithery mcp add <url> # Add an MCP server connectio…”
   • [github] “smithery tool find [query] # Search tools by name or intent smithery tool call <connection> <tool> [args] # Call a tool”
   • [community] “Scanned the top 100 servers on Smithery's registry; 22 of 100 flagged at least one security finding (28 total, 4 critical, 24 high), most co…”

   proof ↗⚑ flag

   Manufact✓fullclaimed7/10

   Manufact lets users paste an MCP server URL and Manufact 'connects directly for Chat, testing, and the publish workflow,' meaning the platform itself invokes the pasted server's tools; testing docs also describe executing across different MCP clients/models. missing for 10: no independent/hands-on confirmation of actual tool invocation behavior, and no detail on tool-call mechanics or error handling.

   • [claimed-docs] “paste your MCP URL. No deploy required: Manufact connects directly for Chat, testing, and the publish workflow.”
   • [claimed-docs] “Manufact executes them across different MCP clients and LLM models”
   • [claimed-docs] “You get one-click installs or copy-paste configuration for”

   proof ↗⚑ flag

4. ai-native userConnect an agent via an official MCP server

   weight 3 · round to Smithery
   Smithery✓fullprobed9/10

   Smithery is a hosting/registry platform (not itself an agent), so the exception applies and publishing an official MCP server endpoint is a fair, evidenced axis: docs describe OAuth-handled connections, deep linking, uplink, and typed SDKs, and independent probes confirm a live hosted MCP endpoint (server.smithery.ai/exa/mcp) requiring OAuth and even the docs site itself served over MCP (smithery.ai/docs/mcp) with a working CLI install. Missing for 10: no independent third-party report validating that connecting via the hosted server works end-to-end for an actual agent (only probe-level confirmation, not user testimonial), and community findings raise tool-description-injection risk that slightly tempers confidence though it doesn't contradict the connection mechanism itself.

   • [claimed-docs] “Smithery gives you a simple REST interface for connecting to MCP servers. Instead of implementing the MCP protocol directly, handling OAuth …”
   • [claimed-docs] “Uplink exposes an MCP server running on any machine as a regular Smithery connection.”
   • [claimed-docs] “Deep links provide a seamless way to integrate Smithery MCPs into supported clients. When a user clicks a deep link from our server page, th…”
   • [claimed-docs] “Smithery handles OAuth, token refresh, and connection management automatically.”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://server.smithery.ai/exa/mcp returned HTTP 401 with an OAuth challenge — the hoste…”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/smithery/): a JSON-RPC initialize POST to https://smithery.ai/docs/mc…”

   proof ↗⚑ flag

   Manufact✓fullprobed8/10

   Manufact ships an official MCP server that exposes the platform (orgs, servers, deployments, logs, observability) as MCP tools so agents like Cursor and Claude can connect and manage everything, with one-click installs/copy-paste config for clients and a dedicated docs page confirmed by probe (manufact-docs-14, manufact-docs-18, manufact-probe-3). Missing for 10: independent/hands-on confirmation that the MCP server connection works smoothly in practice (community evidence is thin/confused, e.g. manufact-comm-3) and clearer detail on auth/session robustness (manufact-comm-2 notes reauth issues in the broader MCP ecosystem).

   • [claimed-docs] “exposes the platform, organizations, servers, deployments, logs, and observability, as MCP tools, so you can manage everything from Cursor, …”
   • [claimed-docs] “You get one-click installs or copy-paste configuration for”
   • [probe] “official MCP server documented at https://docs.manufact.com/mcp”
   • [community] “Noted an issue with the offline_access OAuth scope causing frequent reauthentication requests from clients (citing a related Codex bug); man…”

   proof ↗⚑ flag

5. ai-native userUse an official CLI

   weight 2 · round to Manufact
   Smithery~partialprobed6/10

   There is concrete evidence of a working Smithery CLI (npm install @smithery/cli verified, version 4.11.1) with documented commands for searching the registry, adding MCP connections, finding/calling tools, and installing skills. However, the GitHub source is attributed to 'arcadeai-labs/smithery-cli' rather than an obviously first-party Smithery org, leaving some ambiguity about official first-party ownership, and there is no independent/community corroboration of CLI usage. Missing for 10: clearer first-party branding/ownership confirmation, official docs page dedicated to the CLI, and independent hands-on user reports.

   • [github] “smithery mcp search [term] # Search the Smithery registry smithery mcp add <url> # Add an MCP server connectio…”
   • [github] “smithery tool find [query] # Search tools by name or intent smithery tool call <connection> <tool> [args] # Call a tool”
   • [github] “Browse skills on the [Smithery Skills Registry](https://smithery.ai/skills) and install them with the upstream installer”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/smithery/): a JSON-RPC initialize POST to https://smithery.ai/docs/mc…”

   proof ↗⚑ flag

   Manufact✓fullprobed8/10

   Manufact documents an official CLI with concrete commands (local dev with hot reload/inspector, deploy to Manufact Cloud) and a dedicated cli.md/cli docs page confirmed by probe, fitting the agentic/AI-native workflow of managing MCP servers from the terminal. Missing for 10: independent/hands-on community validation of the CLI itself and a fuller command reference beyond run/deploy.

   • [claimed-docs] “Run your MCP server locally with hot reload + inspector”
   • [claimed-docs] “Deploys the current project to Manufact Cloud.”
   • [probe] “official CLI documented at https://docs.manufact.com/cli”

   proof ↗⚑ flag

6. ai-native userDrive the product through a documented public API

   weight 3 · round to Smithery
   Smithery✓fullprobed8/10

   Smithery exposes a real, documented public API: a registry API (registry.smithery.ai) queryable without keys, a full OpenAPI 3.1.0 spec (35 paths) confirmed live at smithery.ai/docs/openapi.json, and a REST/connection interface with OAuth and token scoping for programmatic/agent use, plus a CLI (smithery mcp/tool commands) for driving it. Initial llms.txt/docs.md probes 404'd but the actual OpenAPI spec and registry endpoints were independently verified live, satisfying 'documented public API' for an AI-native/agentic consumer. missing for 10: no independent third-party corroboration of API stability/versioning, and some docs endpoints (llms.txt) are absent despite the OpenAPI spec existing.

   • [probe] “PROBE registry-api (2026-09-05): GET https://registry.smithery.ai/servers?pageSize=3 with NO API key returned HTTP 200 with a machine-readab…”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://server.smithery.ai/exa/mcp returned HTTP 401 with an OAuth challenge — the hoste…”
   • [probe] “PROBE openapi (2026-09-05): GET https://smithery.ai/docs/openapi.json returned HTTP 200 with an OpenAPI 3.1.0 description (35 documented pat…”
   • [claimed-docs] “Smithery gives you a simple REST interface for connecting to MCP servers. Instead of implementing the MCP protocol directly, handling OAuth …”
   • [claimed-docs] “Service tokens let you safely expose Smithery to browsers, mobile apps, and AI agents without leaking your API key.”
   • [github] “smithery mcp search [term] # Search the Smithery registry smithery mcp add <url> # Add an MCP server connectio…”
   • [github] “smithery tool find [query] # Search tools by name or intent smithery tool call <connection> <tool> [args] # Call a tool”

   proof ↗⚑ flag

   Manufact~partialprobed5/10

   Manufact documents API-key based programmatic access to a REST API for CI/scripts and also exposes platform functionality as MCP tools usable from AI clients like Cursor/Claude, which supports AI-native driving of the product. However, a probe for actual API documentation (OpenAPI/swagger spec) returned 404s across all candidate paths, so there's no verifiable public API reference beyond a passing mention. Missing for 10: a discoverable OpenAPI/swagger spec or full REST API reference docs, and independent confirmation the REST API works as described.

   • [claimed-docs] “API keys authenticate programmatic access to your organization, from the REST API, CI pipelines, and scripts.”
   • [claimed-docs] “exposes the platform, organizations, servers, deployments, logs, and observability, as MCP tools, so you can manage everything from Cursor, …”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.manufact.com/openapi.json, https://docs.manufact.com/swagger.json, https://docs.manufac…”
   • [probe] “official MCP server documented at https://docs.manufact.com/mcp”

   proof ↗⚑ flag

7. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round to Smithery
   Smithery✓fullclaimed7/10

   Smithery docs describe service tokens explicitly scoped to a specific app/connection/user metadata ('list and call tools on connections in my-app where metadata.userId is user-123 — nothing else'), plus encrypted write-only credential storage and namespace-scoped API keys, directly matching the least-privilege credential story for agents. Missing for 10: independent/hands-on verification that scope enforcement actually holds, and documentation of token revocation/expiry/audit controls.

   • [claimed-docs] “Service tokens let you safely expose Smithery to browsers, mobile apps, and AI agents without leaking your API key.”
   • [claimed-docs] “This token can list and call tools on connections in `my-app` where `metadata.userId` is `user-123` — nothing else.”
   • [claimed-docs] “Secure credential storage — Encrypted, write-only storage for API keys and tokens”
   • [claimed-docs] “Namespaces act as a shared workspace. Resources within a namespace can be managed by anyone with the appropriate API key scoped to that name…”

   proof ↗⚑ flag

   Manufact—none0/10

   Docs confirm API keys exist for programmatic/CI access (manufact-docs-17), but there is no evidence of scoping, permission levels, or least-privilege configuration for these keys or for agent-specific credentials. Missing for 10: any mention of scoped/role-based API key permissions, per-agent credential restriction, or least-privilege access controls.

   • [claimed-docs] “API keys authenticate programmatic access to your organization, from the REST API, CI pipelines, and scripts.”

   proof ↗⚑ flag

8. ai-native userBuild against official SDKs

   weight 2 · round to Smithery
   Smithery✓fullprobed7/10

   Smithery auto-generates a typed TypeScript SDK for every published MCP server (smithery-docs-16), ships an official CLI (smithery-cli on GitHub) verified via live npm install (smithery-probe-rt-4), and exposes a documented OpenAPI 3.1 spec for platform API client generation, confirmed live via probe (smithery-probe-rt-3). This gives AI-native builders concrete, verified official SDK/CLI/API surfaces to build against. Missing for 10: SDKs beyond TypeScript (no Python/other language SDK evidence), and independent third-party corroboration of real-world SDK usage beyond Smithery's own docs/probes.

   • [claimed-docs] “Every MCP server published on Smithery gets a typed TypeScript SDK generated from its tool and trigger schemas.”
   • [probe] “PROBE openapi (2026-09-05): GET https://smithery.ai/docs/openapi.json returned HTTP 200 with an OpenAPI 3.1.0 description (35 documented pat…”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/smithery/): a JSON-RPC initialize POST to https://smithery.ai/docs/mc…”
   • [github] “smithery mcp search [term] # Search the Smithery registry smithery mcp add <url> # Add an MCP server connectio…”
   • [github] “smithery tool find [query] # Search tools by name or intent smithery tool call <connection> <tool> [args] # Call a tool”

   proof ↗⚑ flag

   Manufact—none0/10

   Evidence shows an official CLI, REST API with API keys, and an MCP server for managing the platform, but no documented client SDKs (e.g., language libraries) for building against Manufact programmatically; the openapi spec probe also returned 404s, suggesting no formal API/SDK artifact is published.

   • [claimed-docs] “API keys authenticate programmatic access to your organization, from the REST API, CI pipelines, and scripts.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.manufact.com/openapi.json, https://docs.manufact.com/swagger.json, https://docs.manufac…”
   • [probe] “official CLI documented at https://docs.manufact.com/cli”

   proof ↗⚑ flag

9. ai-native userSubscribe to events via webhooks

   weight 2 · round to Smithery
   Smithery~partialclaimed6/10

   Smithery's Triggers feature explicitly lets consumers subscribe to upstream events, proxying subscribe calls through to the server—directly matching webhook/event-subscription functionality for agentic use. However, this is documented only at a conceptual level with no depth on webhook delivery mechanics, retry/reliability guarantees, or independent hands-on verification. missing for 10: detailed webhook payload/delivery docs, independent corroboration of triggers working in practice, and coverage of failure/retry semantics.

   • [claimed-docs] “Triggers let an MCP server surface events from its upstream service. When a consumer activates a trigger via Smithery, Smithery proxies the …”

   proof ↗⚑ flag

   Manufact—none0/10

   No evidence in the pack mentions webhooks or event subscription mechanisms; Manufact's docs cover deployment, observability, MCP tool exposure, and CLI, but nothing about outbound webhook subscriptions for events.

   ⚑ flag

Agentic features

1. ai-native userSet up automations that run autonomously in the background

   weight 2 · round to Manufact
   Smithery~partialclaimed3/10

   Smithery's Triggers feature lets an MCP server surface upstream events which Smithery proxies to a consumer, hinting at event-driven automation, but there is no evidence of a scheduler, persistent background execution engine, or the platform itself running autonomous tasks — automation actually running requires an external agent that consumes the trigger. missing for 10: evidence of a scheduling/cron mechanism, autonomous execution loop hosted by Smithery, and confirmation that triggers fire without an active client agent.

   • [claimed-docs] “Triggers let an MCP server surface events from its upstream service. When a consumer activates a trigger via Smithery, Smithery proxies the …”

   proof ↗⚑ flag

   Manufact~partialclaimed4/10

   Manufact offers some automated background actions tied to deployment events—auto-running a publish checklist after each successful deployment and automated protocol-compliance audits—but these are CI/CD-style triggers, not general-purpose autonomous agent automations that a user configures to run independently in the background. Missing for 10: evidence of user-defined scheduled/triggered autonomous workflows beyond deployment-linked checks, no mention of persistent background agents or cron-like automation configuration, and no independent confirmation that these automations run reliably unattended.

   • [claimed-docs] “you can start a checklist automatically after each successful deployment”
   • [claimed-docs] “runs an automated audit of your server for protocol compliance and platform readiness”
   • [claimed-docs] “end-to-end checks that exercise your server live in ChatGPT and Claude”
   • [claimed-docs] “Trigger deployment: deploy a branch on demand.”

   proof ↗⚑ flag

2. ai-native userOperate the product with natural-language commands

   weight 2 · round to Manufact
   Smithery~partialprobed5/10

   Smithery's CLI exposes intent-based discovery ('smithery tool find [query] # Search tools by name or intent') and its docs are themselves served as an MCP endpoint that an agent can query conversationally, showing some natural-language-friendly interaction. However, the primary interface remains structured CLI subcommands (search, add, call) and REST/OpenAPI calls rather than a general natural-language command layer for operating the whole product. Missing for 10: a documented conversational/chat interface or explicit NL parsing across all product operations, and independent evidence of users successfully issuing free-form natural-language commands rather than structured queries.

   • [github] “smithery tool find [query] # Search tools by name or intent smithery tool call <connection> <tool> [args] # Call a tool”
   • [claimed-docs] “Every MCP server published on Smithery gets a typed TypeScript SDK generated from its tool and trigger schemas.”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/smithery/): a JSON-RPC initialize POST to https://smithery.ai/docs/mc…”
   • [github] “smithery mcp search [term] # Search the Smithery registry smithery mcp add <url> # Add an MCP server connectio…”

   proof ↗⚑ flag

   Manufact✓fullprobed8/10

   Manufact exposes its own platform (orgs, servers, deployments, logs, observability) as MCP tools so users can manage everything via natural language from clients like Cursor or Claude, and also supports copying fix prompts into a coding agent or running built-in autofix. This is a first-party, well-documented agentic control surface over the whole product. Missing for 10: independent hands-on verification of the MCP server actually working end-to-end, and broader third-party confirmation beyond docs/community chatter.

   • [claimed-docs] “exposes the platform, organizations, servers, deployments, logs, and observability, as MCP tools, so you can manage everything from Cursor, …”
   • [probe] “official MCP server documented at https://docs.manufact.com/mcp”
   • [claimed-docs] “copy a fix prompt into your own coding agent, or run the built-in autofix flow”
   • [claimed-docs] “You get one-click installs or copy-paste configuration for”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round to Smithery
   Smithery~partialprobed5/10

   Smithery exposes a full OpenAPI 3.1 spec (35 paths) and even serves its documentation itself as an MCP endpoint that agents can query interactively via JSON-RPC, which supports AI-native exploration of the API surface. However there's no evidence of a classic interactive reference UI (e.g., Swagger/Redoc 'try it out' runnable examples in-browser), and alternate machine-readable doc formats (llms.txt, docs.md) return 404s. missing for 10: an actual interactive/try-it-out API console with executable example requests, evidence of runnable code snippets in docs, independent confirmation of usability of the MCP-served docs.

   • [probe] “PROBE openapi (2026-09-05): GET https://smithery.ai/docs/openapi.json returned HTTP 200 with an OpenAPI 3.1.0 description (35 documented pat…”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/smithery/): a JSON-RPC initialize POST to https://smithery.ai/docs/mc…”
   • [probe] “PROBE llms.txt: HTTP 404 at https://smithery.ai/llms.txt”
   • [probe] “PROBE docs-md: HTTP 404 at https://smithery.ai/docs.md”

   proof ↗⚑ flag

   Manufact—none0/10

   Manufact documents a REST API (via API keys) and a testing dashboard that runs test suites across MCP clients/models, but there is no evidence of an interactive API reference with runnable examples (e.g., a Swagger/OpenAPI explorer) — a direct probe for openapi.json/swagger.json returned 404s. Missing for 10: any interactive API reference UI, runnable code examples for the REST API, or OpenAPI spec exposure.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.manufact.com/openapi.json, https://docs.manufact.com/swagger.json, https://docs.manufac…”
   • [claimed-docs] “API keys authenticate programmatic access to your organization, from the REST API, CI pipelines, and scripts.”
   • [claimed-docs] “Manufact executes them across different MCP clients and LLM models”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round to Smithery
   Smithery✓fullprobed8/10

   A direct hands-on probe confirms a live, machine-readable OpenAPI 3.1.0 specification with 35 documented paths at smithery.ai/docs/openapi.json, which an agent could use to generate a client — squarely satisfying the story. Earlier probe attempts at guessed paths (openapi.json at root, swagger.json, etc.) 404'd, but the actual documented location succeeds. Missing for 10: first-party documentation explicitly announcing/linking the OpenAPI spec, and independent community confirmation of its use.

   • [probe] “PROBE openapi (2026-09-05): GET https://smithery.ai/docs/openapi.json returned HTTP 200 with an OpenAPI 3.1.0 description (35 documented pat…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://smithery.ai/openapi.json, https://smithery.ai/swagger.json, https://smithery.ai/api/openapi.…”

   proof ↗⚑ flag

   Manufact—none0/10

   Manufact provides a REST API and API keys (manufact-docs-17) but the probe explicitly found no OpenAPI/swagger spec at any standard location (manufact-probe-2), and no docs mention a downloadable machine-readable spec.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.manufact.com/openapi.json, https://docs.manufact.com/swagger.json, https://docs.manufac…”
   • [claimed-docs] “API keys authenticate programmatic access to your organization, from the REST API, CI pipelines, and scripts.”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round to Manufact
   Smithery—none0/10

   Evidence shows Smithery's Uplink lets developers test an in-development MCP server against a real agent before publishing, but there is no mention of a distinct sandbox environment, test data isolation, or any mechanism preventing production data exposure during testing.

   • [claimed-docs] “Uplink exposes an MCP server running on any machine as a regular Smithery connection.”
   • [claimed-docs] “You’re developing an MCP server and want to test it against a real agent before publishing.”

   proof ↗⚑ flag

   Manufact~partialclaimed6/10

   Docs show per-branch preview environments with distinct MCP URLs, on-demand deployment triggers, local hot-reload/inspector runs, and an explicit separate production-branch setting — all consistent with testing in isolation from production. However, there's no explicit statement about data isolation or guarantees that sandbox testing cannot touch production data/state. Missing for 10: explicit data-isolation guarantees between preview/local and production environments, and independent confirmation that preview deployments don't share production data stores.

   • [claimed-docs] “Preview environments are created per branch. Each gets its own MCP URL”
   • [claimed-docs] “Trigger deployment: deploy a branch on demand.”
   • [claimed-docs] “Run your MCP server locally with hot reload + inspector”
   • [claimed-docs] “Set which Git branch serves your canonical production URL from the Deployment paths & triggers section”
   • [claimed-docs] “Manufact executes them across different MCP clients and LLM models”

   proof ↗⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   Smithery—none0/10

   Evidence shows Smithery has an OpenAPI 3.1.0 spec and a documented REST/MCP interface, but nothing in the docs or probes describes API versioning conventions or a deprecation policy for breaking changes.

   ⚑ flag

   Manufact—none0/10

   Manufact documents a REST API and API keys for programmatic access, but there is no evidence of API versioning or a documented deprecation policy; an openapi spec probe returned 404s across all candidate paths. Missing for 10: versioned API scheme, changelog/deprecation policy documentation, any mention of backward-compatibility guarantees.

   • [claimed-docs] “API keys authenticate programmatic access to your organization, from the REST API, CI pipelines, and scripts.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.manufact.com/openapi.json, https://docs.manufact.com/swagger.json, https://docs.manufac…”

   proof ↗⚑ flag

Multi tenant

1. platform-engineerMaintain separate connected accounts and scoped tokens per end user of my product, so each user's agent acts only on that user's credentials

   weight 3 · round to Smithery
   Smithery~partialcommunity6/10

   Smithery's token-scoping docs show scoped tokens tied to per-user metadata (e.g. a token limited to connections where metadata.userId is 'user-123'), and secure credential storage/automatic refresh are documented, which directly supports per-end-user credential isolation for a platform-engineer building on top of Smithery. However, there's no first-party documentation of a full per-user account management workflow (provisioning, revocation, auditing across many end users) beyond the single scoping example, and independent security research flagged real vulnerabilities (tool description injection, path traversal) that raise doubts about credential-boundary robustness in practice. missing for 10: dedicated multi-tenant/per-end-user account management docs beyond one scoping example, evidence of token revocation/rotation workflows at scale, and independent verification that scoped tokens hold up against the reported security findings.

   • [claimed-docs] “Service tokens let you safely expose Smithery to browsers, mobile apps, and AI agents without leaking your API key.”
   • [claimed-docs] “This token can list and call tools on connections in `my-app` where `metadata.userId` is `user-123` — nothing else.”
   • [claimed-docs] “Automatic token refresh — Credentials stay valid without manual intervention”
   • [claimed-docs] “Secure credential storage — Encrypted, write-only storage for API keys and tokens”
   • [community] “Scanned the top 100 servers on Smithery's registry; 22 of 100 flagged at least one security finding (28 total, 4 critical, 24 high), most co…”
   • [community] “Security researchers detailed a path traversal vulnerability in Smithery's MCP server hosting that could be escalated into a supply chain at…”

   proof ↗⚑ flag

   Manufact—none0/10

   Manufact's evidence covers deployment, observability, publish checks, API keys, and MCP client connections for the platform-engineer's own organization, but there is no mention of per-end-user account isolation, scoped tokens per end user, or mechanisms for agents to act only on individual end-user credentials. Missing for 10: any documentation of end-user-scoped OAuth/token issuance, multi-tenant credential isolation, or per-user connected account management.

   • [claimed-docs] “API keys authenticate programmatic access to your organization, from the REST API, CI pipelines, and scripts.”
   • [claimed-docs] “exposes the platform, organizations, servers, deployments, logs, and observability, as MCP tools, so you can manage everything from Cursor, …”
   • [claimed-docs] “URL-connected servers that use the Manufact proxy can now be downgraded back to a direct external connection from Settings.”

   proof ↗⚑ flag

Oauth

1. developerLet the platform run OAuth flows for downstream tools and vault the resulting credentials encrypted, with automatic token refresh

   weight 3 · round to Smithery
   Smithery✓fullprobed7/10

   Docs explicitly describe Smithery handling OAuth flows, encrypted write-only credential storage, and automatic token refresh (smithery-docs-1,2,3,13), and this is corroborated by a live probe showing an OAuth challenge gating a hosted MCP endpoint (smithery-probe-rt-2). missing for 10: no independent hands-on confirmation of encryption-at-rest implementation details or token refresh behavior over time, and community evidence focuses on unrelated security findings rather than corroborating credential vaulting.

   • [claimed-docs] “Smithery gives you a simple REST interface for connecting to MCP servers. Instead of implementing the MCP protocol directly, handling OAuth …”
   • [claimed-docs] “Automatic token refresh — Credentials stay valid without manual intervention”
   • [claimed-docs] “Secure credential storage — Encrypted, write-only storage for API keys and tokens”
   • [claimed-docs] “Smithery handles OAuth, token refresh, and connection management automatically.”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://server.smithery.ai/exa/mcp returned HTTP 401 with an OAuth challenge — the hoste…”

   proof ↗⚑ flag

   Manufact—none0/10

   Manufact is an MCP server deployment/observability platform; there is no evidence it runs OAuth flows for downstream tools or vaults credentials with automatic token refresh. The community note about offline_access OAuth scope refers to reauthentication bugs, not a credential-vaulting feature. This is an applicable axis for a platform managing MCP servers/tools, but no supporting evidence exists.

   • [community] “Noted an issue with the offline_access OAuth scope causing frequent reauthentication requests from clients (citing a related Codex bug); man…”

   proof ↗⚑ flag

White label

1. platform-engineerWhite-label the authorization flow with my own branding and my own OAuth app credentials

   weight 1 · round drawn
   Smithery—none0/10

   The evidence pack describes Smithery's own OAuth/credential-management infrastructure (managed auth, token scoping, secure storage) but contains no mention of white-labeling the auth flow with custom branding or bringing your own OAuth app/client credentials to replace Smithery's identity — no BYO-OAuth-app, custom domain, or branding customization feature is documented anywhere.

   ⚑ flag

   Manufact—none0/10

   No evidence in the pack addresses white-labeling the OAuth/authorization flow, custom branding, or bringing your own OAuth app credentials; the docs cover deployment, testing, observability, API keys for programmatic access, but nothing about customizable authorization UI or client credentials. Missing for 10: any mention of white-label branding, custom OAuth client ID/secret configuration, or customizable consent screens.

   ⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round drawn
   Smithery—none0/10

   Evidence shows Smithery lets an agent connect to and aggregate tools from multiple servers, and its registry API can be queried/paginated for many entries, but there is no documented batch/bulk operation capability (e.g., bulk tool calls, mass server management, batch CRUD) that would let a user perform bulk actions across many items at once. missing for 10: batch tool-call API, bulk server management commands, mass connection/config updates, any documented multi-item transaction support.

   ⚑ flag

   Manufact—none0/10

   Manufact's docs describe per-server actions (deploy, test, publish-check) and an MCP tool surface for managing servers/deployments/logs, but nothing describes performing an operation across many items/servers simultaneously (e.g., bulk deploy, bulk test, bulk delete). The axis is plausible for a management platform like this, but no evidence supports it.

   • [claimed-docs] “Trigger deployment: deploy a branch on demand.”
   • [claimed-docs] “Manufact executes them across different MCP clients and LLM models”
   • [claimed-docs] “exposes the platform, organizations, servers, deployments, logs, and observability, as MCP tools, so you can manage everything from Cursor, …”
   • [claimed-docs] “The history table shows every deployment with its MCP URL, source branch and commit, duration, creation time, and the latest completed publi…”

   proof ↗⚑ flag

2. ai-native userSchedule recurring jobs or workflows

   weight 2 · round drawn
   Smithery—none0/10

   Smithery's docs describe on-demand tool connections, OAuth/credential management, and event-driven 'Triggers' that proxy subscribe calls from upstream services, but nothing describes cron-like recurring job scheduling or workflow automation on a timer. Triggers are reactive (event-based), not scheduled/recurring, so the specific capability asked for is unevidenced.

   • [claimed-docs] “Triggers let an MCP server surface events from its upstream service. When a consumer activates a trigger via Smithery, Smithery proxies the …”
   • [claimed-docs] “Uplink exposes an MCP server running on any machine as a regular Smithery connection.”
   • [claimed-docs] “If you’ve already deployed an MCP server elsewhere, you can publish it directly on Smithery via the URL method.”

   proof ↗⚑ flag

   Manufact—none0/10

   Manufact is an MCP server deployment/observability platform; nothing in the evidence describes scheduling recurring jobs, cron-like triggers, or recurring workflow automation—deployments are triggered on push/on-demand, not on a schedule. missing for 10: any cron/scheduler feature, recurring job configuration, or workflow automation trigger based on time intervals.

   • [claimed-docs] “Trigger deployment: deploy a branch on demand.”
   • [claimed-docs] “you can start a checklist automatically after each successful deployment”
   • [claimed-docs] “Set which Git branch serves your canonical production URL from the Deployment paths & triggers section”

   proof ↗⚑ flag

3. ai-native userVersion, review, and roll back my automations

   weight 1 · round to Manufact
   Smithery—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

   Manufact~partialclaimed5/10

   Manufact deploys from GitHub with branch-based preview environments, deployment history showing commit/branch and publish checklist scores, and on-demand trigger deployment—giving a form of version tracking and deployment history, but there is no explicit rollback mechanism (e.g., 'revert to previous deployment') or automation-specific review/approval workflow described. missing for 10: explicit rollback/revert capability for deployments, versioned diff/review UI for automations themselves (not just deployments), and independent confirmation that history table supports reverting to prior versions.

   • [claimed-docs] “Preview environments are created per branch. Each gets its own MCP URL”
   • [claimed-docs] “Trigger deployment: deploy a branch on demand.”
   • [claimed-docs] “The history table shows every deployment with its MCP URL, source branch and commit, duration, creation time, and the latest completed publi…”
   • [claimed-docs] “Set which Git branch serves your canonical production URL from the Deployment paths & triggers section”

   proof ↗⚑ flag

Bridge

1. developerBridge a locally running MCP server into the platform as a managed remote connection without deploying it

   weight 1 · round drawn
   Smithery✓fullclaimed8/10

   Smithery's Uplink feature explicitly exposes an MCP server running on any local machine as a regular Smithery connection, letting developers test/use a locally running server against real agents without deploying it, and this is backed by scoped tokens and managed OAuth/credential handling. missing for 10: independent/hands-on confirmation of Uplink working end-to-end (only first-party docs cited) and no detail on latency/reliability of the bridged local connection.

   • [claimed-docs] “Uplink exposes an MCP server running on any machine as a regular Smithery connection.”
   • [claimed-docs] “You’re developing an MCP server and want to test it against a real agent before publishing.”
   • [claimed-docs] “You want to run a private tool for yourself or your team without hosting it.”
   • [claimed-docs] “Smithery gives you a simple REST interface for connecting to MCP servers. Instead of implementing the MCP protocol directly, handling OAuth …”
   • [claimed-docs] “Service tokens let you safely expose Smithery to browsers, mobile apps, and AI agents without leaking your API key.”

   proof ↗⚑ flag

   Manufact✓fullclaimed8/10

   Docs explicitly state you can 'paste your MCP URL. No deploy required: Manufact connects directly for Chat, testing, and the publish workflow,' and a changelog entry confirms URL-connected servers via proxy can be downgraded to direct external connection, showing this is a real managed remote-connection mode distinct from GitHub deploys. Missing for 10: independent/hands-on confirmation beyond vendor docs and more detail on how 'local' servers specifically get bridged (vs. any external URL).

   • [claimed-docs] “paste your MCP URL. No deploy required: Manufact connects directly for Chat, testing, and the publish workflow.”
   • [claimed-docs] “URL-connected servers that use the Manufact proxy can now be downgraded back to a direct external connection from Settings.”
   • [claimed-docs] “Preview environments are created per branch. Each gets its own MCP URL”

   proof ↗⚑ flag

Access control

1. platform-engineerRestrict which servers and individual tools an agent may call through allowlists or per-tool access control

   weight 2 · round to Smithery
   Smithery~partialclaimed6/10

   Smithery's token-scoping feature lets platform engineers issue tokens restricted to specific connections/namespaces (e.g., 'list and call tools on connections in my-app where metadata.userId is user-123 — nothing else'), which functions as a coarse allowlist mechanism, and namespaces further scope API-key access to a workspace. However, the evidence shows scoping is at the connection/namespace level, not explicit per-tool allow/deny lists within a single server. Missing for 10: documented fine-grained per-tool allow/deny configuration, independent/hands-on verification that tool-level restrictions are enforced, and any admin UI or policy examples showing individual tool blocking within an allowed connection.

   • [claimed-docs] “Service tokens let you safely expose Smithery to browsers, mobile apps, and AI agents without leaking your API key.”
   • [claimed-docs] “This token can list and call tools on connections in `my-app` where `metadata.userId` is `user-123` — nothing else.”
   • [claimed-docs] “Namespaces act as a shared workspace. Resources within a namespace can be managed by anyone with the appropriate API key scoped to that name…”

   proof ↗⚑ flag

   Manufact—none0/10

   Manufact's docs cover deployment, testing, observability, and MCP server management, but there is no mention of allowlisting servers/tools or per-tool access control for agents. Missing for 10: any documentation of tool/server allowlists, per-tool permission scoping, or gateway-level access control policies.

   • [claimed-docs] “shows individual events: every request flowing through the gateway, refreshed every few seconds”
   • [claimed-docs] “exposes the platform, organizations, servers, deployments, logs, and observability, as MCP tools, so you can manage everything from Cursor, …”

   proof ↗⚑ flag

Audit

1. platform-engineerSee a log of every tool call that passed through the platform, with enough detail to audit or replay what an agent did

   weight 2 · round to Manufact
   Smithery—none0/10

   No evidence describes any audit logging, tool-call history, or replay capability for platform engineers—docs cover connection, credential management, publishing, tokens, and namespaces but never mention logs of tool calls passing through the gateway.

   ⚑ flag

   Manufact~partialclaimed6/10

   Manufact's observability dashboard shows individual gateway events refreshed every few seconds, with optional full request/response payload capture, giving platform-engineers a log of tool calls suitable for auditing; the deployment history table also links MCP URLs, commits, and checklist scores. However, there's no explicit mention of replay functionality, retention/export controls, or fine-grained per-tool-call audit trail structure beyond the event list. Missing for 10: explicit replay capability, log retention/export details, and independent/hands-on confirmation of audit completeness.

   • [claimed-docs] “shows individual events: every request flowing through the gateway, refreshed every few seconds”
   • [claimed-docs] “Toggle Capture payloads to also store full request and response bodies for each event”
   • [claimed-docs] “The history table shows every deployment with its MCP URL, source branch and commit, duration, creation time, and the latest completed publi…”

   proof ↗⚑ flag

Connect

1. developerPoint my MCP-compatible client at a hosted remote server URL and reach downstream tools without deploying any infrastructure

   weight 3 · round drawn
   Smithery✓fullprobed8/10

   Docs and live probes confirm Smithery hosts remote MCP server endpoints (server.smithery.ai/<server>/mcp) that clients can point at directly, with OAuth/token handling, deep links, and typed SDKs generated automatically — no server deployment needed by the developer. Independent probe evidence (smithery-probe-rt-2) shows the hosted endpoint is live and gated by managed OAuth rather than requiring self-hosting infrastructure. missing for 10: independent hands-on developer accounts (outside security research) confirming a smooth end-to-end connect experience beyond documentation and infra probes.

   • [claimed-docs] “Smithery gives you a simple REST interface for connecting to MCP servers. Instead of implementing the MCP protocol directly, handling OAuth …”
   • [claimed-docs] “Uplink exposes an MCP server running on any machine as a regular Smithery connection.”
   • [claimed-docs] “If you’ve already deployed an MCP server elsewhere, you can publish it directly on Smithery via the URL method.”
   • [claimed-docs] “Smithery handles OAuth, token refresh, and connection management automatically.”
   • [claimed-docs] “Every MCP server published on Smithery gets a typed TypeScript SDK generated from its tool and trigger schemas.”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://server.smithery.ai/exa/mcp returned HTTP 401 with an OAuth challenge — the hoste…”
   • [probe] “PROBE registry-api (2026-09-05): GET https://registry.smithery.ai/servers?pageSize=3 with NO API key returned HTTP 200 with a machine-readab…”

   proof ↗⚑ flag

   Manufact✓fullclaimed8/10

   Manufact's docs describe hosted MCP servers reachable via a per-deployment MCP URL (docs-4, docs-6), with dashboard flows to paste an existing MCP URL with 'No deploy required' for connecting clients (docs-2), and one-click/copy-paste client configuration (docs-18) so any MCP-compatible client can point at the hosted URL without managing infrastructure. Manufact Cloud handles the deployment/hosting infra itself (docs-1, docs-3, docs-16), fulfilling the 'no infrastructure to deploy' promise for the client side. Missing for 10: independent/hands-on confirmation that a client actually connects and reaches downstream tools end-to-end, and more detail on latency/reliability of the hosted proxy path.

   • [claimed-docs] “paste your MCP URL. No deploy required: Manufact connects directly for Chat, testing, and the publish workflow.”
   • [claimed-docs] “Preview environments are created per branch. Each gets its own MCP URL”
   • [claimed-docs] “The history table shows every deployment with its MCP URL, source branch and commit, duration, creation time, and the latest completed publi…”
   • [claimed-docs] “You get one-click installs or copy-paste configuration for”
   • [claimed-docs] “deploy from GitHub in minutes”

   proof ↗⚑ flag

Generate

1. developerTurn an existing API definition (OpenAPI) or custom code into a hosted MCP server without writing protocol plumbing

   weight 2 · round to Manufact
   Smithery~partialclaimed4/10

   Smithery's docs show a hosting/publishing path for MCP servers already built in code (URL method, MCPB bundles, server-card metadata) which removes protocol plumbing for deployment, but there is no evidence of an OpenAPI-spec ingestion or auto-generation feature that turns an existing REST API definition into an MCP server — the publish flow assumes you already have MCP-shaped code or a deployed server. Missing for 10: OpenAPI-to-MCP conversion tooling, any first-party or independent proof of generating a server from an OpenAPI spec, and confirmation that 'custom code' can be arbitrary non-MCP code rather than code already written against the MCP SDK.

   • [claimed-docs] “If you’ve already deployed an MCP server elsewhere, you can publish it directly on Smithery via the URL method.”
   • [claimed-docs] “For local stdio servers — Smithery distributes a pre-built MCPB bundle that clients download and run locally.”
   • [claimed-docs] “If automatic scanning can’t complete (auth wall, required configuration, or other issues), you can provide server metadata manually via a st…”
   • [claimed-docs] “Every MCP server published on Smithery gets a typed TypeScript SDK generated from its tool and trigger schemas.”

   proof ↗⚑ flag

   Manufact~partialprobed5/10

   Manufact clearly lets developers deploy existing code (via GitHub templates or CLI) into a hosted MCP server without handling protocol plumbing, with deployment history, publish checks, and observability handled by the platform (manufact-docs-3, manufact-docs-16, manufact-docs-7). However, there is no evidence of an OpenAPI-to-MCP conversion feature — the openapi.json probe returned 404s and no docs mention importing an OpenAPI spec to generate an MCP server. missing for 10: OpenAPI spec ingestion/conversion workflow, independent confirmation of code-to-MCP transformation beyond GitHub template deploy.

   • [claimed-docs] “Selecting a template creates a new repository from the template in your connected GitHub account or organization and deploys it”
   • [claimed-docs] “Deploys the current project to Manufact Cloud.”
   • [claimed-docs] “runs an automated audit of your server for protocol compliance and platform readiness”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.manufact.com/openapi.json, https://docs.manufact.com/swagger.json, https://docs.manufac…”

   proof ↗⚑ flag

Publish

1. developerPublish my own MCP server on the platform for distribution, discovery, and installation by others

   weight 2 · round to Smithery
   Smithery✓fullprobed8/10

   Docs clearly describe multiple publishing paths (URL method for already-deployed servers, MCPB bundle distribution for local stdio servers, automatic scanning or manual server-card metadata) resulting in a registry listing with generated TypeScript SDK, and the registry API probe confirms servers are discoverable/searchable by others without auth. Security researchers did find vulnerabilities/prompt-injection risks in some published servers, but this reflects on registry quality, not on whether publishing/discovery/installation works. missing for 10: independent hands-on account of a developer successfully publishing a server end-to-end, and clearer detail on install flow verification post-publish.

   • [claimed-docs] “If you’ve already deployed an MCP server elsewhere, you can publish it directly on Smithery via the URL method.”
   • [claimed-docs] “For local stdio servers — Smithery distributes a pre-built MCPB bundle that clients download and run locally.”
   • [claimed-docs] “If automatic scanning can’t complete (auth wall, required configuration, or other issues), you can provide server metadata manually via a st…”
   • [claimed-docs] “Every MCP server published on Smithery gets a typed TypeScript SDK generated from its tool and trigger schemas.”
   • [probe] “PROBE registry-api (2026-09-05): GET https://registry.smithery.ai/servers?pageSize=3 with NO API key returned HTTP 200 with a machine-readab…”
   • [community] “Scanned the top 100 servers on Smithery's registry; 22 of 100 flagged at least one security finding (28 total, 4 critical, 24 high), most co…”

   proof ↗⚑ flag

   Manufact~partialclaimed6/10

   Manufact provides a full publish workflow (deploy, publish checklist audit, end-to-end checks in ChatGPT/Claude) and one-click install/copy-paste configs for connecting clients, which covers distribution and installation by others. However there is no explicit evidence of a public marketplace, registry, or discovery mechanism where other developers browse and find published servers. Missing for 10: public server marketplace/registry for discovery, evidence of third-party users finding/installing another developer's server organically, independent confirmation of the publish-to-discovery pipeline.

   • [claimed-docs] “paste your MCP URL. No deploy required: Manufact connects directly for Chat, testing, and the publish workflow.”
   • [claimed-docs] “runs an automated audit of your server for protocol compliance and platform readiness”
   • [claimed-docs] “end-to-end checks that exercise your server live in ChatGPT and Claude”
   • [claimed-docs] “You get one-click installs or copy-paste configuration for”
   • [claimed-docs] “copy a fix prompt into your own coding agent, or run the built-in autofix flow”

   proof ↗⚑ flag

Analytics

1. platform-engineerSee usage analytics for the servers and tools my org consumes or publishes — call volumes, users, errors, and cost

   weight 2 · round to Manufact
   Smithery—none0/10

   The evidence pack documents connection, publishing, token scoping, triggers, and SDK generation, but nothing about a dashboard or API for org-level usage analytics (call volumes, per-user breakdowns, error rates, cost). The only quantitative signal found is a per-server 'useCount' field in the public registry listing, which is not org-scoped consumption/error/cost analytics.

   • [probe] “PROBE registry-api (2026-09-05): GET https://registry.smithery.ai/servers?pageSize=3 with NO API key returned HTTP 200 with a machine-readab…”

   proof ↗⚑ flag

   Manufact~partialcommunity5/10

   Manufact's observability dashboard shows per-event traffic through the gateway with optional payload capture, giving call volume and request-level visibility, and community feedback (manufact-comm-1) praises 'analytics, logs' as impressive. However there's no documented breakdown by user, explicit error-rate reporting, or per-server/tool cost analytics — billing shows only org-wide spend limits, not tool-level cost. Missing for 10: per-user attribution, explicit error/failure metrics, and per-server/tool cost breakdowns in the observability view.

   • [claimed-docs] “shows individual events: every request flowing through the gateway, refreshed every few seconds”
   • [claimed-docs] “Toggle Capture payloads to also store full request and response bodies for each event”
   • [claimed-docs] “Owners and admins can set a monthly spend limit for metered usage beyond included credits from the Billing page.”
   • [community] “Impressed with the demo video, particularly the analytics, logs, and test suite features, but pricing page credit system unclear and no visi…”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round to Smithery
   Smithery~partialprobed7/10

   Smithery exposes a documented, live OpenAPI 3.1 spec (35 paths) and a keyless registry API, plus a CLI (`smithery mcp add`, `smithery tool call`) that mirrors core UI actions like discovering, connecting, and invoking MCP servers — showing broad API/CLI parity with the UI for the main workflows (search, connect, publish via URL, call tools). However, no evidence explicitly confirms that secondary UI-only features (team/namespace management, trigger configuration, token scoping setup) are fully API-reachable, and llms.txt/docs.md convenience endpoints 404. missing for 10: explicit confirmation that admin/team features (namespaces, triggers, token scoping) are configurable via the API/CLI rather than only the UI, and independent hands-on verification beyond the vendor's own OpenAPI spec.

   • [probe] “PROBE registry-api (2026-09-05): GET https://registry.smithery.ai/servers?pageSize=3 with NO API key returned HTTP 200 with a machine-readab…”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://server.smithery.ai/exa/mcp returned HTTP 401 with an OAuth challenge — the hoste…”
   • [probe] “PROBE openapi (2026-09-05): GET https://smithery.ai/docs/openapi.json returned HTTP 200 with an OpenAPI 3.1.0 description (35 documented pat…”
   • [github] “smithery mcp search [term] # Search the Smithery registry smithery mcp add <url> # Add an MCP server connectio…”
   • [github] “smithery tool find [query] # Search tools by name or intent smithery tool call <connection> <tool> [args] # Call a tool”
   • [claimed-docs] “If you’ve already deployed an MCP server elsewhere, you can publish it directly on Smithery via the URL method.”
   • [claimed-docs] “Namespaces act as a shared workspace. Resources within a namespace can be managed by anyone with the appropriate API key scoped to that name…”
   • [claimed-docs] “Triggers let an MCP server surface events from its upstream service. When a consumer activates a trigger via Smithery, Smithery proxies the …”

   proof ↗⚑ flag

   Manufact~partialprobed6/10

   Manufact ships an official MCP server exposing platform, orgs, servers, deployments, logs, and observability as tools, plus a CLI and REST API keys for programmatic access (manufact-docs-14,15,16,17), giving broad UI-parity for AI-native users. However, the probe found no discoverable OpenAPI/swagger spec (404s), so full REST API coverage (e.g. billing, connect-clients config, publish checklist) is unverified and some UI actions may lack documented programmatic equivalents. missing for 10: a published OpenAPI/REST API reference confirming full parity, and independent/hands-on confirmation that every UI action is API-reachable.

   • [claimed-docs] “exposes the platform, organizations, servers, deployments, logs, and observability, as MCP tools, so you can manage everything from Cursor, …”
   • [claimed-docs] “Run your MCP server locally with hot reload + inspector”
   • [claimed-docs] “Deploys the current project to Manufact Cloud.”
   • [claimed-docs] “API keys authenticate programmatic access to your organization, from the REST API, CI pipelines, and scripts.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.manufact.com/openapi.json, https://docs.manufact.com/swagger.json, https://docs.manufac…”
   • [probe] “official MCP server documented at https://docs.manufact.com/mcp”
   • [probe] “official CLI documented at https://docs.manufact.com/cli”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round drawn
   Smithery—none0/10

   No evidence anywhere in the pack of a data-export feature, account data portability, or open-format export for users leaving the platform; documentation covers connections, tokens, and MCP server publishing but nothing about exporting stored user data (connections, credentials metadata, namespaces) in an open format.

   ⚑ flag

   Manufact—none0/10

   Evidence shows Manufact deploys MCP servers from user-owned GitHub repos and offers an API/CLI for programmatic access (manufact-docs-16, manufact-docs-17, manufact-probe-4), but there is no documentation of an explicit 'export all your data' feature covering logs, observability payloads, deployment history, or test results in open/portable formats, nor any account-closure data-export workflow.

   • [claimed-docs] “Selecting a template creates a new repository from the template in your connected GitHub account or organization and deploys it”
   • [claimed-docs] “API keys authenticate programmatic access to your organization, from the REST API, CI pipelines, and scripts.”
   • [claimed-docs] “shows individual events: every request flowing through the gateway, refreshed every few seconds”
   • [probe] “official CLI documented at https://docs.manufact.com/cli”

   proof ↗⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round drawn
   Smithery—none0/10

   Evidence shows only a GitHub repo for the Smithery CLI tool (smithery-gh-1/2/3) but no license information, and the core Smithery platform (registry, hosting, OAuth handling) is described only via docs as a hosted SaaS with no mention of source availability or open licensing. No evidence establishes that Smithery's product source is published under an open license.

   • [github] “smithery mcp search [term] # Search the Smithery registry smithery mcp add <url> # Add an MCP server connectio…”
   • [github] “smithery tool find [query] # Search tools by name or intent smithery tool call <connection> <tool> [args] # Call a tool”
   • [github] “Browse skills on the [Smithery Skills Registry](https://smithery.ai/skills) and install them with the upstream installer”

   proof ↗⚑ flag

   Manufact—none0/10

   No evidence anywhere in the pack indicates Manufact's own source code is open-licensed or publicly available; all references are to deploying/managing user MCP servers, not Manufact's own codebase. This is a fair question (SaaS platforms can be open-source) but no evidence supports it.

   ⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round drawn
   Smithery—none0/10

   Evidence describes Smithery only as a hosted registry/platform (registry.smithery.ai, server.smithery.ai) with OAuth-gated hosted MCP endpoints; 'Uplink' lets users expose their own local MCP server as a connection into the hosted Smithery service, but this is not self-hosting the Smithery platform itself. No docs, GitHub repo, or probes indicate a self-hostable/open-source deployment of the core product.

   • [claimed-docs] “Uplink exposes an MCP server running on any machine as a regular Smithery connection.”
   • [probe] “PROBE registry-api (2026-09-05): GET https://registry.smithery.ai/servers?pageSize=3 with NO API key returned HTTP 200 with a machine-readab…”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://server.smithery.ai/exa/mcp returned HTTP 401 with an OAuth challenge — the hoste…”

   proof ↗⚑ flag

   Manufact—none0/10

   All evidence describes deploying MCP servers to 'Manufact Cloud' (docs-1, docs-3, docs-16) and a CLI for local dev/testing (docs-15) plus REST/API keys for programmatic access (docs-17), but nothing indicates the Manufact platform itself (dashboard, gateway, observability, publish-checks) can be self-hosted outside their cloud.

   • [claimed-docs] “deploy from GitHub in minutes”
   • [claimed-docs] “Deploys the current project to Manufact Cloud.”
   • [claimed-docs] “API keys authenticate programmatic access to your organization, from the REST API, CI pipelines, and scripts.”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.manufact.com/llms.txt # Docs - Manufact Cloud: Deploy, monitor, and manage MCP servers on Manufact…”

   proof ↗⚑ flag

Pricing

1. developerPrototype my agent's tool connections on a meaningful free tier before paying anything

   weight 1 · round to Manufact
   Smithery—none0/10

   The evidence pack contains extensive documentation on Smithery's features, registry, and integrations but no mention of pricing tiers, a free tier, usage limits, or billing at all — the axis is applicable to a hosted MCP platform but simply unevidenced.

   ⚑ flag

   Manufact~partialcommunity3/10

   Docs mention 'included credits' and a spend-limit control for usage beyond them, implying some free/starter allotment, but there is no explicit description of a free tier's scope or limits for connecting tools, and community feedback explicitly calls the credit/pricing system unclear ahead of deployment. missing for 10: explicit free-tier definition, limits on tool connections/servers, and confirmation that the free tier is 'meaningful' for prototyping.

   • [claimed-docs] “Owners and admins can set a monthly spend limit for metered usage beyond included credits from the Billing page.”
   • [community] “Impressed with the demo video, particularly the analytics, logs, and test suite features, but pricing page credit system unclear and no visi…”

   proof ↗⚑ flag

2. developerSee transparent pricing per tool call or connection plus documented rate limits before committing to the platform

   weight 2 · round to Manufact
   Smithery—none0/10

   No evidence pack item mentions pricing tiers, per-call costs, or documented rate limits anywhere in Smithery's docs, GitHub, or probes; the documentation covers connection, tokens, and hosting but is silent on pricing/rate limits, and repeated probes (llms.txt, docs.md, openapi) surface no pricing/rate-limit schema.

   ⚑ flag

   Manufact!disputedcontradicted3/10

   Manufact documents a spend-limit/billing feature (monthly cap for metered usage beyond included credits) but no per-tool-call pricing or rate-limit documentation exists, and a community reviewer explicitly reports the credit/pricing system is unclear with no visibility into pay-as-you-go costs after plan limits, contradicting any claim of transparent pricing. missing for 10: per-tool-call/connection pricing breakdown, documented rate limits, resolution of the community-reported cost opacity.

   • [claimed-docs] “Owners and admins can set a monthly spend limit for metered usage beyond included credits from the Billing page.”
   • [community] “Impressed with the demo video, particularly the analytics, logs, and test suite features, but pricing page credit system unclear and no visi…”

   proof ↗⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round drawn
   Smithery—none0/10

   No evidence anywhere in the pack mentions data residency, region selection, or geographic storage controls; Smithery's docs cover credential storage, token scoping, and connection management but not where data is physically stored.

   ⚑ flag

   Manufact—none0/10

   No evidence anywhere in the pack mentions data region selection, residency options, or geographic deployment controls; Manufact Cloud deployment docs discuss branches, environments, and MCP URLs but never data locality. Missing for 10: any mention of region/residency selection, data center location options, or compliance certifications tied to geography.

   ⚑ flag

2. ai-native userControl data retention and deletion

   weight 2 · round drawn
   Smithery—none0/10

   The evidence pack covers credential storage encryption and token scoping but contains no mention of data retention policies, user-initiated data deletion, or account/data export controls. This is a fair privacy-posture question for a platform managing credentials and connections, so the absence of evidence yields 'none' rather than 'na'.

   ⚑ flag

   Manufact—none0/10

   Manufact is an MCP server deployment/observability platform; no evidence in the pack addresses data retention policies, deletion controls, or data lifecycle management for stored logs, payloads, or deployment data. Observability docs mention capturing payloads (manufact-docs-13) but nothing about retention periods or user-initiated deletion.

   ⚑ flag

3. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   Smithery—none0/10

   No evidence anywhere in the pack mentions a telemetry/usage-tracking opt-out setting, privacy policy toggle, or data-collection disclosure for Smithery; the docs cover credential security and OAuth but never address analytics/telemetry opt-out.

   ⚑ flag

   Manufact—none0/10

   No evidence pack item mentions telemetry, usage tracking, or an opt-out/privacy setting for such data collection; the docs cover deployment, observability of user's own MCP traffic, and billing, but nothing about Manufact's own telemetry practices.

   ⚑ flag

Api

1. ai agentQuery the platform's registry or catalog through a documented API to discover servers and tools programmatically

   weight 3 · round to Smithery
   Smithery✓fullprobed9/10

   A live probe confirms an agent can query https://registry.smithery.ai/servers keylessly and get a machine-readable JSON catalog (displayName, qualifiedName, verified, etc.), and a documented OpenAPI 3.1 spec with 35 paths is served at smithery.ai/docs/openapi.json, enabling programmatic client generation. This is corroborated by the smithery-cli's 'mcp search'/'tool find' commands for registry/tool discovery. Missing for 10: no independent third-party report of an agent successfully building a full client purely from the OpenAPI spec, and llms.txt/docs.md convenience endpoints 404.

   • [probe] “PROBE registry-api (2026-09-05): GET https://registry.smithery.ai/servers?pageSize=3 with NO API key returned HTTP 200 with a machine-readab…”
   • [probe] “PROBE openapi (2026-09-05): GET https://smithery.ai/docs/openapi.json returned HTTP 200 with an OpenAPI 3.1.0 description (35 documented pat…”
   • [github] “smithery mcp search [term] # Search the Smithery registry smithery mcp add <url> # Add an MCP server connectio…”
   • [github] “smithery tool find [query] # Search tools by name or intent smithery tool call <connection> <tool> [args] # Call a tool”

   proof ↗⚑ flag

   Manufact~partialprobed5/10

   Manufact exposes an official MCP server (manufact-docs-14) that surfaces platform data—organizations, servers, deployments, logs—as MCP tools, letting an AI agent programmatically query the servers it has deployed from clients like Cursor or Claude. However, there is no documented public REST/OpenAPI catalog for external discovery (the openapi probe returned 404s), and the emphasis is on managing one's own deployed servers rather than browsing a broader tool/server catalog. Missing for 10: a documented REST/OpenAPI registry endpoint, evidence of cross-tenant or public catalog discovery, and independent confirmation of programmatic catalog querying beyond the MCP management tools.

   • [claimed-docs] “exposes the platform, organizations, servers, deployments, logs, and observability, as MCP tools, so you can manage everything from Cursor, …”
   • [claimed-docs] “API keys authenticate programmatic access to your organization, from the REST API, CI pipelines, and scripts.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.manufact.com/openapi.json, https://docs.manufact.com/swagger.json, https://docs.manufac…”
   • [probe] “official MCP server documented at https://docs.manufact.com/mcp”

   proof ↗⚑ flag

Metadata

1. developerInspect a listed server's tools, input schemas, transport, and hosting attributes before connecting to it

   weight 2 · round to Smithery
   Smithery~partialprobed6/10

   The registry API (probe-rt-1) exposes machine-readable metadata per server — displayName, qualifiedName, isDeployed, remote, verified — covering hosting/transport attributes, and community security audits (comm-1, comm-2) confirm that tool description fields are visible and inspectable pre-connection (that's how injected instructions were found). The CLI's `tool find` command (gh-2) and per-server typed SDKs generated from tool schemas (docs-16) further support schema visibility, but there is no single documented endpoint or UI view that surfaces full input schemas plus transport plus hosting attributes together in one inspection step. Missing for 10: a documented unified 'server detail' API/UI showing input schemas explicitly, and independent confirmation that transport type (stdio/http/websocket) is surfaced pre-connect.

   • [probe] “PROBE registry-api (2026-09-05): GET https://registry.smithery.ai/servers?pageSize=3 with NO API key returned HTTP 200 with a machine-readab…”
   • [github] “smithery tool find [query] # Search tools by name or intent smithery tool call <connection> <tool> [args] # Call a tool”
   • [claimed-docs] “Every MCP server published on Smithery gets a typed TypeScript SDK generated from its tool and trigger schemas.”
   • [community] “Scanned the top 100 servers on Smithery's registry; 22 of 100 flagged at least one security finding (28 total, 4 critical, 24 high), most co…”
   • [community] “Real examples pulled from Smithery-hosted servers: Context7, Google Sheets, Senzia, and Brave Search tool descriptions contained agent-direc…”

   proof ↗⚑ flag

   Manufact~partialclaimed4/10

   Manufact's dashboard exposes hosting/deployment attributes (MCP URL, branch, commit, publish-checklist score) and a testing flow that runs across MCP clients/models, which implies some pre-connection inspection, but there's no explicit documentation of viewing a server's tool list or input schemas before connecting. Missing for 10: explicit tool/input-schema inspection UI, transport-type display, and any independent/hands-on confirmation of this inspection workflow.

   • [claimed-docs] “paste your MCP URL. No deploy required: Manufact connects directly for Chat, testing, and the publish workflow.”
   • [claimed-docs] “The history table shows every deployment with its MCP URL, source branch and commit, duration, creation time, and the latest completed publi…”
   • [claimed-docs] “runs an automated audit of your server for protocol compliance and platform readiness”
   • [claimed-docs] “Manufact executes them across different MCP clients and LLM models”

   proof ↗⚑ flag

Playground

1. developerExercise an MCP server's tools from a browser-based inspector or playground before integrating it

   weight 2 · round to Manufact
   Smithery—none0/10

   The evidence pack documents Smithery's registry, REST connect flow, OAuth/token handling, uplink, deep links, and CLI tool-calling, but nothing describes a browser-based inspector or playground UI for interactively testing a server's tools before integration. Since a registry/marketplace product could plausibly ship such a try-before-integrate UI, the axis applies, but no evidence confirms it exists.

   ⚑ flag

   Manufact~partialclaimed6/10

   Manufact's dashboard lets developers paste an MCP URL for direct connection and 'Chat, testing' before publishing (manufact-docs-2), and testing runs across different MCP clients/models (manufact-docs-11), which supports exercising tools pre-integration; the CLI also offers a local inspector (manufact-docs-15). However, this is presented as an integrated dashboard/testing workflow rather than a dedicated browser-based 'inspector/playground' UI, and no screenshots, hands-on walkthrough, or independent corroboration of the playground UX are provided. Missing for 10: concrete UI evidence/screenshots of a dedicated tool-invocation playground, independent hands-on confirmation, and detail on how individual tool calls are exercised interactively.

   • [claimed-docs] “paste your MCP URL. No deploy required: Manufact connects directly for Chat, testing, and the publish workflow.”
   • [claimed-docs] “Manufact executes them across different MCP clients and LLM models”
   • [claimed-docs] “Run your MCP server locally with hot reload + inspector”

   proof ↗⚑ flag

Search

1. developerSearch and browse a large catalog of MCP servers and tools by capability before wiring anything into my agent

   weight 3 · round to Smithery
   Smithery✓fullprobed8/10

   Smithery ships both a CLI (`smithery mcp search`, `smithery tool find [query]`) and a keyless registry REST API (registry.smithery.ai/servers) that returns a machine-readable catalog with names, descriptions, verification status and usage counts, letting developers search/browse servers and tools by capability before connecting; docs also list example servers (Exa, Context7, Browserbase) illustrating capability-based discovery. Missing for 10: no evidence of a rich web UI faceted-browse/filter-by-category experience or independent hands-on review of the discovery UX itself beyond the CLI/API probes.

   • [github] “smithery mcp search [term] # Search the Smithery registry smithery mcp add <url> # Add an MCP server connectio…”
   • [github] “smithery tool find [query] # Search tools by name or intent smithery tool call <connection> <tool> [args] # Call a tool”
   • [probe] “PROBE registry-api (2026-09-05): GET https://registry.smithery.ai/servers?pageSize=3 with NO API key returned HTTP 200 with a machine-readab…”
   • [claimed-docs] “Exa - Search the live web, access LinkedIn profiles, do deep research, and more”
   • [claimed-docs] “Context7 - Reference the latest docs for most major SDKs and frameworks directly in Cursor or Claude Code”
   • [claimed-docs] “Browserbase - Control a remote web browser using Stagehand”

   proof ↗⚑ flag

   Manufact—none0/10

   Manufact is a deploy/manage/observe platform for MCP servers you build and own — there is no evidence of a searchable public catalog/registry of third-party MCP servers or tools that a developer can browse by capability before wiring into an agent. All docs relate to deploying, testing, and monitoring your own server, not discovering existing ones.

   ⚑ flag

Cli

1. developerDrive the publish-and-deploy workflow from an official CLI — authenticate, stage or build a server, push it, and install it into agents

   weight 2 · round drawn
   Smithery~partialprobed6/10

   Smithery ships an official CLI (@smithery/cli, confirmed installable via npm and version-checked) with commands to search/add/call MCP servers, and docs cover publishing servers (URL method, MCPB bundles, server-card metadata) plus auth via OAuth/service tokens and installing into agents via deep links/uplink. However, evidence does not clearly show a unified CLI-driven build-and-push workflow (e.g., 'smithery build' or 'smithery deploy' subcommands) — the CLI evidence only shows search/add/call, while publish/build steps are documented as web/manual flows rather than CLI-driven. missing for 10: CLI subcommands for authenticate/build/push in one flow, independent hands-on confirmation of end-to-end publish-via-CLI, and clarity on whether install-into-agents is CLI-native vs deep-link/manual.

   • [github] “smithery mcp search [term] # Search the Smithery registry smithery mcp add <url> # Add an MCP server connectio…”
   • [github] “smithery tool find [query] # Search tools by name or intent smithery tool call <connection> <tool> [args] # Call a tool”
   • [github] “Browse skills on the [Smithery Skills Registry](https://smithery.ai/skills) and install them with the upstream installer”
   • [claimed-docs] “If you’ve already deployed an MCP server elsewhere, you can publish it directly on Smithery via the URL method.”
   • [claimed-docs] “For local stdio servers — Smithery distributes a pre-built MCPB bundle that clients download and run locally.”
   • [claimed-docs] “If automatic scanning can’t complete (auth wall, required configuration, or other issues), you can provide server metadata manually via a st…”
   • [claimed-docs] “Deep links provide a seamless way to integrate Smithery MCPs into supported clients. When a user clicks a deep link from our server page, th…”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/smithery/): a JSON-RPC initialize POST to https://smithery.ai/docs/mc…”

   proof ↗⚑ flag

   Manufact~partialprobed6/10

   Manufact has an official CLI documented with local run/hot-reload and a deploy command that pushes the project to Manufact Cloud (manufact-docs-15, manufact-docs-16, manufact-probe-4), plus API keys for programmatic/CI authentication (manufact-docs-17) and one-click/copy-paste install into agent clients (manufact-docs-18). However, the docs emphasize a GitHub/dashboard-driven deploy flow rather than a fully CLI-driven auth→build→push→install pipeline, and there's no explicit CLI login/auth command or CLI-based install-into-agent step shown. missing for 10: explicit CLI authenticate command, CLI-triggered install-into-agent step, and independent/hands-on confirmation that the full workflow (auth, build, push, install) works end-to-end via CLI alone.

   • [claimed-docs] “Run your MCP server locally with hot reload + inspector”
   • [claimed-docs] “Deploys the current project to Manufact Cloud.”
   • [claimed-docs] “API keys authenticate programmatic access to your organization, from the REST API, CI pipelines, and scripts.”
   • [claimed-docs] “You get one-click installs or copy-paste configuration for”
   • [probe] “official CLI documented at https://docs.manufact.com/cli”

   proof ↗⚑ flag

Frameworks

1. developerPlug the platform's tools into major agent frameworks and SDKs (Vercel AI SDK, LangChain, CrewAI, OpenAI or Anthropic SDKs) with documented adapters

   weight 2 · round to Smithery
   Smithery~partialclaimed5/10

   Smithery documents a dedicated Vercel AI SDK integration with automatic OAuth/token handling and multi-server tool aggregation (docs-13/14), and every published server gets an auto-generated typed SDK (docs-16), but the evidence pack shows no documented adapters for LangChain, CrewAI, or OpenAI/Anthropic SDKs specifically. Missing for 10: documented LangChain adapter, documented CrewAI adapter, documented OpenAI/Anthropic SDK adapter, and independent/hands-on confirmation beyond first-party docs.

   • [claimed-docs] “Smithery handles OAuth, token refresh, and connection management automatically.”
   • [claimed-docs] “Connect to multiple servers and aggregate their tools”
   • [claimed-docs] “Every MCP server published on Smithery gets a typed TypeScript SDK generated from its tool and trigger schemas.”
   • [claimed-docs] “Smithery gives you a simple REST interface for connecting to MCP servers. Instead of implementing the MCP protocol directly, handling OAuth …”

   proof ↗⚑ flag

   Manufact—none0/10

   Manufact's docs describe connecting hosted MCP servers to MCP clients like Cursor, Claude, and ChatGPT (manufact-docs-14, manufact-docs-18), but there is no evidence of documented adapters or integration guides for agent frameworks/SDKs such as Vercel AI SDK, LangChain, CrewAI, or OpenAI/Anthropic SDKs. Missing for 10: any mention of LangChain, CrewAI, Vercel AI SDK, or OpenAI/Anthropic SDK integration guides or adapters.

   • [claimed-docs] “exposes the platform, organizations, servers, deployments, logs, and observability, as MCP tools, so you can manage everything from Cursor, …”
   • [claimed-docs] “You get one-click installs or copy-paste configuration for”

   proof ↗⚑ flag

Security

1. platform-engineerRely on the platform scanning or inspecting listed servers for malicious or vulnerable behavior before my agents use them

   weight 2 · round to Smithery
   Smithery!disputedcontradicted4/10

   Smithery's docs mention 'automatic scanning' of published servers (smithery-docs-11), implying some vetting pipeline, but independent research directly contradicts its effectiveness: scanning the top 100 registry servers found 22 with security findings (4 critical) including tool-description prompt-injection attacks embedded in widely-used servers like Context7 and Brave Search that execute silently on agent connection (smithery-comm-1, smithery-comm-2), and researchers separately found a path-traversal vulnerability in Smithery's own hosting escalatable to a supply-chain attack (smithery-comm-3). Missing for 10: any documented remediation/removal process for flagged malicious servers, transparency on scan methodology or coverage, and evidence the automatic scanning actually catches behavioral/prompt-injection risks rather than just metadata issues.

   • [claimed-docs] “If automatic scanning can’t complete (auth wall, required configuration, or other issues), you can provide server metadata manually via a st…”
   • [community] “Scanned the top 100 servers on Smithery's registry; 22 of 100 flagged at least one security finding (28 total, 4 critical, 24 high), most co…”
   • [community] “Real examples pulled from Smithery-hosted servers: Context7, Google Sheets, Senzia, and Brave Search tool descriptions contained agent-direc…”
   • [community] “Security researchers detailed a path traversal vulnerability in Smithery's MCP server hosting that could be escalated into a supply chain at…”

   proof ↗⚑ flag

   Manufact—none0/10

   Manufact's publish-checklist audits cover protocol compliance, platform readiness, and end-to-end functional tests (manufact-docs-7, manufact-docs-8), but there is no evidence these checks screen for malicious behavior or security vulnerabilities in listed servers before agent use. Missing for 10: any documented security/malware scanning step, vulnerability detection, or a curated registry vetting process distinct from functional/protocol compliance testing.

   • [claimed-docs] “runs an automated audit of your server for protocol compliance and platform readiness”
   • [claimed-docs] “end-to-end checks that exercise your server live in ChatGPT and Claude”
   • [claimed-docs] “copy a fix prompt into your own coding agent, or run the built-in autofix flow”
   • [claimed-docs] “you can start a checklist automatically after each successful deployment”

   proof ↗⚑ flag

Signals

1. developerCompare servers using published quality signals — usage counts, health or liveness checks, scores, or verification badges

   weight 2 · round to Smithery
   Smithery~partialprobed6/10

   The registry API (probe evidence) exposes useCount and verified fields per server, plus isDeployed/remote status, giving developers concrete usage and verification signals to compare servers; docs also mention automatic scanning during publish which implies some quality vetting. However, there is no documented explicit 'quality score' metric or dedicated health/liveness-check endpoint beyond isDeployed, and independent security scans found unflagged risky servers in the top 100, showing the published signals don't capture all quality concerns. Missing for 10: an explicit scoring/quality-score field, a dedicated liveness/health-check mechanism, and corroboration that verification badges reliably reflect security vetting.

   • [probe] “PROBE registry-api (2026-09-05): GET https://registry.smithery.ai/servers?pageSize=3 with NO API key returned HTTP 200 with a machine-readab…”
   • [claimed-docs] “If automatic scanning can’t complete (auth wall, required configuration, or other issues), you can provide server metadata manually via a st…”
   • [community] “Scanned the top 100 servers on Smithery's registry; 22 of 100 flagged at least one security finding (28 total, 4 critical, 24 high), most co…”

   proof ↗⚑ flag

   Manufact~partialclaimed3/10

   Manufact provides a 'publish checklist score' and automated audits for protocol compliance/readiness on servers you deploy (manufact-docs-6, manufact-docs-7), which is a quality signal, but this is scoped to your own deployed servers rather than a public catalog for comparing third-party MCP servers by usage counts, health/liveness, or verification badges. Missing for 10: cross-server marketplace/directory view, usage/popularity counts, verification badges, and independent evidence that these scores are used to compare multiple distinct servers rather than track one's own deployment quality.

   • [claimed-docs] “The history table shows every deployment with its MCP URL, source branch and commit, duration, creation time, and the latest completed publi…”
   • [claimed-docs] “runs an automated audit of your server for protocol compliance and platform readiness”
   • [claimed-docs] “you can start a checklist automatically after each successful deployment”

   proof ↗⚑ flag

Tool search

1. ai agentSearch and load tools at runtime through meta-tools instead of having thousands of tool definitions stuffed into my context

   weight 2 · round to Smithery
   Smithery✓fullprobed7/10

   Smithery's CLI exposes explicit meta-tools — `mcp search`, `tool find [query]`, and `tool call <connection> <tool>` — letting an agent discover and invoke tools by intent at runtime rather than preloading every tool definition, and the registry API is confirmed live and keylessly searchable in a probe returning a machine-readable catalog. missing for 10: no evidence the search/find/call meta-tools are themselves exposed as callable MCP tools inside a live agent session (vs. CLI-level usage), and no independent benchmark showing reduced context footprint from this pattern.

   • [github] “smithery mcp search [term] # Search the Smithery registry smithery mcp add <url> # Add an MCP server connectio…”
   • [github] “smithery tool find [query] # Search tools by name or intent smithery tool call <connection> <tool> [args] # Call a tool”
   • [probe] “PROBE registry-api (2026-09-05): GET https://registry.smithery.ai/servers?pageSize=3 with NO API key returned HTTP 200 with a machine-readab…”
   • [claimed-docs] “Every MCP server published on Smithery gets a typed TypeScript SDK generated from its tool and trigger schemas.”

   proof ↗⚑ flag

   Manufact—none0/10

   Manufact is a platform for deploying, monitoring, and managing MCP servers (deployments, observability, publish checks), not a runtime tool-search/meta-tool mechanism for agents to dynamically discover and load tools instead of loading full definitions. No evidence describes dynamic tool search/loading or meta-tool patterns to reduce context bloat.

   ⚑ flag