Smithery vs Glama

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round to Glama
   Smithery~partialprobed6/10

   llms.txt and docs.md are confirmed 404, so the specific llms.txt convention isn't supported, but Smithery does expose agent-oriented docs alternatives: docs served live over MCP (JSON-RPC initialize succeeds at /docs/mcp) and a machine-readable OpenAPI 3.1 spec at /docs/openapi.json that an agent could consume to understand the API. missing for 10: an actual llms.txt file, and any docs indicating agents are expected to consume these alternate machine-readable formats as a first-class 'llms.txt-style' workflow.

   • [probe] “PROBE llms.txt: HTTP 404 at https://smithery.ai/llms.txt”
   • [probe] “PROBE docs-md: HTTP 404 at https://smithery.ai/docs.md”
   • [probe] “PROBE openapi (2026-09-05): GET https://smithery.ai/docs/openapi.json returned HTTP 200 with an OpenAPI 3.1.0 description (35 documented pat…”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/smithery/): a JSON-RPC initialize POST to https://smithery.ai/docs/mc…”

   proof ↗⚑ flag

   Glama✓fullprobed8/10

   Glama serves a live llms.txt at the root (HTTP 200) describing the directory, and additionally exposes a machine-discoverable API catalog (.well-known/api-catalog) and an OpenAPI 3.0.3 spec, giving agents multiple agent-oriented discovery docs. Missing for 10: a dedicated docs-md fallback (mcp.md returns 404) and independent confirmation that agents actually consume llms.txt successfully.

   • [probe] “PROBE llms.txt: HTTP 200 at https://glama.ai/llms.txt # Glama > A directory of 81,811 Model Context Protocol servers and 15,990 remote conn…”
   • [probe] “PROBE openapi (2026-09-05): GET https://glama.ai/api/mcp/openapi.json returned HTTP 200 with an OpenAPI 3.0.3 description (9 documented path…”
   • [probe] “PROBE api-catalog (2026-09-05): GET https://glama.ai/.well-known/api-catalog returned HTTP 200 with an RFC 9727 linkset naming the platform …”
   • [probe] “PROBE docs-md: HTTP 404 at https://glama.ai/mcp.md”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round to Smithery
   Smithery✓fullprobed7/10

   Smithery ships a CLI (search/add/call tools) and a documented REST API with an OpenAPI 3.1 spec, service tokens scoped for AI agents/automation, and a live registry API queryable without interactive auth — all of which support scripted/headless use in CI pipelines. Runtime probes confirm the CLI installs and runs (`smithery --version`) and the API/registry endpoints respond programmatically. missing for 10: explicit CI/CD pipeline examples or GitHub Actions integration docs, and independent reports of real-world CI usage.

   • [github] “smithery mcp search [term] # Search the Smithery registry smithery mcp add <url> # Add an MCP server connectio…”
   • [github] “smithery tool find [query] # Search tools by name or intent smithery tool call <connection> <tool> [args] # Call a tool”
   • [claimed-docs] “Smithery gives you a simple REST interface for connecting to MCP servers. Instead of implementing the MCP protocol directly, handling OAuth …”
   • [claimed-docs] “Service tokens let you safely expose Smithery to browsers, mobile apps, and AI agents without leaking your API key.”
   • [probe] “PROBE registry-api (2026-09-05): GET https://registry.smithery.ai/servers?pageSize=3 with NO API key returned HTTP 200 with a machine-readab…”
   • [probe] “PROBE openapi (2026-09-05): GET https://smithery.ai/docs/openapi.json returned HTTP 200 with an OpenAPI 3.1.0 description (35 documented pat…”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/smithery/): a JSON-RPC initialize POST to https://smithery.ai/docs/mc…”

   proof ↗⚑ flag

   Glama!disputedcontradicted4/10

   Glama documents automation hooks (schedules/webhooks, docs-10) and exposes a live registry/platform API with an OpenAPI spec that an agent could script against in CI (glama-probe-rt-1, glama-probe-rt-2, glama-probe-rt-3), suggesting headless automation is possible. However, hands-on community reports show the automated rebuild/rescore trigger silently failed for days despite manual resyncs and API attempts, requiring an undocumented workaround (adding billing credits) to unblock — a concrete case where headless automation did not work as claimed (glama-comm-2, glama-comm-3). Missing for 10: a documented CLI or CI-specific guide, confirmation the API supports full server-management lifecycle non-interactively, and resolution of the demonstrated automation failure.

   • [claimed-docs] “Automate workflows using schedules or webhooks.”
   • [probe] “PROBE registry-api (2026-09-05): GET https://glama.ai/api/mcp/v1/servers?first=3 without credentials returned HTTP 401 with a JSON auth chal…”
   • [probe] “PROBE openapi (2026-09-05): GET https://glama.ai/api/mcp/openapi.json returned HTTP 200 with an OpenAPI 3.0.3 description (9 documented path…”
   • [probe] “PROBE api-catalog (2026-09-05): GET https://glama.ai/.well-known/api-catalog returned HTTP 200 with an RFC 9727 linkset naming the platform …”
   • [community] “After uploading a new MCP version, the quality score test wouldn't rerun for 3 days despite manual resyncs and API attempts. Turned out a cr…”
   • [community] “Neither I nor my AI agent could figure out over 4 days of searching documentation why the quality score retest wasn't triggering on the late…”

   proof ↗⚑ flag

3. ai-native userPlug MCP servers into this product so it can use their tools

   weight 3 · round to Smithery
   Smithery✓fullprobed8/10

   Smithery's core purpose is connecting/aggregating MCP servers: docs and probes confirm a REST/registry interface, hosted MCP endpoints with OAuth, deep links, uplink for local servers, and CLI/SDK tooling for adding and calling tools across connections, with runtime probes corroborating a live registry and per-server endpoints. Community reports of tool-description injection risks are a security caveat, not evidence the connection/tool-use mechanism fails. missing for 10: independent hands-on confirmation that an AI agent (not just probes) successfully invokes tools end-to-end across multiple aggregated servers, and resolution of the tool-description-injection security concerns.

   • [claimed-docs] “Smithery gives you a simple REST interface for connecting to MCP servers. Instead of implementing the MCP protocol directly, handling OAuth …”
   • [claimed-docs] “Uplink exposes an MCP server running on any machine as a regular Smithery connection.”
   • [claimed-docs] “Deep links provide a seamless way to integrate Smithery MCPs into supported clients. When a user clicks a deep link from our server page, th…”
   • [claimed-docs] “Smithery handles OAuth, token refresh, and connection management automatically.”
   • [claimed-docs] “Connect to multiple servers and aggregate their tools”
   • [probe] “PROBE registry-api (2026-09-05): GET https://registry.smithery.ai/servers?pageSize=3 with NO API key returned HTTP 200 with a machine-readab…”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://server.smithery.ai/exa/mcp returned HTTP 401 with an OAuth challenge — the hoste…”
   • [github] “smithery mcp search [term] # Search the Smithery registry smithery mcp add <url> # Add an MCP server connectio…”
   • [github] “smithery tool find [query] # Search tools by name or intent smithery tool call <connection> <tool> [args] # Call a tool”
   • [community] “Scanned the top 100 servers on Smithery's registry; 22 of 100 flagged at least one security finding (28 total, 4 critical, 24 high), most co…”

   proof ↗⚑ flag

   Glama✓fullclaimed7/10

   Glama's MCP gateway explicitly routes agent tool calls through hosted/connected MCP servers, with a catalog to discover servers, an inspector to test them, and per-tool access controls — directly enabling an AI-native user to plug in MCP servers for tool use. Missing for 10: independent hands-on confirmation that agent tool-calling through the gateway works reliably end-to-end (community evidence focuses on quality-score/rebuild issues, not tool invocation itself), and no third-party review specifically validating the plug-in/tool-use flow.

   • [claimed-docs] “Test any MCP server from your browser. Supports every Model Context Protocol feature”
   • [claimed-docs] “Every MCP call from your agents flows through Glama – so you know exactly what your AI is doing, and you decide what it's allowed to do.”
   • [claimed-docs] “Per-tool access control – enable or disable individual tools”
   • [claimed-docs] “Search Glama's curated catalog of open-source servers and remote connectors, inspect their capabilities, and keep your own directory synchro…”
   • [claimed-docs] “MCP clients are the applications that connect to MCP servers.”

   proof ↗⚑ flag

4. ai-native userConnect an agent via an official MCP server

   weight 3 · round to Smithery
   Smithery✓fullprobed9/10

   Smithery is a hosting/registry platform (not itself an agent), so the exception applies and publishing an official MCP server endpoint is a fair, evidenced axis: docs describe OAuth-handled connections, deep linking, uplink, and typed SDKs, and independent probes confirm a live hosted MCP endpoint (server.smithery.ai/exa/mcp) requiring OAuth and even the docs site itself served over MCP (smithery.ai/docs/mcp) with a working CLI install. Missing for 10: no independent third-party report validating that connecting via the hosted server works end-to-end for an actual agent (only probe-level confirmation, not user testimonial), and community findings raise tool-description-injection risk that slightly tempers confidence though it doesn't contradict the connection mechanism itself.

   • [claimed-docs] “Smithery gives you a simple REST interface for connecting to MCP servers. Instead of implementing the MCP protocol directly, handling OAuth …”
   • [claimed-docs] “Uplink exposes an MCP server running on any machine as a regular Smithery connection.”
   • [claimed-docs] “Deep links provide a seamless way to integrate Smithery MCPs into supported clients. When a user clicks a deep link from our server page, th…”
   • [claimed-docs] “Smithery handles OAuth, token refresh, and connection management automatically.”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://server.smithery.ai/exa/mcp returned HTTP 401 with an OAuth challenge — the hoste…”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/smithery/): a JSON-RPC initialize POST to https://smithery.ai/docs/mc…”

   proof ↗⚑ flag

   Glama✓fullprobed7/10

   Glama's docs describe an official MCP gateway/hosting service through which agents connect to MCP servers, with per-tool access control, inspector testing, and a hosted catalog (glama-docs-1,4,5,6). This directly satisfies the story since Glama is a hosting/gateway platform, not an agent, so serving MCP is a core in-scope capability. Missing for 10: independent hands-on verification of an agent actually connecting through the gateway, and community feedback highlights friction in publishing/quality-score workflows (glama-comm-2,3) rather than the connection experience itself.

   • [claimed-docs] “Test any MCP server from your browser. Supports every Model Context Protocol feature”
   • [claimed-docs] “Every MCP call from your agents flows through Glama – so you know exactly what your AI is doing, and you decide what it's allowed to do.”
   • [claimed-docs] “Per-tool access control – enable or disable individual tools”
   • [claimed-docs] “Search Glama's curated catalog of open-source servers and remote connectors, inspect their capabilities, and keep your own directory synchro…”
   • [probe] “PROBE llms.txt: HTTP 200 at https://glama.ai/llms.txt # Glama > A directory of 81,811 Model Context Protocol servers and 15,990 remote conn…”

   proof ↗⚑ flag

5. ai-native userUse an official CLI

   weight 2 · round to Smithery
   Smithery~partialprobed6/10

   There is concrete evidence of a working Smithery CLI (npm install @smithery/cli verified, version 4.11.1) with documented commands for searching the registry, adding MCP connections, finding/calling tools, and installing skills. However, the GitHub source is attributed to 'arcadeai-labs/smithery-cli' rather than an obviously first-party Smithery org, leaving some ambiguity about official first-party ownership, and there is no independent/community corroboration of CLI usage. Missing for 10: clearer first-party branding/ownership confirmation, official docs page dedicated to the CLI, and independent hands-on user reports.

   • [github] “smithery mcp search [term] # Search the Smithery registry smithery mcp add <url> # Add an MCP server connectio…”
   • [github] “smithery tool find [query] # Search tools by name or intent smithery tool call <connection> <tool> [args] # Call a tool”
   • [github] “Browse skills on the [Smithery Skills Registry](https://smithery.ai/skills) and install them with the upstream installer”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/smithery/): a JSON-RPC initialize POST to https://smithery.ai/docs/mc…”

   proof ↗⚑ flag

   Glama—none0/10

   No evidence of an official Glama CLI tool anywhere in the docs, probes, or community discussion; the product exposes web UI, APIs (OpenAPI, registry API), and a browser-based inspector, but nothing indicating a dedicated CLI for AI-native workflows.

   ⚑ flag

6. ai-native userDrive the product through a documented public API

   weight 3 · round to Smithery
   Smithery✓fullprobed8/10

   Smithery exposes a real, documented public API: a registry API (registry.smithery.ai) queryable without keys, a full OpenAPI 3.1.0 spec (35 paths) confirmed live at smithery.ai/docs/openapi.json, and a REST/connection interface with OAuth and token scoping for programmatic/agent use, plus a CLI (smithery mcp/tool commands) for driving it. Initial llms.txt/docs.md probes 404'd but the actual OpenAPI spec and registry endpoints were independently verified live, satisfying 'documented public API' for an AI-native/agentic consumer. missing for 10: no independent third-party corroboration of API stability/versioning, and some docs endpoints (llms.txt) are absent despite the OpenAPI spec existing.

   • [probe] “PROBE registry-api (2026-09-05): GET https://registry.smithery.ai/servers?pageSize=3 with NO API key returned HTTP 200 with a machine-readab…”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://server.smithery.ai/exa/mcp returned HTTP 401 with an OAuth challenge — the hoste…”
   • [probe] “PROBE openapi (2026-09-05): GET https://smithery.ai/docs/openapi.json returned HTTP 200 with an OpenAPI 3.1.0 description (35 documented pat…”
   • [claimed-docs] “Smithery gives you a simple REST interface for connecting to MCP servers. Instead of implementing the MCP protocol directly, handling OAuth …”
   • [claimed-docs] “Service tokens let you safely expose Smithery to browsers, mobile apps, and AI agents without leaking your API key.”
   • [github] “smithery mcp search [term] # Search the Smithery registry smithery mcp add <url> # Add an MCP server connectio…”
   • [github] “smithery tool find [query] # Search tools by name or intent smithery tool call <connection> <tool> [args] # Call a tool”

   proof ↗⚑ flag

   Glama~partialprobed6/10

   Glama exposes a live registry API with an OpenAPI 3.0.3 spec (9 documented paths) and a machine-discoverable API catalog via .well-known/api-catalog, confirming a real documented public API surface. However, the API requires a key and returns 401 without one, no standalone human-readable API docs page was found (mcp.md 404, openapi.json at root 404), and community feedback notes confusion even for API-based automation attempts (quality score retrigger via API failed silently). Missing for 10: a clear top-level API documentation page/portal, broader endpoint coverage beyond 9 paths, and independent confirmation of smooth API-driven workflows.

   • [probe] “PROBE registry-api (2026-09-05): GET https://glama.ai/api/mcp/v1/servers?first=3 without credentials returned HTTP 401 with a JSON auth chal…”
   • [probe] “PROBE openapi (2026-09-05): GET https://glama.ai/api/mcp/openapi.json returned HTTP 200 with an OpenAPI 3.0.3 description (9 documented path…”
   • [probe] “PROBE api-catalog (2026-09-05): GET https://glama.ai/.well-known/api-catalog returned HTTP 200 with an RFC 9727 linkset naming the platform …”
   • [probe] “PROBE docs-md: HTTP 404 at https://glama.ai/mcp.md”
   • [probe] “PROBE openapi: all candidate paths 404 (https://glama.ai/openapi.json, https://glama.ai/swagger.json, https://glama.ai/api/openapi.json, htt…”
   • [community] “Neither I nor my AI agent could figure out over 4 days of searching documentation why the quality score retest wasn't triggering on the late…”

   proof ↗⚑ flag

7. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round to Smithery
   Smithery✓fullclaimed7/10

   Smithery docs describe service tokens explicitly scoped to a specific app/connection/user metadata ('list and call tools on connections in my-app where metadata.userId is user-123 — nothing else'), plus encrypted write-only credential storage and namespace-scoped API keys, directly matching the least-privilege credential story for agents. Missing for 10: independent/hands-on verification that scope enforcement actually holds, and documentation of token revocation/expiry/audit controls.

   • [claimed-docs] “Service tokens let you safely expose Smithery to browsers, mobile apps, and AI agents without leaking your API key.”
   • [claimed-docs] “This token can list and call tools on connections in `my-app` where `metadata.userId` is `user-123` — nothing else.”
   • [claimed-docs] “Secure credential storage — Encrypted, write-only storage for API keys and tokens”
   • [claimed-docs] “Namespaces act as a shared workspace. Resources within a namespace can be managed by anyone with the appropriate API key scoped to that name…”

   proof ↗⚑ flag

   Glama—none0/10

   Evidence shows per-tool access control and gateway visibility into MCP calls, but no mention of scoped/least-privilege API key issuance, credential scoping, or granular permission grants for agent credentials — the API key system found in probes appears to be a single account-level key, not scoped tokens. Missing for 10: scoped credential creation, permission/role definitions per key, evidence of least-privilege token issuance for agents.

   • [claimed-docs] “Per-tool access control – enable or disable individual tools”
   • [claimed-docs] “Every MCP call from your agents flows through Glama – so you know exactly what your AI is doing, and you decide what it's allowed to do.”
   • [probe] “PROBE registry-api (2026-09-05): GET https://glama.ai/api/mcp/v1/servers?first=3 without credentials returned HTTP 401 with a JSON auth chal…”

   proof ↗⚑ flag

8. ai-native userBuild against official SDKs

   weight 2 · round to Smithery
   Smithery✓fullprobed7/10

   Smithery auto-generates a typed TypeScript SDK for every published MCP server (smithery-docs-16), ships an official CLI (smithery-cli on GitHub) verified via live npm install (smithery-probe-rt-4), and exposes a documented OpenAPI 3.1 spec for platform API client generation, confirmed live via probe (smithery-probe-rt-3). This gives AI-native builders concrete, verified official SDK/CLI/API surfaces to build against. Missing for 10: SDKs beyond TypeScript (no Python/other language SDK evidence), and independent third-party corroboration of real-world SDK usage beyond Smithery's own docs/probes.

   • [claimed-docs] “Every MCP server published on Smithery gets a typed TypeScript SDK generated from its tool and trigger schemas.”
   • [probe] “PROBE openapi (2026-09-05): GET https://smithery.ai/docs/openapi.json returned HTTP 200 with an OpenAPI 3.1.0 description (35 documented pat…”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/smithery/): a JSON-RPC initialize POST to https://smithery.ai/docs/mc…”
   • [github] “smithery mcp search [term] # Search the Smithery registry smithery mcp add <url> # Add an MCP server connectio…”
   • [github] “smithery tool find [query] # Search tools by name or intent smithery tool call <connection> <tool> [args] # Call a tool”

   proof ↗⚑ flag

   Glama~partialprobed3/10

   Glama exposes a documented OpenAPI 3.0.3 spec and machine-discoverable API catalog that could be used to generate a client, but there is no evidence of an official first-party SDK (e.g., published npm/PyPI client library) for building against the platform. missing for 10: official SDK packages, SDK usage docs/examples, language coverage, independent confirmation of SDK reliability.

   • [probe] “PROBE openapi (2026-09-05): GET https://glama.ai/api/mcp/openapi.json returned HTTP 200 with an OpenAPI 3.0.3 description (9 documented path…”
   • [probe] “PROBE api-catalog (2026-09-05): GET https://glama.ai/.well-known/api-catalog returned HTTP 200 with an RFC 9727 linkset naming the platform …”
   • [probe] “PROBE registry-api (2026-09-05): GET https://glama.ai/api/mcp/v1/servers?first=3 without credentials returned HTTP 401 with a JSON auth chal…”

   proof ↗⚑ flag

9. ai-native userSubscribe to events via webhooks

   weight 2 · round to Smithery
   Smithery~partialclaimed6/10

   Smithery's Triggers feature explicitly lets consumers subscribe to upstream events, proxying subscribe calls through to the server—directly matching webhook/event-subscription functionality for agentic use. However, this is documented only at a conceptual level with no depth on webhook delivery mechanics, retry/reliability guarantees, or independent hands-on verification. missing for 10: detailed webhook payload/delivery docs, independent corroboration of triggers working in practice, and coverage of failure/retry semantics.

   • [claimed-docs] “Triggers let an MCP server surface events from its upstream service. When a consumer activates a trigger via Smithery, Smithery proxies the …”

   proof ↗⚑ flag

   Glama~partialclaimed4/10

   Glama's pricing page mentions automating workflows via 'schedules or webhooks,' indicating some webhook capability exists, but there is no documentation of a webhook subscription API, event types, payload format, or setup instructions, and no independent/hands-on confirmation. Missing for 10: documented webhook subscription endpoint or event catalog, setup/configuration guide, and community or probe evidence confirming it works in practice.

   • [claimed-docs] “Automate workflows using schedules or webhooks.”

   proof ↗⚑ flag

Agentic features

1. ai-native userSet up automations that run autonomously in the background

   weight 2 · round to Glama
   Smithery~partialclaimed3/10

   Smithery's Triggers feature lets an MCP server surface upstream events which Smithery proxies to a consumer, hinting at event-driven automation, but there is no evidence of a scheduler, persistent background execution engine, or the platform itself running autonomous tasks — automation actually running requires an external agent that consumes the trigger. missing for 10: evidence of a scheduling/cron mechanism, autonomous execution loop hosted by Smithery, and confirmation that triggers fire without an active client agent.

   • [claimed-docs] “Triggers let an MCP server surface events from its upstream service. When a consumer activates a trigger via Smithery, Smithery proxies the …”

   proof ↗⚑ flag

   Glama~partialclaimed4/10

   Glama's pricing page mentions automating workflows via schedules or webhooks (glama-docs-10), which suggests background automation capability, but there is no first-party documentation detailing setup, configuration, or behavior of scheduled/webhook-triggered automations, nor any community or hands-on evidence confirming this works. missing for 10: detailed docs on schedule/webhook automation setup, examples of autonomous background runs, independent confirmation the feature works as claimed.

   • [claimed-docs] “Automate workflows using schedules or webhooks.”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   Smithery~partialprobed5/10

   Smithery exposes a full OpenAPI 3.1 spec (35 paths) and even serves its documentation itself as an MCP endpoint that agents can query interactively via JSON-RPC, which supports AI-native exploration of the API surface. However there's no evidence of a classic interactive reference UI (e.g., Swagger/Redoc 'try it out' runnable examples in-browser), and alternate machine-readable doc formats (llms.txt, docs.md) return 404s. missing for 10: an actual interactive/try-it-out API console with executable example requests, evidence of runnable code snippets in docs, independent confirmation of usability of the MCP-served docs.

   • [probe] “PROBE openapi (2026-09-05): GET https://smithery.ai/docs/openapi.json returned HTTP 200 with an OpenAPI 3.1.0 description (35 documented pat…”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/smithery/): a JSON-RPC initialize POST to https://smithery.ai/docs/mc…”
   • [probe] “PROBE llms.txt: HTTP 404 at https://smithery.ai/llms.txt”
   • [probe] “PROBE docs-md: HTTP 404 at https://smithery.ai/docs.md”

   proof ↗⚑ flag

   Glama~partialprobed5/10

   Glama exposes an OpenAPI 3.0.3 spec (9 documented paths) and an RFC 9727 API catalog, enabling agent-driven client generation, plus an interactive MCP Inspector for testing servers in-browser — but this is not a fully documented, runnable 'API reference' experience with examples for the platform API itself, and the openapi.json/mcp.md at top-level paths 404. missing for 10: a human/agent-facing interactive docs UI (like Swagger UI/Redoc) with runnable examples against the documented OpenAPI paths, and confirmation the 9-path spec covers the full registry API rather than a subset.

   • [probe] “PROBE openapi (2026-09-05): GET https://glama.ai/api/mcp/openapi.json returned HTTP 200 with an OpenAPI 3.0.3 description (9 documented path…”
   • [probe] “PROBE api-catalog (2026-09-05): GET https://glama.ai/.well-known/api-catalog returned HTTP 200 with an RFC 9727 linkset naming the platform …”
   • [claimed-docs] “Test any MCP server from your browser. Supports every Model Context Protocol feature”
   • [probe] “PROBE docs-md: HTTP 404 at https://glama.ai/mcp.md”
   • [probe] “PROBE openapi: all candidate paths 404 (https://glama.ai/openapi.json, https://glama.ai/swagger.json, https://glama.ai/api/openapi.json, htt…”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   Smithery✓fullprobed8/10

   A direct hands-on probe confirms a live, machine-readable OpenAPI 3.1.0 specification with 35 documented paths at smithery.ai/docs/openapi.json, which an agent could use to generate a client — squarely satisfying the story. Earlier probe attempts at guessed paths (openapi.json at root, swagger.json, etc.) 404'd, but the actual documented location succeeds. Missing for 10: first-party documentation explicitly announcing/linking the OpenAPI spec, and independent community confirmation of its use.

   • [probe] “PROBE openapi (2026-09-05): GET https://smithery.ai/docs/openapi.json returned HTTP 200 with an OpenAPI 3.1.0 description (35 documented pat…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://smithery.ai/openapi.json, https://smithery.ai/swagger.json, https://smithery.ai/api/openapi.…”

   proof ↗⚑ flag

   Glama✓fullprobed8/10

   A live probe confirms Glama serves a machine-readable OpenAPI 3.0.3 spec at /api/mcp/openapi.json describing the platform API, plus an RFC 9727 api-catalog linkset for discovery, though an earlier probe found the naive /openapi.json path 404s. missing for 10: no first-party docs page linking/announcing the spec, and no independent/community confirmation of using it to generate a client.

   • [probe] “PROBE openapi (2026-09-05): GET https://glama.ai/api/mcp/openapi.json returned HTTP 200 with an OpenAPI 3.0.3 description (9 documented path…”
   • [probe] “PROBE api-catalog (2026-09-05): GET https://glama.ai/.well-known/api-catalog returned HTTP 200 with an RFC 9727 linkset naming the platform …”
   • [probe] “PROBE openapi: all candidate paths 404 (https://glama.ai/openapi.json, https://glama.ai/swagger.json, https://glama.ai/api/openapi.json, htt…”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round to Glama
   Smithery—none0/10

   Evidence shows Smithery's Uplink lets developers test an in-development MCP server against a real agent before publishing, but there is no mention of a distinct sandbox environment, test data isolation, or any mechanism preventing production data exposure during testing.

   • [claimed-docs] “Uplink exposes an MCP server running on any machine as a regular Smithery connection.”
   • [claimed-docs] “You’re developing an MCP server and want to test it against a real agent before publishing.”

   proof ↗⚑ flag

   Glama~partialclaimed4/10

   Glama's inspector lets users test MCP servers interactively via browser/tunnel and hosting has 'private by default' listing, but there is no explicit sandbox/staging environment concept distinct from production, nor any mention of test data isolation. missing for 10: explicit sandbox environment separate from production, documentation on data isolation/test fixtures, independent confirmation of safe non-production testing.

   • [claimed-docs] “Test any MCP server from your browser. Supports every Model Context Protocol feature”
   • [claimed-docs] “Test your local MCP servers with the inspector by exposing them via a public tunnel using mcp-proxy.”
   • [claimed-docs] “Private by default – flip to public listing when ready.”

   proof ↗⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   Smithery—none0/10

   Evidence shows Smithery has an OpenAPI 3.1.0 spec and a documented REST/MCP interface, but nothing in the docs or probes describes API versioning conventions or a deprecation policy for breaking changes.

   ⚑ flag

   Glama—none0/10

   Evidence shows an OpenAPI spec and API catalog exist (glama-probe-rt-2, glama-probe-rt-3), but there is no mention anywhere of API versioning scheme or a documented deprecation policy for the platform API. Missing for 10: explicit API version numbering, changelog/migration guides, and a stated deprecation/support-lifetime policy.

   • [probe] “PROBE openapi (2026-09-05): GET https://glama.ai/api/mcp/openapi.json returned HTTP 200 with an OpenAPI 3.0.3 description (9 documented path…”
   • [probe] “PROBE api-catalog (2026-09-05): GET https://glama.ai/.well-known/api-catalog returned HTTP 200 with an RFC 9727 linkset naming the platform …”
   • [probe] “PROBE registry-api (2026-09-05): GET https://glama.ai/api/mcp/v1/servers?first=3 without credentials returned HTTP 401 with a JSON auth chal…”

   proof ↗⚑ flag

Multi tenant

1. platform-engineerMaintain separate connected accounts and scoped tokens per end user of my product, so each user's agent acts only on that user's credentials

   weight 3 · round to Smithery
   Smithery~partialcommunity6/10

   Smithery's token-scoping docs show scoped tokens tied to per-user metadata (e.g. a token limited to connections where metadata.userId is 'user-123'), and secure credential storage/automatic refresh are documented, which directly supports per-end-user credential isolation for a platform-engineer building on top of Smithery. However, there's no first-party documentation of a full per-user account management workflow (provisioning, revocation, auditing across many end users) beyond the single scoping example, and independent security research flagged real vulnerabilities (tool description injection, path traversal) that raise doubts about credential-boundary robustness in practice. missing for 10: dedicated multi-tenant/per-end-user account management docs beyond one scoping example, evidence of token revocation/rotation workflows at scale, and independent verification that scoped tokens hold up against the reported security findings.

   • [claimed-docs] “Service tokens let you safely expose Smithery to browsers, mobile apps, and AI agents without leaking your API key.”
   • [claimed-docs] “This token can list and call tools on connections in `my-app` where `metadata.userId` is `user-123` — nothing else.”
   • [claimed-docs] “Automatic token refresh — Credentials stay valid without manual intervention”
   • [claimed-docs] “Secure credential storage — Encrypted, write-only storage for API keys and tokens”
   • [community] “Scanned the top 100 servers on Smithery's registry; 22 of 100 flagged at least one security finding (28 total, 4 critical, 24 high), most co…”
   • [community] “Security researchers detailed a path traversal vulnerability in Smithery's MCP server hosting that could be escalated into a supply chain at…”

   proof ↗⚑ flag

   Glama—none0/10

   No evidence of per-end-user connected accounts or scoped tokens; Glama's gateway/hosting docs describe per-tool access control and a single API-key model for the platform itself, not multi-tenant credential isolation for a product's end users.

   • [claimed-docs] “Every MCP call from your agents flows through Glama – so you know exactly what your AI is doing, and you decide what it's allowed to do.”
   • [claimed-docs] “Per-tool access control – enable or disable individual tools”
   • [probe] “PROBE registry-api (2026-09-05): GET https://glama.ai/api/mcp/v1/servers?first=3 without credentials returned HTTP 401 with a JSON auth chal…”

   proof ↗⚑ flag

Oauth

1. developerLet the platform run OAuth flows for downstream tools and vault the resulting credentials encrypted, with automatic token refresh

   weight 3 · round to Smithery
   Smithery✓fullprobed7/10

   Docs explicitly describe Smithery handling OAuth flows, encrypted write-only credential storage, and automatic token refresh (smithery-docs-1,2,3,13), and this is corroborated by a live probe showing an OAuth challenge gating a hosted MCP endpoint (smithery-probe-rt-2). missing for 10: no independent hands-on confirmation of encryption-at-rest implementation details or token refresh behavior over time, and community evidence focuses on unrelated security findings rather than corroborating credential vaulting.

   • [claimed-docs] “Smithery gives you a simple REST interface for connecting to MCP servers. Instead of implementing the MCP protocol directly, handling OAuth …”
   • [claimed-docs] “Automatic token refresh — Credentials stay valid without manual intervention”
   • [claimed-docs] “Secure credential storage — Encrypted, write-only storage for API keys and tokens”
   • [claimed-docs] “Smithery handles OAuth, token refresh, and connection management automatically.”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://server.smithery.ai/exa/mcp returned HTTP 401 with an OAuth challenge — the hoste…”

   proof ↗⚑ flag

   Glama—none0/10

   No evidence Glama manages OAuth flows for downstream tools or vaults credentials with encrypted storage and automatic refresh; the docs cover hosting, gateway logging, per-tool access control and inspector testing but nothing about OAuth/credential vaulting.

   • [claimed-docs] “Every MCP call from your agents flows through Glama – so you know exactly what your AI is doing, and you decide what it's allowed to do.”
   • [claimed-docs] “Per-tool access control – enable or disable individual tools”
   • [claimed-docs] “Connect a GitHub repository and Glama builds the image straight from source.”
   • [claimed-docs] “Ship a Dockerfile, an npm package, or a PyPI module. Configure Node or Python versions, build steps, and CMD arguments – or let Glama infer …”

   proof ↗⚑ flag

White label

1. platform-engineerWhite-label the authorization flow with my own branding and my own OAuth app credentials

   weight 1 · round drawn
   Smithery—none0/10

   The evidence pack describes Smithery's own OAuth/credential-management infrastructure (managed auth, token scoping, secure storage) but contains no mention of white-labeling the auth flow with custom branding or bringing your own OAuth app/client credentials to replace Smithery's identity — no BYO-OAuth-app, custom domain, or branding customization feature is documented anywhere.

   ⚑ flag

   Glama—none0/10

   No evidence in the pack mentions white-labeling, custom branding, or bringing your own OAuth app credentials for the authorization flow; the docs cover hosting, gateway access control, and inspector tooling but nothing about customizable OAuth/branding. Missing for 10: any mention of custom OAuth client ID/secret configuration, branded consent screens, or white-label auth flow documentation.

   ⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round drawn
   Smithery—none0/10

   Evidence shows Smithery lets an agent connect to and aggregate tools from multiple servers, and its registry API can be queried/paginated for many entries, but there is no documented batch/bulk operation capability (e.g., bulk tool calls, mass server management, batch CRUD) that would let a user perform bulk actions across many items at once. missing for 10: batch tool-call API, bulk server management commands, mass connection/config updates, any documented multi-item transaction support.

   ⚑ flag

   Glama—none0/10

   Glama's evidence covers MCP server hosting, gateway access control, and API discovery, but nothing describes bulk operations across many items (e.g., batch server management, mass tool invocation) for an AI-native user. No evidence of bulk/batch capability exists.

   ⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round to Glama
   Smithery~partialclaimed3/10

   Smithery's 'Triggers' feature lets an MCP server surface upstream events and lets a consumer 'activate' a trigger, which is a thin form of event-driven automation, but there's no evidence of a user-facing rule-definition mechanism (e.g., 'if event X then run action Y') or a rules engine — it only proxies subscribe calls. Missing for 10: a documented rule/condition-action authoring interface, examples of automated action execution on trigger events, and independent confirmation the feature works end-to-end.

   • [claimed-docs] “Triggers let an MCP server surface events from its upstream service. When a consumer activates a trigger via Smithery, Smithery proxies the …”

   proof ↗⚑ flag

   Glama~partialclaimed4/10

   Glama mentions 'automate workflows using schedules or webhooks' on its pricing page, which implies some event/trigger-based automation, but there is no documented rule-definition system, no description of conditions/actions, and no independent evidence of this working in practice. missing for 10: documented rule/condition-action definition interface, examples of triggers mapping to automated actions, and independent corroboration that scheduled/webhook automation works as claimed.

   • [claimed-docs] “Automate workflows using schedules or webhooks.”

   proof ↗⚑ flag

3. ai-native userSchedule recurring jobs or workflows

   weight 2 · round to Glama
   Smithery—none0/10

   Smithery's docs describe on-demand tool connections, OAuth/credential management, and event-driven 'Triggers' that proxy subscribe calls from upstream services, but nothing describes cron-like recurring job scheduling or workflow automation on a timer. Triggers are reactive (event-based), not scheduled/recurring, so the specific capability asked for is unevidenced.

   • [claimed-docs] “Triggers let an MCP server surface events from its upstream service. When a consumer activates a trigger via Smithery, Smithery proxies the …”
   • [claimed-docs] “Uplink exposes an MCP server running on any machine as a regular Smithery connection.”
   • [claimed-docs] “If you’ve already deployed an MCP server elsewhere, you can publish it directly on Smithery via the URL method.”

   proof ↗⚑ flag

   Glama~partialclaimed4/10

   Glama's pricing page mentions automating workflows via schedules or webhooks, but there is no documentation of how to define recurring jobs, what triggers/scheduling syntax exist, or any hands-on/community confirmation of this feature working. Missing for 10: dedicated scheduler docs/UI, workflow-definition examples, independent confirmation of recurring job execution.

   • [claimed-docs] “Automate workflows using schedules or webhooks.”

   proof ↗⚑ flag

4. ai-native userVersion, review, and roll back my automations

   weight 1 · round drawn
   Smithery—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

   Glama—none0/10

   Glama's evidence covers MCP server hosting, gateway routing, access control, and a directory/inspector — nothing about versioning automations, reviewing changes, or rolling back configurations/workflows. The only 'automate workflows' mention (schedules/webhooks) does not touch version control or rollback. missing for 10: version history for automations, diff/review UI, rollback mechanism, any changelog or revision tracking feature.

   • [claimed-docs] “Automate workflows using schedules or webhooks.”

   proof ↗⚑ flag

Bridge

1. developerBridge a locally running MCP server into the platform as a managed remote connection without deploying it

   weight 1 · round to Smithery
   Smithery✓fullclaimed8/10

   Smithery's Uplink feature explicitly exposes an MCP server running on any local machine as a regular Smithery connection, letting developers test/use a locally running server against real agents without deploying it, and this is backed by scoped tokens and managed OAuth/credential handling. missing for 10: independent/hands-on confirmation of Uplink working end-to-end (only first-party docs cited) and no detail on latency/reliability of the bridged local connection.

   • [claimed-docs] “Uplink exposes an MCP server running on any machine as a regular Smithery connection.”
   • [claimed-docs] “You’re developing an MCP server and want to test it against a real agent before publishing.”
   • [claimed-docs] “You want to run a private tool for yourself or your team without hosting it.”
   • [claimed-docs] “Smithery gives you a simple REST interface for connecting to MCP servers. Instead of implementing the MCP protocol directly, handling OAuth …”
   • [claimed-docs] “Service tokens let you safely expose Smithery to browsers, mobile apps, and AI agents without leaking your API key.”

   proof ↗⚑ flag

   Glama~partialclaimed5/10

   Glama's inspector explicitly supports testing local MCP servers by exposing them via a public tunnel (mcp-proxy), which is a bridging mechanism for locally running servers rather than deploying them into Glama's hosting. However, this is framed as a testing/inspection use case, not as a persistent 'managed remote connection' comparable to a hosted server with access control, gateway routing, and catalog listing. Missing for 10: explicit documentation of a persistent managed-connection mode for local servers (vs. one-off inspector testing), independent confirmation that tunneled local servers get the same gateway/access-control/catalog treatment as hosted ones.

   • [claimed-docs] “Test your local MCP servers with the inspector by exposing them via a public tunnel using mcp-proxy.”
   • [claimed-docs] “Test any MCP server from your browser. Supports every Model Context Protocol feature”
   • [claimed-docs] “Every MCP call from your agents flows through Glama – so you know exactly what your AI is doing, and you decide what it's allowed to do.”

   proof ↗⚑ flag

Access control

1. platform-engineerRestrict which servers and individual tools an agent may call through allowlists or per-tool access control

   weight 2 · round drawn
   Smithery~partialclaimed6/10

   Smithery's token-scoping feature lets platform engineers issue tokens restricted to specific connections/namespaces (e.g., 'list and call tools on connections in my-app where metadata.userId is user-123 — nothing else'), which functions as a coarse allowlist mechanism, and namespaces further scope API-key access to a workspace. However, the evidence shows scoping is at the connection/namespace level, not explicit per-tool allow/deny lists within a single server. Missing for 10: documented fine-grained per-tool allow/deny configuration, independent/hands-on verification that tool-level restrictions are enforced, and any admin UI or policy examples showing individual tool blocking within an allowed connection.

   • [claimed-docs] “Service tokens let you safely expose Smithery to browsers, mobile apps, and AI agents without leaking your API key.”
   • [claimed-docs] “This token can list and call tools on connections in `my-app` where `metadata.userId` is `user-123` — nothing else.”
   • [claimed-docs] “Namespaces act as a shared workspace. Resources within a namespace can be managed by anyone with the appropriate API key scoped to that name…”

   proof ↗⚑ flag

   Glama~partialclaimed6/10

   Docs explicitly mention gateway routing all MCP calls and per-tool access control (enable/disable individual tools), directly supporting the story's core requirement of per-tool restriction. However, evidence lacks detail on allowlisting specific servers per-agent, role-based policies, or independent/hands-on confirmation that these controls work as described. missing for 10: server-level allowlist granularity, agent-specific policy configuration examples, independent verification of gateway enforcement in practice.

   • [claimed-docs] “Every MCP call from your agents flows through Glama – so you know exactly what your AI is doing, and you decide what it's allowed to do.”
   • [claimed-docs] “Per-tool access control – enable or disable individual tools”

   proof ↗⚑ flag

Audit

1. platform-engineerSee a log of every tool call that passed through the platform, with enough detail to audit or replay what an agent did

   weight 2 · round to Glama
   Smithery—none0/10

   No evidence describes any audit logging, tool-call history, or replay capability for platform engineers—docs cover connection, credential management, publishing, tokens, and namespaces but never mention logs of tool calls passing through the gateway.

   ⚑ flag

   Glama~partialclaimed4/10

   Glama's gateway routes all MCP calls and gives visibility ('you know exactly what your AI is doing') plus per-tool access control, implying some logging capability, but there is no documented evidence of a persistent, queryable call log with per-call detail (params, responses, timestamps) or replay functionality. missing for 10: detailed audit log schema/UI, replay capability, retention/export controls, independent confirmation of logging fidelity.

   • [claimed-docs] “Every MCP call from your agents flows through Glama – so you know exactly what your AI is doing, and you decide what it's allowed to do.”
   • [claimed-docs] “Per-tool access control – enable or disable individual tools”

   proof ↗⚑ flag

Catalog

1. developerReach a deep catalog of third-party apps and APIs (hundreds to thousands of integrations) through the platform

   weight 3 · round drawn
   Smithery✓fullprobed8/10

   Smithery is a registry/marketplace connecting to a large catalog of third-party MCP servers (Exa, Context7, Browserbase, Google Sheets, Brave Search, etc.), with a live registry API confirmed via probe returning searchable server entries with useCount/verified metadata, plus hosted per-server MCP endpoints (server.smithery.ai) and CLI search/add tooling. Community evidence confirms real scale (100+ servers scanned) though also raises security concerns about tool description injection, which affects trust/quality but not the breadth-of-catalog claim itself. Missing for 10: independently verified total count of hundreds/thousands of listings and first-party data on catalog size/growth.

   • [claimed-docs] “Smithery gives you a simple REST interface for connecting to MCP servers. Instead of implementing the MCP protocol directly, handling OAuth …”
   • [claimed-docs] “Exa - Search the live web, access LinkedIn profiles, do deep research, and more”
   • [claimed-docs] “Context7 - Reference the latest docs for most major SDKs and frameworks directly in Cursor or Claude Code”
   • [claimed-docs] “Browserbase - Control a remote web browser using Stagehand”
   • [github] “smithery mcp search [term] # Search the Smithery registry smithery mcp add <url> # Add an MCP server connectio…”
   • [probe] “PROBE registry-api (2026-09-05): GET https://registry.smithery.ai/servers?pageSize=3 with NO API key returned HTTP 200 with a machine-readab…”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://server.smithery.ai/exa/mcp returned HTTP 401 with an OAuth challenge — the hoste…”
   • [community] “Scanned the top 100 servers on Smithery's registry; 22 of 100 flagged at least one security finding (28 total, 4 critical, 24 high), most co…”

   proof ↗⚑ flag

   Glama✓fullprobed8/10

   Glama's directory contains 81,811 MCP servers and 15,990 remote connectors per its own llms.txt, giving developers access to a massive catalog of third-party integrations, with search and inspection tooling and a registry API. missing for 10: independent verification of catalog usability/quality at scale beyond vendor-reported counts, and community feedback notes it's easy to 'get lost in the sea' of listings suggesting discoverability friction.

   • [probe] “PROBE llms.txt: HTTP 200 at https://glama.ai/llms.txt # Glama > A directory of 81,811 Model Context Protocol servers and 15,990 remote conn…”
   • [claimed-docs] “Search Glama's curated catalog of open-source servers and remote connectors, inspect their capabilities, and keep your own directory synchro…”
   • [community] “Its really easy to get lost in the sea of existing ones on the glama site or github in general - asking for recommendations on how to market…”
   • [probe] “PROBE registry-api (2026-09-05): GET https://glama.ai/api/mcp/v1/servers?first=3 without credentials returned HTTP 401 with a JSON auth chal…”

   proof ↗⚑ flag

Connect

1. developerPoint my MCP-compatible client at a hosted remote server URL and reach downstream tools without deploying any infrastructure

   weight 3 · round to Smithery
   Smithery✓fullprobed8/10

   Docs and live probes confirm Smithery hosts remote MCP server endpoints (server.smithery.ai/<server>/mcp) that clients can point at directly, with OAuth/token handling, deep links, and typed SDKs generated automatically — no server deployment needed by the developer. Independent probe evidence (smithery-probe-rt-2) shows the hosted endpoint is live and gated by managed OAuth rather than requiring self-hosting infrastructure. missing for 10: independent hands-on developer accounts (outside security research) confirming a smooth end-to-end connect experience beyond documentation and infra probes.

   • [claimed-docs] “Smithery gives you a simple REST interface for connecting to MCP servers. Instead of implementing the MCP protocol directly, handling OAuth …”
   • [claimed-docs] “Uplink exposes an MCP server running on any machine as a regular Smithery connection.”
   • [claimed-docs] “If you’ve already deployed an MCP server elsewhere, you can publish it directly on Smithery via the URL method.”
   • [claimed-docs] “Smithery handles OAuth, token refresh, and connection management automatically.”
   • [claimed-docs] “Every MCP server published on Smithery gets a typed TypeScript SDK generated from its tool and trigger schemas.”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://server.smithery.ai/exa/mcp returned HTTP 401 with an OAuth challenge — the hoste…”
   • [probe] “PROBE registry-api (2026-09-05): GET https://registry.smithery.ai/servers?pageSize=3 with NO API key returned HTTP 200 with a machine-readab…”

   proof ↗⚑ flag

   Glama✓fullcommunity7/10

   Glama hosts MCP servers (built from GitHub repos, Dockerfiles, npm/PyPI packages) and exposes them as remote connectors/servers reachable by MCP clients, with a gateway routing all calls and per-tool access control, removing the need for developers to self-host infrastructure. Community evidence confirms real usage of hosting/scoring, though it also reveals friction (undocumented billing gating rebuilds) that adds operational caveats. Missing for 10: independent hands-on confirmation of a client successfully connecting to a hosted remote URL end-to-end, and clearer public docs on connection URL format/reliability.

   • [claimed-docs] “Connect a GitHub repository and Glama builds the image straight from source.”
   • [claimed-docs] “Ship a Dockerfile, an npm package, or a PyPI module. Configure Node or Python versions, build steps, and CMD arguments – or let Glama infer …”
   • [claimed-docs] “Every MCP call from your agents flows through Glama – so you know exactly what your AI is doing, and you decide what it's allowed to do.”
   • [claimed-docs] “Per-tool access control – enable or disable individual tools”
   • [claimed-docs] “Private by default – flip to public listing when ready.”
   • [community] “I love the TDQS system. I've personally revised my MCPs to improve the quality of the score... Really love the idea behind this and Glama in…”
   • [community] “After uploading a new MCP version, the quality score test wouldn't rerun for 3 days despite manual resyncs and API attempts. Turned out a cr…”

   proof ↗⚑ flag

Generate

1. developerTurn an existing API definition (OpenAPI) or custom code into a hosted MCP server without writing protocol plumbing

   weight 2 · round drawn
   Smithery~partialclaimed4/10

   Smithery's docs show a hosting/publishing path for MCP servers already built in code (URL method, MCPB bundles, server-card metadata) which removes protocol plumbing for deployment, but there is no evidence of an OpenAPI-spec ingestion or auto-generation feature that turns an existing REST API definition into an MCP server — the publish flow assumes you already have MCP-shaped code or a deployed server. Missing for 10: OpenAPI-to-MCP conversion tooling, any first-party or independent proof of generating a server from an OpenAPI spec, and confirmation that 'custom code' can be arbitrary non-MCP code rather than code already written against the MCP SDK.

   • [claimed-docs] “If you’ve already deployed an MCP server elsewhere, you can publish it directly on Smithery via the URL method.”
   • [claimed-docs] “For local stdio servers — Smithery distributes a pre-built MCPB bundle that clients download and run locally.”
   • [claimed-docs] “If automatic scanning can’t complete (auth wall, required configuration, or other issues), you can provide server metadata manually via a st…”
   • [claimed-docs] “Every MCP server published on Smithery gets a typed TypeScript SDK generated from its tool and trigger schemas.”

   proof ↗⚑ flag

   Glama~partialclaimed4/10

   Glama's hosting lets you ship a Dockerfile, npm package, PyPI module, or connect a GitHub repo and it builds/runs the server for you (docs-2, docs-3, docs-5, docs-8), which removes protocol plumbing for custom code you already wrote. However, there is no evidence of an OpenAPI-spec-to-MCP-server generator or converter — the docs only describe hosting pre-built code, not turning an API definition into a server. Missing for 10: OpenAPI import/definition-to-server generation feature, documentation or example of spec-driven server creation, independent confirmation of this specific workflow.

   • [claimed-docs] “Connect a GitHub repository and Glama builds the image straight from source.”
   • [claimed-docs] “Ship a Dockerfile, an npm package, or a PyPI module. Configure Node or Python versions, build steps, and CMD arguments – or let Glama infer …”
   • [claimed-docs] “Per-tool access control – enable or disable individual tools”
   • [claimed-docs] “Private by default – flip to public listing when ready.”

   proof ↗⚑ flag

Publish

1. developerPublish my own MCP server on the platform for distribution, discovery, and installation by others

   weight 2 · round to Smithery
   Smithery✓fullprobed8/10

   Docs clearly describe multiple publishing paths (URL method for already-deployed servers, MCPB bundle distribution for local stdio servers, automatic scanning or manual server-card metadata) resulting in a registry listing with generated TypeScript SDK, and the registry API probe confirms servers are discoverable/searchable by others without auth. Security researchers did find vulnerabilities/prompt-injection risks in some published servers, but this reflects on registry quality, not on whether publishing/discovery/installation works. missing for 10: independent hands-on account of a developer successfully publishing a server end-to-end, and clearer detail on install flow verification post-publish.

   • [claimed-docs] “If you’ve already deployed an MCP server elsewhere, you can publish it directly on Smithery via the URL method.”
   • [claimed-docs] “For local stdio servers — Smithery distributes a pre-built MCPB bundle that clients download and run locally.”
   • [claimed-docs] “If automatic scanning can’t complete (auth wall, required configuration, or other issues), you can provide server metadata manually via a st…”
   • [claimed-docs] “Every MCP server published on Smithery gets a typed TypeScript SDK generated from its tool and trigger schemas.”
   • [probe] “PROBE registry-api (2026-09-05): GET https://registry.smithery.ai/servers?pageSize=3 with NO API key returned HTTP 200 with a machine-readab…”
   • [community] “Scanned the top 100 servers on Smithery's registry; 22 of 100 flagged at least one security finding (28 total, 4 critical, 24 high), most co…”

   proof ↗⚑ flag

   Glama✓fullcommunity7/10

   Glama's hosting docs describe end-to-end publishing: connect a GitHub repo or ship a Dockerfile/npm/PyPI package, configure build/run settings, set per-tool access control, and toggle from private to public listing for discovery in Glama's catalog of servers/connectors — matching publish, distribution, discovery, and installation. Community evidence corroborates real users publishing/updating MCP servers and having them scored/listed, though also reports friction (undocumented billing requirement blocking rebuild/rescoring). Missing for 10: independent hands-on confirmation of a smooth end-to-end publish flow without the billing/support friction reported in the community thread, and clearer documentation of the installation step for end users.

   • [claimed-docs] “Connect a GitHub repository and Glama builds the image straight from source.”
   • [claimed-docs] “Ship a Dockerfile, an npm package, or a PyPI module. Configure Node or Python versions, build steps, and CMD arguments – or let Glama infer …”
   • [claimed-docs] “Per-tool access control – enable or disable individual tools”
   • [claimed-docs] “Private by default – flip to public listing when ready.”
   • [claimed-docs] “Search Glama's curated catalog of open-source servers and remote connectors, inspect their capabilities, and keep your own directory synchro…”
   • [community] “I love the TDQS system. I've personally revised my MCPs to improve the quality of the score... Really love the idea behind this and Glama in…”
   • [community] “After uploading a new MCP version, the quality score test wouldn't rerun for 3 days despite manual resyncs and API attempts. Turned out a cr…”
   • [community] “Its really easy to get lost in the sea of existing ones on the glama site or github in general - asking for recommendations on how to market…”

   proof ↗⚑ flag

Triggers

1. developerSubscribe my agent to events from connected downstream apps (triggers/webhooks) instead of polling them

   weight 2 · round to Smithery
   Smithery✓fullclaimed7/10

   Smithery documents a Triggers feature that explicitly lets consumers subscribe to event streams surfaced from an upstream service, with Smithery proxying the subscribe call to the server instead of requiring polling. This directly matches the story of subscribing an agent to downstream app events via webhooks/triggers rather than polling. Missing for 10: independent/hands-on confirmation that triggers work reliably in production, and more detail on delivery guarantees or webhook payload handling.

   • [claimed-docs] “Triggers let an MCP server surface events from its upstream service. When a consumer activates a trigger via Smithery, Smithery proxies the …”

   proof ↗⚑ flag

   Glama~partialclaimed4/10

   Glama's pricing page mentions 'Automate workflows using schedules or webhooks' (glama-docs-10), suggesting some webhook-driven automation exists, but there is no documentation of subscribing an agent to events from downstream connected apps, no event/webhook API reference, and no example of triggers replacing polling. Missing for 10: documented webhook/event subscription API, examples of downstream app event triggers, confirmation this applies to agent-tool integrations rather than just internal automation.

   • [claimed-docs] “Automate workflows using schedules or webhooks.”

   proof ↗⚑ flag

Analytics

1. platform-engineerSee usage analytics for the servers and tools my org consumes or publishes — call volumes, users, errors, and cost

   weight 2 · round drawn
   Smithery—none0/10

   The evidence pack documents connection, publishing, token scoping, triggers, and SDK generation, but nothing about a dashboard or API for org-level usage analytics (call volumes, per-user breakdowns, error rates, cost). The only quantitative signal found is a per-server 'useCount' field in the public registry listing, which is not org-scoped consumption/error/cost analytics.

   • [probe] “PROBE registry-api (2026-09-05): GET https://registry.smithery.ai/servers?pageSize=3 with NO API key returned HTTP 200 with a machine-readab…”

   proof ↗⚑ flag

   Glama—none0/10

   Evidence shows Glama offers a gateway that routes MCP calls (glama-docs-4), hosting, per-tool access control, and a directory/quality scoring system, but there is no mention of usage analytics dashboards, call volume metrics, user-level breakdowns, error rates, or cost tracking for orgs. missing for 10: usage/call-volume dashboards, per-user analytics, error-rate reporting, cost/billing analytics tied to server or tool consumption.

   • [claimed-docs] “Every MCP call from your agents flows through Glama – so you know exactly what your AI is doing, and you decide what it's allowed to do.”
   • [claimed-docs] “Per-tool access control – enable or disable individual tools”
   • [claimed-docs] “Search Glama's curated catalog of open-source servers and remote connectors, inspect their capabilities, and keep your own directory synchro…”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round to Smithery
   Smithery~partialprobed7/10

   Smithery exposes a documented, live OpenAPI 3.1 spec (35 paths) and a keyless registry API, plus a CLI (`smithery mcp add`, `smithery tool call`) that mirrors core UI actions like discovering, connecting, and invoking MCP servers — showing broad API/CLI parity with the UI for the main workflows (search, connect, publish via URL, call tools). However, no evidence explicitly confirms that secondary UI-only features (team/namespace management, trigger configuration, token scoping setup) are fully API-reachable, and llms.txt/docs.md convenience endpoints 404. missing for 10: explicit confirmation that admin/team features (namespaces, triggers, token scoping) are configurable via the API/CLI rather than only the UI, and independent hands-on verification beyond the vendor's own OpenAPI spec.

   • [probe] “PROBE registry-api (2026-09-05): GET https://registry.smithery.ai/servers?pageSize=3 with NO API key returned HTTP 200 with a machine-readab…”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://server.smithery.ai/exa/mcp returned HTTP 401 with an OAuth challenge — the hoste…”
   • [probe] “PROBE openapi (2026-09-05): GET https://smithery.ai/docs/openapi.json returned HTTP 200 with an OpenAPI 3.1.0 description (35 documented pat…”
   • [github] “smithery mcp search [term] # Search the Smithery registry smithery mcp add <url> # Add an MCP server connectio…”
   • [github] “smithery tool find [query] # Search tools by name or intent smithery tool call <connection> <tool> [args] # Call a tool”
   • [claimed-docs] “If you’ve already deployed an MCP server elsewhere, you can publish it directly on Smithery via the URL method.”
   • [claimed-docs] “Namespaces act as a shared workspace. Resources within a namespace can be managed by anyone with the appropriate API key scoped to that name…”
   • [claimed-docs] “Triggers let an MCP server surface events from its upstream service. When a consumer activates a trigger via Smithery, Smithery proxies the …”

   proof ↗⚑ flag

   Glama~partialprobed5/10

   Glama exposes a real, documented API (OpenAPI spec at /api/mcp/openapi.json, RFC 9727 api-catalog, registry endpoint requiring API key) covering server search/registry data, but the evidence shows only 9 documented API paths versus a much richer UI feature set (hosting/build config, per-tool access control, gateway policy, scheduling/webhooks, inspector) with no confirmation these UI-only features are API-accessible. Community reports also show real friction (undocumented billing gate blocking rebuild triggers via API/manual resync) indicating parity gaps in practice. Missing for 10: evidence that hosting/build config, access control, gateway rules, and automation/webhooks are exposed via the documented API, and confirmation the community-reported API rebuild friction has been resolved.

   • [probe] “PROBE openapi (2026-09-05): GET https://glama.ai/api/mcp/openapi.json returned HTTP 200 with an OpenAPI 3.0.3 description (9 documented path…”
   • [probe] “PROBE api-catalog (2026-09-05): GET https://glama.ai/.well-known/api-catalog returned HTTP 200 with an RFC 9727 linkset naming the platform …”
   • [probe] “PROBE registry-api (2026-09-05): GET https://glama.ai/api/mcp/v1/servers?first=3 without credentials returned HTTP 401 with a JSON auth chal…”
   • [claimed-docs] “Ship a Dockerfile, an npm package, or a PyPI module. Configure Node or Python versions, build steps, and CMD arguments – or let Glama infer …”
   • [claimed-docs] “Per-tool access control – enable or disable individual tools”
   • [claimed-docs] “Automate workflows using schedules or webhooks.”
   • [community] “After uploading a new MCP version, the quality score test wouldn't rerun for 3 days despite manual resyncs and API attempts. Turned out a cr…”
   • [community] “Neither I nor my AI agent could figure out over 4 days of searching documentation why the quality score retest wasn't triggering on the late…”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round drawn
   Smithery—none0/10

   No evidence anywhere in the pack of a data-export feature, account data portability, or open-format export for users leaving the platform; documentation covers connections, tokens, and MCP server publishing but nothing about exporting stored user data (connections, credentials metadata, namespaces) in an open format.

   ⚑ flag

   Glama—none0/10

   No evidence of a data-export feature, open-format export tools, or account-portability mechanism for user-uploaded MCP configurations, server catalogs, or usage data; evidence covers hosting, API access, and catalog browsing but not exporting/leaving with one's own data.

   ⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round drawn
   Smithery—none0/10

   Evidence shows only a GitHub repo for the Smithery CLI tool (smithery-gh-1/2/3) but no license information, and the core Smithery platform (registry, hosting, OAuth handling) is described only via docs as a hosted SaaS with no mention of source availability or open licensing. No evidence establishes that Smithery's product source is published under an open license.

   • [github] “smithery mcp search [term] # Search the Smithery registry smithery mcp add <url> # Add an MCP server connectio…”
   • [github] “smithery tool find [query] # Search tools by name or intent smithery tool call <connection> <tool> [args] # Call a tool”
   • [github] “Browse skills on the [Smithery Skills Registry](https://smithery.ai/skills) and install them with the upstream installer”

   proof ↗⚑ flag

   Glama—none0/10

   Glama is a hosted directory/gateway platform for MCP servers; there is no evidence of its own source code being published under an open license — the evidence pack only covers API access, hosting features, and a curated catalog of third-party open-source MCP servers, not Glama's own source availability.

   ⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round drawn
   Smithery—none0/10

   Evidence describes Smithery only as a hosted registry/platform (registry.smithery.ai, server.smithery.ai) with OAuth-gated hosted MCP endpoints; 'Uplink' lets users expose their own local MCP server as a connection into the hosted Smithery service, but this is not self-hosting the Smithery platform itself. No docs, GitHub repo, or probes indicate a self-hostable/open-source deployment of the core product.

   • [claimed-docs] “Uplink exposes an MCP server running on any machine as a regular Smithery connection.”
   • [probe] “PROBE registry-api (2026-09-05): GET https://registry.smithery.ai/servers?pageSize=3 with NO API key returned HTTP 200 with a machine-readab…”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://server.smithery.ai/exa/mcp returned HTTP 401 with an OAuth challenge — the hoste…”

   proof ↗⚑ flag

   Glama—none0/10

   Glama is presented as a hosted SaaS directory/gateway for MCP servers with no evidence of a self-hostable core product, open-source release, or on-prem deployment option; all evidence points to a cloud-hosted service with API access, not self-hosting. Missing for 10: any mention of open-source code, self-host installer/docker-compose for the core platform, or on-prem deployment instructions.

   • [claimed-docs] “Connect a GitHub repository and Glama builds the image straight from source.”
   • [claimed-docs] “Ship a Dockerfile, an npm package, or a PyPI module. Configure Node or Python versions, build steps, and CMD arguments – or let Glama infer …”
   • [probe] “PROBE llms.txt: HTTP 200 at https://glama.ai/llms.txt # Glama > A directory of 81,811 Model Context Protocol servers and 15,990 remote conn…”
   • [probe] “PROBE registry-api (2026-09-05): GET https://glama.ai/api/mcp/v1/servers?first=3 without credentials returned HTTP 401 with a JSON auth chal…”

   proof ↗⚑ flag

Pricing

1. developerPrototype my agent's tool connections on a meaningful free tier before paying anything

   weight 1 · round to Glama
   Smithery—none0/10

   The evidence pack contains extensive documentation on Smithery's features, registry, and integrations but no mention of pricing tiers, a free tier, usage limits, or billing at all — the axis is applicable to a hosted MCP platform but simply unevidenced.

   ⚑ flag

   Glama!disputedcontradicted4/10

   Glama's docs promote free-sounding capabilities like testing MCP servers and browsing the catalog (glama-docs-1, glama-docs-6), but no evidence pack item actually describes a free pricing tier or its limits, and a hands-on report shows a user's MCP rebuild/rescan silently failed for days until a credit card was added — an undocumented paywall blocking core functionality (glama-comm-2, glama-comm-3). This directly contradicts the idea of a functional free prototyping tier. Missing for 10: explicit free-tier plan details/limits, confirmation that tool-connection prototyping works fully without payment, and resolution of the undocumented credit-card requirement.

   • [claimed-docs] “Test any MCP server from your browser. Supports every Model Context Protocol feature”
   • [claimed-docs] “Search Glama's curated catalog of open-source servers and remote connectors, inspect their capabilities, and keep your own directory synchro…”
   • [community] “After uploading a new MCP version, the quality score test wouldn't rerun for 3 days despite manual resyncs and API attempts. Turned out a cr…”
   • [community] “Neither I nor my AI agent could figure out over 4 days of searching documentation why the quality score retest wasn't triggering on the late…”

   proof ↗⚑ flag

2. developerSee transparent pricing per tool call or connection plus documented rate limits before committing to the platform

   weight 2 · round drawn
   Smithery—none0/10

   No evidence pack item mentions pricing tiers, per-call costs, or documented rate limits anywhere in Smithery's docs, GitHub, or probes; the documentation covers connection, tokens, and hosting but is silent on pricing/rate limits, and repeated probes (llms.txt, docs.md, openapi) surface no pricing/rate-limit schema.

   ⚑ flag

   Glama—none0/10

   The evidence only shows a generic /pricing page reference (workflow automation) with no per-tool-call or per-connection pricing figures, and no documented rate limits anywhere in the docs or probes. Community reports (glama-comm-2, glama-comm-3) actually describe an undocumented billing requirement (a credit card needed for rebuilds) that surprised users, further indicating the platform lacks the transparent, documented pricing/rate-limit information the story asks for.

   • [claimed-docs] “Automate workflows using schedules or webhooks.”
   • [community] “After uploading a new MCP version, the quality score test wouldn't rerun for 3 days despite manual resyncs and API attempts. Turned out a cr…”
   • [community] “Neither I nor my AI agent could figure out over 4 days of searching documentation why the quality score retest wasn't triggering on the late…”

   proof ↗⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round drawn
   Smithery—none0/10

   No evidence anywhere in the pack mentions data residency, region selection, or geographic storage controls; Smithery's docs cover credential storage, token scoping, and connection management but not where data is physically stored.

   ⚑ flag

   Glama—none0/10

   No evidence anywhere in the pack mentions data residency, region selection, or storage location controls for hosted MCP servers or user data; the axis applies to a hosting platform but is unaddressed. Missing for 10: any mention of region/data-residency options, storage location controls, or compliance certifications (e.g., GDPR/data locality) that would let a user choose where data is stored.

   ⚑ flag

2. ai-native userPrevent my data from being used to train AI models

   weight 3 · round drawn
   Smithery—none0/10

   No evidence in the pack addresses AI training data usage, opt-out controls, or any data-privacy policy regarding model training; docs focus on credential storage, OAuth, and connection routing, not training-data opt-out.

   ⚑ flag

   Glama—none0/10

   No evidence in the pack addresses data usage for AI training, opt-out settings, or any privacy policy regarding model training; the material only covers MCP hosting, gateway, and directory features. This is an applicable axis for an AI-native platform handling user data/tools, but nothing shows a mechanism to prevent training use, so it is 'none'. Missing for 10: any training-data opt-out policy, privacy documentation, or terms-of-service statement on data usage.

   ⚑ flag

3. ai-native userControl data retention and deletion

   weight 2 · round drawn
   Smithery—none0/10

   The evidence pack covers credential storage encryption and token scoping but contains no mention of data retention policies, user-initiated data deletion, or account/data export controls. This is a fair privacy-posture question for a platform managing credentials and connections, so the absence of evidence yields 'none' rather than 'na'.

   ⚑ flag

   Glama—none0/10

   No evidence describes data retention policies, deletion controls, or export/erasure mechanisms for user data or MCP call logs; the evidence covers hosting, gateway routing, and API access but nothing about retention/deletion settings. missing for 10: retention period disclosure, deletion/export controls, data lifecycle policy documentation.

   ⚑ flag

4. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   Smithery—none0/10

   No evidence anywhere in the pack mentions a telemetry/usage-tracking opt-out setting, privacy policy toggle, or data-collection disclosure for Smithery; the docs cover credential security and OAuth but never address analytics/telemetry opt-out.

   ⚑ flag

   Glama—none0/10

   No evidence in the pack addresses telemetry/usage-tracking settings or an opt-out mechanism for Glama; the docs cover MCP hosting, gateway, and inspector features only. missing for 10: any mention of telemetry collection, privacy policy, or a user-facing opt-out/disable-tracking setting.

   ⚑ flag

Api

1. ai agentQuery the platform's registry or catalog through a documented API to discover servers and tools programmatically

   weight 3 · round to Smithery
   Smithery✓fullprobed9/10

   A live probe confirms an agent can query https://registry.smithery.ai/servers keylessly and get a machine-readable JSON catalog (displayName, qualifiedName, verified, etc.), and a documented OpenAPI 3.1 spec with 35 paths is served at smithery.ai/docs/openapi.json, enabling programmatic client generation. This is corroborated by the smithery-cli's 'mcp search'/'tool find' commands for registry/tool discovery. Missing for 10: no independent third-party report of an agent successfully building a full client purely from the OpenAPI spec, and llms.txt/docs.md convenience endpoints 404.

   • [probe] “PROBE registry-api (2026-09-05): GET https://registry.smithery.ai/servers?pageSize=3 with NO API key returned HTTP 200 with a machine-readab…”
   • [probe] “PROBE openapi (2026-09-05): GET https://smithery.ai/docs/openapi.json returned HTTP 200 with an OpenAPI 3.1.0 description (35 documented pat…”
   • [github] “smithery mcp search [term] # Search the Smithery registry smithery mcp add <url> # Add an MCP server connectio…”
   • [github] “smithery tool find [query] # Search tools by name or intent smithery tool call <connection> <tool> [args] # Call a tool”

   proof ↗⚑ flag

   Glama✓fullprobed8/10

   Glama exposes a live registry API (glama.ai/api/mcp/v1/servers) documented via an OpenAPI 3.0.3 spec with 9 paths and an RFC 9727 well-known API catalog for machine discovery, confirming programmatic, documented access to its server/tool catalog; docs also describe the curated catalog of servers and connectors. Missing for 10: independent third-party developer confirmation of successful client generation/integration and clearer public documentation of the API key auth flow beyond the probe's 401 challenge.

   • [probe] “PROBE registry-api (2026-09-05): GET https://glama.ai/api/mcp/v1/servers?first=3 without credentials returned HTTP 401 with a JSON auth chal…”
   • [probe] “PROBE openapi (2026-09-05): GET https://glama.ai/api/mcp/openapi.json returned HTTP 200 with an OpenAPI 3.0.3 description (9 documented path…”
   • [probe] “PROBE api-catalog (2026-09-05): GET https://glama.ai/.well-known/api-catalog returned HTTP 200 with an RFC 9727 linkset naming the platform …”
   • [claimed-docs] “Search Glama's curated catalog of open-source servers and remote connectors, inspect their capabilities, and keep your own directory synchro…”
   • [probe] “PROBE llms.txt: HTTP 200 at https://glama.ai/llms.txt # Glama > A directory of 81,811 Model Context Protocol servers and 15,990 remote conn…”

   proof ↗⚑ flag

Metadata

1. developerInspect a listed server's tools, input schemas, transport, and hosting attributes before connecting to it

   weight 2 · round drawn
   Smithery~partialprobed6/10

   The registry API (probe-rt-1) exposes machine-readable metadata per server — displayName, qualifiedName, isDeployed, remote, verified — covering hosting/transport attributes, and community security audits (comm-1, comm-2) confirm that tool description fields are visible and inspectable pre-connection (that's how injected instructions were found). The CLI's `tool find` command (gh-2) and per-server typed SDKs generated from tool schemas (docs-16) further support schema visibility, but there is no single documented endpoint or UI view that surfaces full input schemas plus transport plus hosting attributes together in one inspection step. Missing for 10: a documented unified 'server detail' API/UI showing input schemas explicitly, and independent confirmation that transport type (stdio/http/websocket) is surfaced pre-connect.

   • [probe] “PROBE registry-api (2026-09-05): GET https://registry.smithery.ai/servers?pageSize=3 with NO API key returned HTTP 200 with a machine-readab…”
   • [github] “smithery tool find [query] # Search tools by name or intent smithery tool call <connection> <tool> [args] # Call a tool”
   • [claimed-docs] “Every MCP server published on Smithery gets a typed TypeScript SDK generated from its tool and trigger schemas.”
   • [community] “Scanned the top 100 servers on Smithery's registry; 22 of 100 flagged at least one security finding (28 total, 4 critical, 24 high), most co…”
   • [community] “Real examples pulled from Smithery-hosted servers: Context7, Google Sheets, Senzia, and Brave Search tool descriptions contained agent-direc…”

   proof ↗⚑ flag

   Glama~partialprobed6/10

   Glama's inspector lets developers test/inspect any listed MCP server's tools and capabilities from the browser, and the directory/reference docs mention inspecting capabilities and searching the catalog with hosting attributes (Dockerfile/npm/PyPI, private/public status) documented per-server. However, there's no concrete evidence of a structured, per-server schema view showing input schemas or transport type explicitly, and the registry API requires an API key, limiting easy inspection before connecting. Missing for 10: explicit UI/API display of input schemas and transport protocol per listing, and independent hands-on confirmation of pre-connection inspection depth.

   • [claimed-docs] “Test any MCP server from your browser. Supports every Model Context Protocol feature”
   • [claimed-docs] “Search Glama's curated catalog of open-source servers and remote connectors, inspect their capabilities, and keep your own directory synchro…”
   • [claimed-docs] “Ship a Dockerfile, an npm package, or a PyPI module. Configure Node or Python versions, build steps, and CMD arguments – or let Glama infer …”
   • [claimed-docs] “Private by default – flip to public listing when ready.”
   • [probe] “PROBE registry-api (2026-09-05): GET https://glama.ai/api/mcp/v1/servers?first=3 without credentials returned HTTP 401 with a JSON auth chal…”

   proof ↗⚑ flag

Playground

1. developerExercise an MCP server's tools from a browser-based inspector or playground before integrating it

   weight 2 · round to Glama
   Smithery—none0/10

   The evidence pack documents Smithery's registry, REST connect flow, OAuth/token handling, uplink, deep links, and CLI tool-calling, but nothing describes a browser-based inspector or playground UI for interactively testing a server's tools before integration. Since a registry/marketplace product could plausibly ship such a try-before-integrate UI, the axis applies, but no evidence confirms it exists.

   ⚑ flag

   Glama✓fullclaimed8/10

   Glama documents a browser-based inspector that can test any MCP server (local via tunnel or remote) covering every MCP feature, directly matching the story's ask to try tools before integration. Missing for 10: independent hands-on confirmation of the inspector's UX/tool-invocation flow beyond vendor docs.

   • [claimed-docs] “Test any MCP server from your browser. Supports every Model Context Protocol feature”
   • [claimed-docs] “Test your local MCP servers with the inspector by exposing them via a public tunnel using mcp-proxy.”

   proof ↗⚑ flag

Search

1. developerSearch and browse a large catalog of MCP servers and tools by capability before wiring anything into my agent

   weight 3 · round drawn
   Smithery✓fullprobed8/10

   Smithery ships both a CLI (`smithery mcp search`, `smithery tool find [query]`) and a keyless registry REST API (registry.smithery.ai/servers) that returns a machine-readable catalog with names, descriptions, verification status and usage counts, letting developers search/browse servers and tools by capability before connecting; docs also list example servers (Exa, Context7, Browserbase) illustrating capability-based discovery. Missing for 10: no evidence of a rich web UI faceted-browse/filter-by-category experience or independent hands-on review of the discovery UX itself beyond the CLI/API probes.

   • [github] “smithery mcp search [term] # Search the Smithery registry smithery mcp add <url> # Add an MCP server connectio…”
   • [github] “smithery tool find [query] # Search tools by name or intent smithery tool call <connection> <tool> [args] # Call a tool”
   • [probe] “PROBE registry-api (2026-09-05): GET https://registry.smithery.ai/servers?pageSize=3 with NO API key returned HTTP 200 with a machine-readab…”
   • [claimed-docs] “Exa - Search the live web, access LinkedIn profiles, do deep research, and more”
   • [claimed-docs] “Context7 - Reference the latest docs for most major SDKs and frameworks directly in Cursor or Claude Code”
   • [claimed-docs] “Browserbase - Control a remote web browser using Stagehand”

   proof ↗⚑ flag

   Glama✓fullprobed8/10

   Glama's core product is a directory/catalog of 81,811 MCP servers and 15,990 remote connectors, with search, inspection of capabilities, and syncing to a directory (glama-docs-6, glama-probe-1); the inspector lets developers test capabilities before wiring in (glama-docs-1). Community feedback confirms the catalog is large and used, though some find it hard to navigate. Missing for 10: no evidence of structured capability-based filtering/faceted search UI details, and independent hands-on browsing experience beyond a marketing/docs description.

   • [claimed-docs] “Search Glama's curated catalog of open-source servers and remote connectors, inspect their capabilities, and keep your own directory synchro…”
   • [claimed-docs] “Test any MCP server from your browser. Supports every Model Context Protocol feature”
   • [probe] “PROBE llms.txt: HTTP 200 at https://glama.ai/llms.txt # Glama > A directory of 81,811 Model Context Protocol servers and 15,990 remote conn…”
   • [community] “Its really easy to get lost in the sea of existing ones on the glama site or github in general - asking for recommendations on how to market…”

   proof ↗⚑ flag

Cli

1. developerDrive the publish-and-deploy workflow from an official CLI — authenticate, stage or build a server, push it, and install it into agents

   weight 2 · round to Smithery
   Smithery~partialprobed6/10

   Smithery ships an official CLI (@smithery/cli, confirmed installable via npm and version-checked) with commands to search/add/call MCP servers, and docs cover publishing servers (URL method, MCPB bundles, server-card metadata) plus auth via OAuth/service tokens and installing into agents via deep links/uplink. However, evidence does not clearly show a unified CLI-driven build-and-push workflow (e.g., 'smithery build' or 'smithery deploy' subcommands) — the CLI evidence only shows search/add/call, while publish/build steps are documented as web/manual flows rather than CLI-driven. missing for 10: CLI subcommands for authenticate/build/push in one flow, independent hands-on confirmation of end-to-end publish-via-CLI, and clarity on whether install-into-agents is CLI-native vs deep-link/manual.

   • [github] “smithery mcp search [term] # Search the Smithery registry smithery mcp add <url> # Add an MCP server connectio…”
   • [github] “smithery tool find [query] # Search tools by name or intent smithery tool call <connection> <tool> [args] # Call a tool”
   • [github] “Browse skills on the [Smithery Skills Registry](https://smithery.ai/skills) and install them with the upstream installer”
   • [claimed-docs] “If you’ve already deployed an MCP server elsewhere, you can publish it directly on Smithery via the URL method.”
   • [claimed-docs] “For local stdio servers — Smithery distributes a pre-built MCPB bundle that clients download and run locally.”
   • [claimed-docs] “If automatic scanning can’t complete (auth wall, required configuration, or other issues), you can provide server metadata manually via a st…”
   • [claimed-docs] “Deep links provide a seamless way to integrate Smithery MCPs into supported clients. When a user clicks a deep link from our server page, th…”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/smithery/): a JSON-RPC initialize POST to https://smithery.ai/docs/mc…”

   proof ↗⚑ flag

   Glama—none0/10

   Glama documents web-based hosting (connect GitHub repo, Dockerfile/npm/PyPI builds) and a REST/OpenAPI web API, but there is no evidence of an official CLI tool for authenticating, staging/building, pushing, or installing servers into agents — all workflows described are browser/dashboard driven. Missing for 10: any CLI binary or package, CLI-based auth flow, CLI build/push commands, CLI install-into-agent step.

   • [claimed-docs] “Connect a GitHub repository and Glama builds the image straight from source.”
   • [claimed-docs] “Ship a Dockerfile, an npm package, or a PyPI module. Configure Node or Python versions, build steps, and CMD arguments – or let Glama infer …”
   • [probe] “PROBE registry-api (2026-09-05): GET https://glama.ai/api/mcp/v1/servers?first=3 without credentials returned HTTP 401 with a JSON auth chal…”
   • [probe] “PROBE openapi (2026-09-05): GET https://glama.ai/api/mcp/openapi.json returned HTTP 200 with an OpenAPI 3.0.3 description (9 documented path…”

   proof ↗⚑ flag

Frameworks

1. developerPlug the platform's tools into major agent frameworks and SDKs (Vercel AI SDK, LangChain, CrewAI, OpenAI or Anthropic SDKs) with documented adapters

   weight 2 · round to Smithery
   Smithery~partialclaimed5/10

   Smithery documents a dedicated Vercel AI SDK integration with automatic OAuth/token handling and multi-server tool aggregation (docs-13/14), and every published server gets an auto-generated typed SDK (docs-16), but the evidence pack shows no documented adapters for LangChain, CrewAI, or OpenAI/Anthropic SDKs specifically. Missing for 10: documented LangChain adapter, documented CrewAI adapter, documented OpenAI/Anthropic SDK adapter, and independent/hands-on confirmation beyond first-party docs.

   • [claimed-docs] “Smithery handles OAuth, token refresh, and connection management automatically.”
   • [claimed-docs] “Connect to multiple servers and aggregate their tools”
   • [claimed-docs] “Every MCP server published on Smithery gets a typed TypeScript SDK generated from its tool and trigger schemas.”
   • [claimed-docs] “Smithery gives you a simple REST interface for connecting to MCP servers. Instead of implementing the MCP protocol directly, handling OAuth …”

   proof ↗⚑ flag

   Glama—none0/10

   Glama is an MCP server directory/hosting/gateway platform; evidence covers MCP server hosting, inspection, and a generic 'MCP clients' page, but there is no documented adapter or integration guide for Vercel AI SDK, LangChain, CrewAI, or OpenAI/Anthropic SDKs. No SDK-specific code samples or adapter docs appear anywhere in the pack.

   • [claimed-docs] “MCP clients are the applications that connect to MCP servers.”
   • [claimed-docs] “Every MCP call from your agents flows through Glama – so you know exactly what your AI is doing, and you decide what it's allowed to do.”
   • [probe] “PROBE docs-md: HTTP 404 at https://glama.ai/mcp.md”
   • [probe] “PROBE openapi: all candidate paths 404 (https://glama.ai/openapi.json, https://glama.ai/swagger.json, https://glama.ai/api/openapi.json, htt…”

   proof ↗⚑ flag

Security

1. platform-engineerRely on the platform scanning or inspecting listed servers for malicious or vulnerable behavior before my agents use them

   weight 2 · round to Glama
   Smithery!disputedcontradicted4/10

   Smithery's docs mention 'automatic scanning' of published servers (smithery-docs-11), implying some vetting pipeline, but independent research directly contradicts its effectiveness: scanning the top 100 registry servers found 22 with security findings (4 critical) including tool-description prompt-injection attacks embedded in widely-used servers like Context7 and Brave Search that execute silently on agent connection (smithery-comm-1, smithery-comm-2), and researchers separately found a path-traversal vulnerability in Smithery's own hosting escalatable to a supply-chain attack (smithery-comm-3). Missing for 10: any documented remediation/removal process for flagged malicious servers, transparency on scan methodology or coverage, and evidence the automatic scanning actually catches behavioral/prompt-injection risks rather than just metadata issues.

   • [claimed-docs] “If automatic scanning can’t complete (auth wall, required configuration, or other issues), you can provide server metadata manually via a st…”
   • [community] “Scanned the top 100 servers on Smithery's registry; 22 of 100 flagged at least one security finding (28 total, 4 critical, 24 high), most co…”
   • [community] “Real examples pulled from Smithery-hosted servers: Context7, Google Sheets, Senzia, and Brave Search tool descriptions contained agent-direc…”
   • [community] “Security researchers detailed a path traversal vulnerability in Smithery's MCP server hosting that could be escalated into a supply chain at…”

   proof ↗⚑ flag

   Glama~partialprobed5/10

   Glama's llms.txt claims every listed server is 'scanned, scored' and it offers a TDQS quality-score system plus an inspector tool to test servers manually, which supports platform-engineers wanting some assurance before use. However, community evidence shows the quality-score rescan mechanism is opaque, gated behind an undocumented billing requirement, and can silently fail to update for days, undermining confidence in reliability of the scanning pipeline; no evidence describes what 'scanning' actually checks for (malware, vulnerabilities, supply-chain risk) beyond a marketing claim. Missing for 10: technical documentation of what the scan/score actually detects (security vs. quality), independent verification that malicious servers are caught or blocked, and confirmation the rescan pipeline works reliably without hidden gating.

   • [probe] “PROBE llms.txt: HTTP 200 at https://glama.ai/llms.txt # Glama > A directory of 81,811 Model Context Protocol servers and 15,990 remote conn…”
   • [claimed-docs] “Test any MCP server from your browser. Supports every Model Context Protocol feature”
   • [claimed-docs] “Search Glama's curated catalog of open-source servers and remote connectors, inspect their capabilities, and keep your own directory synchro…”
   • [community] “I love the TDQS system. I've personally revised my MCPs to improve the quality of the score... Really love the idea behind this and Glama in…”
   • [community] “After uploading a new MCP version, the quality score test wouldn't rerun for 3 days despite manual resyncs and API attempts. Turned out a cr…”
   • [community] “Neither I nor my AI agent could figure out over 4 days of searching documentation why the quality score retest wasn't triggering on the late…”

   proof ↗⚑ flag

Signals

1. developerCompare servers using published quality signals — usage counts, health or liveness checks, scores, or verification badges

   weight 2 · round to Smithery
   Smithery~partialprobed6/10

   The registry API (probe evidence) exposes useCount and verified fields per server, plus isDeployed/remote status, giving developers concrete usage and verification signals to compare servers; docs also mention automatic scanning during publish which implies some quality vetting. However, there is no documented explicit 'quality score' metric or dedicated health/liveness-check endpoint beyond isDeployed, and independent security scans found unflagged risky servers in the top 100, showing the published signals don't capture all quality concerns. Missing for 10: an explicit scoring/quality-score field, a dedicated liveness/health-check mechanism, and corroboration that verification badges reliably reflect security vetting.

   • [probe] “PROBE registry-api (2026-09-05): GET https://registry.smithery.ai/servers?pageSize=3 with NO API key returned HTTP 200 with a machine-readab…”
   • [claimed-docs] “If automatic scanning can’t complete (auth wall, required configuration, or other issues), you can provide server metadata manually via a st…”
   • [community] “Scanned the top 100 servers on Smithery's registry; 22 of 100 flagged at least one security finding (28 total, 4 critical, 24 high), most co…”

   proof ↗⚑ flag

   Glama!disputedcontradicted5/10

   Glama's directory scans and scores servers (TDQS) and the probe confirms a large catalog with scoring metadata (glama-probe-1), and a community member praises the quality-score system as a signal for comparison (glama-comm-1). However, hands-on reports show the score can silently fail to refresh for days due to an undocumented credit-card requirement, undermining trust that displayed scores are current (glama-comm-2, glama-comm-3). Missing for 10: documented usage-count metrics, uptime/liveness check indicators, verification badges, and independent confirmation that scores reliably update.

   • [probe] “PROBE llms.txt: HTTP 200 at https://glama.ai/llms.txt # Glama > A directory of 81,811 Model Context Protocol servers and 15,990 remote conn…”
   • [community] “I love the TDQS system. I've personally revised my MCPs to improve the quality of the score... Really love the idea behind this and Glama in…”
   • [community] “After uploading a new MCP version, the quality score test wouldn't rerun for 3 days despite manual resyncs and API attempts. Turned out a cr…”
   • [community] “Neither I nor my AI agent could figure out over 4 days of searching documentation why the quality score retest wasn't triggering on the late…”

   proof ↗⚑ flag

Tool search

1. ai agentSearch and load tools at runtime through meta-tools instead of having thousands of tool definitions stuffed into my context

   weight 2 · round to Smithery
   Smithery✓fullprobed7/10

   Smithery's CLI exposes explicit meta-tools — `mcp search`, `tool find [query]`, and `tool call <connection> <tool>` — letting an agent discover and invoke tools by intent at runtime rather than preloading every tool definition, and the registry API is confirmed live and keylessly searchable in a probe returning a machine-readable catalog. missing for 10: no evidence the search/find/call meta-tools are themselves exposed as callable MCP tools inside a live agent session (vs. CLI-level usage), and no independent benchmark showing reduced context footprint from this pattern.

   • [github] “smithery mcp search [term] # Search the Smithery registry smithery mcp add <url> # Add an MCP server connectio…”
   • [github] “smithery tool find [query] # Search tools by name or intent smithery tool call <connection> <tool> [args] # Call a tool”
   • [probe] “PROBE registry-api (2026-09-05): GET https://registry.smithery.ai/servers?pageSize=3 with NO API key returned HTTP 200 with a machine-readab…”
   • [claimed-docs] “Every MCP server published on Smithery gets a typed TypeScript SDK generated from its tool and trigger schemas.”

   proof ↗⚑ flag

   Glama—none0/10

   Glama is a directory/hosting/gateway for MCP servers with search and per-tool access control, but there is no evidence of runtime meta-tool search/load capability (e.g., a 'search_tools'/'load_tool' meta-tool that lets an agent dynamically discover and inject tool definitions instead of preloading them into context). The catalog search (glama-docs-6) is a human/dev-facing directory feature, not an agent-callable meta-tool for dynamic tool loading.

   • [claimed-docs] “Search Glama's curated catalog of open-source servers and remote connectors, inspect their capabilities, and keep your own directory synchro…”
   • [claimed-docs] “Every MCP call from your agents flows through Glama – so you know exactly what your AI is doing, and you decide what it's allowed to do.”
   • [claimed-docs] “Per-tool access control – enable or disable individual tools”

   proof ↗⚑ flag