Pipedream MCP vs Gram (Speakeasy)

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round drawn
   Pipedream MCP✓fullprobed8/10

   A live probe confirms Pipedream serves markdown-formatted docs (mcp.md) that explicitly reference and point to a full llms.txt documentation index at pipedream.com/docs/llms.txt, directly matching the agent-oriented docs story. missing for 10: no independent confirmation that an agent actually consumed llms.txt successfully end-to-end (only that it is served/discoverable).

   • [probe] “PROBE docs-md: HTTP 200 at https://pipedream.com/docs/connect/mcp.md > ## Documentation Index > Fetch the complete documentation index at: h…”
   • [claimed-docs] “Add Pipedream MCP to your app or agent to make tool calls on behalf of your users to 3,000+ APIs and 10,000+ tools.”

   proof ↗⚑ flag

   Gram (Speakeasy)✓fullprobed8/10

   Speakeasy directly hosts a working llms.txt (HTTP 200, confirmed by probe) plus a docs.md markdown-native page, meaning an agent can be pointed at agent-oriented docs formats; the docs also emphasize AI-native/agent workflows throughout (SKILL.md, platform MCP server, CLI). missing for 10: no evidence of llms.txt existing for Gram product docs specifically (only the root Speakeasy domain), and no independent/hands-on confirmation an agent actually consumed and acted on the llms.txt content.

   • [probe] “PROBE llms.txt: HTTP 200 at https://www.speakeasy.com/llms.txt # Speakeasy > Speakeasy is the AI control plane for the AI-native enterprise…”
   • [probe] “PROBE docs-md: HTTP 200 at https://www.speakeasy.com/docs/ai-control-plane.md # AI Control Plane Documentation {/* * Renders through the …”
   • [claimed-docs] “A skill is a versioned `SKILL.md` manifest: reusable instructions that agents load on demand.”
   • [claimed-docs] “An org admin working in Claude Code, Cursor, Codex, or another MCP client can find and add MCP servers, finish their setup, put them on a pl…”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round drawn
   Pipedream MCP~partialprobed6/10

   Pipedream MCP exposes a remote HTTP/SSE server and an SDK (@pipedream/sdk) that can be scripted programmatically, which supports headless/CI use, and the API is designed for developers to embed in their own apps/agents rather than requiring a GUI. However, the server requires a per-end-user binding (external user id) and OAuth-based account connection flows, which complicate fully unattended CI automation, and there is no explicit CI/headless-mode documentation or example. missing for 10: explicit CI/headless automation guide, evidence of non-interactive auth/token provisioning for unattended runs, and confirmation of stable long-running headless operation.

   • [claimed-docs] “tools: await mcpClient.tools(), // Automatically loads all available tools”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://remote.mcp.pipedream.net answered a JSON-RPC error over HTTP 400 ({"jsonrpc":"2.…”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/pipedream-mcp/): `npm install @pipedream/sdk` completed into a throwa…”
   • [claimed-docs] “To use Pipedream MCP in your own app or agent to make tool calls on behalf of your users, you can get started for free in development mode”

   proof ↗⚑ flag

   Gram (Speakeasy)~partialprobed6/10

   Gram ships an official CLI (verified runnable via runtime probe) that can build/publish MCP servers and toolsets, which supports CI/headless automation, and docs confirm a working command-line reference. However, there is no explicit CI/CD pipeline documentation, no GitHub Actions example, and no headless auth/service-account guidance for unattended runs. Missing for 10: documented CI examples (e.g., GitHub Actions/pipeline configs), non-interactive/service-account authentication flow for the CLI, and independent hands-on confirmation of full automation beyond version-check.

   • [claimed-docs] “install toolsets as MCP servers in AI agents, and keep the CLI itself up to date”
   • [probe] “official CLI documented at https://www.speakeasy.com/docs/ai-control-plane/reference/command-line”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/gram/): downloaded the official cli@0.16.0 release artifact (gram_dar…”

   proof ↗⚑ flag

3. ai-native userPlug MCP servers into this product so it can use their tools

   weight 3 · round to Gram (Speakeasy)
   Pipedream MCP—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

   Gram (Speakeasy)✓fullclaimed7/10

   Gram documents fronting/governing third-party MCP servers ('put a governed endpoint in front of an MCP server someone else runs') and an in-dashboard harness to connect, authenticate, and chat with a model that calls tools from a connected server, plus a real-world case (MoonPay, 200+ MCP servers aggregated under the gateway). This directly satisfies plugging external MCP servers in so their tools become usable. Missing for 10: independent/hands-on verification of third-party server integration in production and more detail on tool-call fidelity across diverse upstream servers.

   • [claimed-docs] “Put a governed endpoint in front of an MCP server someone else runs, so access control, audit, and upstream credentials apply without changi…”
   • [claimed-docs] “An in-dashboard agent harness for testing MCP servers before connecting a real client: pick a server, authenticate, chat with a model that c…”
   • [claimed-docs] “MoonPay brought 200+ MCP servers under the MCP Gateway, going from proof of concept to a company-wide rollout in 30 days.”
   • [claimed-docs] “unsanctioned shadow MCP servers are blocked by default”

   proof ↗⚑ flag

4. ai-native userConnect an agent via an official MCP server

   weight 3 · round drawn
   Pipedream MCP✓fullprobed9/10

   Pipedream ships an official documented MCP server (remote.mcp.pipedream.net / mcp.pipedream.com) explicitly designed for agents/apps to connect and make tool calls across 3,000+ APIs, with SSE/streamable HTTP support, built-in auth, and an SDK — and a live probe confirms the endpoint is real and responds to JSON-RPC per the documented protocol. Missing for 10: an end-to-end probe showing a fully successful tool call (the live test only reached the 'external user id required' auth step, not actual tool invocation).

   • [claimed-docs] “Add Pipedream MCP to your app or agent to make tool calls on behalf of your users to 3,000+ APIs and 10,000+ tools.”
   • [claimed-docs] “This is handled automatically by Pipedream’s MCP server and there’s no additional implementation required”
   • [claimed-docs] “The Pipedream MCP server supports both SSE and streamable HTTP transport types dynamically, with no configuration required by the developer …”
   • [claimed-docs] “As an end user: Connect your accounts through Pipedream MCP at mcp.pipedream.com”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://remote.mcp.pipedream.net answered a JSON-RPC error over HTTP 400 ({"jsonrpc":"2.…”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/pipedream-mcp/): `npm install @pipedream/sdk` completed into a throwa…”

   proof ↗⚑ flag

   Gram (Speakeasy)✓fullprobed9/10

   Gram documents and exposes an official 'Platform MCP' server that lets MCP clients (Claude Code, Cursor, Codex, etc.) connect directly, plus generates/hosts MCP servers as its core product function, and a CLI/runtime probe confirms the tooling actually installs and runs. missing for 10: independent (non-vendor) hands-on confirmation that an external agent successfully connects to and operates the Platform MCP server end-to-end.

   • [claimed-docs] “An org admin working in Claude Code, Cursor, Codex, or another MCP client can find and add MCP servers, finish their setup, put them on a pl…”
   • [probe] “official MCP server documented at https://www.speakeasy.com/docs/ai-control-plane/reference/platform-mcp”
   • [claimed-docs] “Built — the Control Plane generates the server itself from a first-party source: an OpenAPI document or a TypeScript Functions project.”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/gram/): downloaded the official cli@0.16.0 release artifact (gram_dar…”
   • [claimed-docs] “An in-dashboard agent harness for testing MCP servers before connecting a real client: pick a server, authenticate, chat with a model that c…”

   proof ↗⚑ flag

5. ai-native userUse an official CLI

   weight 2 · round to Gram (Speakeasy)
   Pipedream MCP—none0/10

   Evidence covers Pipedream's MCP server, SDK (@pipedream/sdk), and chat app, but no mention of an official CLI tool for AI-native workflows. Missing for 10: any documentation or reference to a Pipedream CLI product.

   ⚑ flag

   Gram (Speakeasy)✓fullprobed8/10

   Gram ships an official CLI documented at reference/command-line, used to install toolsets as MCP servers and keep itself updated, and a runtime probe confirms the binary actually downloads and runs (`gram --version`). missing for 10: independent/third-party review of CLI usage beyond the vendor docs and a single runtime probe, and more detail on the full breadth of CLI commands beyond install/update.

   • [claimed-docs] “install toolsets as MCP servers in AI agents, and keep the CLI itself up to date”
   • [probe] “official CLI documented at https://www.speakeasy.com/docs/ai-control-plane/reference/command-line”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/gram/): downloaded the official cli@0.16.0 release artifact (gram_dar…”

   proof ↗⚑ flag

6. ai-native userDrive the product through a documented public API

   weight 3 · round to Pipedream MCP
   Pipedream MCP✓fullprobed8/10

   Pipedream publishes detailed developer docs, an SDK (@pipedream/sdk), and a live MCP JSON-RPC endpoint (remote.mcp.pipedream.net) confirmed reachable and behaving per spec, giving AI-native developers a documented public API/protocol to drive tool calls programmatically. Missing for 10: independent third-party developer testimonials or case studies beyond vendor docs/probes.

   • [claimed-docs] “Add Pipedream MCP to your app or agent to make tool calls on behalf of your users to 3,000+ APIs and 10,000+ tools.”
   • [claimed-docs] “tools: await mcpClient.tools(), // Automatically loads all available tools”
   • [claimed-docs] “To use Pipedream MCP in your own app or agent to make tool calls on behalf of your users, you can get started for free in development mode”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://remote.mcp.pipedream.net answered a JSON-RPC error over HTTP 400 ({"jsonrpc":"2.…”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/pipedream-mcp/): `npm install @pipedream/sdk` completed into a throwa…”

   proof ↗⚑ flag

   Gram (Speakeasy)~partialprobed7/10

   Gram exposes a documented Platform MCP server and CLI that let an AI-native user/agent drive the product programmatically (find/add MCP servers, author skills, manage policies, read logs) without the dashboard, and the CLI was independently verified to install and run. However, a direct probe for a conventional public REST/OpenAPI API (openapi.json, swagger.json, etc.) returned 404s, so there is no evidence of a general-purpose documented HTTP API beyond the MCP/CLI surfaces. Missing for 10: a documented REST/OpenAPI public API endpoint, independent confirmation of the Platform MCP server's functional breadth beyond docs.

   • [claimed-docs] “An org admin working in Claude Code, Cursor, Codex, or another MCP client can find and add MCP servers, finish their setup, put them on a pl…”
   • [probe] “official MCP server documented at https://www.speakeasy.com/docs/ai-control-plane/reference/platform-mcp”
   • [claimed-docs] “install toolsets as MCP servers in AI agents, and keep the CLI itself up to date”
   • [probe] “official CLI documented at https://www.speakeasy.com/docs/ai-control-plane/reference/command-line”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/gram/): downloaded the official cli@0.16.0 release artifact (gram_dar…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://www.speakeasy.com/openapi.json, https://www.speakeasy.com/swagger.json, https://www.speakeas…”

   proof ↗⚑ flag

7. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round to Pipedream MCP
   Pipedream MCP~partialprobed7/10

   Pipedream MCP issues per-end-user, revocable credentials via built-in OAuth-style account connections, and the live server enforces an external-user-id binding before serving tools, showing real per-user credential scoping rather than a single shared key. However, there is no explicit documentation of fine-grained scope selection (e.g., choosing specific read/write permissions per tool or API) that would constitute true least-privilege configuration. missing for 10: explicit least-privilege scope controls (choosing minimal permission sets per credential), independent/hands-on confirmation of revocation actually restricting access.

   • [claimed-docs] “Pipedream Connect includes built-in user authentication for more than APIs via MCP, which means you don’t need to build any authorization fl…”
   • [claimed-docs] “As an end user: Connect your accounts through Pipedream MCP at mcp.pipedream.com”
   • [claimed-docs] “Revocable access: Users can revoke access to their connected accounts at any time”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://remote.mcp.pipedream.net answered a JSON-RPC error over HTTP 400 ({"jsonrpc":"2.…”

   proof ↗⚑ flag

   Gram (Speakeasy)~partialclaimed6/10

   Gram supports OAuth 2.0/2.1 integration with external providers, RBAC enforced at team/server/tool level synced to enterprise IDPs, and governs upstream credentials for third-party MCP servers with shadow-server blocking — collectively enabling scoped access control for agent tool use. However, there's no explicit documentation of per-agent least-privilege token minting/vending or fine-grained scope selection at credential-issuance time; the emphasis is on gateway-level RBAC and OAuth pass-through rather than dynamically scoped credential issuance. Missing for 10: explicit per-agent credential scoping/token minting mechanism, documented least-privilege scope configuration UI, and independent verification of enforcement.

   • [claimed-docs] “you can use any OAuth 2.0 provider you have set up (such as Auth0, Okta, Keycloak, or your own OAuth server)”
   • [claimed-docs] “Put a governed endpoint in front of an MCP server someone else runs, so access control, audit, and upstream credentials apply without changi…”
   • [github] “Team, server, and tool level permissions enforced through RBAC and Oauth2.1. Synced to your enterprise IDP (Okta, Azure AD, Google Workspace…”
   • [claimed-docs] “unsanctioned shadow MCP servers are blocked by default”

   proof ↗⚑ flag

8. ai-native userBuild against official SDKs

   weight 2 · round to Pipedream MCP
   Pipedream MCP✓fullprobed8/10

   Pipedream documents and ships an official SDK (@pipedream/sdk) with a PipedreamClient class, integration guides for AI frameworks (mcpClient.tools()), and a runtime probe confirms the package installs and exports the client as documented. Missing for 10: coverage of SDKs in multiple languages beyond JS/TS and independent third-party corroboration of SDK usage in production apps.

   • [claimed-docs] “tools: await mcpClient.tools(), // Automatically loads all available tools”
   • [claimed-docs] “To use Pipedream MCP in your own app or agent to make tool calls on behalf of your users, you can get started for free in development mode”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/pipedream-mcp/): `npm install @pipedream/sdk` completed into a throwa…”
   • [claimed-docs] “Add Pipedream MCP to your app or agent to make tool calls on behalf of your users to 3,000+ APIs and 10,000+ tools.”

   proof ↗⚑ flag

   Gram (Speakeasy)—none0/10

   The evidence pack documents Gram's CLI, MCP server distribution, OAuth, skills, and observability features, but nowhere mentions official client SDKs (e.g., Python/TypeScript/Go libraries) for building against Gram's platform itself — this is a fair question for a platform product but is simply unaddressed.

   ⚑ flag

9. ai-native userSubscribe to events via webhooks

   weight 2 · round drawn
   Pipedream MCP—none0/10

   The evidence pack describes Pipedream MCP purely as a tool-calling interface (auth, transport, tool listing) with no mention of webhook subscriptions or event-driven notifications delivered through MCP itself.

   ⚑ flag

   Gram (Speakeasy)—none0/10

   Gram's evidence covers MCP server generation, tool logs, CLI, and RBAC, but nothing describes webhook subscriptions or event-driven push notifications for AI agents; missing for 10: webhook subscription mechanism, event schema/documentation, or any mention of push-based event delivery.

   ⚑ flag

Agentic features

1. ai-native userSet up automations that run autonomously in the background

   weight 2 · round drawn
   Pipedream MCP—none0/10

   The evidence describes Pipedream MCP as a tool-calling gateway that lets agents invoke APIs on behalf of users in real time, with no mention of scheduled, triggered, or persistently-running background automations. Missing for 10: any documentation of triggers, cron schedules, event-driven workflows, or autonomous execution outside an active agent session.

   • [claimed-docs] “Add Pipedream MCP to your app or agent to make tool calls on behalf of your users to 3,000+ APIs and 10,000+ tools.”
   • [claimed-docs] “This is handled automatically by Pipedream’s MCP server and there’s no additional implementation required”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://remote.mcp.pipedream.net answered a JSON-RPC error over HTTP 400 ({"jsonrpc":"2.…”

   proof ↗⚑ flag

   Gram (Speakeasy)—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

2. ai-native userOperate the product with natural-language commands

   weight 2 · round to Gram (Speakeasy)
   Pipedream MCP~partialprobed6/10

   Pipedream MCP exposes tools via natural-language-capable chat interface (chat.pipedream.com) and MCP clients can invoke 3,000+ APIs through natural-language tool calls, with automatic tool loading and no manual auth flows needed. However, evidence is mostly first-party docs describing the underlying architecture rather than direct demonstration of natural-language command handling, and the live endpoint probe shows a per-user binding requirement that adds friction beyond pure NL commands. Missing for 10: independent/hands-on evidence of natural-language interaction quality, examples of actual NL prompts being translated to tool calls, and confirmation the chat app works smoothly without technical setup for end users.

   • [claimed-docs] “Add Pipedream MCP to your app or agent to make tool calls on behalf of your users to 3,000+ APIs and 10,000+ tools.”
   • [claimed-docs] “tools: await mcpClient.tools(), // Automatically loads all available tools”
   • [claimed-docs] “Try out Pipedream MCP in our chat app at chat.pipedream.com”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://remote.mcp.pipedream.net answered a JSON-RPC error over HTTP 400 ({"jsonrpc":"2.…”

   proof ↗⚑ flag

   Gram (Speakeasy)✓fullprobed7/10

   Gram documents a platform MCP server that lets an org admin operate the product (find/add servers, manage risk policies, author skills, read observability) entirely from within an MCP client like Claude Code or Cursor using natural language, without opening the dashboard — this is direct agentic/NL control of the product itself. Missing for 10: independent/hands-on verification that these natural-language admin operations actually execute correctly (only the CLI binary itself was runtime-probed, not the platform-mcp NL workflow).

   • [claimed-docs] “An org admin working in Claude Code, Cursor, Codex, or another MCP client can find and add MCP servers, finish their setup, put them on a pl…”
   • [probe] “official MCP server documented at https://www.speakeasy.com/docs/ai-control-plane/reference/platform-mcp”
   • [claimed-docs] “An in-dashboard agent harness for testing MCP servers before connecting a real client: pick a server, authenticate, chat with a model that c…”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round to Pipedream MCP
   Pipedream MCP~partialclaimed3/10

   Pipedream provides docs with code snippets (docs-4, docs-5) and a live chat app (chat.pipedream.com) where users can try MCP tool calls interactively (docs-9), but there is no evidence of a dedicated interactive API reference/playground with runnable examples for the 3,000+ APIs. missing for 10: an actual interactive API explorer or sandboxed runnable code examples beyond a chat demo, independent confirmation of such a reference existing.

   • [claimed-docs] “Try out Pipedream MCP in our chat app at chat.pipedream.com”
   • [claimed-docs] “tools: await mcpClient.tools(), // Automatically loads all available tools”
   • [claimed-docs] “The Pipedream MCP server supports both SSE and streamable HTTP transport types dynamically, with no configuration required by the developer …”

   proof ↗⚑ flag

   Gram (Speakeasy)—none0/10

   Gram is positioned as an AI control plane/MCP gateway focused on server generation, governance, and observability; there is no evidence of an interactive API reference UI with runnable examples for exploring endpoints, nor documentation of such a feature.

   ⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   Pipedream MCP—none0/10

   The evidence describes Pipedream's MCP server, SDK, and docs (including an llms.txt index), but nowhere mentions a downloadable OpenAPI or equivalent machine-readable API spec for its endpoints/tools.

   ⚑ flag

   Gram (Speakeasy)—none0/10

   The evidence shows Gram consumes OpenAPI documents as input to build MCP servers, but there is no evidence Gram itself publishes a downloadable machine-readable spec of its own API — the direct probe for openapi.json/swagger.json/well-known paths all returned 404.

   • [probe] “PROBE openapi: all candidate paths 404 (https://www.speakeasy.com/openapi.json, https://www.speakeasy.com/swagger.json, https://www.speakeas…”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round to Gram (Speakeasy)
   Pipedream MCP~partialclaimed3/10

   Docs mention getting started in a free 'development mode' for testing tool calls (docs-10), implying some separation from production usage, but there is no explicit documentation of a sandbox environment, test data isolation, or guarantees that dev-mode calls don't touch real production accounts/data. Missing for 10: explicit sandbox/test-vs-prod environment docs, confirmation that test accounts are isolated from real user data, and independent verification of this isolation.

   • [claimed-docs] “To use Pipedream MCP in your own app or agent to make tool calls on behalf of your users, you can get started for free in development mode”

   proof ↗⚑ flag

   Gram (Speakeasy)~partialclaimed4/10

   Gram provides an in-dashboard 'Connect' harness to test MCP servers with a model before wiring up a real client, which offers some sandbox-like isolation from production agent traffic, but there is no explicit mention of separate sandbox vs. production data/environments, test fixtures, or guarantees that tool calls in this harness avoid hitting live production data sources. missing for 10: explicit sandbox/staging environment concept, isolation guarantees from production data, mock or test data support during agent testing.

   • [claimed-docs] “An in-dashboard agent harness for testing MCP servers before connecting a real client: pick a server, authenticate, chat with a model that c…”

   proof ↗⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   Pipedream MCP—none0/10

   The evidence pack covers MCP tool usage, authentication, and transport, but contains no mention of API versioning scheme, version numbers, or a documented deprecation policy for the MCP server or SDK. missing for 10: any documented API version scheme, changelog, deprecation notice process, or backward-compatibility guarantees.

   ⚑ flag

   Gram (Speakeasy)—none0/10

   The evidence pack covers OpenAPI-generated MCP servers, OAuth, governance, CLI, observability, and skills, but contains no documentation of API versioning practices or a deprecation policy for Gram's own APIs/servers.

   ⚑ flag

Multi tenant

1. platform-engineerMaintain separate connected accounts and scoped tokens per end user of my product, so each user's agent acts only on that user's credentials

   weight 3 · round to Pipedream MCP
   Pipedream MCP✓fullprobed8/10

   Docs describe per-end-user connected accounts with built-in auth/token storage/refresh handled by Pipedream, external_user_id scoping so agents act only on that user's credentials, and user-facing revocation of connected accounts; the live probe confirms the MCP server enforces an external user id binding before serving tools, corroborating the scoping claim. missing for 10: independent multi-tenant audit or case study showing token isolation at scale, and detail on scoped token permission granularity beyond per-account connection.

   • [claimed-docs] “Pipedream Connect includes built-in user authentication for more than APIs via MCP, which means you don’t need to build any authorization fl…”
   • [claimed-docs] “This is handled automatically by Pipedream’s MCP server and there’s no additional implementation required”
   • [claimed-docs] “As an end user: Connect your accounts through Pipedream MCP at mcp.pipedream.com”
   • [claimed-docs] “Revocable access: Users can revoke access to their connected accounts at any time”
   • [claimed-docs] “To use Pipedream MCP in your own app or agent to make tool calls on behalf of your users, you can get started for free in development mode”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://remote.mcp.pipedream.net answered a JSON-RPC error over HTTP 400 ({"jsonrpc":"2.…”

   proof ↗⚑ flag

   Gram (Speakeasy)—none0/10

   Gram's evidence describes OAuth 2.0 provider integration and team/server/tool-level RBAC synced to an enterprise IDP (gram-docs-2, gram-gh-2), but nothing documents per-end-user connected accounts or scoped tokens so that each of a platform-engineer's own product's end users gets isolated credentials — the RBAC described is org/team-level, not end-user-level multi-tenant credential isolation. Missing for 10: explicit documentation of per-end-user connected-account/token scoping, any multi-tenant credential isolation guide, and evidence of enforcement that one end user's agent cannot use another's credentials.

   • [claimed-docs] “you can use any OAuth 2.0 provider you have set up (such as Auth0, Okta, Keycloak, or your own OAuth server)”
   • [github] “Team, server, and tool level permissions enforced through RBAC and Oauth2.1. Synced to your enterprise IDP (Okta, Azure AD, Google Workspace…”
   • [claimed-docs] “Put a governed endpoint in front of an MCP server someone else runs, so access control, audit, and upstream credentials apply without changi…”

   proof ↗⚑ flag

Oauth

1. developerLet the platform run OAuth flows for downstream tools and vault the resulting credentials encrypted, with automatic token refresh

   weight 3 · round to Pipedream MCP
   Pipedream MCP✓fullprobed8/10

   Docs explicitly state Pipedream Connect/MCP handles OAuth/user authentication, token storage, and refresh automatically with no additional implementation required, and this is corroborated by a live probe showing the remote MCP endpoint enforcing per-user auth binding. Revocable access confirms credential lifecycle management beyond just initial OAuth. Missing for 10: explicit documentation of encryption-at-rest/vaulting mechanics for stored credentials and independent third-party verification of refresh behavior over time.

   • [claimed-docs] “Pipedream Connect includes built-in user authentication for more than APIs via MCP, which means you don’t need to build any authorization fl…”
   • [claimed-docs] “This is handled automatically by Pipedream’s MCP server and there’s no additional implementation required”
   • [claimed-docs] “Revocable access: Users can revoke access to their connected accounts at any time”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://remote.mcp.pipedream.net answered a JSON-RPC error over HTTP 400 ({"jsonrpc":"2.…”

   proof ↗⚑ flag

   Gram (Speakeasy)~partialclaimed5/10

   Docs confirm Gram lets you configure external OAuth 2.0 providers (Auth0, Okta, Keycloak, custom) for downstream tool auth and enforces OAuth2.1/RBAC for platform access, implying it manages OAuth flows for connected servers, but there is no explicit documentation of encrypted credential vaulting or automatic token refresh mechanics. Missing for 10: explicit description of credential encryption at rest, automatic refresh-token handling, and any hands-on/independent verification of these security claims.

   • [claimed-docs] “you can use any OAuth 2.0 provider you have set up (such as Auth0, Okta, Keycloak, or your own OAuth server)”
   • [github] “Team, server, and tool level permissions enforced through RBAC and Oauth2.1. Synced to your enterprise IDP (Okta, Azure AD, Google Workspace…”
   • [claimed-docs] “Put a governed endpoint in front of an MCP server someone else runs, so access control, audit, and upstream credentials apply without changi…”

   proof ↗⚑ flag

White label

1. platform-engineerWhite-label the authorization flow with my own branding and my own OAuth app credentials

   weight 1 · round to Gram (Speakeasy)
   Pipedream MCP—none0/10

   The evidence describes Pipedream's built-in authentication handling and revocable user connections, but nothing in the pack mentions white-labeling the OAuth flow with custom branding or letting platform engineers substitute their own OAuth app credentials. This is a fair axis for a connect/auth platform, so absence of evidence yields 'none' rather than 'na'.

   • [claimed-docs] “Pipedream Connect includes built-in user authentication for more than APIs via MCP, which means you don’t need to build any authorization fl…”
   • [claimed-docs] “As an end user: Connect your accounts through Pipedream MCP at mcp.pipedream.com”
   • [claimed-docs] “Revocable access: Users can revoke access to their connected accounts at any time”

   proof ↗⚑ flag

   Gram (Speakeasy)~partialclaimed6/10

   Gram supports bringing your own OAuth 2.0 provider (Auth0, Okta, Keycloak, or a custom server) so the underlying credentials are fully controlled by the platform-engineer, and RBAC/OAuth2.1 syncs to enterprise IDPs. However, there is no documented evidence of UI-level 'white-labeling' — custom branding/logo/domain on the consent or login screen itself — only backend provider configuration. Missing for 10: evidence of custom branding/theming of the OAuth consent screen or hosted login page, and confirmation that the authorization UI itself (not just the provider) can be rebranded.

   • [claimed-docs] “you can use any OAuth 2.0 provider you have set up (such as Auth0, Okta, Keycloak, or your own OAuth server)”
   • [github] “Team, server, and tool level permissions enforced through RBAC and Oauth2.1. Synced to your enterprise IDP (Okta, Azure AD, Google Workspace…”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round drawn
   Pipedream MCP—none0/10

   The evidence describes Pipedream MCP exposing thousands of individual API tools and handling auth/transport automatically, but nothing in the pack documents a bulk/batch operation capability (e.g., applying one action across many items or records in a single call). Missing for 10: any documentation of batch/bulk tool calls, looping over item sets, or multi-item transaction support.

   • [claimed-docs] “Add Pipedream MCP to your app or agent to make tool calls on behalf of your users to 3,000+ APIs and 10,000+ tools.”
   • [claimed-docs] “tools: await mcpClient.tools(), // Automatically loads all available tools”
   • [claimed-docs] “Slack: Send messages, manage channels, create reminders, and more”

   proof ↗⚑ flag

   Gram (Speakeasy)—none0/10

   The evidence pack covers MCP server hosting, governance, observability, RBAC, and CLI/skills features, but nowhere describes any bulk or batch operation capability (e.g., bulk approving tools, bulk managing servers, or batch tool invocation across many items). This is a plausible axis for a control-plane product managing many servers/toolsets, but no evidence shows it delivers.

   ⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round drawn
   Pipedream MCP—none0/10

   The evidence pack describes Pipedream MCP purely as a tool-calling/connector layer (auth, transport, tool listing) for agents to invoke APIs on demand — it contains no mention of defining rules, event triggers, or automatic actions firing on events. Missing for 10: any documentation of trigger/rule definitions, event-based workflow automation, or scheduled/conditional execution within the MCP server itself.

   • [claimed-docs] “Add Pipedream MCP to your app or agent to make tool calls on behalf of your users to 3,000+ APIs and 10,000+ tools.”
   • [claimed-docs] “tools: await mcpClient.tools(), // Automatically loads all available tools”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://remote.mcp.pipedream.net answered a JSON-RPC error over HTTP 400 ({"jsonrpc":"2.…”

   proof ↗⚑ flag

   Gram (Speakeasy)—none0/10

   Gram's evidence covers governance, observability, tool logging, RBAC, and MCP server management, but there is no mention of user-definable rule/trigger systems that automatically fire actions in response to events (e.g., event-driven automation, if-this-then-that rules, webhooks-to-action). Risk policies and access controls are governance constructs, not event-triggered automation rules.

   ⚑ flag

3. ai-native userSchedule recurring jobs or workflows

   weight 2 · round drawn
   Pipedream MCP—none0/10

   The evidence pack describes Pipedream MCP purely as a tool-calling gateway (auth, tool discovery, transport) for invoking 3,000+ APIs on demand, with no mention of scheduling, cron triggers, or recurring workflow execution via MCP.

   ⚑ flag

   Gram (Speakeasy)—none0/10

   Gram is an MCP control plane focused on generating, governing, observing, and distributing MCP servers/toolsets/skills, but nothing in the evidence describes scheduling recurring jobs or workflows (e.g., cron-like triggers or automated recurring execution) — it only covers request-time tool calls, logging, and access control.

   ⚑ flag

4. ai-native userVersion, review, and roll back my automations

   weight 1 · round to Gram (Speakeasy)
   Pipedream MCP—none0/10

   Evidence covers tool access, auth, and transport for Pipedream MCP, but nothing addresses versioning of automations, review workflows, or rollback capability. No mention of change history, diffing, approval flows, or reverting to prior configurations.

   ⚑ flag

   Gram (Speakeasy)~partialclaimed4/10

   Gram documents that skills are versioned (`SKILL.md` manifest) and provides audit/observability logs of tool calls, giving some traceability, but there is no evidence of a review workflow or an explicit rollback mechanism for automations/toolsets. missing for 10: explicit review process, rollback/revert functionality, version history UI for toolsets or MCP server configs.

   • [claimed-docs] “A skill is a versioned `SKILL.md` manifest: reusable instructions that agents load on demand.”
   • [claimed-docs] “the raw execution log for every tool call the platform observes across hosted MCP servers, tunneled and shadow MCP servers, skills, and loca…”

   proof ↗⚑ flag

Bridge

1. developerBridge a locally running MCP server into the platform as a managed remote connection without deploying it

   weight 1 · round to Gram (Speakeasy)
   Pipedream MCP—none0/10

   All evidence describes Pipedream operating its own hosted remote MCP server (exposing 3,000+ APIs/tools) that clients connect to — not a mechanism for a developer to take a locally-running custom MCP server and have Pipedream bridge/host it remotely without deployment. No evidence pack item mentions importing, registering, or proxying an external/local MCP server through Pipedream's platform.

   ⚑ flag

   Gram (Speakeasy)~partialclaimed5/10

   Docs mention 'tunneled MCP servers' alongside hosted and shadow servers in the observability/tool-logs reference, implying a mechanism to bridge a locally running server into the platform without redeploying it, and the third-party governance guide covers fronting externally-run servers. However, no dedicated guide, CLI command, or hands-on walkthrough for setting up a local-to-remote tunnel is included in the evidence pack. Missing for 10: a dedicated tunnel setup guide/CLI reference, concrete steps for exposing a local dev MCP server, and independent confirmation the tunnel actually works.

   • [claimed-docs] “the raw execution log for every tool call the platform observes across hosted MCP servers, tunneled and shadow MCP servers, skills, and loca…”
   • [claimed-docs] “Put a governed endpoint in front of an MCP server someone else runs, so access control, audit, and upstream credentials apply without changi…”
   • [claimed-docs] “install toolsets as MCP servers in AI agents, and keep the CLI itself up to date”

   proof ↗⚑ flag

Access control

1. platform-engineerRestrict which servers and individual tools an agent may call through allowlists or per-tool access control

   weight 2 · round to Gram (Speakeasy)
   Pipedream MCP—none0/10

   The evidence describes authentication, revocable account access, and automatic tool loading, but nowhere documents allowlisting of specific MCP servers or granular per-tool access control for platform engineers. The docs even state tools are 'automatically loaded' rather than filtered, and no admin/allowlist configuration is described. missing for 10: allowlist configuration UI/API, per-tool enable/disable controls, evidence of restricting which servers an agent may reach.

   • [claimed-docs] “tools: await mcpClient.tools(), // Automatically loads all available tools”
   • [claimed-docs] “Revocable access: Users can revoke access to their connected accounts at any time”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://remote.mcp.pipedream.net answered a JSON-RPC error over HTTP 400 ({"jsonrpc":"2.…”

   proof ↗⚑ flag

   Gram (Speakeasy)✓fullclaimed8/10

   Gram documents team/server/tool-level RBAC enforced with OAuth2.1 synced to enterprise IDPs, and explicitly blocks unsanctioned shadow MCP servers by default, directly matching allowlist/per-tool access control for a platform-engineer persona. This is corroborated by third-party rollout evidence (MoonPay) and governance guides for wrapping third-party MCP servers with access control. Missing for 10: independent hands-on verification of the RBAC/permission enforcement mechanics beyond vendor docs and GitHub README claims.

   • [github] “Team, server, and tool level permissions enforced through RBAC and Oauth2.1. Synced to your enterprise IDP (Okta, Azure AD, Google Workspace…”
   • [claimed-docs] “unsanctioned shadow MCP servers are blocked by default”
   • [claimed-docs] “MoonPay brought 200+ MCP servers under the MCP Gateway, going from proof of concept to a company-wide rollout in 30 days.”
   • [claimed-docs] “Put a governed endpoint in front of an MCP server someone else runs, so access control, audit, and upstream credentials apply without changi…”

   proof ↗⚑ flag

Audit

1. platform-engineerSee a log of every tool call that passed through the platform, with enough detail to audit or replay what an agent did

   weight 2 · round to Gram (Speakeasy)
   Pipedream MCP—none0/10

   No evidence describes any logging, audit trail, or replay capability for tool calls made through Pipedream MCP; documentation covers auth, transport, and connectivity but is silent on observability/audit logs. Missing for 10: any mention of a tool-call log, audit trail UI/API, replay mechanism, or retention/detail guarantees for platform engineers.

   ⚑ flag

   Gram (Speakeasy)✓fullclaimed8/10

   Gram documents a raw execution log capturing every tool call across hosted, tunneled, shadow MCP servers, skills, and local tools ([gram-docs-5]), plus usage/cost tracking and session deep-dives ([gram-gh-1]) that support audit and replay-style analysis. RBAC/OAuth-scoped access adds context for auditability of who called what ([gram-gh-2]). Missing for 10: independent/hands-on verification of log completeness or replay functionality specifically, and detail on log retention/export format.

   • [claimed-docs] “the raw execution log for every tool call the platform observes across hosted MCP servers, tunneled and shadow MCP servers, skills, and loca…”
   • [github] “Track AI usage across teams and measure impact with either tokens or cost. Deep dive expensive sessions, create budgets and measure tool eff…”
   • [github] “Team, server, and tool level permissions enforced through RBAC and Oauth2.1. Synced to your enterprise IDP (Okta, Azure AD, Google Workspace…”
   • [claimed-docs] “unsanctioned shadow MCP servers are blocked by default”

   proof ↗⚑ flag

Catalog

1. developerReach a deep catalog of third-party apps and APIs (hundreds to thousands of integrations) through the platform

   weight 3 · round to Pipedream MCP
   Pipedream MCP✓fullprobed9/10

   Docs and live probe confirm Pipedream MCP exposes 3,000+ APIs and 10,000+ tools with built-in auth, and the remote MCP endpoint is live and functioning (requiring per-user binding as documented). Missing for 10: independent third-party verification of the full catalog size beyond vendor docs.

   • [claimed-docs] “Add Pipedream MCP to your app or agent to make tool calls on behalf of your users to 3,000+ APIs and 10,000+ tools.”
   • [claimed-docs] “Pipedream Connect includes built-in user authentication for more than APIs via MCP, which means you don’t need to build any authorization fl…”
   • [claimed-docs] “tools: await mcpClient.tools(), // Automatically loads all available tools”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://remote.mcp.pipedream.net answered a JSON-RPC error over HTTP 400 ({"jsonrpc":"2.…”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/pipedream-mcp/): `npm install @pipedream/sdk` completed into a throwa…”

   proof ↗⚑ flag

   Gram (Speakeasy)—none0/10

   Gram's documented model is generating MCP servers from your own OpenAPI/TypeScript source or governing a third-party MCP server you already run — there is no evidence of a pre-built catalog of hundreds/thousands of ready-made third-party app/API integrations comparable to a connector marketplace. The MoonPay case study mentions bringing 200+ existing internally-run MCP servers under governance, not a Gram-provided catalog to browse and connect.

   • [claimed-docs] “Built — the Control Plane generates the server itself from a first-party source: an OpenAPI document or a TypeScript Functions project.”
   • [claimed-docs] “Put a governed endpoint in front of an MCP server someone else runs, so access control, audit, and upstream credentials apply without changi…”
   • [claimed-docs] “MoonPay brought 200+ MCP servers under the MCP Gateway, going from proof of concept to a company-wide rollout in 30 days.”

   proof ↗⚑ flag

Connect

1. developerPoint my MCP-compatible client at a hosted remote server URL and reach downstream tools without deploying any infrastructure

   weight 3 · round to Pipedream MCP
   Pipedream MCP✓fullprobed9/10

   Pipedream documents a hosted remote MCP endpoint (mcp.pipedream.com / remote.mcp.pipedream.net) that clients can point at without deploying infrastructure, with SSE/streamable HTTP transport auto-negotiated and tool loading handled server-side. A live probe confirms the endpoint is reachable and responds per the JSON-RPC protocol (requiring only a per-end-user binding parameter, not extra infra). Missing for 10: independent third-party (non-vendor) confirmation of a full end-to-end tool call succeeding beyond the initialize handshake.

   • [claimed-docs] “Add Pipedream MCP to your app or agent to make tool calls on behalf of your users to 3,000+ APIs and 10,000+ tools.”
   • [claimed-docs] “The Pipedream MCP server supports both SSE and streamable HTTP transport types dynamically, with no configuration required by the developer …”
   • [claimed-docs] “As an end user: Connect your accounts through Pipedream MCP at mcp.pipedream.com”
   • [claimed-docs] “Try out Pipedream MCP in our chat app at chat.pipedream.com”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://remote.mcp.pipedream.net answered a JSON-RPC error over HTTP 400 ({"jsonrpc":"2.…”
   • [probe] “official MCP server documented at https://pipedream.com/docs/connect/mcp”

   proof ↗⚑ flag

   Gram (Speakeasy)✓fullprobed8/10

   Gram's Control Plane auto-generates hosted MCP servers from an OpenAPI doc or TypeScript project, so developers get a remote server URL to point MCP clients at without deploying infra themselves, and this is corroborated by the CLI runtime probe and a real customer rollout (MoonPay's 200+ servers under the MCP Gateway). missing for 10: independent hands-on confirmation of an end-to-end client connecting to a hosted URL (only the CLI tool itself was runtime-verified, not the hosted server reachability).

   • [claimed-docs] “Built — the Control Plane generates the server itself from a first-party source: an OpenAPI document or a TypeScript Functions project.”
   • [claimed-docs] “MoonPay brought 200+ MCP servers under the MCP Gateway, going from proof of concept to a company-wide rollout in 30 days.”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/gram/): downloaded the official cli@0.16.0 release artifact (gram_dar…”
   • [claimed-docs] “the raw execution log for every tool call the platform observes across hosted MCP servers, tunneled and shadow MCP servers, skills, and loca…”

   proof ↗⚑ flag

Generate

1. developerTurn an existing API definition (OpenAPI) or custom code into a hosted MCP server without writing protocol plumbing

   weight 2 · round to Gram (Speakeasy)
   Pipedream MCP—none0/10

   All evidence describes consuming Pipedream's existing hosted MCP server (mcp.pipedream.com / remote.mcp.pipedream.net) to call Pipedream's pre-built 3,000+ API integrations on behalf of end users — none of it shows a developer importing their own OpenAPI spec or custom code to generate a new hosted MCP server without protocol plumbing. Missing for 10: any documentation of OpenAPI import, custom-code-to-MCP conversion, or a 'create your own MCP server' workflow.

   • [claimed-docs] “Add Pipedream MCP to your app or agent to make tool calls on behalf of your users to 3,000+ APIs and 10,000+ tools.”
   • [claimed-docs] “Pipedream Connect includes built-in user authentication for more than APIs via MCP, which means you don’t need to build any authorization fl…”
   • [claimed-docs] “This is handled automatically by Pipedream’s MCP server and there’s no additional implementation required”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://remote.mcp.pipedream.net answered a JSON-RPC error over HTTP 400 ({"jsonrpc":"2.…”

   proof ↗⚑ flag

   Gram (Speakeasy)✓fullprobed8/10

   Gram's Control Plane generates a hosted MCP server directly from an OpenAPI document or a TypeScript Functions project, with no protocol plumbing required from the developer, and a CLI/runtime probe confirms the tooling actually installs and runs. missing for 10: independent (non-vendor) hands-on account of the OpenAPI-to-MCP generation flow itself, and more detail on custom-code/TypeScript function support beyond the docs mention.

   • [claimed-docs] “Built — the Control Plane generates the server itself from a first-party source: an OpenAPI document or a TypeScript Functions project.”
   • [probe] “official CLI documented at https://www.speakeasy.com/docs/ai-control-plane/reference/command-line”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/gram/): downloaded the official cli@0.16.0 release artifact (gram_dar…”

   proof ↗⚑ flag

Publish

1. developerPublish my own MCP server on the platform for distribution, discovery, and installation by others

   weight 2 · round to Gram (Speakeasy)
   Pipedream MCP—none0/10

   All evidence describes Pipedream's own hosted MCP server (remote.mcp.pipedream.net) that exposes Pipedream's 3,000+ pre-built API integrations to developers' apps/agents — it does not show any mechanism for a third-party developer to publish their own custom MCP server on the platform for other users to discover and install. Missing for 10: any publishing/registry workflow, discovery/marketplace listing feature, or installation flow for user-authored MCP servers.

   ⚑ flag

   Gram (Speakeasy)✓fullprobed8/10

   Gram lets developers generate/host MCP servers from OpenAPI or TypeScript sources on its Control Plane, with governance, RBAC, tool logs, and a CLI/dashboard to publish, discover, and install servers into MCP clients, and the CLI was independently verified to install and run. missing for 10: no independent (non-vendor) evidence of a public discovery/marketplace listing showing other developers actually finding and installing a third party's published server, and no detail on versioning/publishing workflow specifics beyond docs.

   • [claimed-docs] “Built — the Control Plane generates the server itself from a first-party source: an OpenAPI document or a TypeScript Functions project.”
   • [claimed-docs] “install toolsets as MCP servers in AI agents, and keep the CLI itself up to date”
   • [claimed-docs] “An org admin working in Claude Code, Cursor, Codex, or another MCP client can find and add MCP servers, finish their setup, put them on a pl…”
   • [claimed-docs] “MoonPay brought 200+ MCP servers under the MCP Gateway, going from proof of concept to a company-wide rollout in 30 days.”
   • [github] “Team, server, and tool level permissions enforced through RBAC and Oauth2.1. Synced to your enterprise IDP (Okta, Azure AD, Google Workspace…”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/gram/): downloaded the official cli@0.16.0 release artifact (gram_dar…”
   • [probe] “official MCP server documented at https://www.speakeasy.com/docs/ai-control-plane/reference/platform-mcp”

   proof ↗⚑ flag

Triggers

1. developerSubscribe my agent to events from connected downstream apps (triggers/webhooks) instead of polling them

   weight 2 · round drawn
   Pipedream MCP—none0/10

   The evidence pack only describes Pipedream MCP as a tool-calling interface (action/tool invocation via JSON-RPC, authentication, transport details) — nothing shows agents subscribing to webhook/trigger events pushed from downstream apps via MCP; this is a request/response tool model, not an event-subscription model.

   • [claimed-docs] “Add Pipedream MCP to your app or agent to make tool calls on behalf of your users to 3,000+ APIs and 10,000+ tools.”
   • [claimed-docs] “tools: await mcpClient.tools(), // Automatically loads all available tools”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://remote.mcp.pipedream.net answered a JSON-RPC error over HTTP 400 ({"jsonrpc":"2.…”

   proof ↗⚑ flag

   Gram (Speakeasy)—none0/10

   Gram's documented capabilities center on generating/govern/observe MCP servers from OpenAPI specs and exposing tools for agents to call (pull-based); there is no mention anywhere in the evidence of webhook subscriptions, event triggers, or push notifications from downstream apps to agents. This is a plausible axis for an MCP gateway/control-plane product, but no evidence shows it delivers it.

   ⚑ flag

Analytics

1. platform-engineerSee usage analytics for the servers and tools my org consumes or publishes — call volumes, users, errors, and cost

   weight 2 · round to Gram (Speakeasy)
   Pipedream MCP—none0/10

   Evidence covers MCP connectivity, auth, transport, and tool loading, but nothing addresses usage analytics dashboards for call volumes, user counts, error rates, or cost tracking across servers/tools an org consumes or publishes.

   ⚑ flag

   Gram (Speakeasy)✓fullclaimed7/10

   Gram documents tool-call logs across hosted/tunneled/shadow servers and local tools, and the GitHub repo description explicitly claims usage tracking across teams by tokens/cost, budgets, and tool effectiveness measurement, directly matching call volumes/cost/usage. Errors are implied via raw execution logs but not explicitly broken out as an error-rate metric, and there's no independent/hands-on corroboration of the analytics dashboard itself. Missing for 10: explicit error-rate reporting UI evidence, independent third-party validation of the analytics dashboard beyond vendor docs/GitHub description.

   • [github] “Track AI usage across teams and measure impact with either tokens or cost. Deep dive expensive sessions, create budgets and measure tool eff…”
   • [claimed-docs] “the raw execution log for every tool call the platform observes across hosted MCP servers, tunneled and shadow MCP servers, skills, and loca…”
   • [claimed-docs] “unsanctioned shadow MCP servers are blocked by default”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round to Gram (Speakeasy)
   Pipedream MCP~partialprobed5/10

   Pipedream MCP exposes tool-calling and account-connection via API/MCP server (remote.mcp.pipedream.net, SDK), and end users can connect/revoke accounts via mcp.pipedream.com, but the evidence pack does not show that all UI-driven configuration (e.g., building/publishing custom workflows, managing app settings, full admin/dashboard functionality) is exposed through the API/MCP interface. missing for 10: evidence that non-tool-calling UI features (workflow building, org/team admin, billing, dashboard analytics) are also accessible via API/MCP, and independent confirmation beyond vendor docs.

   • [claimed-docs] “Add Pipedream MCP to your app or agent to make tool calls on behalf of your users to 3,000+ APIs and 10,000+ tools.”
   • [claimed-docs] “tools: await mcpClient.tools(), // Automatically loads all available tools”
   • [claimed-docs] “As an end user: Connect your accounts through Pipedream MCP at mcp.pipedream.com”
   • [claimed-docs] “Revocable access: Users can revoke access to their connected accounts at any time”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://remote.mcp.pipedream.net answered a JSON-RPC error over HTTP 400 ({"jsonrpc":"2.…”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/pipedream-mcp/): `npm install @pipedream/sdk` completed into a throwa…”

   proof ↗⚑ flag

   Gram (Speakeasy)~partialprobed6/10

   Gram exposes a Platform MCP server and CLI that let an org admin add/configure MCP servers, author skills, manage risk policies, and read observability data without opening the dashboard (gram-docs-4, gram-docs-7, gram-probe-5), and the CLI is confirmed to actually install and run (gram-probe-rt-1). However, no public REST/OpenAPI surface was found (gram-probe-3: openapi.json paths all 404), so full UI-parity via a general API is unconfirmed. Missing for 10: a documented general-purpose REST/OpenAPI API covering all dashboard actions, and independent confirmation that every UI feature (not just admin/config tasks) is reachable programmatically.

   • [claimed-docs] “install toolsets as MCP servers in AI agents, and keep the CLI itself up to date”
   • [claimed-docs] “An org admin working in Claude Code, Cursor, Codex, or another MCP client can find and add MCP servers, finish their setup, put them on a pl…”
   • [probe] “official CLI documented at https://www.speakeasy.com/docs/ai-control-plane/reference/command-line”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/gram/): downloaded the official cli@0.16.0 release artifact (gram_dar…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://www.speakeasy.com/openapi.json, https://www.speakeasy.com/swagger.json, https://www.speakeas…”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round drawn
   Pipedream MCP—none0/10

   Pipedream MCP is a connectivity/integration layer for tool calls to third-party APIs; the evidence covers authentication, tool loading, and revocable access, but nothing addresses exporting a user's own data from Pipedream in open formats or account/data portability for leaving the platform.

   ⚑ flag

   Gram (Speakeasy)—none0/10

   The evidence describes Gram's inputs (OpenAPI specs, SKILL.md manifests) and platform features like RBAC, logging, and CLI, but there is no documented data-export feature letting a user pull out their configured toolsets, logs, policies, or org data in open formats to migrate away. Axis applies to a platform like this but no evidence supports it.

   ⚑ flag

3. ai-native userSelf-host the core product

   weight 3 · round drawn
   Pipedream MCP—none0/10

   All evidence describes Pipedream MCP as a hosted service (mcp.pipedream.com, remote.mcp.pipedream.net) with no mention of a self-hosted or open-source deployment option for the core MCP server; the SDK/npm package is a client library, not a self-hostable server.

   • [claimed-docs] “As an end user: Connect your accounts through Pipedream MCP at mcp.pipedream.com”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://remote.mcp.pipedream.net answered a JSON-RPC error over HTTP 400 ({"jsonrpc":"2.…”
   • [claimed-docs] “To use Pipedream MCP in your own app or agent to make tool calls on behalf of your users, you can get started for free in development mode”

   proof ↗⚑ flag

   Gram (Speakeasy)—none0/10

   While Gram's code is hosted on GitHub (gram-gh-1, gram-gh-2), the evidence pack contains no documentation, guide, or deployment instructions describing a supported self-hosted deployment of the core Control Plane — all docs describe the hosted SaaS platform, dashboard, and CLI/skill features rather than self-hosting.

   • [github] “Track AI usage across teams and measure impact with either tokens or cost. Deep dive expensive sessions, create budgets and measure tool eff…”
   • [github] “Team, server, and tool level permissions enforced through RBAC and Oauth2.1. Synced to your enterprise IDP (Okta, Azure AD, Google Workspace…”

   proof ↗⚑ flag

Pricing

1. developerPrototype my agent's tool connections on a meaningful free tier before paying anything

   weight 1 · round to Pipedream MCP
   Pipedream MCP~partialprobed5/10

   Docs explicitly state developers can 'get started for free in development mode' and try the hosted chat app, and the SDK/server are confirmed live and installable without payment friction. However, there's no detail on what limits/quotas define the free tier (rate limits, tool-call caps, upgrade triggers), so it's unclear how 'meaningful' the free tier is for real prototyping at scale. Missing for 10: specific free-tier quotas/limits, comparison to paid tiers, and independent developer accounts confirming the free tier suffices for non-trivial prototyping.

   • [claimed-docs] “To use Pipedream MCP in your own app or agent to make tool calls on behalf of your users, you can get started for free in development mode”
   • [claimed-docs] “Try out Pipedream MCP in our chat app at chat.pipedream.com”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://remote.mcp.pipedream.net answered a JSON-RPC error over HTTP 400 ({"jsonrpc":"2.…”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/pipedream-mcp/): `npm install @pipedream/sdk` completed into a throwa…”

   proof ↗⚑ flag

   Gram (Speakeasy)—none0/10

   The evidence pack contains no mention of pricing tiers, a free tier, or cost thresholds for Gram — nothing describes what is available without payment. This is a fair pricing-plans axis for any commercial dev tool, but there's simply no documentation or citation about free-tier limits or trial terms.

   ⚑ flag

2. developerSee transparent pricing per tool call or connection plus documented rate limits before committing to the platform

   weight 2 · round drawn
   Pipedream MCP—none0/10

   Evidence only covers MCP setup, auth, and transport mechanics; no citation discloses per-tool-call pricing, connection pricing tiers, or documented rate limits. A passing mention of 'free in development mode' does not constitute transparent pricing or rate-limit documentation.

   ⚑ flag

   Gram (Speakeasy)—none0/10

   No evidence pack item mentions pricing, cost per tool call/connection, or documented rate limits; all citations concern architecture, RBAC, skills, and CLI features. Absence of evidence for an applicable and standard pricing-transparency axis means this must be scored none.

   ⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round drawn
   Pipedream MCP—none0/10

   No evidence anywhere in the pack mentions data residency, regional storage options, or compliance controls for where data is stored; the docs focus on tool access, authentication, and transport mechanics.

   ⚑ flag

   Gram (Speakeasy)—none0/10

   No evidence in the pack mentions data residency, region selection, or geographic storage controls for Gram; the documentation focuses on governance, RBAC, tool logs, and MCP server hosting but never addresses where data is physically stored.

   ⚑ flag

2. ai-native userPrevent my data from being used to train AI models

   weight 3 · round drawn
   Pipedream MCP—none0/10

   The evidence pack covers MCP connectivity, authentication, and revocable account access, but contains no mention of AI-training data usage policies or opt-out controls for Pipedream MCP.

   ⚑ flag

   Gram (Speakeasy)—none0/10

   Gram's evidence pack covers MCP server governance, RBAC, observability, and tool logging, but nothing addresses whether user/tool-call data is used to train AI models or an opt-out policy for such training — missing for 10: any data-training policy statement, opt-out controls, or model-training disclosure.

   ⚑ flag

3. ai-native userControl data retention and deletion

   weight 2 · round to Pipedream MCP
   Pipedream MCP~partialclaimed4/10

   Docs confirm users can revoke connected-account access at any time, which covers a piece of deletion control, but there is no documented policy on data retention periods, deletion of stored request/response logs, or export/erasure guarantees. missing for 10: explicit data retention timelines, log/data deletion mechanisms beyond account revocation, compliance/GDPR-style deletion guarantees.

   • [claimed-docs] “Revocable access: Users can revoke access to their connected accounts at any time”
   • [claimed-docs] “As an end user: Connect your accounts through Pipedream MCP at mcp.pipedream.com”

   proof ↗⚑ flag

   Gram (Speakeasy)—none0/10

   The evidence pack covers governance, RBAC, audit logs, OAuth, and shadow-server blocking, but no citation addresses data retention policies, deletion controls, or user data lifecycle management for the AI-native platform.

   ⚑ flag

4. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   Pipedream MCP—none0/10

   No evidence anywhere in the pack addresses telemetry, analytics, or usage-tracking opt-out settings for Pipedream MCP; docs cover authentication, transport, and account revocation but not telemetry controls.

   ⚑ flag

   Gram (Speakeasy)—none0/10

   No evidence in the pack addresses whether Gram's CLI, dashboard, or platform collects vendor-side telemetry/usage data or offers an opt-out; the only related items describe customer-facing usage analytics (tracking AI agent activity), not the product's own telemetry practices. Missing for 10: any mention of a telemetry setting, opt-out flag, or privacy policy statement about data collection from the CLI/platform itself.

   • [github] “Track AI usage across teams and measure impact with either tokens or cost. Deep dive expensive sessions, create budgets and measure tool eff…”

   proof ↗⚑ flag

Api

1. ai agentQuery the platform's registry or catalog through a documented API to discover servers and tools programmatically

   weight 3 · round to Gram (Speakeasy)
   Pipedream MCP~partialprobed5/10

   Pipedream documents a programmatic `mcpClient.tools()` call that 'automatically loads all available tools' for a connected app, and the live endpoint probe confirms the MCP server responds to JSON-RPC requests, showing some machine-readable tool discovery. However, there is no documented registry/catalog API for browsing the full list of 3,000+ integrated apps/servers before connecting — discovery is scoped per-user/per-app rather than a global catalog query. Missing for 10: a documented catalog/registry-listing endpoint (not just per-connection tools()), independent confirmation of full catalog enumeration.

   • [claimed-docs] “tools: await mcpClient.tools(), // Automatically loads all available tools”
   • [claimed-docs] “Add Pipedream MCP to your app or agent to make tool calls on behalf of your users to 3,000+ APIs and 10,000+ tools.”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://remote.mcp.pipedream.net answered a JSON-RPC error over HTTP 400 ({"jsonrpc":"2.…”

   proof ↗⚑ flag

   Gram (Speakeasy)~partialprobed6/10

   Gram documents an official 'platform-mcp' server that lets an agent working through an MCP client find and add MCP servers, manage toolsets, and read observability data programmatically (gram-docs-7, gram-probe-4), and the CLI can install toolsets as MCP servers (gram-docs-4/5). This covers the discovery story via the MCP protocol itself acting as the documented API, but there is no confirmed general-purpose REST/OpenAPI catalog endpoint (the openapi probe returned 404s) and no independent hands-on proof of the discovery flow itself. Missing for 10: a documented REST/OpenAPI registry endpoint, and third-party/hands-on verification that an agent can actually enumerate servers/tools via the platform MCP server.

   • [claimed-docs] “An org admin working in Claude Code, Cursor, Codex, or another MCP client can find and add MCP servers, finish their setup, put them on a pl…”
   • [claimed-docs] “install toolsets as MCP servers in AI agents, and keep the CLI itself up to date”
   • [probe] “official MCP server documented at https://www.speakeasy.com/docs/ai-control-plane/reference/platform-mcp”
   • [probe] “PROBE openapi: all candidate paths 404 (https://www.speakeasy.com/openapi.json, https://www.speakeasy.com/swagger.json, https://www.speakeas…”

   proof ↗⚑ flag

Metadata

1. developerInspect a listed server's tools, input schemas, transport, and hosting attributes before connecting to it

   weight 2 · round to Gram (Speakeasy)
   Pipedream MCP~partialclaimed4/10

   Docs show that tools can be listed/loaded via mcpClient.tools() and that transport (SSE/streamable HTTP) is documented, giving partial visibility into a server's capabilities before use, but there is no evidence of a registry listing with explicit input schemas or hosting attributes for inspection prior to connecting. missing for 10: schema inspection UI/API, hosting attribute metadata, and a registry-style browsing view for listed servers before connection.

   • [claimed-docs] “tools: await mcpClient.tools(), // Automatically loads all available tools”
   • [claimed-docs] “The Pipedream MCP server supports both SSE and streamable HTTP transport types dynamically, with no configuration required by the developer …”
   • [claimed-docs] “Slack: Send messages, manage channels, create reminders, and more”

   proof ↗⚑ flag

   Gram (Speakeasy)~partialclaimed5/10

   Gram's in-dashboard 'Connect' harness lets a developer pick a listed server, authenticate, and chat with a model that calls its tools while inspecting logs, which implies pre-connection tool inspection, and servers are generated from OpenAPI/TypeScript sources implying schema definitions exist. However, no evidence explicitly documents a UI or API for viewing input schemas, transport type, or hosting attributes for a listed server. Missing for 10: explicit documentation of schema viewer, transport metadata display, hosting attribute display, and independent/hands-on confirmation of this inspection workflow.

   • [claimed-docs] “An in-dashboard agent harness for testing MCP servers before connecting a real client: pick a server, authenticate, chat with a model that c…”
   • [claimed-docs] “Built — the Control Plane generates the server itself from a first-party source: an OpenAPI document or a TypeScript Functions project.”

   proof ↗⚑ flag

Playground

1. developerExercise an MCP server's tools from a browser-based inspector or playground before integrating it

   weight 2 · round to Gram (Speakeasy)
   Pipedream MCP~partialprobed6/10

   Pipedream explicitly offers a browser-based 'chat app' at chat.pipedream.com for trying out its MCP tools before integrating, and mcp.pipedream.com lets users connect/authorize accounts, which functions as a lightweight playground. However, there's no dedicated 'MCP inspector' UI shown, no independent hands-on report confirming the chat app's tool-exercising experience, and the live endpoint requires per-user binding before serving tools. Missing for 10: independent/hands-on verification of the chat playground's tool-calling UX, and a dedicated inspector-style tool browser beyond the chat interface.

   • [claimed-docs] “Try out Pipedream MCP in our chat app at chat.pipedream.com”
   • [claimed-docs] “As an end user: Connect your accounts through Pipedream MCP at mcp.pipedream.com”
   • [probe] “PROBE mcp-endpoint (2026-09-05): POST initialize to https://remote.mcp.pipedream.net answered a JSON-RPC error over HTTP 400 ({"jsonrpc":"2.…”

   proof ↗⚑ flag

   Gram (Speakeasy)✓fullclaimed8/10

   Gram provides an in-dashboard agent harness (gram-docs-8) explicitly described as a way to pick a server, authenticate, chat with a model that invokes tools, and inspect logs before connecting a real client — directly matching the story of exercising tools from a browser-based playground pre-integration. This is first-party documentation with a clear feature name ('Connect') dedicated to this exact workflow. Missing for 10: independent/hands-on user reports confirming the playground works as described, and more detail on tool-level interaction granularity within the UI.

   • [claimed-docs] “An in-dashboard agent harness for testing MCP servers before connecting a real client: pick a server, authenticate, chat with a model that c…”

   proof ↗⚑ flag

Search

1. developerSearch and browse a large catalog of MCP servers and tools by capability before wiring anything into my agent

   weight 3 · round to Pipedream MCP
   Pipedream MCP~partialclaimed4/10

   Pipedream advertises a large catalog (3,000+ APIs, 10,000+ tools) and lists example apps like Slack with their supported actions, implying some browsable catalog exists, but there is no direct evidence of a search/browse UI or capability-based filtering mechanism for developers to explore before integration. missing for 10: explicit search/filter interface, capability-tagging or query API, independent confirmation of browsing experience.

   • [claimed-docs] “Add Pipedream MCP to your app or agent to make tool calls on behalf of your users to 3,000+ APIs and 10,000+ tools.”
   • [claimed-docs] “Slack: Send messages, manage channels, create reminders, and more”
   • [claimed-docs] “Try out Pipedream MCP in our chat app at chat.pipedream.com”

   proof ↗⚑ flag

   Gram (Speakeasy)~partialclaimed3/10

   Gram documents that an admin can 'find and add MCP servers' via the platform MCP client integration, implying some discovery capability, but there is no evidence of a searchable/browsable catalog of MCP servers by capability, catalog size, or a discovery UI — Gram is primarily framed as a control plane for generating/governing your own servers rather than a registry of third-party servers. missing for 10: a documented catalog/search UI, evidence of catalog scale or third-party server listings, and any capability-based filtering/browsing mechanism.

   • [claimed-docs] “An org admin working in Claude Code, Cursor, Codex, or another MCP client can find and add MCP servers, finish their setup, put them on a pl…”
   • [claimed-docs] “Put a governed endpoint in front of an MCP server someone else runs, so access control, audit, and upstream credentials apply without changi…”

   proof ↗⚑ flag

Cli

1. developerDrive the publish-and-deploy workflow from an official CLI — authenticate, stage or build a server, push it, and install it into agents

   weight 2 · round to Gram (Speakeasy)
   Pipedream MCP—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

   Gram (Speakeasy)~partialprobed7/10

   There's a documented, runtime-verified official CLI (gram-docs-4, gram-probe-5, gram-probe-rt-1) that supports installing toolsets as MCP servers into agents, plus docs on building servers from OpenAPI/TS and platform-mcp integration for org admins to add/configure servers from within MCP clients. However, the evidence doesn't clearly walk through explicit CLI subcommands for authenticate/stage/build/push in one coherent flow — most detail is on install and general CLI existence rather than a full documented publish pipeline. Missing for 10: explicit CLI command reference showing login/auth, build/stage, and push subcommands with example output, and independent (non-vendor) confirmation of the full workflow beyond version-check.

   • [claimed-docs] “install toolsets as MCP servers in AI agents, and keep the CLI itself up to date”
   • [claimed-docs] “Built — the Control Plane generates the server itself from a first-party source: an OpenAPI document or a TypeScript Functions project.”
   • [probe] “official CLI documented at https://www.speakeasy.com/docs/ai-control-plane/reference/command-line”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/gram/): downloaded the official cli@0.16.0 release artifact (gram_dar…”
   • [claimed-docs] “An org admin working in Claude Code, Cursor, Codex, or another MCP client can find and add MCP servers, finish their setup, put them on a pl…”

   proof ↗⚑ flag

Frameworks

1. developerPlug the platform's tools into major agent frameworks and SDKs (Vercel AI SDK, LangChain, CrewAI, OpenAI or Anthropic SDKs) with documented adapters

   weight 2 · round to Pipedream MCP
   Pipedream MCP~partialprobed5/10

   Pipedream documents a generic MCP client integration page (docs/connect/mcp/ai-frameworks) showing a code snippet (mcpClient.tools()) for loading tools automatically, implying framework-agnostic SDK integration, and ships an official @pipedream/sdk package confirmed via runtime probe. However, the evidence pack never names or shows adapters for specific major frameworks like Vercel AI SDK, LangChain, CrewAI, or OpenAI/Anthropic SDKs explicitly — only a generic MCP client pattern is shown. Missing for 10: explicit documented code samples/adapters per named framework (LangChain, CrewAI, Vercel AI SDK, OpenAI/Anthropic SDKs), independent hands-on confirmation of each integration working.

   • [claimed-docs] “tools: await mcpClient.tools(), // Automatically loads all available tools”
   • [claimed-docs] “Add Pipedream MCP to your app or agent to make tool calls on behalf of your users to 3,000+ APIs and 10,000+ tools.”
   • [probe] “PROBE runtime (recorded 2026-09-04, see data/mcp-infrastructure/proofs/pipedream-mcp/): `npm install @pipedream/sdk` completed into a throwa…”

   proof ↗⚑ flag

   Gram (Speakeasy)—none0/10

   Gram documents MCP server generation, its own CLI, and MCP-client compatibility (Claude Code, Cursor, Codex), but there is no evidence of documented adapters or integration guides for Vercel AI SDK, LangChain, CrewAI, or OpenAI/Anthropic SDKs specifically.

   ⚑ flag

Security

1. platform-engineerRely on the platform scanning or inspecting listed servers for malicious or vulnerable behavior before my agents use them

   weight 2 · round to Gram (Speakeasy)
   Pipedream MCP—none0/10

   Evidence covers authentication, transport, tool loading, and revocable access, but nothing in the pack describes any scanning, vetting, or security review process for listed servers/tools before agents use them.

   ⚑ flag

   Gram (Speakeasy)~partialclaimed5/10

   Gram provides governance controls relevant to tool curation — RBAC/OAuth2.1 access control, full audit/tool-call logging across hosted, tunneled and shadow servers, 'risk policies' management via the platform MCP server, and blocking of unsanctioned shadow MCP servers by default — plus an in-dashboard harness to test a server's tool calls before connecting a real client. However none of this is documented as active scanning for malicious code or known vulnerabilities in listed servers; it's closer to access governance, logging and manual testing than automated security scanning. Missing for 10: explicit vulnerability/malware scanning of server code or behavior, third-party security audit integration, and independent evidence that risk policies actually catch malicious servers.

   • [claimed-docs] “unsanctioned shadow MCP servers are blocked by default”
   • [claimed-docs] “An org admin working in Claude Code, Cursor, Codex, or another MCP client can find and add MCP servers, finish their setup, put them on a pl…”
   • [claimed-docs] “the raw execution log for every tool call the platform observes across hosted MCP servers, tunneled and shadow MCP servers, skills, and loca…”
   • [claimed-docs] “An in-dashboard agent harness for testing MCP servers before connecting a real client: pick a server, authenticate, chat with a model that c…”
   • [github] “Team, server, and tool level permissions enforced through RBAC and Oauth2.1. Synced to your enterprise IDP (Okta, Azure AD, Google Workspace…”

   proof ↗⚑ flag

Signals

1. developerCompare servers using published quality signals — usage counts, health or liveness checks, scores, or verification badges

   weight 2 · round drawn
   Pipedream MCP—none0/10

   Evidence only covers Pipedream MCP's functionality (auth, transport, tool listing) but contains no usage counts, health/liveness checks, quality scores, or verification badges for comparing servers.

   ⚑ flag

   Gram (Speakeasy)—none0/10

   Gram provides usage/cost tracking (gram-gh-1) and tool-call logs (gram-docs-5), plus governance controls like blocking shadow servers (gram-docs-10), but there is no evidence of published quality signals such as health/liveness checks, verification badges, or comparative quality scores that a developer could use to compare servers side-by-side. Missing for 10: liveness/health-check indicators, explicit quality/verification badges, and any server-comparison or ranking UI.

   • [github] “Track AI usage across teams and measure impact with either tokens or cost. Deep dive expensive sessions, create budgets and measure tool eff…”
   • [claimed-docs] “the raw execution log for every tool call the platform observes across hosted MCP servers, tunneled and shadow MCP servers, skills, and loca…”
   • [claimed-docs] “unsanctioned shadow MCP servers are blocked by default”

   proof ↗⚑ flag

Tool search

1. ai agentSearch and load tools at runtime through meta-tools instead of having thousands of tool definitions stuffed into my context

   weight 2 · round drawn
   Pipedream MCP—none0/10

   Pipedream MCP's own docs describe `mcpClient.tools()` as automatically loading *all* available tools rather than exposing meta-tools (e.g., search_tools/load_tool) for on-demand discovery, and no evidence pack item mentions a meta-tool/dynamic-search pattern for reducing context load.

   • [claimed-docs] “tools: await mcpClient.tools(), // Automatically loads all available tools”
   • [claimed-docs] “Add Pipedream MCP to your app or agent to make tool calls on behalf of your users to 3,000+ APIs and 10,000+ tools.”

   proof ↗⚑ flag

   Gram (Speakeasy)—none0/10

   Gram's evidence covers MCP server hosting, toolset governance, OAuth, RBAC, and 'skills' loaded on demand, but none of it describes a runtime meta-tool (e.g., search_tools/load_tool) mechanism that lets an agent dynamically discover and load individual tool definitions instead of receiving a large static toolset. The closest analog (SKILL.md manifests loaded on demand) is a different mechanism from runtime tool search across a large tool catalog.

   ⚑ flag