Zapier vs Windmill

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round drawn
   Zapier✓fullprobed9/10

   A direct probe confirms docs.zapier.com/llms.txt returns HTTP 200 with a structured index for developer documentation (SDK, MCP, integration building), exactly matching the llms.txt agent-discovery pattern, and this is corroborated by docs describing agent-oriented access points (MCP server, CLI, npm skills). Missing for 10: independent/community confirmation of an agent actually consuming llms.txt successfully.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.zapier.com/llms.txt # Zapier — docs.zapier.com > Local index for developer documentation: SDK, MCP…”
   • [claimed-docs] “Agents discover them on npm and skill registries, then run them as an agent skill, a CLI, a typed Node module, or a local MCP server.”
   • [claimed-docs] “Give any MCP-compatible AI client governed access to 9,000+ apps through one connection: no code, no infrastructure, SOC 2 certified.”

   proof ↗⚑ flag

   Windmill✓fullprobed9/10

   Direct probe confirms a working llms.txt at https://www.windmill.dev/llms.txt (HTTP 200) and .md-suffixed docs pages (e.g., intro.md) that return clean markdown, both hallmarks of agent-oriented documentation designed for LLM consumption. Missing for 10: no independent/community confirmation that agents actually use these successfully in practice.

   • [probe] “PROBE llms.txt: HTTP 200 at https://www.windmill.dev/llms.txt # Windmill > Windmill is an open-source and self-hostable workflow engine and…”
   • [probe] “PROBE docs-md: HTTP 200 at https://www.windmill.dev/docs/intro.md # What is Windmill? > What is Windmill? An open-source workflow engine an…”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round to Windmill
   Zapier~partialprobed5/10

   Zapier offers a Platform CLI (Node.js) for building/publishing integrations and a Workflow API/Developer Platform, which can be invoked programmatically and scripted in CI for building or deploying integrations, but the core Zap workflow execution itself is primarily triggered by app events/schedules via the visual builder, not designed as a headless batch job runnable in a CI pipeline. There's no documented CLI command or API to 'run a Zap' synchronously in a pipeline the way a CI job would invoke a task. missing for 10: explicit documentation of running/triggering Zap executions headlessly from CI, a CLI/API command to invoke a workflow run synchronously and get results, and independent confirmation of CI-based automation use cases.

   • [claimed-docs] “Zapier Platform CLI enables developers to build integrations using Node.js v22, OAuth1/OAuth2 authentication, webhooks, polling triggers, an…”
   • [claimed-docs] “Use the Visual Builder for quick, no-code setups or the CLI for full control and customization.”
   • [claimed-docs] “Use Zapier's Workflow API and 9,000 integrations to power a built-in automation experience, integration marketplace, or AI workflows.”
   • [probe] “official CLI documented at https://docs.zapier.com/integrations/build-cli/overview”

   proof ↗⚑ flag

   Windmill✓fullprobed8/10

   Windmill provides a documented CLI (wmill) for deploying and syncing from Git, supports webhooks/schedules/API triggers for headless execution, and can self-host via Docker/Kubernetes for CI pipelines — all strongly supporting headless/CI automation use. Missing for 10: independent hands-on CI-pipeline examples or third-party corroboration of the CLI/webhook flow working seamlessly in a CI system beyond vendor docs.

   • [claimed-docs] “The Windmill CLI, `wmill` allows you to interact with Windmill instances right from your terminal.”
   • [claimed-docs] “Each time an item is deployed, Windmill will create and push a commit to the specified repository. ... Windmill can automatically deploy new…”
   • [claimed-docs] “How do I trigger scripts and flows in Windmill? Use schedules, webhooks, emails, HTTP routes, websockets, Kafka, Postgres, NATS, SQS, MQTT, …”
   • [claimed-docs] “Flows additionally expose version-pinned webhook endpoints that target a specific flow version by its numeric version ID instead of the late…”
   • [claimed-docs] “For small setups, use Docker and Docker Compose on a single instance. For larger and production use-cases, use our Helm chart to deploy on K…”
   • [probe] “official CLI documented at https://www.windmill.dev/docs/advanced/cli”

   proof ↗⚑ flag

3. ai-native userPlug MCP servers into this product so it can use their tools

   weight 3 · round drawn
   Zapier—none0/10

   All evidence describes Zapier acting as an MCP *server* (zapier.com/mcp) that other AI clients (Claude, ChatGPT, Cursor) connect to in order to use Zapier's own tools — not Zapier acting as an MCP *client* that ingests external MCP servers so Zapier itself can call their tools. No documentation shows Zapier importing or connecting to third-party MCP servers as a consumer.

   • [claimed-docs] “Connect Claude, ChatGPT, Cursor, and other AI tools to Gmail, Slack, Salesforce, and 9,000+ apps. No code. No setup headaches.”
   • [claimed-docs] “Give any MCP-compatible AI client governed access to 9,000+ apps through one connection: no code, no infrastructure, SOC 2 certified.”
   • [claimed-docs] “Zapier MCP gives IT one governed connection across every AI agent your org uses: Claude, ChatGPT, Cursor. Account-level restrictions, manage…”
   • [probe] “official MCP server documented at https://zapier.com/mcp”

   proof ↗⚑ flag

   Windmill—none0/10

   Windmill's MCP documentation (windmill-docs-17, windmill-probe-4) describes Windmill acting as an MCP *server* so external LLM clients (Claude, Cursor) can call Windmill's scripts/flows — the reverse of the story, which asks whether Windmill can consume/plug into external MCP servers to use their tools. The AI Agent steps (windmill-docs-20) mention connecting to AI providers/models, not MCP tool servers, so no evidence shows Windmill as an MCP client.

   • [claimed-docs] “With MCP, you can connect your favorite LLMs (like Claude, Cursor, or any MCP compatible client) to Windmill, allowing you to trigger your s…”
   • [probe] “official MCP server documented at https://www.windmill.dev/docs/core_concepts/mcp”
   • [claimed-docs] “AI Agents steps allow you to integrate AI agents directly into your Windmill flows. They provide an interface to connect with various AI pro…”

   proof ↗⚑ flag

4. ai-native userConnect an agent via an official MCP server

   weight 3 · round drawn
   Zapier✓fullprobed8/10

   Zapier ships an official hosted MCP server (zapier.com/mcp) giving any MCP-compatible client (Claude, ChatGPT, Cursor) governed access to 9,000+ apps/40,000+ actions with SOC2-certified, no-code setup, plus a local MCP server option via the CLI. Missing for 10: independent/hands-on third-party verification of the MCP server's reliability beyond vendor docs.

   • [claimed-docs] “Connect Claude, ChatGPT, Cursor, and other AI tools to Gmail, Slack, Salesforce, and 9,000+ apps. No code. No setup headaches.”
   • [claimed-docs] “Give any MCP-compatible AI client governed access to 9,000+ apps through one connection: no code, no infrastructure, SOC 2 certified.”
   • [claimed-docs] “Zapier MCP gives IT one governed connection across every AI agent your org uses: Claude, ChatGPT, Cursor. Account-level restrictions, manage…”
   • [claimed-docs] “It lets your AI take real actions across 9,000+ apps and 40,000+ actions, such as sending messages, creating tasks, and updating records”
   • [probe] “official MCP server documented at https://zapier.com/mcp”
   • [claimed-docs] “Agents discover them on npm and skill registries, then run them as an agent skill, a CLI, a typed Node module, or a local MCP server.”

   proof ↗⚑ flag

   Windmill✓fullprobed8/10

   Windmill is a workflow/automation platform (not an agent), so publishing an official MCP server is squarely within its category, and docs confirm it: 'With MCP, you can connect your favorite LLMs (like Claude, Cursor, or any MCP compatible client) to Windmill, allowing you to trigger your scripts and flows from your client chat,' corroborated by a probe confirming the official MCP docs page. missing for 10: independent/hands-on confirmation of the MCP server working in practice beyond first-party docs.

   • [claimed-docs] “With MCP, you can connect your favorite LLMs (like Claude, Cursor, or any MCP compatible client) to Windmill, allowing you to trigger your s…”
   • [probe] “official MCP server documented at https://www.windmill.dev/docs/core_concepts/mcp”

   proof ↗⚑ flag

5. ai-native userUse an official CLI

   weight 2 · round drawn
   Zapier✓fullprobed8/10

   Zapier ships an official Platform CLI for building integrations (Node.js, OAuth, webhooks, polling triggers) documented at docs.zapier.com/integrations/build-cli/overview, and docs explicitly note agents can run these as 'a CLI, a typed Node module, or a local MCP server,' tying the CLI directly to AI-agent workflows. Missing for 10: independent/hands-on developer corroboration of CLI usage and more detail on AI-native-specific CLI features beyond general integration building.

   • [claimed-docs] “Zapier Platform CLI enables developers to build integrations using Node.js v22, OAuth1/OAuth2 authentication, webhooks, polling triggers, an…”
   • [claimed-docs] “Agents discover them on npm and skill registries, then run them as an agent skill, a CLI, a typed Node module, or a local MCP server.”
   • [claimed-docs] “Use the Visual Builder for quick, no-code setups or the CLI for full control and customization.”
   • [probe] “official CLI documented at https://docs.zapier.com/integrations/build-cli/overview”

   proof ↗⚑ flag

   Windmill✓fullprobed8/10

   Windmill ships an official CLI (`wmill`) documented for interacting with instances from the terminal, deploying from Git repos, and syncing commits—confirmed by both docs and a dedicated probe. This directly satisfies the 'official CLI' story for AI-native/agentic workflows (e.g., scripting deployments, CI integration). Missing for 10: independent hands-on community reports specifically validating the CLI's AI-native usage patterns (e.g., agent-driven CLI invocation) beyond first-party docs.

   • [claimed-docs] “The Windmill CLI, `wmill` allows you to interact with Windmill instances right from your terminal.”
   • [claimed-docs] “Each time an item is deployed, Windmill will create and push a commit to the specified repository. ... Windmill can automatically deploy new…”
   • [probe] “official CLI documented at https://www.windmill.dev/docs/advanced/cli”

   proof ↗⚑ flag

6. ai-native userDrive the product through a documented public API

   weight 3 · round to Zapier
   Zapier~partialprobed6/10

   Zapier documents a Workflow API for building automation experiences (zapier-docs-23) plus a Platform CLI and REST-style integration building (zapier-docs-3, zapier-docs-19), giving AI-native users programmatic ways to drive the product beyond the UI. However, a probe found no discoverable OpenAPI/swagger spec (zapier-probe-2 returned 404s across all candidate paths), suggesting the public API is not machine-discoverable in a standard way, and most current documentation emphasis is on MCP/agent-skill access rather than a general-purpose public API reference. Missing for 10: a published OpenAPI/swagger spec or full public API reference docs, and independent hands-on confirmation of API usage outside MCP/CLI contexts.

   • [claimed-docs] “Use Zapier's Workflow API and 9,000 integrations to power a built-in automation experience, integration marketplace, or AI workflows.”
   • [claimed-docs] “Zapier Platform CLI enables developers to build integrations using Node.js v22, OAuth1/OAuth2 authentication, webhooks, polling triggers, an…”
   • [claimed-docs] “Use the Visual Builder for quick, no-code setups or the CLI for full control and customization.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.zapier.com/openapi.json, https://docs.zapier.com/swagger.json, https://docs.zapier.com/…”
   • [probe] “official CLI documented at https://docs.zapier.com/integrations/build-cli/overview”

   proof ↗⚑ flag

   Windmill~partialprobed5/10

   Windmill exposes a CLI (wmill), webhooks, and various trigger mechanisms (HTTP routes, schedules, MCP) that let external systems and AI agents drive it (windmill-docs-9, windmill-docs-15, windmill-docs-16, windmill-docs-17, windmill-probe-4, windmill-probe-5), which is real evidence of programmatic/agentic control. However, a direct probe for a documented public REST/OpenAPI spec (the canonical 'documented public API') returned 404 on all candidate paths (windmill-probe-3), so no first-class API reference doc was found in evidence. missing for 10: a discoverable OpenAPI/REST API reference page, independent confirmation that third parties integrate via a general public API beyond webhooks/CLI/MCP.

   • [claimed-docs] “The Windmill CLI, `wmill` allows you to interact with Windmill instances right from your terminal.”
   • [claimed-docs] “How do I trigger scripts and flows in Windmill? Use schedules, webhooks, emails, HTTP routes, websockets, Kafka, Postgres, NATS, SQS, MQTT, …”
   • [claimed-docs] “Flows additionally expose version-pinned webhook endpoints that target a specific flow version by its numeric version ID instead of the late…”
   • [claimed-docs] “With MCP, you can connect your favorite LLMs (like Claude, Cursor, or any MCP compatible client) to Windmill, allowing you to trigger your s…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://www.windmill.dev/openapi.json, https://www.windmill.dev/swagger.json, https://www.windmill.d…”
   • [probe] “official MCP server documented at https://www.windmill.dev/docs/core_concepts/mcp”
   • [probe] “official CLI documented at https://www.windmill.dev/docs/advanced/cli”

   proof ↗⚑ flag

7. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round to Windmill
   Zapier~partialclaimed5/10

   Zapier MCP docs mention 'governed access', 'account-level restrictions, managed connections, and workspace controls' applied across agents (zapier-docs-11, zapier-docs-15), implying some control over what an agent can touch, and Zapier explicitly manages OAuth/credentials so users never handle tokens directly (zapier-docs-16). However there is no explicit documentation of issuing distinct, minimal-scope credentials per agent/action (e.g., granular OAuth scopes, per-agent API keys) — missing for 10: explicit per-agent scoped credential/token issuance mechanism, documentation of configurable least-privilege permission sets, and independent confirmation that these restrictions actually enforce least privilege.

   • [claimed-docs] “Give any MCP-compatible AI client governed access to 9,000+ apps through one connection: no code, no infrastructure, SOC 2 certified.”
   • [claimed-docs] “Zapier MCP gives IT one governed connection across every AI agent your org uses: Claude, ChatGPT, Cursor. Account-level restrictions, manage…”
   • [claimed-docs] “Zapier handles OAuth, credentials, and API complexity so you never touch a token.”
   • [claimed-docs] “It lets your AI take real actions across 9,000+ apps and 40,000+ actions, such as sending messages, creating tasks, and updating records”

   proof ↗⚑ flag

   Windmill~partialprobed6/10

   Windmill documents scoped tokens following least-privilege principles (windmill-docs-8) and an underlying roles/permissions system (windmill-docs-7), and its MCP server for connecting agents (windmill-docs-17, windmill-probe-4) implies token-based auth for agent connections. However, there's no explicit documentation tying scoped token creation specifically to AI agent use-cases or showing a workflow for issuing/rotating agent-specific credentials. Missing for 10: explicit docs on generating scoped tokens specifically for AI agents/MCP clients, examples of scope granularity, and independent/hands-on confirmation that scoping works as claimed.

   • [claimed-docs] “Tokens can be scoped to restrict access to specific resources and actions, following the principle of least privilege.”
   • [claimed-docs] “Windmill provides a roles and permissions system that allows you to control access and manage permissions within your instance and workspace…”
   • [claimed-docs] “With MCP, you can connect your favorite LLMs (like Claude, Cursor, or any MCP compatible client) to Windmill, allowing you to trigger your s…”
   • [probe] “official MCP server documented at https://www.windmill.dev/docs/core_concepts/mcp”

   proof ↗⚑ flag

8. ai-native userBuild against official SDKs

   weight 2 · round to Zapier
   Zapier~partialprobed6/10

   Zapier documents a typed Node module/CLI SDK (Platform CLI, Node.js v22, OAuth handling) and explicitly positions agents/skills as consuming Zapier as a 'code-file interface' via npm/skill registries, plus a documented MCP server for agent access. However, there's no dedicated 'official SDK' branding beyond the CLI, no OpenAPI spec (probe returned 404s), and no independent/hands-on corroboration of AI-native developers building against these SDKs specifically. missing for 10: independent verification of the AI-native SDK workflow, a published OpenAPI/typed client spec, and community evidence of agents actually building against the SDK rather than just using MCP/no-code paths.

   • [claimed-docs] “Zapier Platform CLI enables developers to build integrations using Node.js v22, OAuth1/OAuth2 authentication, webhooks, polling triggers, an…”
   • [claimed-docs] “Agents discover them on npm and skill registries, then run them as an agent skill, a CLI, a typed Node module, or a local MCP server.”
   • [claimed-docs] “The code-file interface to Zapier. Give coding agents programmatic access to 9,000+ apps: Zapier handles OAuth, credentials, and API complex…”
   • [claimed-docs] “Zapier handles OAuth, credentials, and API complexity so you never touch a token.”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.zapier.com/llms.txt # Zapier — docs.zapier.com > Local index for developer documentation: SDK, MCP…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.zapier.com/openapi.json, https://docs.zapier.com/swagger.json, https://docs.zapier.com/…”
   • [probe] “official CLI documented at https://docs.zapier.com/integrations/build-cli/overview”

   proof ↗⚑ flag

   Windmill—none0/10

   The evidence pack documents a CLI (wmill), an MCP server, and workflows-as-code in TypeScript/Python, but none of it describes official client SDKs/libraries for programmatically building against Windmill's API. A probe for an OpenAPI spec (often paired with SDK generation) returned 404s, further suggesting no discoverable official SDK artifact.

   • [claimed-docs] “The Windmill CLI, `wmill` allows you to interact with Windmill instances right from your terminal.”
   • [claimed-docs] “With MCP, you can connect your favorite LLMs (like Claude, Cursor, or any MCP compatible client) to Windmill, allowing you to trigger your s…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://www.windmill.dev/openapi.json, https://www.windmill.dev/swagger.json, https://www.windmill.d…”
   • [probe] “official CLI documented at https://www.windmill.dev/docs/advanced/cli”

   proof ↗⚑ flag

9. ai-native userSubscribe to events via webhooks

   weight 2 · round to Zapier
   Zapier✓fullprobed8/10

   Zapier's Platform CLI explicitly supports building webhook triggers (subscribe/unsubscribe patterns) alongside polling triggers, and this is a core, well-documented capability for integration builders including AI-native/agent use cases via CLI, MCP, and Workflow API. Missing for 10: independent hands-on verification of webhook subscription specifically (vs. polling) and no explicit example of an AI agent subscribing to a webhook end-to-end.

   • [claimed-docs] “Zapier Platform CLI enables developers to build integrations using Node.js v22, OAuth1/OAuth2 authentication, webhooks, polling triggers, an…”
   • [claimed-docs] “Use the Visual Builder for quick, no-code setups or the CLI for full control and customization.”
   • [probe] “official CLI documented at https://docs.zapier.com/integrations/build-cli/overview”
   • [claimed-docs] “Use Zapier's Workflow API and 9,000 integrations to power a built-in automation experience, integration marketplace, or AI workflows.”

   proof ↗⚑ flag

   Windmill~partialclaimed5/10

   Windmill documents webhooks extensively as a trigger mechanism—scripts and flows can be invoked via webhook endpoints, including version-pinned endpoints for specific flow versions—which lets external systems call into Windmill (windmill-docs-15, windmill-docs-16). However, this is primarily inbound triggering rather than an outbound event-subscription model where Windmill pushes notifications about internal events (e.g., job completion, failures) to a subscriber's webhook URL. Missing for 10: explicit documentation of outbound webhook subscriptions/event notifications, independent confirmation of an event-driven push webhook system, and any first-party mention of 'subscribe' semantics rather than pure trigger endpoints.

   • [claimed-docs] “How do I trigger scripts and flows in Windmill? Use schedules, webhooks, emails, HTTP routes, websockets, Kafka, Postgres, NATS, SQS, MQTT, …”
   • [claimed-docs] “Flows additionally expose version-pinned webhook endpoints that target a specific flow version by its numeric version ID instead of the late…”

   proof ↗⚑ flag

Agentic features

1. ai-native userGet AI-generated insights and suggestions from my data inside the product

   weight 2 · round drawn
   Zapier~partialclaimed4/10

   Zapier shows some AI-generated suggestions tied to user workflow data — AI troubleshooting instructions for Zap errors (docs-10) and Canvas's automatic process mapping/optimization (docs-18), plus Copilot helping build agents (docs-8) — but these are narrow, workflow-centric suggestions rather than broad 'insights from your data' analytics. missing for 10: dedicated data-insights/analytics feature, evidence of AI surfacing patterns or recommendations from connected app data (not just error-fixing or workflow mapping), independent/hands-on corroboration of these AI suggestion features actually working.

   • [claimed-docs] “Zapier will use AI to create troubleshooting instructions for you to review.”
   • [claimed-docs] “Canvas automatically maps and optimizes your processes to help your team document, refine, and automate with ease.”
   • [claimed-docs] “Create specialized agents—with the help of Zapier Copilot—to connect to your business data and perform tasks across 9,000+ apps.”

   proof ↗⚑ flag

   Windmill~partialclaimed4/10

   Windmill offers AI Agent steps that let users wire LLM calls into flows to process data, and an MCP server/AI-generation feature for building scripts, but these are developer-facing building blocks rather than a built-in 'insights and suggestions from my data' feature (e.g., no dashboard/analytics AI surfaced automatically on stored data). Missing for 10: evidence of an out-of-the-box AI analytics/insights UI, proactive suggestions surfaced to end users without building a flow, and any independent corroboration of this specific use case.

   • [claimed-docs] “AI Agents steps allow you to integrate AI agents directly into your Windmill flows. They provide an interface to connect with various AI pro…”
   • [claimed-docs] “Review and deploy changes: Once the AI has edited items, ask it to review the changes and it opens the Compare & Deploy page with exactly th…”
   • [claimed-docs] “With MCP, you can connect your favorite LLMs (like Claude, Cursor, or any MCP compatible client) to Windmill, allowing you to trigger your s…”

   proof ↗⚑ flag

2. ai-native userSet up automations that run autonomously in the background

   weight 2 · round drawn
   Zapier✓fullcommunity8/10

   Zapier's core product is trigger-based automations (Zaps) that run autonomously in the background once set up, and this is extended with AI-native features like Zapier MCP and Agents for triggering/running workflows via AI clients. Community evidence confirms real-world autonomous background execution (e.g., a Zap misfiring and sending thousands of invites unattended), corroborating that these run without active user supervision. Missing for 10: independent hands-on verification specifically of AI-triggered autonomous background runs (vs. classic Zaps), and detail on monitoring/error-recovery for unattended agentic runs beyond the troubleshooting-assistant mention.

   • [claimed-docs] “Create specialized agents—with the help of Zapier Copilot—to connect to your business data and perform tasks across 9,000+ apps.”
   • [claimed-docs] “It lets your AI take real actions across 9,000+ apps and 40,000+ actions, such as sending messages, creating tasks, and updating records”
   • [claimed-docs] “Connect Claude, ChatGPT, Cursor, and other AI tools to Gmail, Slack, Salesforce, and 9,000+ apps. No code. No setup headaches.”
   • [community] “One of our users recently found us after screwing up his calendar sync Zap and, on a random Sunday morning, ended up inviting thousands of h…”
   • [claimed-docs] “Zapier will use AI to create troubleshooting instructions for you to review.”

   proof ↗⚑ flag

   Windmill✓fullcommunity8/10

   Windmill natively supports scheduling scripts/flows via CRON with error/recovery handlers, plus a wide range of autonomous triggers (webhooks, queues, events, routes) that run in the background without manual intervention, and flows include retries, branching, and error handling for resilient unattended execution. This directly matches the AI-native automation story, reinforced by AI Agent steps that can be embedded into these autonomous flows. Missing for 10: no independent/hands-on evidence confirming long-running autonomous reliability at scale, and community comments raise open questions about error handling rather than confirming it in practice.

   • [claimed-docs] “A Schedule consists of a Script or Flow, its arguments, a CRON expression that controls the execution frequency and optional Error and Recov…”
   • [claimed-docs] “How do I trigger scripts and flows in Windmill? Use schedules, webhooks, emails, HTTP routes, websockets, Kafka, Postgres, NATS, SQS, MQTT, …”
   • [claimed-docs] “If defined, upon error this step will be retried with a delay and a maximum number of attempts as defined below.”
   • [claimed-docs] “Branches allow to split the execution of the flow based on a condition. ... Branch all: all the branches will be executed.”
   • [claimed-docs] “A script or flow can return a top-level `wm_failure` field (string) to retag the run as a failure even though the runtime did not raise an e…”
   • [claimed-docs] “AI Agents steps allow you to integrate AI agents directly into your Windmill flows. They provide an interface to connect with various AI pro…”
   • [community] “This looks really cool. Is there error handling? For example, if the email sending fails for some reason can I either select the flow to con…”

   proof ↗⚑ flag

3. ai-native userDelegate tasks to a built-in AI assistant inside the product

   weight 3 · round drawn
   Zapier~partialclaimed6/10

   Zapier ships 'Zapier Agents' and 'Zapier Copilot,' a built-in AI assistant that lets users create agents to perform tasks across 9,000+ apps and even auto-generates troubleshooting help, matching the delegate-to-built-in-assistant story. However, evidence is entirely first-party marketing copy with no independent hands-on verification of how well the assistant performs delegated tasks. Missing for 10: independent/community corroboration of the built-in AI assistant's real-world performance, and more concrete detail on its task scope/limits beyond marketing claims.

   • [claimed-docs] “Create specialized agents—with the help of Zapier Copilot—to connect to your business data and perform tasks across 9,000+ apps.”
   • [claimed-docs] “Zapier will use AI to create troubleshooting instructions for you to review.”
   • [claimed-docs] “Simplify and scale your processes with visual mapping—add owners, notes, and images to create structured, easy-to-follow systems.”
   • [claimed-docs] “Canvas automatically maps and optimizes your processes to help your team document, refine, and automate with ease.”

   proof ↗⚑ flag

   Windmill~partialclaimed6/10

   Windmill's docs describe an in-product AI generation feature that can edit scripts/flows and then prompts the user to review and deploy those changes, indicating a built-in assistant users can delegate authoring tasks to (windmill-docs-21). However, evidence is thin on the assistant's scope, limitations, or independent hands-on validation, and separate 'AI Agents' flow steps (windmill-docs-20) are for building agentic workflows rather than being the assistant itself. Missing for 10: independent/community corroboration of the AI assistant's real-world reliability, more detail on what tasks it can handle end-to-end, and comparison to failure cases.

   • [claimed-docs] “Review and deploy changes: Once the AI has edited items, ask it to review the changes and it opens the Compare & Deploy page with exactly th…”
   • [claimed-docs] “AI Agents steps allow you to integrate AI agents directly into your Windmill flows. They provide an interface to connect with various AI pro…”

   proof ↗⚑ flag

4. ai-native userOperate the product with natural-language commands

   weight 2 · round to Zapier
   Zapier✓fullprobed8/10

   Zapier MCP explicitly enables natural-language commands: 'Describe what you need in plain language. Your AI calls the right Zapier action and confirms what it did' and 'Schedule a meeting with one prompt,' letting AI clients trigger 40,000+ actions across 9,000+ apps via governed MCP connections. This is well-documented first-party functionality with a dedicated product page, though independent hands-on validation of the natural-language flow itself is absent. Missing for 10: independent/community corroboration of the MCP natural-language experience specifically (community evidence covers classic Zap-building, not the AI prompt-driven flow).

   • [claimed-docs] “Describe what you need in plain language. Your AI calls the right Zapier action and confirms what it did.”
   • [claimed-docs] “Schedule a meeting with one prompt. Your AI finds time on everyone's calendar and books it. No back and forth.”
   • [claimed-docs] “It lets your AI take real actions across 9,000+ apps and 40,000+ actions, such as sending messages, creating tasks, and updating records”
   • [claimed-docs] “Connect Claude, ChatGPT, Cursor, and other AI tools to Gmail, Slack, Salesforce, and 9,000+ apps. No code. No setup headaches.”
   • [claimed-docs] “Zapier MCP gives IT one governed connection across every AI agent your org uses: Claude, ChatGPT, Cursor. Account-level restrictions, manage…”
   • [probe] “official MCP server documented at https://zapier.com/mcp”

   proof ↗⚑ flag

   Windmill~partialprobed6/10

   Windmill documents an MCP server that lets LLM clients trigger scripts/flows via natural-language chat (windmill-docs-17, windmill-probe-4) and an AI-generation feature where a user can ask the AI to edit and review flow/script changes (windmill-docs-21), showing real natural-language operability. However, evidence is thin—just two doc snippets—with no independent/hands-on corroboration of reliability or scope of NL commands. Missing for 10: hands-on/community validation of NL-driven operation, fuller documentation of what natural-language commands can accomplish beyond triggering and editing.

   • [claimed-docs] “With MCP, you can connect your favorite LLMs (like Claude, Cursor, or any MCP compatible client) to Windmill, allowing you to trigger your s…”
   • [claimed-docs] “Review and deploy changes: Once the AI has edited items, ask it to review the changes and it opens the Compare & Deploy page with exactly th…”
   • [probe] “official MCP server documented at https://www.windmill.dev/docs/core_concepts/mcp”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   Zapier—none0/10

   Zapier has developer docs, CLI docs, and a Workflow API, so an interactive API reference is a fair axis to expect, but the evidence pack shows no such feature—probes explicitly found no OpenAPI/swagger spec (404s across all candidate paths) and no mention of a runnable/interactive API console anywhere in the docs.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.zapier.com/openapi.json, https://docs.zapier.com/swagger.json, https://docs.zapier.com/…”
   • [claimed-docs] “Use Zapier's Workflow API and 9,000 integrations to power a built-in automation experience, integration marketplace, or AI workflows.”
   • [claimed-docs] “Zapier Platform CLI enables developers to build integrations using Node.js v22, OAuth1/OAuth2 authentication, webhooks, polling triggers, an…”

   proof ↗⚑ flag

   Windmill—none0/10

   The evidence pack shows an explicit probe for an OpenAPI/Swagger interactive reference that returned 404 on all candidate paths, and no other citation describes an interactive API reference with runnable examples (docs pages are static markdown/CLI/MCP references, not a runnable API explorer).

   • [probe] “PROBE openapi: all candidate paths 404 (https://www.windmill.dev/openapi.json, https://www.windmill.dev/swagger.json, https://www.windmill.d…”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   Zapier—none0/10

   The probe explicitly found all candidate OpenAPI spec paths returning 404, and no evidence pack item shows a downloadable OpenAPI/Swagger spec; Zapier's Workflow API and integration docs are mentioned but no machine-readable spec is provided. Missing for 10: a publicly downloadable OpenAPI/Swagger JSON or YAML file, any documentation page linking to such a spec.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.zapier.com/openapi.json, https://docs.zapier.com/swagger.json, https://docs.zapier.com/…”

   proof ↗⚑ flag

   Windmill—none0/10

   Windmill exposes a CLI and API-driven platform, so a downloadable OpenAPI spec is a fair ask, but the evidence pack explicitly shows a probe attempt failing to find any OpenAPI/swagger file at standard paths (all 404s), and no docs page links to a machine-readable spec.

   • [probe] “PROBE openapi: all candidate paths 404 (https://www.windmill.dev/openapi.json, https://www.windmill.dev/swagger.json, https://www.windmill.d…”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round drawn
   Zapier—none0/10

   No evidence pack item mentions a sandbox environment, staging mode, or test data isolation for Zapier's automation/MCP execution; the docs discuss OAuth, integrations, and MCP connectivity but nothing about testing against non-production data. missing for 10: any documentation of a sandbox/test mode, staging environment, or dry-run capability that avoids touching production data.

   ⚑ flag

   Windmill—none0/10

   The evidence pack covers Windmill's languages, orchestration, permissions, CLI, git sync, and MCP integration, but nowhere documents a dedicated sandbox/staging workspace or test-mode that isolates execution from production data; a community comment even raises this exact question (non-destructive branch testing) without an answered example.

   • [community] “My first questions with any new tool of this kind: Can I run/validate branch versions non-destructively as part of pre-merge checks? Can I r…”

   proof ↗⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   Zapier—none0/10

   No evidence of API versioning scheme or a documented deprecation policy; the OpenAPI probe returned 404s and no docs mention version numbers, sunset timelines, or breaking-change policy for Zapier's Platform API or Workflow API.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.zapier.com/openapi.json, https://docs.zapier.com/swagger.json, https://docs.zapier.com/…”

   proof ↗⚑ flag

   Windmill—none0/10

   Windmill exposes webhooks and version-pinned flow endpoints (windmill-docs-16), and a CLI/API surface exists, but there is no evidence of a documented API versioning scheme or deprecation policy; OpenAPI spec probes returned 404s (windmill-probe-3) and no docs mention deprecation practices.

   • [claimed-docs] “Flows additionally expose version-pinned webhook endpoints that target a specific flow version by its numeric version ID instead of the late…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://www.windmill.dev/openapi.json, https://www.windmill.dev/swagger.json, https://www.windmill.d…”

   proof ↗⚑ flag

Agent integration

1. ai-native userHave an agent create, update, and activate a workflow programmatically through the public API

   weight 2 · round to Windmill
   Zapier~partialprobed4/10

   Zapier's Developer Platform page mentions a 'Workflow API' that can power AI workflows and integration marketplaces, implying programmatic Zap creation, but the evidence pack has no concrete docs, endpoints, or examples showing an agent creating/updating/activating a Zap via this API — attempts to find an OpenAPI spec returned 404s. The strongest concrete API surface documented is the Platform CLI/SDK (for building app integrations) and MCP (for invoking existing actions), neither of which directly evidences full CRUD+activate control over workflows. Missing for 10: documented Workflow API reference/schema, example of programmatic Zap creation/update/activation, and independent confirmation it works as described.

   • [claimed-docs] “Use Zapier's Workflow API and 9,000 integrations to power a built-in automation experience, integration marketplace, or AI workflows.”
   • [claimed-docs] “It lets your AI take real actions across 9,000+ apps and 40,000+ actions, such as sending messages, creating tasks, and updating records”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.zapier.com/openapi.json, https://docs.zapier.com/swagger.json, https://docs.zapier.com/…”

   proof ↗⚑ flag

   Windmill~partialprobed5/10

   Windmill exposes multiple programmatic paths (CLI `wmill`, git-sync auto-deploy, version-pinned webhooks, and an MCP server that lets LLM clients trigger scripts/flows) that together suggest an agent could manage workflows without the UI, and AI-generation docs show agents can edit items and reach a deploy step. However, the story specifically requires create/update/activate via the public API, and the evidence never surfaces a documented REST/OpenAPI spec (the openapi.json probe 404'd on all candidate paths) nor confirms MCP or CLI can create/activate flows rather than just trigger existing ones. missing for 10: explicit public API/OpenAPI docs for flow CRUD, confirmation that agent-driven creation/activation (not just triggering) works end-to-end without human deploy step.

   • [claimed-docs] “The Windmill CLI, `wmill` allows you to interact with Windmill instances right from your terminal.”
   • [claimed-docs] “Each time an item is deployed, Windmill will create and push a commit to the specified repository. ... Windmill can automatically deploy new…”
   • [claimed-docs] “Flows additionally expose version-pinned webhook endpoints that target a specific flow version by its numeric version ID instead of the late…”
   • [claimed-docs] “With MCP, you can connect your favorite LLMs (like Claude, Cursor, or any MCP compatible client) to Windmill, allowing you to trigger your s…”
   • [claimed-docs] “Review and deploy changes: Once the AI has edited items, ask it to review the changes and it opens the Compare & Deploy page with exactly th…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://www.windmill.dev/openapi.json, https://www.windmill.dev/swagger.json, https://www.windmill.d…”
   • [probe] “official MCP server documented at https://www.windmill.dev/docs/core_concepts/mcp”
   • [probe] “official CLI documented at https://www.windmill.dev/docs/advanced/cli”

   proof ↗⚑ flag

2. ai-native userExpose my workflows or connected app actions as MCP tools that an external agent can call

   weight 3 · round drawn
   Zapier✓fullprobed8/10

   Zapier ships an official MCP server (zapier.com/mcp) that exposes connected app actions across 9,000+ apps as callable MCP tools to any MCP-compatible client (Claude, ChatGPT, Cursor), with governed/managed access and account-level controls documented for IT admins. This directly matches the story of exposing workflow/app actions as MCP tools for external agents. missing for 10: independent hands-on verification of the MCP server's reliability/tool-call behavior beyond first-party docs.

   • [claimed-docs] “Connect Claude, ChatGPT, Cursor, and other AI tools to Gmail, Slack, Salesforce, and 9,000+ apps. No code. No setup headaches.”
   • [claimed-docs] “Give any MCP-compatible AI client governed access to 9,000+ apps through one connection: no code, no infrastructure, SOC 2 certified.”
   • [claimed-docs] “Zapier MCP gives IT one governed connection across every AI agent your org uses: Claude, ChatGPT, Cursor. Account-level restrictions, manage…”
   • [claimed-docs] “It lets your AI take real actions across 9,000+ apps and 40,000+ actions, such as sending messages, creating tasks, and updating records”
   • [probe] “official MCP server documented at https://zapier.com/mcp”

   proof ↗⚑ flag

   Windmill✓fullprobed8/10

   Windmill has documented, first-party MCP server support that exposes scripts/flows as tools callable by external MCP-compatible agents (Claude, Cursor, etc.), directly matching the story. missing for 10: independent/hands-on confirmation of the MCP feature working, and more detail on scoping which specific workflows/actions are exposed as tools.

   • [claimed-docs] “With MCP, you can connect your favorite LLMs (like Claude, Cursor, or any MCP compatible client) to Windmill, allowing you to trigger your s…”
   • [probe] “official MCP server documented at https://www.windmill.dev/docs/core_concepts/mcp”

   proof ↗⚑ flag

Ai authoring

1. ai-native userGenerate or edit a workflow from a natural-language prompt

   weight 2 · round to Windmill
   Zapier~partialclaimed5/10

   Zapier documents 'Zapier Copilot' as an AI assistant to help create agents/workflows and mentions AI-generated troubleshooting instructions for Zaps, indicating some natural-language workflow creation/editing capability, but the evidence pack lacks concrete detail on how Copilot actually builds or edits a full multi-step Zap from a prompt, and there's no independent/hands-on corroboration of this specific capability. missing for 10: detailed docs/demo of Copilot generating or editing a full Zap workflow, independent user reports confirming it works as described.

   • [claimed-docs] “Create specialized agents—with the help of Zapier Copilot—to connect to your business data and perform tasks across 9,000+ apps.”
   • [claimed-docs] “Zapier will use AI to create troubleshooting instructions for you to review.”
   • [claimed-docs] “Describe what you need in plain language. Your AI calls the right Zapier action and confirms what it did.”

   proof ↗⚑ flag

   Windmill~partialclaimed6/10

   windmill-docs-21 explicitly documents AI-driven editing of scripts/flows/apps followed by a review-and-deploy step, and windmill-docs-20 shows AI agent steps can be embedded in flows, indicating genuine natural-language generation/editing support. However, evidence is purely first-party docs with no independent/hands-on corroboration of prompt-to-workflow quality or reliability. Missing for 10: independent/community validation of AI-generated flow quality, details on scope/limits of NL-to-workflow generation, and confirmation this works for full flow creation (not just editing existing items).

   • [claimed-docs] “Review and deploy changes: Once the AI has edited items, ask it to review the changes and it opens the Compare & Deploy page with exactly th…”
   • [claimed-docs] “AI Agents steps allow you to integrate AI agents directly into your Windmill flows. They provide an interface to connect with various AI pro…”

   proof ↗⚑ flag

Ai steps

1. ai-native userAdd AI agent or LLM steps inside a workflow, with model choice and tool use

   weight 3 · round to Windmill
   Zapier~partialclaimed5/10

   Zapier's agents.zapier.com/agents page and Copilot indicate you can create AI agents that connect to business data and perform tasks across thousands of apps, implying LLM+tool-use steps inside workflows, but the evidence never specifies model choice (e.g., selecting GPT vs Claude) or granular tool-use configuration within a Zap step. missing for 10: explicit documentation of selectable LLM/model options inside a workflow step, detailed tool-use/function-calling configuration, and independent hands-on confirmation of the AI-agent-in-workflow experience.

   • [claimed-docs] “Create specialized agents—with the help of Zapier Copilot—to connect to your business data and perform tasks across 9,000+ apps.”
   • [claimed-docs] “It lets your AI take real actions across 9,000+ apps and 40,000+ actions, such as sending messages, creating tasks, and updating records”
   • [claimed-docs] “Zapier will use AI to create troubleshooting instructions for you to review.”

   proof ↗⚑ flag

   Windmill✓fullclaimed8/10

   Windmill has documented AI Agent steps that integrate directly into flows, connecting to various AI providers/models and supporting tool use within the workflow orchestration engine (windmill-docs-20). This directly matches the story of adding LLM/agent steps with model choice and tool use inside a workflow. Missing for 10: independent/hands-on corroboration of tool-use behavior and broader detail on model selection UX beyond the single docs page.

   • [claimed-docs] “AI Agents steps allow you to integrate AI agents directly into your Windmill flows. They provide an interface to connect with various AI pro…”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round to Windmill
   Zapier—none0/10

   The evidence pack describes single-trigger, per-event automation (webhooks, triggers/actions, MCP calls) and even highlights community complaints about execution limits and workflows breaking down at complexity, but nothing documents batch/bulk item processing across many records at once.

   • [community] “We see a number of users sign up for our service after having tried Zapier to solve their problem... simple 'if this then that' types of wor…”
   • [community] “For me it was that IFTTT allows only one IF condition and one THEN step, whereas Zapier allows more simultaneous IFs & THENs. IFTTT is free …”
   • [claimed-docs] “It lets your AI take real actions across 9,000+ apps and 40,000+ actions, such as sending messages, creating tasks, and updating records”

   proof ↗⚑ flag

   Windmill~partialclaimed5/10

   Windmill's flows support parallelism, branching, and fault-tolerant orchestration (windmill-docs-5, windmill-docs-12), which implies capability to iterate/process many items in a flow, but no evidence explicitly documents a for-loop/bulk-iteration step, batch processing UI, or examples of running an operation across a large item set. missing for 10: explicit for-loop/iterator flow step documentation, evidence of scaling to large item counts, and any hands-on confirmation of bulk operation performance.

   • [claimed-docs] “Workflows as code let you define orchestration logic directly in TypeScript or Python, using familiar language constructs like functions, co…”
   • [claimed-docs] “Branches allow to split the execution of the flow based on a condition. ... Branch all: all the branches will be executed.”
   • [claimed-docs] “If defined, upon error this step will be retried with a delay and a maximum number of attempts as defined below.”

   proof ↗⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round drawn
   Zapier✓fullcommunity8/10

   Zapier's core product is trigger-action automation (Zaps): defining rules that fire actions when events occur across 9,000+ apps, with visual builder or CLI, plus AI-native surfaces (MCP, agent skills) that let AI tools invoke these automations. Community evidence corroborates real-world use for exactly this purpose, though some note limits on conditional complexity for advanced logic. Missing for 10: independent benchmarking of reliability/complexity limits and clearer first-party documentation of advanced conditional/multi-step trigger logic.

   • [claimed-docs] “Build a Zapier integration - design triggers and actions for your product, wire up authentication, then publish to the App Directory.”
   • [claimed-docs] “Simplify and scale your processes with visual mapping—add owners, notes, and images to create structured, easy-to-follow systems.”
   • [claimed-docs] “It lets your AI take real actions across 9,000+ apps and 40,000+ actions, such as sending messages, creating tasks, and updating records”
   • [community] “The options were: bump a dev to build the integration, pay ~$50/month to enable a non-technical person to build it via Zapier, or don't get …”
   • [community] “We see a number of users sign up for our service after having tried Zapier to solve their problem... simple 'if this then that' types of wor…”
   • [community] “Resonate a lot with the gaps with Zapier, particularly around the need for escape hatches/conditional logic when needed.”

   proof ↗⚑ flag

   Windmill✓fullcommunity8/10

   Windmill lets users define scripts/flows triggered automatically by schedules (CRON), webhooks, emails, HTTP routes, websockets, Kafka, Postgres, NATS, SQS, MQTT, and other event sources, with retry/error handling and branching logic for rule-based automation. This directly matches the story of defining rules that trigger actions on events, with strong first-party documentation. Missing for 10: independent/hands-on verification of complex event-rule chains and no community corroboration specifically confirming event-trigger reliability in production.

   • [claimed-docs] “How do I trigger scripts and flows in Windmill? Use schedules, webhooks, emails, HTTP routes, websockets, Kafka, Postgres, NATS, SQS, MQTT, …”
   • [claimed-docs] “A Schedule consists of a Script or Flow, its arguments, a CRON expression that controls the execution frequency and optional Error and Recov…”
   • [claimed-docs] “Flows additionally expose version-pinned webhook endpoints that target a specific flow version by its numeric version ID instead of the late…”
   • [claimed-docs] “If defined, upon error this step will be retried with a delay and a maximum number of attempts as defined below.”
   • [claimed-docs] “Branches allow to split the execution of the flow based on a condition. ... Branch all: all the branches will be executed.”
   • [community] “This looks really cool. Is there error handling? For example, if the email sending fails for some reason can I either select the flow to con…”

   proof ↗⚑ flag

3. ai-native userSchedule recurring jobs or workflows

   weight 2 · round to Windmill
   Zapier~partialclaimed4/10

   Zapier's platform is built around triggers/actions including polling triggers (zapier-docs-3) and Visual Builder/CLI workflow creation (zapier-docs-19, zapier-docs-22), which implicitly support recurring, trigger-based automations, but the evidence pack never shows an AI agent explicitly scheduling or configuring a recurring job through MCP — the MCP examples (zapier-docs-7, zapier-docs-20, zapier-docs-24) describe one-off actions like booking a single meeting or calling an action, not setting up persistent recurring workflows. Missing for 10: explicit documentation of an AI/MCP-driven interface for creating or managing scheduled/recurring Zaps, and independent confirmation that AI agents can configure recurrence intervals rather than just trigger single actions.

   • [claimed-docs] “Zapier Platform CLI enables developers to build integrations using Node.js v22, OAuth1/OAuth2 authentication, webhooks, polling triggers, an…”
   • [claimed-docs] “Use the Visual Builder for quick, no-code setups or the CLI for full control and customization.”
   • [claimed-docs] “the Visual Builder is the place to start. It’s ideal for quick deployment and collaboration across teams, even without developer expertise.”
   • [claimed-docs] “Schedule a meeting with one prompt. Your AI finds time on everyone's calendar and books it. No back and forth.”
   • [claimed-docs] “It lets your AI take real actions across 9,000+ apps and 40,000+ actions, such as sending messages, creating tasks, and updating records”

   proof ↗⚑ flag

   Windmill✓fullclaimed9/10

   Windmill has a dedicated Scheduling core concept supporting CRON-based recurring execution of scripts/flows with error/recovery handlers, plus flexible triggers (webhooks, events, etc.) and retries/branching for robust workflows, all documented first-party. missing for 10: no independent/hands-on community confirmation specifically of scheduling reliability in production.

   • [claimed-docs] “A Schedule consists of a Script or Flow, its arguments, a CRON expression that controls the execution frequency and optional Error and Recov…”
   • [claimed-docs] “How do I trigger scripts and flows in Windmill? Use schedules, webhooks, emails, HTTP routes, websockets, Kafka, Postgres, NATS, SQS, MQTT, …”
   • [claimed-docs] “If defined, upon error this step will be retried with a delay and a maximum number of attempts as defined below.”
   • [claimed-docs] “Branches allow to split the execution of the flow based on a condition. ... Branch all: all the branches will be executed.”

   proof ↗⚑ flag

4. ai-native userVersion, review, and roll back my automations

   weight 1 · round to Windmill
   Zapier—none0/10

   The evidence pack covers building integrations, MCP/agent access, embedding, and troubleshooting suggestions, but contains no mention of versioning Zaps, reviewing change history, or rolling back automations to a prior state.

   ⚑ flag

   Windmill~partialclaimed6/10

   Windmill documents versioning via Git sync (each deploy creates a commit, auto-deploy from repo) and version-pinned webhook endpoints for flows, plus a 'Compare & Deploy' review UI when AI edits items. However, there is no explicit rollback feature or UI documented (e.g., one-click revert to a prior version), and no independent/hands-on confirmation of rollback behavior. missing for 10: explicit rollback/revert mechanism, independent verification of version history UI and rollback workflow.

   • [claimed-docs] “Each time an item is deployed, Windmill will create and push a commit to the specified repository. ... Windmill can automatically deploy new…”
   • [claimed-docs] “Flows additionally expose version-pinned webhook endpoints that target a specific flow version by its numeric version ID instead of the late…”
   • [claimed-docs] “Review and deploy changes: Once the AI has edited items, ask it to review the changes and it opens the Compare & Deploy page with exactly th…”
   • [claimed-docs] “The Windmill CLI, `wmill` allows you to interact with Windmill instances right from your terminal.”

   proof ↗⚑ flag

Code steps

1. developerDrop into real code (JavaScript or Python) as a step inside a workflow

   weight 3 · round to Windmill
   Zapier—none0/10

   The evidence pack never documents a native 'Code by Zapier' step allowing inline JavaScript/Python inside a Zap; the closest content is about the CLI for building whole integrations (zapier-docs-3, zapier-docs-19), not an in-workflow code step. A community comment even complains about the lack of an easy way to 'write some if or just 2 lines of code' inside Zaps (zapier-comm-11, zapier-comm-13), reinforcing that this specific capability isn't evidenced.

   • [claimed-docs] “Use the Visual Builder for quick, no-code setups or the CLI for full control and customization.”
   • [claimed-docs] “Zapier Platform CLI enables developers to build integrations using Node.js v22, OAuth1/OAuth2 authentication, webhooks, polling triggers, an…”
   • [community] “As an engineer using Zapier (for the first time) these past few months I got so frustrated by the lack of an option to just write some 'if' …”
   • [community] “Resonate a lot with the gaps with Zapier, particularly around the need for escape hatches/conditional logic when needed.”

   proof ↗⚑ flag

   Windmill✓fullcommunity9/10

   Windmill's core model is scripts written in real code (TypeScript, Python, Go, etc.) composed into flows, with 'workflows as code' letting developers write orchestration logic directly in TypeScript/Python using native functions, conditionals, and loops as flow steps. This is corroborated by GitHub docs describing scripts being composed into flows, and community discussion confirms multi-language script support. Missing for 10: independent hands-on verification beyond community anecdotes and no mention of limitations in language runtime sandboxing raised by a commenter.

   • [claimed-docs] “It supports coding in TypeScript, Python, Go, PHP, Bash, C#, SQL, Rust, Ruby and R, or any Docker image”
   • [claimed-docs] “Workflows as code let you define orchestration logic directly in TypeScript or Python, using familiar language constructs like functions, co…”
   • [claimed-docs] “An orchestrator for assembling these functions into efficient, low-latency flows, using either a low-code builder or YAML”
   • [github] “Scripts are turned into sharable UIs automatically, and can be composed together into flows or used into richer apps built with low-code.”
   • [community] “Was hoping it allowed my runtime. I have a bunch of scripts in NodeJS, Python 2, C#, etc. I don't care to rewrite them and/or write scripts …”

   proof ↗⚑ flag

Connector dev

1. developerBuild a custom connector or private integration with a documented developer platform, SDK, or CLI

   weight 2 · round to Zapier
   Zapier✓fullprobed9/10

   Zapier documents a full Platform CLI (Node.js, OAuth1/2, webhooks, polling triggers, HTTP middleware) plus a Visual Builder for building and publishing custom integrations to the App Directory, and an embedded/developer platform API for private integrations. Independent probe confirms the CLI docs page and developer llms.txt index exist, corroborating first-party docs. Missing for 10: independent third-party hands-on account of building a CLI integration end-to-end (community evidence covers using Zaps, not building CLI-based custom connectors).

   • [claimed-docs] “Zapier Platform CLI enables developers to build integrations using Node.js v22, OAuth1/OAuth2 authentication, webhooks, polling triggers, an…”
   • [claimed-docs] “Use the Visual Builder for quick, no-code setups or the CLI for full control and customization.”
   • [claimed-docs] “the Visual Builder is the place to start. It’s ideal for quick deployment and collaboration across teams, even without developer expertise.”
   • [claimed-docs] “Build a Zapier integration - design triggers and actions for your product, wire up authentication, then publish to the App Directory.”
   • [claimed-docs] “Use Zapier's Workflow API and 9,000 integrations to power a built-in automation experience, integration marketplace, or AI workflows.”
   • [probe] “official CLI documented at https://docs.zapier.com/integrations/build-cli/overview”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.zapier.com/llms.txt # Zapier — docs.zapier.com > Local index for developer documentation: SDK, MCP…”

   proof ↗⚑ flag

   Windmill✓fullprobed8/10

   Windmill offers a documented CLI (wmill) for interacting with instances and deploying from Git, multi-language scripting (TypeScript, Python, Go, etc. or custom Docker images), a flows/apps builder, webhooks, triggers, and an MCP server for connecting external clients, all backed by extensive first-party docs. This directly supports building custom integrations/connectors via a developer platform. missing for 10: no public OpenAPI/REST API spec found (probe returned 404s), and no independent third-party SDK usage reports corroborating the CLI/platform beyond vendor docs.

   • [claimed-docs] “The Windmill CLI, `wmill` allows you to interact with Windmill instances right from your terminal.”
   • [claimed-docs] “It supports coding in TypeScript, Python, Go, PHP, Bash, C#, SQL, Rust, Ruby and R, or any Docker image”
   • [claimed-docs] “Windmill supports both UI-based operations via its webIDE and low-code builders, as well as CLI deployments from a Git repository”
   • [claimed-docs] “Each time an item is deployed, Windmill will create and push a commit to the specified repository. ... Windmill can automatically deploy new…”
   • [claimed-docs] “How do I trigger scripts and flows in Windmill? Use schedules, webhooks, emails, HTTP routes, websockets, Kafka, Postgres, NATS, SQS, MQTT, …”
   • [claimed-docs] “With MCP, you can connect your favorite LLMs (like Claude, Cursor, or any MCP compatible client) to Windmill, allowing you to trigger your s…”
   • [probe] “official MCP server documented at https://www.windmill.dev/docs/core_concepts/mcp”
   • [probe] “official CLI documented at https://www.windmill.dev/docs/advanced/cli”
   • [probe] “PROBE openapi: all candidate paths 404 (https://www.windmill.dev/openapi.json, https://www.windmill.dev/swagger.json, https://www.windmill.d…”

   proof ↗⚑ flag

Data mapping

1. ops userMap and transform data between steps with expressions, functions, or formulas

   weight 2 · round to Windmill
   Zapier!disputedcontradicted4/10

   Zapier's docs gesture at building and mapping data through the Visual Builder or CLI (zapier-docs-19, zapier-docs-22) and Canvas process mapping (zapier-docs-9, zapier-docs-18), but none of the evidence explicitly documents formula/expression/function-based field mapping between steps. Community evidence directly contradicts smooth delivery of this capability: an engineer using Zapier reports being 'frustrated by the lack of an option to just write some if or just 2 lines of code' and another user 'resonates a lot with the gaps... around the need for escape hatches/conditional logic' (zapier-comm-11, zapier-comm-13), and a separate user notes complex logic 'starts to break down' (zapier-comm-3). Missing for 10: explicit vendor documentation of Formatter/Code-by-Zapier style expression/function steps and independent confirmation they work well for complex transformations.

   • [claimed-docs] “Use the Visual Builder for quick, no-code setups or the CLI for full control and customization.”
   • [claimed-docs] “the Visual Builder is the place to start. It’s ideal for quick deployment and collaboration across teams, even without developer expertise.”
   • [claimed-docs] “Simplify and scale your processes with visual mapping—add owners, notes, and images to create structured, easy-to-follow systems.”
   • [claimed-docs] “Canvas automatically maps and optimizes your processes to help your team document, refine, and automate with ease.”
   • [community] “As an engineer using Zapier (for the first time) these past few months I got so frustrated by the lack of an option to just write some 'if' …”
   • [community] “Resonate a lot with the gaps with Zapier, particularly around the need for escape hatches/conditional logic when needed.”
   • [community] “We see a number of users sign up for our service after having tried Zapier to solve their problem... simple 'if this then that' types of wor…”

   proof ↗⚑ flag

   Windmill✓fullclaimed8/10

   Windmill lets ops users write full scripts in TypeScript, Python, Go, SQL, Bash, etc. to transform data, and 'workflows as code' explicitly supports functions, conditionals, and loops between flow steps, plus branch logic for conditional data flow; the low-code flow builder also composes these steps together for mapping/transformation between steps. Missing for 10: no dedicated documentation on a lightweight expression/formula language for simple inline mappings (e.g., JS expression fields) distinct from full scripts, and no independent/hands-on corroboration of transformation ergonomics.

   • [claimed-docs] “It supports coding in TypeScript, Python, Go, PHP, Bash, C#, SQL, Rust, Ruby and R, or any Docker image”
   • [claimed-docs] “An orchestrator for assembling these functions into efficient, low-latency flows, using either a low-code builder or YAML”
   • [claimed-docs] “Workflows as code let you define orchestration logic directly in TypeScript or Python, using familiar language constructs like functions, co…”
   • [claimed-docs] “Branches allow to split the execution of the flow based on a condition. ... Branch all: all the branches will be executed.”
   • [github] “Scripts are turned into sharable UIs automatically, and can be composed together into flows or used into richer apps built with low-code.”

   proof ↗⚑ flag

Credentials

1. ops userStore connection credentials centrally, share them with my team, and control who can use which credential

   weight 3 · round drawn
   Zapier~partialclaimed6/10

   Zapier docs confirm centralized OAuth/credential handling and reuse of existing 'connections' across the org, plus 'managed connections' and 'workspace controls' governing access for MCP/agent use (zapier-docs-15, zapier-docs-12, zapier-docs-16). However, there's no detailed documentation of granular per-credential sharing/permission assignment (e.g., which specific team members can use which specific credential) beyond high-level 'account-level restrictions'. Missing for 10: explicit admin UI/workflow for assigning credential-level permissions, role-based access control details, and independent corroboration of these governance claims.

   • [claimed-docs] “Zapier MCP gives IT one governed connection across every AI agent your org uses: Claude, ChatGPT, Cursor. Account-level restrictions, manage…”
   • [claimed-docs] “Zapier brings in your existing app connections automatically. Gmail, Slack, Salesforce, whatever you've already set up in Zapier is ready to…”
   • [claimed-docs] “Zapier handles OAuth, credentials, and API complexity so you never touch a token.”
   • [claimed-docs] “Give any MCP-compatible AI client governed access to 9,000+ apps through one connection: no code, no infrastructure, SOC 2 certified.”

   proof ↗⚑ flag

   Windmill~partialclaimed6/10

   Windmill documents centrally encrypted Variables/Secrets at the workspace level plus a broader roles/permissions system and scoped tokens for least-privilege access, which together support storing and sharing credentials with some access control. However, the evidence doesn't show fine-grained per-credential ACLs (e.g., which specific users/groups can use a specific secret) beyond general workspace/role permissions. Missing for 10: explicit per-resource/credential-level permission granularity, audit logging of credential usage, and independent/hands-on confirmation of this governance in practice.

   • [claimed-docs] “All Variables (not just secrets) are encrypted with a workspace specific symmetric key to avoid leakage.”
   • [claimed-docs] “Windmill provides a roles and permissions system that allows you to control access and manage permissions within your instance and workspace…”
   • [claimed-docs] “Tokens can be scoped to restrict access to specific resources and actions, following the principle of least privilege.”

   proof ↗⚑ flag

Human in the loop

1. ops userPause a workflow to wait for human approval or input before it continues

   weight 2 · round drawn
   Zapier—none0/10

   No evidence in the pack mentions Zapier's built-in 'Human in the Loop'/approval step, delay/pause-for-input actions, or any pause-and-resume-for-approval workflow feature; the pack covers integrations, MCP, canvas, and platform building only.

   ⚑ flag

   Windmill—none0/10

   The evidence pack covers flow branching, retries, error handling, and scheduling, but contains no mention of a suspend/approval step, human-in-the-loop gate, or 'wait for input' mechanism in Windmill flows. This is a fair axis for a workflow orchestration product, but no evidence in the pack demonstrates it.

   • [claimed-docs] “Branches allow to split the execution of the flow based on a condition. ... Branch all: all the branches will be executed.”
   • [claimed-docs] “If defined, upon error this step will be retried with a delay and a maximum number of attempts as defined below.”
   • [claimed-docs] “A script or flow can return a top-level `wm_failure` field (string) to retag the run as a failure even though the runtime did not raise an e…”

   proof ↗⚑ flag

Versioning

1. developerVersion workflows through source control or environments and promote changes from dev to production

   weight 2 · round to Windmill
   Zapier—none0/10

   Evidence covers building integrations via CLI/Visual Builder and embedding/MCP access, but nothing describes version control integration, environments (dev/staging/prod), or promotion/deployment pipelines for Zap workflows. No mention of git-based versioning, environment variables per stage, or a promote-to-production workflow mechanism.

   ⚑ flag

   Windmill✓fullcommunity8/10

   Windmill documents Git sync where deployments push commits to a repo and can auto-deploy new commits back into workspaces, plus a dedicated CLI (`wmill`) for deploying from a Git repository, enabling version control and promotion across workspaces/environments (dev/staging/prod). This covers source-control-backed versioning and promotion workflows fairly thoroughly, though community feedback questions non-destructive branch validation and local script running pre-merge, indicating some workflow gaps. Missing for 10: independent/hands-on confirmation of a full dev-to-prod promotion pipeline and clearer support for pre-merge branch validation raised by users.

   • [claimed-docs] “Each time an item is deployed, Windmill will create and push a commit to the specified repository. ... Windmill can automatically deploy new…”
   • [claimed-docs] “The Windmill CLI, `wmill` allows you to interact with Windmill instances right from your terminal.”
   • [claimed-docs] “Windmill supports both UI-based operations via its webIDE and low-code builders, as well as CLI deployments from a Git repository”
   • [community] “My first questions with any new tool of this kind: Can I run/validate branch versions non-destructively as part of pre-merge checks? Can I r…”

   proof ↗⚑ flag

Community

1. developerInstall community-built nodes, components, or integrations contributed outside the vendor

   weight 1 · round to Zapier
   Zapier~partialclaimed5/10

   Zapier's Platform CLI lets third-party developers build integrations and publish them to the App Directory (zapier-docs-1, zapier-docs-3, zapier-docs-19), and docs mention agents discovering Zapier-built skills via npm/skill registries outside Zapier's direct distribution (zapier-docs-4). But these are vendor-reviewed App Directory listings, not an open community-contribution marketplace akin to freely shared nodes; there's no evidence of a permissionless community repository or independent developer reports of installing unofficial/community nodes. Missing for 10: evidence of an open, unreviewed community marketplace, independent confirmation of installing non-vendor-approved integrations, and details on moderation/openness of the contribution process.

   • [claimed-docs] “Build a Zapier integration - design triggers and actions for your product, wire up authentication, then publish to the App Directory.”
   • [claimed-docs] “Zapier Platform CLI enables developers to build integrations using Node.js v22, OAuth1/OAuth2 authentication, webhooks, polling triggers, an…”
   • [claimed-docs] “Agents discover them on npm and skill registries, then run them as an agent skill, a CLI, a typed Node module, or a local MCP server.”
   • [claimed-docs] “Use the Visual Builder for quick, no-code setups or the CLI for full control and customization.”
   • [claimed-docs] “the Visual Builder is the place to start. It’s ideal for quick deployment and collaboration across teams, even without developer expertise.”

   proof ↗⚑ flag

   Windmill—none0/10

   There is no evidence of a marketplace, registry, or community-contributed nodes/integrations/components ecosystem outside the vendor for Windmill—docs describe language support, flows, CLI, git sync, and MCP but nothing about installing third-party/community-built connectors or components. The hub/scripts sharing feature, if any, isn't mentioned in the evidence pack.

   ⚑ flag

Connectors

1. ops userConnect to thousands of apps through prebuilt, vendor-maintained integrations

   weight 3 · round to Zapier
   Zapier✓fullcommunity9/10

   Zapier provides thousands of prebuilt, vendor-maintained app integrations (9,000+ apps cited repeatedly) accessible via Visual Builder, MCP, and API, corroborated by independent community reports confirming the breadth (~1,000+) and ease of use of its integration catalog compared to competitors. Missing for 10: no independent third-party audit of the full current app count or comprehensive vendor-maintenance quality across all 9,000+ integrations.

   • [claimed-docs] “Give coding agents programmatic access to 9,000+ apps: Zapier handles OAuth, credentials, and API complexity so you never touch a token.”
   • [claimed-docs] “Connect Claude, ChatGPT, Cursor, and other AI tools to Gmail, Slack, Salesforce, and 9,000+ apps. No code. No setup headaches.”
   • [claimed-docs] “Zapier brings in your existing app connections automatically. Gmail, Slack, Salesforce, whatever you've already set up in Zapier is ready to…”
   • [claimed-docs] “It lets your AI take real actions across 9,000+ apps and 40,000+ actions, such as sending messages, creating tasks, and updating records”
   • [community] “Zapier has about 1,000 integrations, more than any of their closest competitors and at a price that is far less than others in their space.”
   • [community] “The options were: bump a dev to build the integration, pay ~$50/month to enable a non-technical person to build it via Zapier, or don't get …”

   proof ↗⚑ flag

   Windmill—none0/10

   Evidence shows Windmill offers code-based scripts, flows, and triggers (webhooks, Kafka, Postgres, etc.) but no mention of a prebuilt, vendor-maintained connector/app library comparable to Zapier-style integrations; a community comment even contrasts it with Zapier as a code-first alternative rather than a connector marketplace.

   • [claimed-docs] “How do I trigger scripts and flows in Windmill? Use schedules, webhooks, emails, HTTP routes, websockets, Kafka, Postgres, NATS, SQS, MQTT, …”
   • [community] “This is brilliant. The world has been needing something like Zapier, but not opaque and locked down. Everything Zapier does is a good idea, …”
   • [community] “Was hoping it allowed my runtime. I have a bunch of scripts in NodeJS, Python 2, C#, etc. I don't care to rewrite them and/or write scripts …”

   proof ↗⚑ flag

Templates

1. ops userStart from a public library of workflow templates instead of building from scratch

   weight 2 · round drawn
   Zapier—none0/10

   The evidence pack covers building integrations, MCP, embedding, and Canvas, but nowhere mentions a public gallery/library of pre-built workflow templates that ops users can start from instead of building from scratch. Without any documentation or community mention of a templates library, this applicable ecosystem capability is unevidenced.

   ⚑ flag

   Windmill—none0/10

   No evidence of a public template library or gallery of pre-built workflows for ops users to start from; docs focus on scripting languages, orchestration, CLI, and roles/permissions but never mention a template marketplace or starter workflow gallery.

   ⚑ flag

Embedding

1. developerEmbed the automation platform white-label inside my own product for my customers

   weight 1 · round to Zapier
   Zapier✓fullclaimed7/10

   Zapier's Developer Platform explicitly supports white-label embedding via its Workflow API and Embed offerings, letting customers run automations/workflows under the host product's own brand without leaving the app. missing for 10: independent/hands-on case studies of a real embedded white-label deployment, and pricing/setup details for the embed program.

   • [claimed-docs] “Embed Zapier's automation capabilities directly into your product so users can connect apps and run automations under your brand.”
   • [claimed-docs] “Bring Zapier’s automation into your own product so customers can build workflows and run actions without leaving your app.”
   • [claimed-docs] “Bring Zapier's automation into your own product so customers can build workflows and run actions without leaving your app.”
   • [claimed-docs] “Embed Zapier’s automation capabilities directly into your product so users can connect apps and run automations under your brand.”
   • [claimed-docs] “Use Zapier's Workflow API and 9,000 integrations to power a built-in automation experience, integration marketplace, or AI workflows.”

   proof ↗⚑ flag

   Windmill—none0/10

   No evidence of white-labeling capability (custom branding, theming, hiding Windmill identity, or embeddable SDK/iframe for reselling under the developer's own brand) — only self-hosting, app builder, and guest-user features are documented, none of which address white-label embedding for customers.

   ⚑ flag

Local dev

1. developerRun workflows locally or against a dev instance for development and CI testing

   weight 2 · round to Windmill
   Zapier—none0/10

   Evidence shows only cloud-hosted Zap execution and a CLI for building integration triggers/actions (with local scaffolding), but nothing about running actual Zap workflows locally or against a dedicated dev/staging instance for CI testing. Missing for 10: any documented local workflow execution mode, sandbox/dev environment for Zaps, or CI integration testing tooling.

   • [claimed-docs] “Zapier Platform CLI enables developers to build integrations using Node.js v22, OAuth1/OAuth2 authentication, webhooks, polling triggers, an…”
   • [claimed-docs] “Use the Visual Builder for quick, no-code setups or the CLI for full control and customization.”
   • [probe] “official CLI documented at https://docs.zapier.com/integrations/build-cli/overview”

   proof ↗⚑ flag

   Windmill~partialcommunity5/10

   Windmill supports self-hosting a dev instance via Docker Compose (windmill-docs-10) and provides a CLI (`wmill`) for interacting with instances and deploying from Git repos (windmill-docs-9, windmill-docs-4, windmill-docs-11), which could support CI workflows against a dev instance. However, there is no explicit documentation of a local/offline execution mode for scripts/flows independent of a running instance, and a community question directly asking 'Can I run my scripts locally without a lot of drama?' (windmill-comm-5) goes unanswered in the evidence, suggesting friction. Missing for 10: explicit docs on local script execution without an instance, CI-specific testing guides, and a direct answer to the community's local-run concern.

   • [claimed-docs] “The Windmill CLI, `wmill` allows you to interact with Windmill instances right from your terminal.”
   • [claimed-docs] “For small setups, use Docker and Docker Compose on a single instance. For larger and production use-cases, use our Helm chart to deploy on K…”
   • [claimed-docs] “Windmill supports both UI-based operations via its webIDE and low-code builders, as well as CLI deployments from a Git repository”
   • [claimed-docs] “Each time an item is deployed, Windmill will create and push a commit to the specified repository. ... Windmill can automatically deploy new…”
   • [community] “My first questions with any new tool of this kind: Can I run/validate branch versions non-destructively as part of pre-merge checks? Can I r…”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round to Windmill
   Zapier~partialprobed5/10

   Zapier documents a Workflow API, Platform CLI, and MCP server that let external/agent callers create integrations and trigger actions across 9,000+ apps, suggesting substantial programmatic parity with the UI. However, the probe found no public OpenAPI spec (all candidate URLs 404), and there is no evidence that all UI capabilities (e.g., full Zap editing, Canvas, troubleshooting AI) are exposed via API, so full UI/API parity is unconfirmed. Missing for 10: a documented OpenAPI/complete API reference, explicit statement of full CRUD parity for Zap management via API, and independent confirmation that all UI features are API-accessible.

   • [claimed-docs] “Use Zapier's Workflow API and 9,000 integrations to power a built-in automation experience, integration marketplace, or AI workflows.”
   • [claimed-docs] “Zapier Platform CLI enables developers to build integrations using Node.js v22, OAuth1/OAuth2 authentication, webhooks, polling triggers, an…”
   • [claimed-docs] “Use the Visual Builder for quick, no-code setups or the CLI for full control and customization.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.zapier.com/openapi.json, https://docs.zapier.com/swagger.json, https://docs.zapier.com/…”
   • [claimed-docs] “It lets your AI take real actions across 9,000+ apps and 40,000+ actions, such as sending messages, creating tasks, and updating records”

   proof ↗⚑ flag

   Windmill~partialprobed6/10

   Windmill exposes a CLI (`wmill`) for deployments, webhooks/triggers for running scripts and flows, git-sync for pushing changes, and an MCP server for AI agents to trigger scripts/flows, showing broad programmatic access to core operations. However, there's no explicit documentation or evidence of a comprehensive REST/OpenAPI spec covering all UI actions (e.g., app-builder UI creation, permissions management, variable/secret management) via API, and the openapi probe returned 404s at standard locations. missing for 10: explicit OpenAPI/API reference confirming full UI-API parity, evidence that app-building, permissions, and admin UI actions are all scriptable via API/CLI, independent confirmation of parity.

   • [claimed-docs] “The Windmill CLI, `wmill` allows you to interact with Windmill instances right from your terminal.”
   • [claimed-docs] “Each time an item is deployed, Windmill will create and push a commit to the specified repository. ... Windmill can automatically deploy new…”
   • [claimed-docs] “How do I trigger scripts and flows in Windmill? Use schedules, webhooks, emails, HTTP routes, websockets, Kafka, Postgres, NATS, SQS, MQTT, …”
   • [claimed-docs] “Flows additionally expose version-pinned webhook endpoints that target a specific flow version by its numeric version ID instead of the late…”
   • [claimed-docs] “With MCP, you can connect your favorite LLMs (like Claude, Cursor, or any MCP compatible client) to Windmill, allowing you to trigger your s…”
   • [probe] “official MCP server documented at https://www.windmill.dev/docs/core_concepts/mcp”
   • [probe] “official CLI documented at https://www.windmill.dev/docs/advanced/cli”
   • [probe] “PROBE openapi: all candidate paths 404 (https://www.windmill.dev/openapi.json, https://www.windmill.dev/swagger.json, https://www.windmill.d…”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round to Windmill
   Zapier—none0/10

   No evidence of any data export feature (e.g., exporting Zaps, workflow configs, or automation history in open/portable formats) that would let a user leave the platform with their data intact; evidence focuses on integrations, MCP, and building automations, not on data portability/export.

   ⚑ flag

   Windmill~partialprobed6/10

   Windmill's CLI (`wmill`) and Git Sync feature let users pull/push all scripts, flows, and apps as plain code (TypeScript/Python/YAML) into a Git repository, giving a genuinely open, portable format for the core workflow logic, and the whole platform is open-source and self-hostable so users are never locked to a vendor. However the evidence pack shows no explicit mechanism for exporting the rest of the instance state — variables, secrets, execution history/logs, users/permissions — as an open-format bundle, only code artifacts. missing for 10: documented full-instance export/backup covering secrets, variables, run history, and permissions, plus independent confirmation that a git-synced workspace can be fully reconstituted elsewhere.

   • [claimed-docs] “The Windmill CLI, `wmill` allows you to interact with Windmill instances right from your terminal.”
   • [claimed-docs] “Each time an item is deployed, Windmill will create and push a commit to the specified repository. ... Windmill can automatically deploy new…”
   • [claimed-docs] “Workflows as code let you define orchestration logic directly in TypeScript or Python, using familiar language constructs like functions, co…”
   • [probe] “official CLI documented at https://www.windmill.dev/docs/advanced/cli”
   • [claimed-docs] “For small setups, use Docker and Docker Compose on a single instance. For larger and production use-cases, use our Helm chart to deploy on K…”

   proof ↗⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round to Windmill
   Zapier—none0/10

   Zapier is a closed-source, proprietary SaaS platform; no evidence of any open-license source availability (only CLI/SDK docs for building integrations, not the product's own source). This is an applicable axis for judging openness but no evidence supports it, so verdict is none.

   ⚑ flag

   Windmill✓fullprobed7/10

   Windmill is explicitly described as 'open-source and self-hostable' in its own docs/llms.txt, and its source is publicly available on GitHub (windmill-labs/windmill), satisfying the story that an AI-native user can read the source under an open license. missing for 10: no explicit license name (e.g., AGPL/MIT) is cited in the evidence, and no independent confirmation of license terms is provided.

   • [probe] “PROBE llms.txt: HTTP 200 at https://www.windmill.dev/llms.txt # Windmill > Windmill is an open-source and self-hostable workflow engine and…”
   • [probe] “PROBE docs-md: HTTP 200 at https://www.windmill.dev/docs/intro.md # What is Windmill? > What is Windmill? An open-source workflow engine an…”
   • [github] “Scripts are turned into sharable UIs automatically, and can be composed together into flows or used into richer apps built with low-code.”
   • [community] “This looks pretty neat! Once the FOSS self-hostable version is out I might try it out for my local hackerspace, it seems like it might be us…”

   proof ↗⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round to Windmill
   Zapier—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

   Windmill✓fullprobed9/10

   Windmill provides official self-hosting docs with Docker Compose for small setups and Helm/Kubernetes for production, corroborated by independent community reports of successful self-hosting via Caprover and Podman+Docker Compose. Missing for 10: more recent/large-scale production self-host case studies beyond community hobbyist reports.

   • [claimed-docs] “For small setups, use Docker and Docker Compose on a single instance. For larger and production use-cases, use our Helm chart to deploy on K…”
   • [community] “Yesterday I submitted a one-click install to Caprover, never been so easy to self host. It's really fast on Oracle's free 4vCPU, 24GB RAM ma…”
   • [community] “Pretty easy to selfhost as I deployed it on a Alpine Linux server using Podman + Docker Compose binary. Really nice piece of software.”
   • [probe] “PROBE llms.txt: HTTP 200 at https://www.windmill.dev/llms.txt # Windmill > Windmill is an open-source and self-hostable workflow engine and…”

   proof ↗⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round to Windmill
   Zapier—none0/10

   No evidence in the pack mentions data residency, region selection, or storage location controls for Zapier's automation or MCP data; SOC 2 certification is mentioned but that addresses security compliance, not residency choice.

   ⚑ flag

   Windmill~partialcommunity4/10

   Windmill is self-hostable via Docker/Docker Compose or Kubernetes Helm chart, which implicitly lets a user control where their data resides by deploying in any region/infrastructure of their choosing. However, there is no explicit documentation of a region-selection feature for Windmill's hosted/cloud offering or any formal data-residency guarantee. Missing for 10: explicit cloud region selection options, formal data residency/compliance statements, and independent confirmation of self-hosted deployments meeting residency requirements.

   • [claimed-docs] “For small setups, use Docker and Docker Compose on a single instance. For larger and production use-cases, use our Helm chart to deploy on K…”
   • [community] “Yesterday I submitted a one-click install to Caprover, never been so easy to self host. It's really fast on Oracle's free 4vCPU, 24GB RAM ma…”
   • [community] “Pretty easy to selfhost as I deployed it on a Alpine Linux server using Podman + Docker Compose binary. Really nice piece of software.”

   proof ↗⚑ flag

2. ai-native userPrevent my data from being used to train AI models

   weight 3 · round drawn
   Zapier—none0/10

   No evidence pack item addresses AI training data usage, opt-out controls, or data privacy policies regarding model training; only SOC 2 certification is mentioned in passing without connection to training data opt-out.

   ⚑ flag

   Windmill—none0/10

   No evidence in the pack addresses AI-training data usage or opt-out policies for Windmill; the product is a workflow/automation engine and could plausibly publish a data-usage/privacy policy, but none is documented here.

   ⚑ flag

3. ai-native userControl data retention and deletion

   weight 2 · round drawn
   Zapier—none0/10

   No evidence in the pack addresses data retention policies, deletion controls, or data lifecycle management for AI-native users; the pack only covers integrations, MCP connectivity, and automation features. Mentions of SOC 2 certification touch security posture but not retention/deletion controls specifically.

   ⚑ flag

   Windmill—none0/10

   The evidence pack covers encryption of variables, roles/permissions, and token scoping, but there is no mention of data retention policies, data deletion controls, or export/purge capabilities for an AI-native user's data. No documentation or community evidence addresses this axis.

   ⚑ flag

4. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   Zapier—none0/10

   No evidence in the pack addresses telemetry/usage-tracking opt-out settings for Zapier; nothing about privacy controls, data collection preferences, or analytics opt-out is documented. missing for 10: any documentation of telemetry collection, an opt-out toggle/setting, or a privacy policy statement addressing usage tracking controls.

   ⚑ flag

   Windmill—none0/10

   No evidence in the pack mentions telemetry, usage tracking, or an opt-out mechanism for Windmill; the docs focus on features, self-hosting, and permissions but never address telemetry data collection. Since Windmill is self-hostable software where telemetry opt-out is a plausible and fair question, absence of evidence yields 'none' rather than 'na'. missing for 10: any mention of telemetry collection, opt-out settings/env vars, or privacy documentation addressing usage tracking.

   ⚑ flag

Durability

1. developerThrottle or queue workflow executions to respect downstream rate limits

   weight 1 · round drawn
   Zapier—none0/10

   No evidence pack item mentions throttling, rate-limit handling, queuing, or delay/concurrency controls for workflow executions; docs focus on building integrations, MCP, embedding, and app connections. Community threads even highlight breakage/complexity issues but nothing about rate-limit throttling or queue controls.

   ⚑ flag

   Windmill—none0/10

   The evidence pack shows retries, scheduling, and branching (windmill-docs-13, windmill-docs-18, windmill-docs-12) but contains no mention of concurrency limits, execution throttling, or queuing mechanisms to respect downstream rate limits. Missing for 10: any documentation of concurrency-limit settings, rate-limit-aware queuing, or throttling controls on flows/scripts.

   • [claimed-docs] “If defined, upon error this step will be retried with a delay and a maximum number of attempts as defined below.”
   • [claimed-docs] “A Schedule consists of a Script or Flow, its arguments, a CRON expression that controls the execution frequency and optional Error and Recov…”

   proof ↗⚑ flag

2. developerRun long-lived workflows that wait for days and survive worker or platform restarts without losing state

   weight 2 · round to Windmill
   Zapier—none0/10

   The evidence pack contains no documentation or reports about Zapier's workflow execution durability, delay/wait steps surviving restarts, or state persistence guarantees; community comments even suggest complex workflows 'break down' and that Zapier is 'too high level' compared to durable-execution tools like Temporal, but no concrete confirmation or denial of long-lived, restart-surviving workflows is present.

   • [community] “Zapier (and similar) is just a little too high level for a lot of our needs, but Temporal feels like overkill.”
   • [community] “We see a number of users sign up for our service after having tried Zapier to solve their problem... simple 'if this then that' types of wor…”

   proof ↗⚑ flag

   Windmill~partialclaimed5/10

   Windmill's docs describe 'workflows as code' with built-in checkpointing, parallelism, and fault tolerance, plus retry/error-handling mechanisms (windmill-docs-5, windmill-docs-13, windmill-docs-19), which imply support for durable, resumable flows. However, there is no explicit documentation or independent confirmation of long-duration (multi-day) sleeps/waits or of state surviving worker/platform restarts specifically. Missing for 10: explicit docs on sleep/suspend semantics for days-long waits, and hands-on/community evidence confirming state survives worker or platform restarts.

   • [claimed-docs] “Workflows as code let you define orchestration logic directly in TypeScript or Python, using familiar language constructs like functions, co…”
   • [claimed-docs] “If defined, upon error this step will be retried with a delay and a maximum number of attempts as defined below.”
   • [claimed-docs] “A script or flow can return a top-level `wm_failure` field (string) to retag the run as a failure even though the runtime did not raise an e…”
   • [claimed-docs] “A Schedule consists of a Script or Flow, its arguments, a CRON expression that controls the execution frequency and optional Error and Recov…”

   proof ↗⚑ flag

Error handling

1. developerConfigure automatic retries with backoff for failed steps or activities

   weight 3 · round to Windmill
   Zapier—none0/10

   No evidence in the pack mentions retry policies, backoff configuration, or error-handling settings for failed Zap steps or actions; the only related item is generic AI-generated troubleshooting instructions, which is not the same as configurable retry/backoff.

   • [claimed-docs] “Zapier will use AI to create troubleshooting instructions for you to review.”

   proof ↗⚑ flag

   Windmill✓fullclaimed9/10

   Windmill's docs explicitly document a retries feature for flow steps with configurable delay and maximum attempts on error, directly matching the story, and error/recovery handlers extend this to scheduled flows. Missing for 10: independent hands-on confirmation of retry/backoff behavior in production use beyond docs.

   • [claimed-docs] “If defined, upon error this step will be retried with a delay and a maximum number of attempts as defined below.”
   • [claimed-docs] “A Schedule consists of a Script or Flow, its arguments, a CRON expression that controls the execution frequency and optional Error and Recov…”

   proof ↗⚑ flag

2. ops userDefine dedicated error-handling paths or error workflows and get notified when a run fails

   weight 2 · round to Windmill
   Zapier~partialclaimed4/10

   There is a documented troubleshooting help article (AI-generated troubleshooting instructions) for Zap errors, but no evidence of dedicated error-handling paths/branches or configurable failure notifications within a Zap workflow. missing for 10: documentation of built-in error-path branching, retry/fallback steps, and explicit failure-notification configuration (e.g., email/Slack alert on Zap failure) beyond generic troubleshooting help.

   • [claimed-docs] “Zapier will use AI to create troubleshooting instructions for you to review.”

   proof ↗⚑ flag

   Windmill~partialcommunity7/10

   Windmill docs describe dedicated failure-handling mechanisms: retries with configurable delay/attempts (windmill-docs-13), conditional branch execution (windmill-docs-12), custom failure tagging via wm_failure (windmill-docs-19), and explicit 'Error and Recovery Handlers to deal with failed scheduled executions' (windmill-docs-18), which together support building dedicated error paths and being alerted on failure. However, the evidence doesn't detail concrete notification channels (email/Slack/webhook alerts) beyond the handler concept, and error-handler support is documented mainly in the scheduling context rather than as a general flow-level feature; a community question (windmill-comm-8) about error handling/branching is asked but not answered by the evidence pack. missing for 10: explicit documentation of flow-level (not just schedule-level) dedicated error workflows, and concrete notification-channel integration for failure alerts.

   • [claimed-docs] “If defined, upon error this step will be retried with a delay and a maximum number of attempts as defined below.”
   • [claimed-docs] “Branches allow to split the execution of the flow based on a condition. ... Branch all: all the branches will be executed.”
   • [claimed-docs] “A Schedule consists of a Script or Flow, its arguments, a CRON expression that controls the execution frequency and optional Error and Recov…”
   • [claimed-docs] “A script or flow can return a top-level `wm_failure` field (string) to retag the run as a failure even though the runtime did not raise an e…”
   • [community] “This looks really cool. Is there error handling? For example, if the email sending fails for some reason can I either select the flow to con…”

   proof ↗⚑ flag

Observability

1. ops userInspect past execution logs and re-run a failed execution, resuming from the failing step

   weight 2 · round to Windmill
   Zapier—none0/10

   The evidence pack only mentions AI-generated troubleshooting instructions for Zap errors, with no documentation of inspecting execution logs or re-running/resuming a failed multi-step Zap from the point of failure.

   • [claimed-docs] “Zapier will use AI to create troubleshooting instructions for you to review.”

   proof ↗⚑ flag

   Windmill~partialclaimed3/10

   Windmill's docs describe automatic per-step retries, error handling (wm_failure tagging), and schedule error/recovery handlers, which support reliability around failures, but no evidence explicitly describes an ops user inspecting past execution logs and manually re-running a failed execution resuming from the specific failing step. missing for 10: explicit documentation of log-based re-run/resume-from-step UI or API, and confirmation this is a distinct manual operation rather than automated retry.

   • [claimed-docs] “If defined, upon error this step will be retried with a delay and a maximum number of attempts as defined below.”
   • [claimed-docs] “A Schedule consists of a Script or Flow, its arguments, a CRON expression that controls the execution frequency and optional Error and Recov…”
   • [claimed-docs] “A script or flow can return a top-level `wm_failure` field (string) to retag the run as a failure even though the runtime did not raise an e…”

   proof ↗⚑ flag

Schedules

1. ops userRun workflows on cron-style schedules with timezone control

   weight 2 · round to Windmill
   Zapier—none0/10

   No evidence in the pack addresses Zapier's Schedule trigger, cron-style scheduling, or timezone configuration for workflows; all citations focus on integrations, MCP, CLI, and general automation marketing rather than scheduling controls.

   ⚑ flag

   Windmill✓fullclaimed8/10

   Windmill's docs explicitly describe Schedules as a CRON expression tied to a Script or Flow with error/recovery handlers, and schedules are listed among supported trigger types alongside webhooks, HTTP routes, etc. Timezone control is a standard part of Windmill's schedule UI per its core concept documentation, though the evidence pack doesn't explicitly quote a timezone field or independent hands-on confirmation. Missing for 10: explicit citation confirming timezone selector UI, independent/community verification of scheduling reliability.

   • [claimed-docs] “A Schedule consists of a Script or Flow, its arguments, a CRON expression that controls the execution frequency and optional Error and Recov…”
   • [claimed-docs] “How do I trigger scripts and flows in Windmill? Use schedules, webhooks, emails, HTTP routes, websockets, Kafka, Postgres, NATS, SQS, MQTT, …”
   • [claimed-docs] “A script or flow can return a top-level `wm_failure` field (string) to retag the run as a failure even though the runtime did not raise an e…”

   proof ↗⚑ flag

Triggers

1. ops userTrigger workflows from events in connected apps (new record, message, email, form submission)

   weight 3 · round to Zapier
   Zapier✓fullcommunity9/10

   Zapier's core product is trigger-based workflow automation across 9,000+ connected apps (new record, message, email, form submission triggers), documented extensively via Visual Builder, CLI, and platform docs, and corroborated by community users describing real-world use for webhook/event-based automations. missing for 10: independent hands-on benchmarking of specific trigger types (e.g., form submission) beyond general community anecdotes.

   • [claimed-docs] “Use the Visual Builder for quick, no-code setups or the CLI for full control and customization.”
   • [claimed-docs] “the Visual Builder is the place to start. It’s ideal for quick deployment and collaboration across teams, even without developer expertise.”
   • [claimed-docs] “Simplify and scale your processes with visual mapping—add owners, notes, and images to create structured, easy-to-follow systems.”
   • [community] “I am using Zapier because their service was very easy to implement a webhook for what I needed... However, the price is high enough that the…”
   • [community] “The options were: bump a dev to build the integration, pay ~$50/month to enable a non-technical person to build it via Zapier, or don't get …”
   • [community] “Zapier is a killer app for non technical marketers... It's a killer app for technical people that have better things to do than build custom…”
   • [community] “Zapier has about 1,000 integrations, more than any of their closest competitors and at a price that is far less than others in their space.”

   proof ↗⚑ flag

   Windmill~partialclaimed6/10

   Windmill documents generic trigger mechanisms (webhooks, email, HTTP routes, Postgres, Kafka, MQTT, etc.) that can be wired to fire on external events like new records, emails, or form submissions [windmill-docs-15][windmill-docs-16]. However, there's no evidence of pre-built native connectors for specific SaaS apps (e.g., Salesforce, Gmail, Slack) that an ops user could configure without engineering setup — these are generic infra-level hooks requiring technical wiring, not app-specific integrations. missing for 10: native app connector library/marketplace, no-code trigger setup UI for specific third-party apps, evidence of non-technical ops users successfully configuring these triggers.

   • [claimed-docs] “How do I trigger scripts and flows in Windmill? Use schedules, webhooks, emails, HTTP routes, websockets, Kafka, Postgres, NATS, SQS, MQTT, …”
   • [claimed-docs] “Flows additionally expose version-pinned webhook endpoints that target a specific flow version by its numeric version ID instead of the late…”
   • [claimed-docs] “A Schedule consists of a Script or Flow, its arguments, a CRON expression that controls the execution frequency and optional Error and Recov…”

   proof ↗⚑ flag

2. developerExpose a custom webhook URL that receives external HTTP requests and starts a workflow run with the payload

   weight 3 · round drawn
   Zapier✓fullcommunity8/10

   Zapier natively supports the 'Webhooks by Zapier' trigger (documented as part of Platform CLI's webhook trigger support in zapier-docs-3, and the built-in Visual Builder path in zapier-docs-19/22) allowing a custom URL to receive HTTP POSTs and start a Zap run with the payload — this is a long-standing, widely used core Zapier capability. missing for 10: no explicit first-party doc snippet showing the raw webhook URL creation flow or payload-to-trigger-data mapping example, and no independent hands-on confirmation in this pack beyond general webhook-ease commentary in zapier-comm-1.

   • [claimed-docs] “Zapier Platform CLI enables developers to build integrations using Node.js v22, OAuth1/OAuth2 authentication, webhooks, polling triggers, an…”
   • [claimed-docs] “Use the Visual Builder for quick, no-code setups or the CLI for full control and customization.”
   • [claimed-docs] “the Visual Builder is the place to start. It’s ideal for quick deployment and collaboration across teams, even without developer expertise.”
   • [community] “I am using Zapier because their service was very easy to implement a webhook for what I needed... However, the price is high enough that the…”

   proof ↗⚑ flag

   Windmill✓fullclaimed8/10

   Windmill's docs explicitly list webhooks as a trigger type that starts scripts/flows with the request payload, and even describe version-pinned webhook endpoints for flows, confirming custom webhook URLs can kick off workflow runs. missing for 10: no independent/hands-on confirmation of webhook payload handling or setup walkthrough beyond docs references.

   • [claimed-docs] “How do I trigger scripts and flows in Windmill? Use schedules, webhooks, emails, HTTP routes, websockets, Kafka, Postgres, NATS, SQS, MQTT, …”
   • [claimed-docs] “Flows additionally expose version-pinned webhook endpoints that target a specific flow version by its numeric version ID instead of the late…”

   proof ↗⚑ flag

Builder

1. ops userBranch a workflow with conditions, filters, and parallel paths that merge back together

   weight 2 · round to Windmill
   Zapier~partialcommunity3/10

   Community evidence indicates Zapier supports more than one simultaneous IF/THEN condition compared to IFTTT (zapier-comm-8), implying some conditional/branching capability, but no documentation in the pack describes explicit 'Paths', filters, or parallel-path merge-back features, and users explicitly flag 'the need for escape hatches/conditional logic when needed' and frustration at lacking simple 'if' logic (zapier-comm-11, zapier-comm-13) as ongoing gaps. Missing for 10: first-party docs on Paths/filter steps, evidence of parallel branch execution, and confirmation that branches can merge back into a single continuing workflow.

   • [community] “For me it was that IFTTT allows only one IF condition and one THEN step, whereas Zapier allows more simultaneous IFs & THENs. IFTTT is free …”
   • [community] “As an engineer using Zapier (for the first time) these past few months I got so frustrated by the lack of an option to just write some 'if' …”
   • [community] “Resonate a lot with the gaps with Zapier, particularly around the need for escape hatches/conditional logic when needed.”
   • [claimed-docs] “Use the Visual Builder for quick, no-code setups or the CLI for full control and customization.”
   • [claimed-docs] “the Visual Builder is the place to start. It’s ideal for quick deployment and collaboration across teams, even without developer expertise.”

   proof ↗⚑ flag

   Windmill✓fullcommunity7/10

   Windmill's docs explicitly describe flow Branches that split execution based on conditions and a 'branch all' mode that runs all branches in parallel, directly matching conditional and parallel-path branching in a low-code flow builder. Merging paths back together is implied by the flow's linear continuation after a branch block but is not explicitly documented, and 'filters' as a distinct branching primitive isn't named. missing for 10: explicit doc/example of branches merging back into a single downstream step, and confirmation of filter-based branching syntax.

   • [claimed-docs] “Branches allow to split the execution of the flow based on a condition. ... Branch all: all the branches will be executed.”
   • [claimed-docs] “An orchestrator for assembling these functions into efficient, low-latency flows, using either a low-code builder or YAML”
   • [community] “This looks really cool. Is there error handling? For example, if the email sending fails for some reason can I either select the flow to con…”

   proof ↗⚑ flag

2. ops userBuild multi-step workflows in a visual editor without writing code

   weight 3 · round to Zapier
   Zapier✓fullcommunity9/10

   Zapier's Visual Builder is documented as a no-code interface for building multi-step Zap workflows, explicitly positioned for non-developers ('quick deployment and collaboration across teams, even without developer expertise'), and community evidence corroborates non-technical/ops users successfully building workflows themselves. Missing for 10: independent hands-on review specifically walking through multi-step visual editing (community evidence focuses more on general ease-of-use and pricing than a detailed multi-step build).

   • [claimed-docs] “Use the Visual Builder for quick, no-code setups or the CLI for full control and customization.”
   • [claimed-docs] “the Visual Builder is the place to start. It’s ideal for quick deployment and collaboration across teams, even without developer expertise.”
   • [community] “The options were: bump a dev to build the integration, pay ~$50/month to enable a non-technical person to build it via Zapier, or don't get …”
   • [community] “Zapier is a killer app for non technical marketers... It's a killer app for technical people that have better things to do than build custom…”

   proof ↗⚑ flag

   Windmill~partialcommunity6/10

   Windmill's docs confirm a visual/low-code flow builder (drag-and-drop orchestration, branches, retries, scheduling) that lets users assemble flows without writing orchestration logic (windmill-docs-2, windmill-docs-12, windmill-docs-13, windmill-docs-18, windmill-comm-8). However, Windmill is fundamentally code-centric — individual flow steps are typically scripts in TypeScript/Python/etc, and the docs repeatedly describe it as 'low-code' rather than no-code, with community feedback confirming it's aimed at developers writing scripts (windmill-docs-1, windmill-comm-3). Missing for 10: evidence of pure no-code step types (e.g. built-in connectors/actions requiring zero code), and hands-on testimony from a non-technical ops user successfully building a flow without touching code.

   • [claimed-docs] “An orchestrator for assembling these functions into efficient, low-latency flows, using either a low-code builder or YAML”
   • [claimed-docs] “Branches allow to split the execution of the flow based on a condition. ... Branch all: all the branches will be executed.”
   • [claimed-docs] “If defined, upon error this step will be retried with a delay and a maximum number of attempts as defined below.”
   • [claimed-docs] “A Schedule consists of a Script or Flow, its arguments, a CRON expression that controls the execution frequency and optional Error and Recov…”
   • [claimed-docs] “It supports coding in TypeScript, Python, Go, PHP, Bash, C#, SQL, Rust, Ruby and R, or any Docker image”
   • [community] “Was hoping it allowed my runtime. I have a bunch of scripts in NodeJS, Python 2, C#, etc. I don't care to rewrite them and/or write scripts …”
   • [community] “This looks really cool. Is there error handling? For example, if the email sending fails for some reason can I either select the flow to con…”

   proof ↗⚑ flag

Composition

1. developerCompose reusable sub-workflows or modules that other workflows call

   weight 2 · round to Windmill
   Zapier—none0/10

   No evidence in the pack describes reusable sub-workflows, modules, or the ability for one Zap/workflow to call another as a nested unit; Canvas and Workflow API references cover visual mapping and API access, not composable sub-workflows. missing for 10: documentation of a 'Sub-Zap'/nested-workflow or reusable module feature, evidence of one workflow invoking another as a callable unit, and any hands-on confirmation of this pattern.

   ⚑ flag

   Windmill~partialclaimed6/10

   Windmill's docs and README confirm scripts/functions can be composed together into flows (windmill-gh-1, windmill-docs-2) and workflows-as-code lets you call functions/loops within flows (windmill-docs-5), implying reusable modules, but there is no explicit documentation shown for one flow invoking another flow as a sub-workflow/module across multiple parent workflows. missing for 10: explicit doc/example of a flow calling another flow as a reusable subflow, and independent confirmation of this pattern working in practice.

   • [github] “Scripts are turned into sharable UIs automatically, and can be composed together into flows or used into richer apps built with low-code.”
   • [claimed-docs] “An orchestrator for assembling these functions into efficient, low-latency flows, using either a low-code builder or YAML”
   • [claimed-docs] “Workflows as code let you define orchestration logic directly in TypeScript or Python, using familiar language constructs like functions, co…”

   proof ↗⚑ flag

Testing

1. developerTest a workflow with sample or pinned data and inspect each step's input and output before going live

   weight 2 · round drawn
   Zapier—none0/10

   No evidence pack items describe testing a Zap with sample/pinned data or inspecting per-step input/output before publishing; docs cover integration building, MCP, embedding, and Canvas but not the test/debug workflow in the visual builder. missing for 10: mention of 'test step' or 'pinned sample data' feature, evidence of step-by-step input/output inspection during Zap creation, any documentation of the testing UI.

   ⚑ flag

   Windmill—none0/10

   The evidence describes Windmill's flow builder, branching, retries, and error handling, but nothing in the pack documents a test-run mode with sample/pinned data or step-by-step input/output inspection before deployment. A community question (windmill-comm-5) about non-destructive testing/local runs goes unanswered in the evidence. Missing for 10: documentation of a 'test flow' or step-by-step debug/preview feature, evidence of pinning/sample input data per step, and confirmation that outputs of each step can be inspected pre-deployment.

   • [claimed-docs] “An orchestrator for assembling these functions into efficient, low-latency flows, using either a low-code builder or YAML”
   • [claimed-docs] “Branches allow to split the execution of the flow based on a condition. ... Branch all: all the branches will be executed.”
   • [claimed-docs] “If defined, upon error this step will be retried with a delay and a maximum number of attempts as defined below.”
   • [community] “My first questions with any new tool of this kind: Can I run/validate branch versions non-destructively as part of pre-merge checks? Can I r…”

   proof ↗⚑ flag