Make vs Windmill

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round to Windmill
   Make✓fullprobed8/10

   Make hosts a live llms.txt at developers.make.com/llms.txt (HTTP 200, confirmed by probe) plus Markdown-formatted 'Make Skills' docs designed for AI assistants, directly supporting agent-oriented documentation consumption. missing for 10: confirmation that all doc subpages (not just the root) resolve as .md, and independent/third-party evidence of agents successfully using llms.txt in practice.

   • [probe] “PROBE llms.txt: HTTP 200 at https://developers.make.com/llms.txt # Make Developer Hub ## Home - [Make Developer Hub](https://developers.ma…”
   • [claimed-docs] “Make Skills are Markdown files that help your assistant reliably perform Make-specific tasks, including building scenarios, configuring modu…”
   • [claimed-docs] “Make Skills are Markdown files that help your assistant reliably perform Make-specific tasks, including building scenarios, configuring modu…”
   • [probe] “PROBE docs-md: HTTP 200 at https://developers.make.com/api-documentation.md # Page Not Found The URL `api-documentation` does not exist. Th…”

   proof ↗⚑ flag

   Windmill✓fullprobed9/10

   Direct probe confirms a working llms.txt at https://www.windmill.dev/llms.txt (HTTP 200) and .md-suffixed docs pages (e.g., intro.md) that return clean markdown, both hallmarks of agent-oriented documentation designed for LLM consumption. Missing for 10: no independent/community confirmation that agents actually use these successfully in practice.

   • [probe] “PROBE llms.txt: HTTP 200 at https://www.windmill.dev/llms.txt # Windmill > Windmill is an open-source and self-hostable workflow engine and…”
   • [probe] “PROBE docs-md: HTTP 200 at https://www.windmill.dev/docs/intro.md # What is Windmill? > What is Windmill? An open-source workflow engine an…”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round to Windmill
   Make~partialprobed6/10

   Make exposes a REST API (with auth/scopes) and webhooks that let external systems or CI pipelines trigger, schedule, and manage scenarios without using the UI, which supports headless/automated invocation. However, there is no evidence of an official CLI, containerized runner, or CI-specific integration guide (e.g., GitHub Actions), and the openapi probe returned 404s, suggesting weaker machine-readable API tooling. Missing for 10: dedicated CLI or CI/CD integration docs, confirmed OpenAPI spec, and hands-on evidence of running scenarios in a pipeline.

   • [claimed-docs] “All requests to the Make API require authentication with the user authentication token with the relevant scopes enabled.”
   • [claimed-docs] “The Make API follows the REST API design. Make API is organized into resource-oriented URLs”
   • [claimed-docs] “custom webhooks allow you to create a url to which you can send any data”
   • [claimed-docs] “webhooks create a url that you can call from an external app or service, or from another use webhooks to trigger the execution”
   • [claimed-docs] “If you don't want to run your immediately after a webhook receives data, you can schedule your to process all webhook requests periodically”
   • [claimed-docs] “you can schedule your to process all webhook requests periodically”
   • [probe] “PROBE openapi: all candidate paths 404 (https://developers.make.com/openapi.json, https://developers.make.com/swagger.json, https://develope…”

   proof ↗⚑ flag

   Windmill✓fullprobed8/10

   Windmill provides a documented CLI (wmill) for deploying and syncing from Git, supports webhooks/schedules/API triggers for headless execution, and can self-host via Docker/Kubernetes for CI pipelines — all strongly supporting headless/CI automation use. Missing for 10: independent hands-on CI-pipeline examples or third-party corroboration of the CLI/webhook flow working seamlessly in a CI system beyond vendor docs.

   • [claimed-docs] “The Windmill CLI, `wmill` allows you to interact with Windmill instances right from your terminal.”
   • [claimed-docs] “Each time an item is deployed, Windmill will create and push a commit to the specified repository. ... Windmill can automatically deploy new…”
   • [claimed-docs] “How do I trigger scripts and flows in Windmill? Use schedules, webhooks, emails, HTTP routes, websockets, Kafka, Postgres, NATS, SQS, MQTT, …”
   • [claimed-docs] “Flows additionally expose version-pinned webhook endpoints that target a specific flow version by its numeric version ID instead of the late…”
   • [claimed-docs] “For small setups, use Docker and Docker Compose on a single instance. For larger and production use-cases, use our Helm chart to deploy on K…”
   • [probe] “official CLI documented at https://www.windmill.dev/docs/advanced/cli”

   proof ↗⚑ flag

3. ai-native userPlug MCP servers into this product so it can use their tools

   weight 3 · round drawn
   Make—none0/10

   All evidence describes Make exposing its own scenarios as an MCP server to external AI clients (Claude, ChatGPT), the reverse of this story's requirement that Make itself act as an MCP client consuming external MCP servers' tools. No documentation shows Make importing or calling third-party MCP servers as tool providers within its scenarios/agents.

   • [claimed-docs] “Turns your **active** and **on-demand** scenarios into callable tools for AI”
   • [claimed-docs] “AI systems like Claude and ChatGPT act as MCP clients of Make MCP server. The server provides them access to **scenario run** and **manageme…”
   • [claimed-docs] “AI systems like Claude and ChatGPT act as MCP clients of Make MCP server.”
   • [claimed-docs] “Make MCP server allows AI systems, such as large language models (LLMs), to run scenarios and manage the contents of your Make account.”
   • [claimed-docs] “learn about agentic automation with the make ai agents app”

   proof ↗⚑ flag

   Windmill—none0/10

   Windmill's MCP documentation (windmill-docs-17, windmill-probe-4) describes Windmill acting as an MCP *server* so external LLM clients (Claude, Cursor) can call Windmill's scripts/flows — the reverse of the story, which asks whether Windmill can consume/plug into external MCP servers to use their tools. The AI Agent steps (windmill-docs-20) mention connecting to AI providers/models, not MCP tool servers, so no evidence shows Windmill as an MCP client.

   • [claimed-docs] “With MCP, you can connect your favorite LLMs (like Claude, Cursor, or any MCP compatible client) to Windmill, allowing you to trigger your s…”
   • [probe] “official MCP server documented at https://www.windmill.dev/docs/core_concepts/mcp”
   • [claimed-docs] “AI Agents steps allow you to integrate AI agents directly into your Windmill flows. They provide an interface to connect with various AI pro…”

   proof ↗⚑ flag

4. ai-native userConnect an agent via an official MCP server

   weight 3 · round drawn
   Make✓fullprobed8/10

   Make documents an official MCP server that turns scenarios into callable tools for AI systems like Claude/ChatGPT, exposing scenario run and management capabilities, plus Make Skills to help agents connect. Missing for 10: independent/hands-on third-party corroboration and details on auth scopes specific to MCP server usage.

   • [claimed-docs] “Turns your **active** and **on-demand** scenarios into callable tools for AI”
   • [claimed-docs] “AI systems like Claude and ChatGPT act as MCP clients of Make MCP server. The server provides them access to **scenario run** and **manageme…”
   • [claimed-docs] “AI systems like Claude and ChatGPT act as MCP clients of Make MCP server.”
   • [claimed-docs] “Make MCP server allows AI systems, such as large language models (LLMs), to run scenarios and manage the contents of your Make account.”
   • [claimed-docs] “Make Skills are Markdown files that help your assistant reliably perform Make-specific tasks, including building scenarios, configuring modu…”
   • [probe] “official MCP server documented at https://developers.make.com/mcp-server”

   proof ↗⚑ flag

   Windmill✓fullprobed8/10

   Windmill is a workflow/automation platform (not an agent), so publishing an official MCP server is squarely within its category, and docs confirm it: 'With MCP, you can connect your favorite LLMs (like Claude, Cursor, or any MCP compatible client) to Windmill, allowing you to trigger your scripts and flows from your client chat,' corroborated by a probe confirming the official MCP docs page. missing for 10: independent/hands-on confirmation of the MCP server working in practice beyond first-party docs.

   • [claimed-docs] “With MCP, you can connect your favorite LLMs (like Claude, Cursor, or any MCP compatible client) to Windmill, allowing you to trigger your s…”
   • [probe] “official MCP server documented at https://www.windmill.dev/docs/core_concepts/mcp”

   proof ↗⚑ flag

5. ai-native userUse an official CLI

   weight 2 · round to Windmill
   Make—none0/10

   The evidence pack documents Make's REST API, MCP server, webhooks, and Skills, but contains no mention of an official command-line interface (CLI) tool for Make. Since automation platforms could plausibly ship a CLI, this axis applies, but no evidence supports it being delivered.

   ⚑ flag

   Windmill✓fullprobed8/10

   Windmill ships an official CLI (`wmill`) documented for interacting with instances from the terminal, deploying from Git repos, and syncing commits—confirmed by both docs and a dedicated probe. This directly satisfies the 'official CLI' story for AI-native/agentic workflows (e.g., scripting deployments, CI integration). Missing for 10: independent hands-on community reports specifically validating the CLI's AI-native usage patterns (e.g., agent-driven CLI invocation) beyond first-party docs.

   • [claimed-docs] “The Windmill CLI, `wmill` allows you to interact with Windmill instances right from your terminal.”
   • [claimed-docs] “Each time an item is deployed, Windmill will create and push a commit to the specified repository. ... Windmill can automatically deploy new…”
   • [probe] “official CLI documented at https://www.windmill.dev/docs/advanced/cli”

   proof ↗⚑ flag

6. ai-native userDrive the product through a documented public API

   weight 3 · round to Make
   Make✓fullprobed8/10

   Make documents a public REST API (resource-oriented URLs, token auth, troubleshooting section) plus a first-party MCP server exposing scenario run/management as callable tools, enabling AI-native driving of the product beyond just the raw API. Missing for 10: a working OpenAPI/swagger spec (probe found 404s) and independent hands-on corroboration of the API's completeness/reliability.

   • [claimed-docs] “All requests to the Make API require authentication with the user authentication token with the relevant scopes enabled.”
   • [claimed-docs] “The Make API follows the REST API design. Make API is organized into resource-oriented URLs”
   • [claimed-docs] “If you are experiencing issues with the Make API, go to the Troubleshooting section for tips about what could go wrong and how to fix it.”
   • [claimed-docs] “Turns your **active** and **on-demand** scenarios into callable tools for AI”
   • [claimed-docs] “AI systems like Claude and ChatGPT act as MCP clients of Make MCP server. The server provides them access to **scenario run** and **manageme…”
   • [claimed-docs] “Make MCP server allows AI systems, such as large language models (LLMs), to run scenarios and manage the contents of your Make account.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://developers.make.com/openapi.json, https://developers.make.com/swagger.json, https://develope…”
   • [probe] “official MCP server documented at https://developers.make.com/mcp-server”

   proof ↗⚑ flag

   Windmill~partialprobed5/10

   Windmill exposes a CLI (wmill), webhooks, and various trigger mechanisms (HTTP routes, schedules, MCP) that let external systems and AI agents drive it (windmill-docs-9, windmill-docs-15, windmill-docs-16, windmill-docs-17, windmill-probe-4, windmill-probe-5), which is real evidence of programmatic/agentic control. However, a direct probe for a documented public REST/OpenAPI spec (the canonical 'documented public API') returned 404 on all candidate paths (windmill-probe-3), so no first-class API reference doc was found in evidence. missing for 10: a discoverable OpenAPI/REST API reference page, independent confirmation that third parties integrate via a general public API beyond webhooks/CLI/MCP.

   • [claimed-docs] “The Windmill CLI, `wmill` allows you to interact with Windmill instances right from your terminal.”
   • [claimed-docs] “How do I trigger scripts and flows in Windmill? Use schedules, webhooks, emails, HTTP routes, websockets, Kafka, Postgres, NATS, SQS, MQTT, …”
   • [claimed-docs] “Flows additionally expose version-pinned webhook endpoints that target a specific flow version by its numeric version ID instead of the late…”
   • [claimed-docs] “With MCP, you can connect your favorite LLMs (like Claude, Cursor, or any MCP compatible client) to Windmill, allowing you to trigger your s…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://www.windmill.dev/openapi.json, https://www.windmill.dev/swagger.json, https://www.windmill.d…”
   • [probe] “official MCP server documented at https://www.windmill.dev/docs/core_concepts/mcp”
   • [probe] “official CLI documented at https://www.windmill.dev/docs/advanced/cli”

   proof ↗⚑ flag

7. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round to Windmill
   Make~partialclaimed5/10

   Make API tokens support 'relevant scopes enabled' for authentication, indicating some scoped credential capability, and MCP server access can be scoped to specific scenarios/tools rather than full account access. However there's no documentation of granular least-privilege scope definitions, no scope list, no per-agent credential issuance workflow, and no independent verification of enforcement. Missing for 10: a documented list of available API scopes, evidence of fine-grained least-privilege controls (e.g., read-only vs write, resource-level restrictions), and independent/hands-on confirmation that scoped tokens actually restrict agent access as claimed.

   • [claimed-docs] “All requests to the Make API require authentication with the user authentication token with the relevant scopes enabled.”
   • [claimed-docs] “The Make API follows the REST API design. Make API is organized into resource-oriented URLs”
   • [claimed-docs] “Make MCP server allows AI systems, such as large language models (LLMs), to run scenarios and manage the contents of your Make account.”
   • [claimed-docs] “View and modify scenarios and their related entities (e.g., connections, webhooks, and data stores)”

   proof ↗⚑ flag

   Windmill~partialprobed6/10

   Windmill documents scoped tokens following least-privilege principles (windmill-docs-8) and an underlying roles/permissions system (windmill-docs-7), and its MCP server for connecting agents (windmill-docs-17, windmill-probe-4) implies token-based auth for agent connections. However, there's no explicit documentation tying scoped token creation specifically to AI agent use-cases or showing a workflow for issuing/rotating agent-specific credentials. Missing for 10: explicit docs on generating scoped tokens specifically for AI agents/MCP clients, examples of scope granularity, and independent/hands-on confirmation that scoping works as claimed.

   • [claimed-docs] “Tokens can be scoped to restrict access to specific resources and actions, following the principle of least privilege.”
   • [claimed-docs] “Windmill provides a roles and permissions system that allows you to control access and manage permissions within your instance and workspace…”
   • [claimed-docs] “With MCP, you can connect your favorite LLMs (like Claude, Cursor, or any MCP compatible client) to Windmill, allowing you to trigger your s…”
   • [probe] “official MCP server documented at https://www.windmill.dev/docs/core_concepts/mcp”

   proof ↗⚑ flag

8. ai-native userBuild against official SDKs

   weight 2 · round drawn
   Make—none0/10

   Evidence shows a REST API and an MCP server, but no official client SDKs (Python/JS/etc.) or OpenAPI spec are documented—probes for openapi/swagger endpoints returned 404, indicating no formal SDK-generation artifact. The API docs describe raw REST endpoints and auth tokens, not packaged SDK libraries.

   • [claimed-docs] “All requests to the Make API require authentication with the user authentication token with the relevant scopes enabled.”
   • [claimed-docs] “The Make API follows the REST API design. Make API is organized into resource-oriented URLs”
   • [probe] “PROBE openapi: all candidate paths 404 (https://developers.make.com/openapi.json, https://developers.make.com/swagger.json, https://develope…”

   proof ↗⚑ flag

   Windmill—none0/10

   The evidence pack documents a CLI (wmill), an MCP server, and workflows-as-code in TypeScript/Python, but none of it describes official client SDKs/libraries for programmatically building against Windmill's API. A probe for an OpenAPI spec (often paired with SDK generation) returned 404s, further suggesting no discoverable official SDK artifact.

   • [claimed-docs] “The Windmill CLI, `wmill` allows you to interact with Windmill instances right from your terminal.”
   • [claimed-docs] “With MCP, you can connect your favorite LLMs (like Claude, Cursor, or any MCP compatible client) to Windmill, allowing you to trigger your s…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://www.windmill.dev/openapi.json, https://www.windmill.dev/swagger.json, https://www.windmill.d…”
   • [probe] “official CLI documented at https://www.windmill.dev/docs/advanced/cli”

   proof ↗⚑ flag

9. ai-native userSubscribe to events via webhooks

   weight 2 · round to Make
   Make~partialclaimed6/10

   Make's custom webhooks let a scenario expose a URL that receives events from external systems and trigger scenario runs, effectively letting a user 'subscribe' to external events via webhook (make-docs-5, make-docs-15, make-docs-25), with queuing/scheduling options (make-docs-6, make-docs-21). However, this is a general automation feature, not something exposed or documented specifically for AI-native/agentic consumption (e.g., no MCP tool or agent-specific API for creating/managing webhook subscriptions is shown). Missing for 10: evidence of webhook subscription management being agent-callable via MCP or API, and any outbound event-notification (pub/sub) webhook mechanism for external AI systems.

   • [claimed-docs] “custom webhooks allow you to create a url to which you can send any data”
   • [claimed-docs] “webhooks create a url that you can call from an external app or service, or from another use webhooks to trigger the execution”
   • [claimed-docs] “webhooks create a url that you can call from an external app or service, or from another use webhooks to trigger the execution of”
   • [claimed-docs] “If you don't want to run your immediately after a webhook receives data, you can schedule your to process all webhook requests periodically”
   • [claimed-docs] “the whole queue is then processed every time your schedule criteria are met”

   proof ↗⚑ flag

   Windmill~partialclaimed5/10

   Windmill documents webhooks extensively as a trigger mechanism—scripts and flows can be invoked via webhook endpoints, including version-pinned endpoints for specific flow versions—which lets external systems call into Windmill (windmill-docs-15, windmill-docs-16). However, this is primarily inbound triggering rather than an outbound event-subscription model where Windmill pushes notifications about internal events (e.g., job completion, failures) to a subscriber's webhook URL. Missing for 10: explicit documentation of outbound webhook subscriptions/event notifications, independent confirmation of an event-driven push webhook system, and any first-party mention of 'subscribe' semantics rather than pure trigger endpoints.

   • [claimed-docs] “How do I trigger scripts and flows in Windmill? Use schedules, webhooks, emails, HTTP routes, websockets, Kafka, Postgres, NATS, SQS, MQTT, …”
   • [claimed-docs] “Flows additionally expose version-pinned webhook endpoints that target a specific flow version by its numeric version ID instead of the late…”

   proof ↗⚑ flag

Agentic features

1. ai-native userGet AI-generated insights and suggestions from my data inside the product

   weight 2 · round to Windmill
   Make—none0/10

   Evidence shows Make's AI Agents app and MCP server let external AI systems build/run automations and manage scenarios, but there is no evidence of a feature that surfaces AI-generated insights or suggestions from a user's own data inside the product UI.

   • [claimed-docs] “learn about agentic automation with the make ai agents app”
   • [claimed-docs] “Make MCP server allows AI systems, such as large language models (LLMs), to run scenarios and manage the contents of your Make account.”

   proof ↗⚑ flag

   Windmill~partialclaimed4/10

   Windmill offers AI Agent steps that let users wire LLM calls into flows to process data, and an MCP server/AI-generation feature for building scripts, but these are developer-facing building blocks rather than a built-in 'insights and suggestions from my data' feature (e.g., no dashboard/analytics AI surfaced automatically on stored data). Missing for 10: evidence of an out-of-the-box AI analytics/insights UI, proactive suggestions surfaced to end users without building a flow, and any independent corroboration of this specific use case.

   • [claimed-docs] “AI Agents steps allow you to integrate AI agents directly into your Windmill flows. They provide an interface to connect with various AI pro…”
   • [claimed-docs] “Review and deploy changes: Once the AI has edited items, ask it to review the changes and it opens the Compare & Deploy page with exactly th…”
   • [claimed-docs] “With MCP, you can connect your favorite LLMs (like Claude, Cursor, or any MCP compatible client) to Windmill, allowing you to trigger your s…”

   proof ↗⚑ flag

2. ai-native userSet up automations that run autonomously in the background

   weight 2 · round drawn
   Make✓fullclaimed8/10

   Make scenarios are core automations that run autonomously on triggers (webhooks, schedules) in the background without human intervention, with error handling and history/monitoring built in, and can even be exposed as callable tools via MCP for AI orchestration. missing for 10: independent/hands-on evidence of long-running autonomous scenarios at scale, and no third-party corroboration beyond vendor docs.

   • [claimed-docs] “custom webhooks allow you to create a url to which you can send any data”
   • [claimed-docs] “If you don't want to run your immediately after a webhook receives data, you can schedule your to process all webhook requests periodically”
   • [claimed-docs] “webhooks create a url that you can call from an external app or service, or from another use webhooks to trigger the execution”
   • [claimed-docs] “you can schedule your to process all webhook requests periodically”
   • [claimed-docs] “the whole queue is then processed every time your schedule criteria are met”
   • [claimed-docs] “this page helps users navigate errors, diagnose and resolve issues in by providing detailed information on common errors and warnings, error…”
   • [claimed-docs] “scenario history”
   • [claimed-docs] “Turns your **active** and **on-demand** scenarios into callable tools for AI”
   • [claimed-docs] “Make MCP server allows AI systems, such as large language models (LLMs), to run scenarios and manage the contents of your Make account.”

   proof ↗⚑ flag

   Windmill✓fullcommunity8/10

   Windmill natively supports scheduling scripts/flows via CRON with error/recovery handlers, plus a wide range of autonomous triggers (webhooks, queues, events, routes) that run in the background without manual intervention, and flows include retries, branching, and error handling for resilient unattended execution. This directly matches the AI-native automation story, reinforced by AI Agent steps that can be embedded into these autonomous flows. Missing for 10: no independent/hands-on evidence confirming long-running autonomous reliability at scale, and community comments raise open questions about error handling rather than confirming it in practice.

   • [claimed-docs] “A Schedule consists of a Script or Flow, its arguments, a CRON expression that controls the execution frequency and optional Error and Recov…”
   • [claimed-docs] “How do I trigger scripts and flows in Windmill? Use schedules, webhooks, emails, HTTP routes, websockets, Kafka, Postgres, NATS, SQS, MQTT, …”
   • [claimed-docs] “If defined, upon error this step will be retried with a delay and a maximum number of attempts as defined below.”
   • [claimed-docs] “Branches allow to split the execution of the flow based on a condition. ... Branch all: all the branches will be executed.”
   • [claimed-docs] “A script or flow can return a top-level `wm_failure` field (string) to retag the run as a failure even though the runtime did not raise an e…”
   • [claimed-docs] “AI Agents steps allow you to integrate AI agents directly into your Windmill flows. They provide an interface to connect with various AI pro…”
   • [community] “This looks really cool. Is there error handling? For example, if the email sending fails for some reason can I either select the flow to con…”

   proof ↗⚑ flag

3. ai-native userDelegate tasks to a built-in AI assistant inside the product

   weight 3 · round to Windmill
   Make~partialclaimed3/10

   Make documents a 'Make AI Agents' app for 'agentic automation' inside the platform, suggesting a built-in AI agent/assistant capability, but the evidence is a single glancing mention with no detail on how tasks are delegated or what the assistant can do. Missing for 10: detailed docs on the AI Agents app's task-delegation UX, concrete examples of use, and independent/hands-on corroboration of its capabilities.

   • [claimed-docs] “learn about agentic automation with the make ai agents app”

   proof ↗⚑ flag

   Windmill~partialclaimed6/10

   Windmill's docs describe an in-product AI generation feature that can edit scripts/flows and then prompts the user to review and deploy those changes, indicating a built-in assistant users can delegate authoring tasks to (windmill-docs-21). However, evidence is thin on the assistant's scope, limitations, or independent hands-on validation, and separate 'AI Agents' flow steps (windmill-docs-20) are for building agentic workflows rather than being the assistant itself. Missing for 10: independent/community corroboration of the AI assistant's real-world reliability, more detail on what tasks it can handle end-to-end, and comparison to failure cases.

   • [claimed-docs] “Review and deploy changes: Once the AI has edited items, ask it to review the changes and it opens the Compare & Deploy page with exactly th…”
   • [claimed-docs] “AI Agents steps allow you to integrate AI agents directly into your Windmill flows. They provide an interface to connect with various AI pro…”

   proof ↗⚑ flag

4. ai-native userOperate the product with natural-language commands

   weight 2 · round drawn
   Make~partialprobed6/10

   Make ships an official MCP server and 'Make AI Agents' app that let LLMs (Claude, ChatGPT) run and manage scenarios via natural-language tool calls, plus 'Make Skills' markdown files to guide assistants — this is real agentic/NL operation support. However, this capability is mediated entirely through external AI clients rather than a built-in natural-language command interface in Make itself, and there is no independent/hands-on corroboration of reliability. Missing for 10: evidence of a native in-product NL command bar/assistant, and third-party/hands-on validation of the MCP-driven workflow.

   • [claimed-docs] “Turns your **active** and **on-demand** scenarios into callable tools for AI”
   • [claimed-docs] “AI systems like Claude and ChatGPT act as MCP clients of Make MCP server. The server provides them access to **scenario run** and **manageme…”
   • [claimed-docs] “learn about agentic automation with the make ai agents app”
   • [claimed-docs] “Make MCP server allows AI systems, such as large language models (LLMs), to run scenarios and manage the contents of your Make account.”
   • [claimed-docs] “Make Skills are Markdown files that help your assistant reliably perform Make-specific tasks, including building scenarios, configuring modu…”
   • [probe] “official MCP server documented at https://developers.make.com/mcp-server”

   proof ↗⚑ flag

   Windmill~partialprobed6/10

   Windmill documents an MCP server that lets LLM clients trigger scripts/flows via natural-language chat (windmill-docs-17, windmill-probe-4) and an AI-generation feature where a user can ask the AI to edit and review flow/script changes (windmill-docs-21), showing real natural-language operability. However, evidence is thin—just two doc snippets—with no independent/hands-on corroboration of reliability or scope of NL commands. Missing for 10: hands-on/community validation of NL-driven operation, fuller documentation of what natural-language commands can accomplish beyond triggering and editing.

   • [claimed-docs] “With MCP, you can connect your favorite LLMs (like Claude, Cursor, or any MCP compatible client) to Windmill, allowing you to trigger your s…”
   • [claimed-docs] “Review and deploy changes: Once the AI has edited items, ask it to review the changes and it opens the Compare & Deploy page with exactly th…”
   • [probe] “official MCP server documented at https://www.windmill.dev/docs/core_concepts/mcp”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   Make—none0/10

   Make provides REST API documentation describing resource-oriented endpoints and authentication (make-docs-1, make-docs-11, make-docs-23), but no evidence shows an interactive reference with runnable/try-it examples; probes for OpenAPI/Swagger specs and even the docs page in machine-readable form returned 404s (make-probe-2, make-probe-3), suggesting no such interactive tooling exists.

   • [claimed-docs] “All requests to the Make API require authentication with the user authentication token with the relevant scopes enabled.”
   • [claimed-docs] “The Make API follows the REST API design. Make API is organized into resource-oriented URLs”
   • [claimed-docs] “If you are experiencing issues with the Make API, go to the Troubleshooting section for tips about what could go wrong and how to fix it.”
   • [probe] “PROBE docs-md: HTTP 200 at https://developers.make.com/api-documentation.md # Page Not Found The URL `api-documentation` does not exist. Th…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://developers.make.com/openapi.json, https://developers.make.com/swagger.json, https://develope…”

   proof ↗⚑ flag

   Windmill—none0/10

   The evidence pack shows an explicit probe for an OpenAPI/Swagger interactive reference that returned 404 on all candidate paths, and no other citation describes an interactive API reference with runnable examples (docs pages are static markdown/CLI/MCP references, not a runnable API explorer).

   • [probe] “PROBE openapi: all candidate paths 404 (https://www.windmill.dev/openapi.json, https://www.windmill.dev/swagger.json, https://www.windmill.d…”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   Make—none0/10

   Make documents its REST API but explicit probes for OpenAPI/swagger specs at standard paths all returned 404, and no downloadable machine-readable spec is referenced anywhere in the docs pack.

   • [claimed-docs] “The Make API follows the REST API design. Make API is organized into resource-oriented URLs”
   • [probe] “PROBE docs-md: HTTP 200 at https://developers.make.com/api-documentation.md # Page Not Found The URL `api-documentation` does not exist. Th…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://developers.make.com/openapi.json, https://developers.make.com/swagger.json, https://develope…”

   proof ↗⚑ flag

   Windmill—none0/10

   Windmill exposes a CLI and API-driven platform, so a downloadable OpenAPI spec is a fair ask, but the evidence pack explicitly shows a probe attempt failing to find any OpenAPI/swagger file at standard paths (all 404s), and no docs page links to a machine-readable spec.

   • [probe] “PROBE openapi: all candidate paths 404 (https://www.windmill.dev/openapi.json, https://www.windmill.dev/swagger.json, https://www.windmill.d…”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round drawn
   Make—none0/10

   No evidence of a sandbox environment, staging account, or test/production data separation for Make; the docs cover webhooks, scenarios, error handling, and MCP server but nothing about isolating test runs from production data.

   ⚑ flag

   Windmill—none0/10

   The evidence pack covers Windmill's languages, orchestration, permissions, CLI, git sync, and MCP integration, but nowhere documents a dedicated sandbox/staging workspace or test-mode that isolates execution from production data; a community comment even raises this exact question (non-destructive branch testing) without an answered example.

   • [community] “My first questions with any new tool of this kind: Can I run/validate branch versions non-destructively as part of pre-merge checks? Can I r…”

   proof ↗⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   Make—none0/10

   The evidence shows Make has a REST API and MCP server documentation, but nowhere is there mention of API versioning scheme, version numbers, or a documented deprecation policy for breaking changes. Probes even show broken/missing OpenAPI spec links, further suggesting no formal versioning artifact is exposed.

   • [claimed-docs] “The Make API follows the REST API design. Make API is organized into resource-oriented URLs”
   • [claimed-docs] “If you are experiencing issues with the Make API, go to the Troubleshooting section for tips about what could go wrong and how to fix it.”
   • [probe] “PROBE docs-md: HTTP 200 at https://developers.make.com/api-documentation.md # Page Not Found The URL `api-documentation` does not exist. Th…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://developers.make.com/openapi.json, https://developers.make.com/swagger.json, https://develope…”

   proof ↗⚑ flag

   Windmill—none0/10

   Windmill exposes webhooks and version-pinned flow endpoints (windmill-docs-16), and a CLI/API surface exists, but there is no evidence of a documented API versioning scheme or deprecation policy; OpenAPI spec probes returned 404s (windmill-probe-3) and no docs mention deprecation practices.

   • [claimed-docs] “Flows additionally expose version-pinned webhook endpoints that target a specific flow version by its numeric version ID instead of the late…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://www.windmill.dev/openapi.json, https://www.windmill.dev/swagger.json, https://www.windmill.d…”

   proof ↗⚑ flag

Agent integration

1. ai-native userHave an agent create, update, and activate a workflow programmatically through the public API

   weight 2 · round drawn
   Make~partialprobed5/10

   Make's public REST API is documented as resource-oriented and requires token auth, and the MCP server explicitly allows AI systems to view and modify scenarios, run them, and manage account contents, implying create/update/activate capabilities. However, no evidence explicitly documents an API endpoint or example for creating a new scenario or toggling activation state, and the docs snippets are largely generic descriptions rather than concrete workflow-lifecycle examples. missing for 10: explicit API endpoint documentation for scenario creation, explicit 'activate' endpoint/example, and independent confirmation of programmatic activation.

   • [claimed-docs] “The Make API follows the REST API design. Make API is organized into resource-oriented URLs”
   • [claimed-docs] “View and modify scenarios and their related entities (e.g., connections, webhooks, and data stores)”
   • [claimed-docs] “Make MCP server allows AI systems, such as large language models (LLMs), to run scenarios and manage the contents of your Make account.”
   • [claimed-docs] “All requests to the Make API require authentication with the user authentication token with the relevant scopes enabled.”
   • [probe] “official MCP server documented at https://developers.make.com/mcp-server”

   proof ↗⚑ flag

   Windmill~partialprobed5/10

   Windmill exposes multiple programmatic paths (CLI `wmill`, git-sync auto-deploy, version-pinned webhooks, and an MCP server that lets LLM clients trigger scripts/flows) that together suggest an agent could manage workflows without the UI, and AI-generation docs show agents can edit items and reach a deploy step. However, the story specifically requires create/update/activate via the public API, and the evidence never surfaces a documented REST/OpenAPI spec (the openapi.json probe 404'd on all candidate paths) nor confirms MCP or CLI can create/activate flows rather than just trigger existing ones. missing for 10: explicit public API/OpenAPI docs for flow CRUD, confirmation that agent-driven creation/activation (not just triggering) works end-to-end without human deploy step.

   • [claimed-docs] “The Windmill CLI, `wmill` allows you to interact with Windmill instances right from your terminal.”
   • [claimed-docs] “Each time an item is deployed, Windmill will create and push a commit to the specified repository. ... Windmill can automatically deploy new…”
   • [claimed-docs] “Flows additionally expose version-pinned webhook endpoints that target a specific flow version by its numeric version ID instead of the late…”
   • [claimed-docs] “With MCP, you can connect your favorite LLMs (like Claude, Cursor, or any MCP compatible client) to Windmill, allowing you to trigger your s…”
   • [claimed-docs] “Review and deploy changes: Once the AI has edited items, ask it to review the changes and it opens the Compare & Deploy page with exactly th…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://www.windmill.dev/openapi.json, https://www.windmill.dev/swagger.json, https://www.windmill.d…”
   • [probe] “official MCP server documented at https://www.windmill.dev/docs/core_concepts/mcp”
   • [probe] “official CLI documented at https://www.windmill.dev/docs/advanced/cli”

   proof ↗⚑ flag

2. ai-native userExpose my workflows or connected app actions as MCP tools that an external agent can call

   weight 3 · round drawn
   Make✓fullprobed8/10

   Make offers an official MCP server that turns active/on-demand scenarios into callable tools for AI agents like Claude/ChatGPT, exposing scenario run and management capabilities via a documented protocol; scenarios can connect to third-party app actions, making them exposable as tools. Missing for 10: independent hands-on validation of the MCP server beyond first-party docs and more detail on granular scoping/security of exposed tools.

   • [claimed-docs] “Turns your **active** and **on-demand** scenarios into callable tools for AI”
   • [claimed-docs] “AI systems like Claude and ChatGPT act as MCP clients of Make MCP server. The server provides them access to **scenario run** and **manageme…”
   • [claimed-docs] “AI systems like Claude and ChatGPT act as MCP clients of Make MCP server.”
   • [claimed-docs] “Turns your active and on-demand scenarios into callable tools for AI”
   • [claimed-docs] “Make MCP server allows AI systems, such as large language models (LLMs), to run scenarios and manage the contents of your Make account.”
   • [probe] “official MCP server documented at https://developers.make.com/mcp-server”

   proof ↗⚑ flag

   Windmill✓fullprobed8/10

   Windmill has documented, first-party MCP server support that exposes scripts/flows as tools callable by external MCP-compatible agents (Claude, Cursor, etc.), directly matching the story. missing for 10: independent/hands-on confirmation of the MCP feature working, and more detail on scoping which specific workflows/actions are exposed as tools.

   • [claimed-docs] “With MCP, you can connect your favorite LLMs (like Claude, Cursor, or any MCP compatible client) to Windmill, allowing you to trigger your s…”
   • [probe] “official MCP server documented at https://www.windmill.dev/docs/core_concepts/mcp”

   proof ↗⚑ flag

Ai authoring

1. ai-native userGenerate or edit a workflow from a natural-language prompt

   weight 2 · round to Windmill
   Make~partialclaimed5/10

   Make's docs show 'Make Skills' (Markdown files) enabling an AI assistant to build and configure scenarios via the MCP server, and the MCP server lets AI systems like Claude/ChatGPT manage scenario contents — implying prompt-driven workflow creation/editing through external AI clients. However, this is an indirect, third-party-assistant-mediated path rather than a documented native in-product 'type a prompt, get a workflow' feature, and there's no hands-on or independent verification of it working. Missing for 10: a native first-party natural-language-to-workflow builder in the Make UI, concrete examples/screenshots of prompt-generated scenarios, and independent hands-on corroboration.

   • [claimed-docs] “Make Skills are Markdown files that help your assistant reliably perform Make-specific tasks, including building scenarios, configuring modu…”
   • [claimed-docs] “Make Skills are Markdown files that help your assistant reliably perform Make-specific tasks, including building scenarios, configuring modu…”
   • [claimed-docs] “Make MCP server allows AI systems, such as large language models (LLMs), to run scenarios and manage the contents of your Make account.”
   • [claimed-docs] “learn about agentic automation with the make ai agents app”

   proof ↗⚑ flag

   Windmill~partialclaimed6/10

   windmill-docs-21 explicitly documents AI-driven editing of scripts/flows/apps followed by a review-and-deploy step, and windmill-docs-20 shows AI agent steps can be embedded in flows, indicating genuine natural-language generation/editing support. However, evidence is purely first-party docs with no independent/hands-on corroboration of prompt-to-workflow quality or reliability. Missing for 10: independent/community validation of AI-generated flow quality, details on scope/limits of NL-to-workflow generation, and confirmation this works for full flow creation (not just editing existing items).

   • [claimed-docs] “Review and deploy changes: Once the AI has edited items, ask it to review the changes and it opens the Compare & Deploy page with exactly th…”
   • [claimed-docs] “AI Agents steps allow you to integrate AI agents directly into your Windmill flows. They provide an interface to connect with various AI pro…”

   proof ↗⚑ flag

Ai steps

1. ai-native userAdd AI agent or LLM steps inside a workflow, with model choice and tool use

   weight 3 · round to Windmill
   Make~partialclaimed5/10

   Make has a dedicated 'AI Agents' app (make-docs-9) for agentic automation within workflows, implying model choice and tool use, but the evidence pack lacks first-party documentation detailing how to configure model selection or tool/function definitions within an agent step — most evidence instead focuses on the MCP server (which exposes Make scenarios as tools to external AI clients like Claude/ChatGPT, the reverse direction). missing for 10: detailed docs on adding an AI/LLM step inside a scenario, configuring model provider/choice, and defining tool use within that step, plus independent/hands-on confirmation of this workflow.

   • [claimed-docs] “learn about agentic automation with the make ai agents app”
   • [claimed-docs] “Turns your **active** and **on-demand** scenarios into callable tools for AI”
   • [claimed-docs] “Make MCP server allows AI systems, such as large language models (LLMs), to run scenarios and manage the contents of your Make account.”

   proof ↗⚑ flag

   Windmill✓fullclaimed8/10

   Windmill has documented AI Agent steps that integrate directly into flows, connecting to various AI providers/models and supporting tool use within the workflow orchestration engine (windmill-docs-20). This directly matches the story of adding LLM/agent steps with model choice and tool use inside a workflow. Missing for 10: independent/hands-on corroboration of tool-use behavior and broader detail on model selection UX beyond the single docs page.

   • [claimed-docs] “AI Agents steps allow you to integrate AI agents directly into your Windmill flows. They provide an interface to connect with various AI pro…”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round drawn
   Make~partialclaimed5/10

   Make's webhook queue processing and API/data-store functions imply some capacity for handling batches of items via scenarios, and the REST API with scoped tokens could be scripted for bulk actions, but no evidence documents a native bulk-operation feature (e.g., batch endpoints, bulk item processing UI, or iterator-based bulk automation) for AI-native use. missing for 10: explicit documentation of batch/bulk API endpoints or iterator modules for processing many items in one call, evidence of AI agents invoking bulk operations via MCP server, and any hands-on confirmation of bulk throughput/performance.

   • [claimed-docs] “custom webhooks allow you to create a url to which you can send any data”
   • [claimed-docs] “If you don't want to run your immediately after a webhook receives data, you can schedule your to process all webhook requests periodically”
   • [claimed-docs] “the whole queue is then processed every time your schedule criteria are met”
   • [claimed-docs] “transform and format data using our range of functions”
   • [claimed-docs] “The Make API follows the REST API design. Make API is organized into resource-oriented URLs”
   • [claimed-docs] “Make MCP server allows AI systems, such as large language models (LLMs), to run scenarios and manage the contents of your Make account.”

   proof ↗⚑ flag

   Windmill~partialclaimed5/10

   Windmill's flows support parallelism, branching, and fault-tolerant orchestration (windmill-docs-5, windmill-docs-12), which implies capability to iterate/process many items in a flow, but no evidence explicitly documents a for-loop/bulk-iteration step, batch processing UI, or examples of running an operation across a large item set. missing for 10: explicit for-loop/iterator flow step documentation, evidence of scaling to large item counts, and any hands-on confirmation of bulk operation performance.

   • [claimed-docs] “Workflows as code let you define orchestration logic directly in TypeScript or Python, using familiar language constructs like functions, co…”
   • [claimed-docs] “Branches allow to split the execution of the flow based on a condition. ... Branch all: all the branches will be executed.”
   • [claimed-docs] “If defined, upon error this step will be retried with a delay and a maximum number of attempts as defined below.”

   proof ↗⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round to Windmill
   Make✓fullclaimed7/10

   Make's core scenario model is built on trigger-action automation: webhooks let users create URLs that trigger scenario execution on external events, schedules can batch-process queued events, and error handlers manage automated fault-response actions. This directly matches the story of defining rules that fire actions on events, though evidence is entirely first-party docs. Missing for 10: independent/hands-on verification and concrete examples of complex conditional rule logic beyond webhooks/schedules.

   • [claimed-docs] “custom webhooks allow you to create a url to which you can send any data”
   • [claimed-docs] “If you don't want to run your immediately after a webhook receives data, you can schedule your to process all webhook requests periodically”
   • [claimed-docs] “webhooks create a url that you can call from an external app or service, or from another use webhooks to trigger the execution”
   • [claimed-docs] “you can schedule your to process all webhook requests periodically”
   • [claimed-docs] “the whole queue is then processed every time your schedule criteria are met”
   • [claimed-docs] “webhooks create a url that you can call from an external app or service, or from another use webhooks to trigger the execution of”
   • [claimed-docs] “this page helps users navigate errors, diagnose and resolve issues in by providing detailed information on common errors and warnings, error…”
   • [claimed-docs] “diagnose and resolve issues in by providing detailed information on common errors and warnings, error handlers”

   proof ↗⚑ flag

   Windmill✓fullcommunity8/10

   Windmill lets users define scripts/flows triggered automatically by schedules (CRON), webhooks, emails, HTTP routes, websockets, Kafka, Postgres, NATS, SQS, MQTT, and other event sources, with retry/error handling and branching logic for rule-based automation. This directly matches the story of defining rules that trigger actions on events, with strong first-party documentation. Missing for 10: independent/hands-on verification of complex event-rule chains and no community corroboration specifically confirming event-trigger reliability in production.

   • [claimed-docs] “How do I trigger scripts and flows in Windmill? Use schedules, webhooks, emails, HTTP routes, websockets, Kafka, Postgres, NATS, SQS, MQTT, …”
   • [claimed-docs] “A Schedule consists of a Script or Flow, its arguments, a CRON expression that controls the execution frequency and optional Error and Recov…”
   • [claimed-docs] “Flows additionally expose version-pinned webhook endpoints that target a specific flow version by its numeric version ID instead of the late…”
   • [claimed-docs] “If defined, upon error this step will be retried with a delay and a maximum number of attempts as defined below.”
   • [claimed-docs] “Branches allow to split the execution of the flow based on a condition. ... Branch all: all the branches will be executed.”
   • [community] “This looks really cool. Is there error handling? For example, if the email sending fails for some reason can I either select the flow to con…”

   proof ↗⚑ flag

3. ai-native userSchedule recurring jobs or workflows

   weight 2 · round to Windmill
   Make✓fullclaimed7/10

   Make's docs confirm native scenario scheduling ('schedule a scenario') and webhook queues that can be processed on a periodic schedule, directly supporting recurring automated workflows. Missing for 10: independent/hands-on confirmation of scheduling reliability, details on schedule granularity/frequency options, and any AI-specific scheduling interface beyond generic scenario scheduling.

   • [claimed-docs] “schedule a scenario docid 8rwfo krohjlepg4qhx3 clone a scenario docid\ c8f35kwpicyg1az5vlgehdelete a scenario”
   • [claimed-docs] “If you don't want to run your immediately after a webhook receives data, you can schedule your to process all webhook requests periodically”
   • [claimed-docs] “you can schedule your to process all webhook requests periodically”
   • [claimed-docs] “the whole queue is then processed every time your schedule criteria are met”

   proof ↗⚑ flag

   Windmill✓fullclaimed9/10

   Windmill has a dedicated Scheduling core concept supporting CRON-based recurring execution of scripts/flows with error/recovery handlers, plus flexible triggers (webhooks, events, etc.) and retries/branching for robust workflows, all documented first-party. missing for 10: no independent/hands-on community confirmation specifically of scheduling reliability in production.

   • [claimed-docs] “A Schedule consists of a Script or Flow, its arguments, a CRON expression that controls the execution frequency and optional Error and Recov…”
   • [claimed-docs] “How do I trigger scripts and flows in Windmill? Use schedules, webhooks, emails, HTTP routes, websockets, Kafka, Postgres, NATS, SQS, MQTT, …”
   • [claimed-docs] “If defined, upon error this step will be retried with a delay and a maximum number of attempts as defined below.”
   • [claimed-docs] “Branches allow to split the execution of the flow based on a condition. ... Branch all: all the branches will be executed.”

   proof ↗⚑ flag

4. ai-native userVersion, review, and roll back my automations

   weight 1 · round to Windmill
   Make~partialclaimed5/10

   Make documents 'scenario history' and the ability to 'clone a scenario', which suggest some versioning/backup capability, but there is no explicit documentation of a review/diff UI or a true rollback mechanism restoring a prior version. missing for 10: explicit rollback/revert feature docs, version diff/review UI, and independent confirmation that history can restore a scenario to an earlier state.

   • [claimed-docs] “clone a scenario”
   • [claimed-docs] “scenario history”
   • [claimed-docs] “schedule a scenario docid 8rwfo krohjlepg4qhx3 clone a scenario docid\ c8f35kwpicyg1az5vlgehdelete a scenario”

   proof ↗⚑ flag

   Windmill~partialclaimed6/10

   Windmill documents versioning via Git sync (each deploy creates a commit, auto-deploy from repo) and version-pinned webhook endpoints for flows, plus a 'Compare & Deploy' review UI when AI edits items. However, there is no explicit rollback feature or UI documented (e.g., one-click revert to a prior version), and no independent/hands-on confirmation of rollback behavior. missing for 10: explicit rollback/revert mechanism, independent verification of version history UI and rollback workflow.

   • [claimed-docs] “Each time an item is deployed, Windmill will create and push a commit to the specified repository. ... Windmill can automatically deploy new…”
   • [claimed-docs] “Flows additionally expose version-pinned webhook endpoints that target a specific flow version by its numeric version ID instead of the late…”
   • [claimed-docs] “Review and deploy changes: Once the AI has edited items, ask it to review the changes and it opens the Compare & Deploy page with exactly th…”
   • [claimed-docs] “The Windmill CLI, `wmill` allows you to interact with Windmill instances right from your terminal.”

   proof ↗⚑ flag

Code steps

1. developerDrop into real code (JavaScript or Python) as a step inside a workflow

   weight 3 · round to Windmill
   Make—none0/10

   The evidence pack covers Make's API, webhooks, MCP server, error handling, and functions, but contains no mention of a custom code step, JavaScript/Python module, or code-execution capability inside a Make scenario. No evidence supports the ability to write and run custom code as a workflow step.

   ⚑ flag

   Windmill✓fullcommunity9/10

   Windmill's core model is scripts written in real code (TypeScript, Python, Go, etc.) composed into flows, with 'workflows as code' letting developers write orchestration logic directly in TypeScript/Python using native functions, conditionals, and loops as flow steps. This is corroborated by GitHub docs describing scripts being composed into flows, and community discussion confirms multi-language script support. Missing for 10: independent hands-on verification beyond community anecdotes and no mention of limitations in language runtime sandboxing raised by a commenter.

   • [claimed-docs] “It supports coding in TypeScript, Python, Go, PHP, Bash, C#, SQL, Rust, Ruby and R, or any Docker image”
   • [claimed-docs] “Workflows as code let you define orchestration logic directly in TypeScript or Python, using familiar language constructs like functions, co…”
   • [claimed-docs] “An orchestrator for assembling these functions into efficient, low-latency flows, using either a low-code builder or YAML”
   • [github] “Scripts are turned into sharable UIs automatically, and can be composed together into flows or used into richer apps built with low-code.”
   • [community] “Was hoping it allowed my runtime. I have a bunch of scripts in NodeJS, Python 2, C#, etc. I don't care to rewrite them and/or write scripts …”

   proof ↗⚑ flag

Connector dev

1. developerBuild a custom connector or private integration with a documented developer platform, SDK, or CLI

   weight 2 · round to Windmill
   Make✓fullprobed7/10

   Make provides a documented Custom Apps platform (App components: Base, Connections, Webhooks, Modules, Remote Procedure Calls) plus a REST API for building custom connectors/integrations, directly matching the story. Missing for 10: no confirmed official CLI or OpenAPI spec (probe found 404s for openapi.json/swagger.json and a broken docs-md page), and no independent/hands-on corroboration of connector-building success.

   • [claimed-docs] “App components * [Base] * [Connections] * [Webhooks] * [Modules] * [Remote Procedure Calls]”
   • [claimed-docs] “The Make API follows the REST API design. Make API is organized into resource-oriented URLs”
   • [claimed-docs] “All requests to the Make API require authentication with the user authentication token with the relevant scopes enabled.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://developers.make.com/openapi.json, https://developers.make.com/swagger.json, https://develope…”
   • [probe] “PROBE docs-md: HTTP 200 at https://developers.make.com/api-documentation.md # Page Not Found The URL `api-documentation` does not exist. Th…”

   proof ↗⚑ flag

   Windmill✓fullprobed8/10

   Windmill offers a documented CLI (wmill) for interacting with instances and deploying from Git, multi-language scripting (TypeScript, Python, Go, etc. or custom Docker images), a flows/apps builder, webhooks, triggers, and an MCP server for connecting external clients, all backed by extensive first-party docs. This directly supports building custom integrations/connectors via a developer platform. missing for 10: no public OpenAPI/REST API spec found (probe returned 404s), and no independent third-party SDK usage reports corroborating the CLI/platform beyond vendor docs.

   • [claimed-docs] “The Windmill CLI, `wmill` allows you to interact with Windmill instances right from your terminal.”
   • [claimed-docs] “It supports coding in TypeScript, Python, Go, PHP, Bash, C#, SQL, Rust, Ruby and R, or any Docker image”
   • [claimed-docs] “Windmill supports both UI-based operations via its webIDE and low-code builders, as well as CLI deployments from a Git repository”
   • [claimed-docs] “Each time an item is deployed, Windmill will create and push a commit to the specified repository. ... Windmill can automatically deploy new…”
   • [claimed-docs] “How do I trigger scripts and flows in Windmill? Use schedules, webhooks, emails, HTTP routes, websockets, Kafka, Postgres, NATS, SQS, MQTT, …”
   • [claimed-docs] “With MCP, you can connect your favorite LLMs (like Claude, Cursor, or any MCP compatible client) to Windmill, allowing you to trigger your s…”
   • [probe] “official MCP server documented at https://www.windmill.dev/docs/core_concepts/mcp”
   • [probe] “official CLI documented at https://www.windmill.dev/docs/advanced/cli”
   • [probe] “PROBE openapi: all candidate paths 404 (https://www.windmill.dev/openapi.json, https://www.windmill.dev/swagger.json, https://www.windmill.d…”

   proof ↗⚑ flag

Data mapping

1. ops userMap and transform data between steps with expressions, functions, or formulas

   weight 2 · round to Windmill
   Make~partialclaimed6/10

   Make docs confirm a dedicated functions system for transforming/formatting data between modules (make-docs-8), which is the core mapping/expression mechanism ops users use between steps. However, evidence lacks depth on formula syntax, custom function scripting (e.g., IML/JS custom functions), or hands-on examples of complex transformations. missing for 10: detailed docs/examples of expression syntax, custom JS functions, and independent validation of transformation capabilities.

   • [claimed-docs] “transform and format data using our range of functions”

   proof ↗⚑ flag

   Windmill✓fullclaimed8/10

   Windmill lets ops users write full scripts in TypeScript, Python, Go, SQL, Bash, etc. to transform data, and 'workflows as code' explicitly supports functions, conditionals, and loops between flow steps, plus branch logic for conditional data flow; the low-code flow builder also composes these steps together for mapping/transformation between steps. Missing for 10: no dedicated documentation on a lightweight expression/formula language for simple inline mappings (e.g., JS expression fields) distinct from full scripts, and no independent/hands-on corroboration of transformation ergonomics.

   • [claimed-docs] “It supports coding in TypeScript, Python, Go, PHP, Bash, C#, SQL, Rust, Ruby and R, or any Docker image”
   • [claimed-docs] “An orchestrator for assembling these functions into efficient, low-latency flows, using either a low-code builder or YAML”
   • [claimed-docs] “Workflows as code let you define orchestration logic directly in TypeScript or Python, using familiar language constructs like functions, co…”
   • [claimed-docs] “Branches allow to split the execution of the flow based on a condition. ... Branch all: all the branches will be executed.”
   • [github] “Scripts are turned into sharable UIs automatically, and can be composed together into flows or used into richer apps built with low-code.”

   proof ↗⚑ flag

Credentials

1. ops userStore connection credentials centrally, share them with my team, and control who can use which credential

   weight 3 · round to Windmill
   Make—none0/10

   The evidence pack only lists 'Connections' as an app component category in custom-app developer docs (make-docs-10); there is no evidence describing centralized credential storage, team sharing, or per-credential access control for ops users.

   • [claimed-docs] “App components * [Base] * [Connections] * [Webhooks] * [Modules] * [Remote Procedure Calls]”

   proof ↗⚑ flag

   Windmill~partialclaimed6/10

   Windmill documents centrally encrypted Variables/Secrets at the workspace level plus a broader roles/permissions system and scoped tokens for least-privilege access, which together support storing and sharing credentials with some access control. However, the evidence doesn't show fine-grained per-credential ACLs (e.g., which specific users/groups can use a specific secret) beyond general workspace/role permissions. Missing for 10: explicit per-resource/credential-level permission granularity, audit logging of credential usage, and independent/hands-on confirmation of this governance in practice.

   • [claimed-docs] “All Variables (not just secrets) are encrypted with a workspace specific symmetric key to avoid leakage.”
   • [claimed-docs] “Windmill provides a roles and permissions system that allows you to control access and manage permissions within your instance and workspace…”
   • [claimed-docs] “Tokens can be scoped to restrict access to specific resources and actions, following the principle of least privilege.”

   proof ↗⚑ flag

Human in the loop

1. ops userPause a workflow to wait for human approval or input before it continues

   weight 2 · round drawn
   Make—none0/10

   The evidence pack covers webhooks, error handling, MCP server, and API docs, but contains no mention of a human-in-the-loop approval step, pause/resume mechanism, or manual confirmation module within Make scenarios. missing for 10: any documentation of a 'wait for input/approval' or manual confirmation module, pause/resume scenario capability, or timeout-based human approval feature.

   ⚑ flag

   Windmill—none0/10

   The evidence pack covers flow branching, retries, error handling, and scheduling, but contains no mention of a suspend/approval step, human-in-the-loop gate, or 'wait for input' mechanism in Windmill flows. This is a fair axis for a workflow orchestration product, but no evidence in the pack demonstrates it.

   • [claimed-docs] “Branches allow to split the execution of the flow based on a condition. ... Branch all: all the branches will be executed.”
   • [claimed-docs] “If defined, upon error this step will be retried with a delay and a maximum number of attempts as defined below.”
   • [claimed-docs] “A script or flow can return a top-level `wm_failure` field (string) to retag the run as a failure even though the runtime did not raise an e…”

   proof ↗⚑ flag

Versioning

1. developerVersion workflows through source control or environments and promote changes from dev to production

   weight 2 · round to Windmill
   Make—none0/10

   Evidence only shows scenario cloning and scenario history features, with no documentation of source-control integration, environment separation, or dev-to-production promotion workflows for Make scenarios.

   • [claimed-docs] “clone a scenario”
   • [claimed-docs] “scenario history”
   • [claimed-docs] “schedule a scenario docid 8rwfo krohjlepg4qhx3 clone a scenario docid\ c8f35kwpicyg1az5vlgehdelete a scenario”

   proof ↗⚑ flag

   Windmill✓fullcommunity8/10

   Windmill documents Git sync where deployments push commits to a repo and can auto-deploy new commits back into workspaces, plus a dedicated CLI (`wmill`) for deploying from a Git repository, enabling version control and promotion across workspaces/environments (dev/staging/prod). This covers source-control-backed versioning and promotion workflows fairly thoroughly, though community feedback questions non-destructive branch validation and local script running pre-merge, indicating some workflow gaps. Missing for 10: independent/hands-on confirmation of a full dev-to-prod promotion pipeline and clearer support for pre-merge branch validation raised by users.

   • [claimed-docs] “Each time an item is deployed, Windmill will create and push a commit to the specified repository. ... Windmill can automatically deploy new…”
   • [claimed-docs] “The Windmill CLI, `wmill` allows you to interact with Windmill instances right from your terminal.”
   • [claimed-docs] “Windmill supports both UI-based operations via its webIDE and low-code builders, as well as CLI deployments from a Git repository”
   • [community] “My first questions with any new tool of this kind: Can I run/validate branch versions non-destructively as part of pre-merge checks? Can I r…”

   proof ↗⚑ flag

Community

1. developerInstall community-built nodes, components, or integrations contributed outside the vendor

   weight 1 · round to Make
   Make~partialclaimed4/10

   Make provides a Custom Apps SDK (Base, Connections, Webhooks, Modules, RPCs) that lets developers build custom integrations, implying a framework where community-built apps could exist, but there is no evidence of a marketplace, directory, or install flow for discovering and installing third-party/community-contributed apps or modules into a Make account. missing for 10: evidence of a public app marketplace/store, discovery of community apps, one-click install of third-party nodes, and any review/vetting process for community contributions.

   • [claimed-docs] “App components * [Base] * [Connections] * [Webhooks] * [Modules] * [Remote Procedure Calls]”

   proof ↗⚑ flag

   Windmill—none0/10

   There is no evidence of a marketplace, registry, or community-contributed nodes/integrations/components ecosystem outside the vendor for Windmill—docs describe language support, flows, CLI, git sync, and MCP but nothing about installing third-party/community-built connectors or components. The hub/scripts sharing feature, if any, isn't mentioned in the evidence pack.

   ⚑ flag

Connectors

1. ops userConnect to thousands of apps through prebuilt, vendor-maintained integrations

   weight 3 · round drawn
   Make—none0/10

   The evidence pack covers Make's API, webhooks, MCP server, and custom-app-building framework (Base/Connections/Webhooks/Modules/RPC), but contains no citation confirming a marketplace of thousands of prebuilt, vendor-maintained app connectors. Missing for 10: any documentation or listing of the app/connector marketplace, connector count, or vendor-maintenance claims.

   • [claimed-docs] “App components * [Base] * [Connections] * [Webhooks] * [Modules] * [Remote Procedure Calls]”
   • [claimed-docs] “The Make API follows the REST API design. Make API is organized into resource-oriented URLs”

   proof ↗⚑ flag

   Windmill—none0/10

   Evidence shows Windmill offers code-based scripts, flows, and triggers (webhooks, Kafka, Postgres, etc.) but no mention of a prebuilt, vendor-maintained connector/app library comparable to Zapier-style integrations; a community comment even contrasts it with Zapier as a code-first alternative rather than a connector marketplace.

   • [claimed-docs] “How do I trigger scripts and flows in Windmill? Use schedules, webhooks, emails, HTTP routes, websockets, Kafka, Postgres, NATS, SQS, MQTT, …”
   • [community] “This is brilliant. The world has been needing something like Zapier, but not opaque and locked down. Everything Zapier does is a good idea, …”
   • [community] “Was hoping it allowed my runtime. I have a bunch of scripts in NodeJS, Python 2, C#, etc. I don't care to rewrite them and/or write scripts …”

   proof ↗⚑ flag

Templates

1. ops userStart from a public library of workflow templates instead of building from scratch

   weight 2 · round drawn
   Make—none0/10

   The evidence pack covers Make's API, MCP server, webhooks, error handling, and scenario management, but contains no mention of a public template library or gallery that ops users could start from instead of building scenarios from scratch. Missing for 10: any documentation of a templates gallery, marketplace, or pre-built scenario library, and evidence of browsing/importing templates.

   ⚑ flag

   Windmill—none0/10

   No evidence of a public template library or gallery of pre-built workflows for ops users to start from; docs focus on scripting languages, orchestration, CLI, and roles/permissions but never mention a template marketplace or starter workflow gallery.

   ⚑ flag

Embedding

1. developerEmbed the automation platform white-label inside my own product for my customers

   weight 1 · round drawn
   Make—none0/10

   The evidence pack covers Make's API, webhooks, MCP server, and app-building docs, but contains no mention of white-labeling, embeddable widgets, custom branding, or an embed SDK for reselling Make inside a third-party product. Absence of evidence for this applicable capability yields 'none'.

   ⚑ flag

   Windmill—none0/10

   No evidence of white-labeling capability (custom branding, theming, hiding Windmill identity, or embeddable SDK/iframe for reselling under the developer's own brand) — only self-hosting, app builder, and guest-user features are documented, none of which address white-label embedding for customers.

   ⚑ flag

Local dev

1. developerRun workflows locally or against a dev instance for development and CI testing

   weight 2 · round to Windmill
   Make—none0/10

   Make is a cloud-hosted automation platform; the evidence pack shows only cloud scenario execution, webhooks, REST API, and MCP server management — there is no mention of a local runtime, CLI, self-hosted/dev instance, or CI testing workflow for scenarios.

   ⚑ flag

   Windmill~partialcommunity5/10

   Windmill supports self-hosting a dev instance via Docker Compose (windmill-docs-10) and provides a CLI (`wmill`) for interacting with instances and deploying from Git repos (windmill-docs-9, windmill-docs-4, windmill-docs-11), which could support CI workflows against a dev instance. However, there is no explicit documentation of a local/offline execution mode for scripts/flows independent of a running instance, and a community question directly asking 'Can I run my scripts locally without a lot of drama?' (windmill-comm-5) goes unanswered in the evidence, suggesting friction. Missing for 10: explicit docs on local script execution without an instance, CI-specific testing guides, and a direct answer to the community's local-run concern.

   • [claimed-docs] “The Windmill CLI, `wmill` allows you to interact with Windmill instances right from your terminal.”
   • [claimed-docs] “For small setups, use Docker and Docker Compose on a single instance. For larger and production use-cases, use our Helm chart to deploy on K…”
   • [claimed-docs] “Windmill supports both UI-based operations via its webIDE and low-code builders, as well as CLI deployments from a Git repository”
   • [claimed-docs] “Each time an item is deployed, Windmill will create and push a commit to the specified repository. ... Windmill can automatically deploy new…”
   • [community] “My first questions with any new tool of this kind: Can I run/validate branch versions non-destructively as part of pre-merge checks? Can I r…”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round drawn
   Make~partialprobed6/10

   Make ships a documented, resource-oriented REST API covering scenarios, connections, webhooks, and data stores (make-docs-11, make-docs-14), plus an MCP server that lets AI systems run and manage scenarios (make-docs-3, make-docs-20). However, there's no evidence of a formal, discoverable OpenAPI spec (probe found 404s at standard locations, make-probe-3) and no explicit documentation confirming full parity for scenario-building/editing logic via API as opposed to the UI or Make Skills workflow. missing for 10: explicit API endpoints for full scenario creation/editing parity, public OpenAPI schema, independent confirmation of 1:1 UI/API feature parity.

   • [claimed-docs] “The Make API follows the REST API design. Make API is organized into resource-oriented URLs”
   • [claimed-docs] “View and modify scenarios and their related entities (e.g., connections, webhooks, and data stores)”
   • [claimed-docs] “AI systems like Claude and ChatGPT act as MCP clients of Make MCP server. The server provides them access to **scenario run** and **manageme…”
   • [claimed-docs] “Make MCP server allows AI systems, such as large language models (LLMs), to run scenarios and manage the contents of your Make account.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://developers.make.com/openapi.json, https://developers.make.com/swagger.json, https://develope…”

   proof ↗⚑ flag

   Windmill~partialprobed6/10

   Windmill exposes a CLI (`wmill`) for deployments, webhooks/triggers for running scripts and flows, git-sync for pushing changes, and an MCP server for AI agents to trigger scripts/flows, showing broad programmatic access to core operations. However, there's no explicit documentation or evidence of a comprehensive REST/OpenAPI spec covering all UI actions (e.g., app-builder UI creation, permissions management, variable/secret management) via API, and the openapi probe returned 404s at standard locations. missing for 10: explicit OpenAPI/API reference confirming full UI-API parity, evidence that app-building, permissions, and admin UI actions are all scriptable via API/CLI, independent confirmation of parity.

   • [claimed-docs] “The Windmill CLI, `wmill` allows you to interact with Windmill instances right from your terminal.”
   • [claimed-docs] “Each time an item is deployed, Windmill will create and push a commit to the specified repository. ... Windmill can automatically deploy new…”
   • [claimed-docs] “How do I trigger scripts and flows in Windmill? Use schedules, webhooks, emails, HTTP routes, websockets, Kafka, Postgres, NATS, SQS, MQTT, …”
   • [claimed-docs] “Flows additionally expose version-pinned webhook endpoints that target a specific flow version by its numeric version ID instead of the late…”
   • [claimed-docs] “With MCP, you can connect your favorite LLMs (like Claude, Cursor, or any MCP compatible client) to Windmill, allowing you to trigger your s…”
   • [probe] “official MCP server documented at https://www.windmill.dev/docs/core_concepts/mcp”
   • [probe] “official CLI documented at https://www.windmill.dev/docs/advanced/cli”
   • [probe] “PROBE openapi: all candidate paths 404 (https://www.windmill.dev/openapi.json, https://www.windmill.dev/swagger.json, https://www.windmill.d…”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round to Windmill
   Make—none0/10

   Evidence covers Make's REST API, webhooks, MCP server, and scenario management, but nothing addresses a bulk/full account data export feature or open-format portability guarantee that would let a user leave with all their data. Missing for 10: documented data export/download feature, open format (e.g., JSON/CSV) export of scenarios and data stores, any GDPR-style account export or migration tooling.

   ⚑ flag

   Windmill~partialprobed6/10

   Windmill's CLI (`wmill`) and Git Sync feature let users pull/push all scripts, flows, and apps as plain code (TypeScript/Python/YAML) into a Git repository, giving a genuinely open, portable format for the core workflow logic, and the whole platform is open-source and self-hostable so users are never locked to a vendor. However the evidence pack shows no explicit mechanism for exporting the rest of the instance state — variables, secrets, execution history/logs, users/permissions — as an open-format bundle, only code artifacts. missing for 10: documented full-instance export/backup covering secrets, variables, run history, and permissions, plus independent confirmation that a git-synced workspace can be fully reconstituted elsewhere.

   • [claimed-docs] “The Windmill CLI, `wmill` allows you to interact with Windmill instances right from your terminal.”
   • [claimed-docs] “Each time an item is deployed, Windmill will create and push a commit to the specified repository. ... Windmill can automatically deploy new…”
   • [claimed-docs] “Workflows as code let you define orchestration logic directly in TypeScript or Python, using familiar language constructs like functions, co…”
   • [probe] “official CLI documented at https://www.windmill.dev/docs/advanced/cli”
   • [claimed-docs] “For small setups, use Docker and Docker Compose on a single instance. For larger and production use-cases, use our Helm chart to deploy on K…”

   proof ↗⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round to Windmill
   Make—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

   Windmill✓fullprobed7/10

   Windmill is explicitly described as 'open-source and self-hostable' in its own docs/llms.txt, and its source is publicly available on GitHub (windmill-labs/windmill), satisfying the story that an AI-native user can read the source under an open license. missing for 10: no explicit license name (e.g., AGPL/MIT) is cited in the evidence, and no independent confirmation of license terms is provided.

   • [probe] “PROBE llms.txt: HTTP 200 at https://www.windmill.dev/llms.txt # Windmill > Windmill is an open-source and self-hostable workflow engine and…”
   • [probe] “PROBE docs-md: HTTP 200 at https://www.windmill.dev/docs/intro.md # What is Windmill? > What is Windmill? An open-source workflow engine an…”
   • [github] “Scripts are turned into sharable UIs automatically, and can be composed together into flows or used into richer apps built with low-code.”
   • [community] “This looks pretty neat! Once the FOSS self-hostable version is out I might try it out for my local hackerspace, it seems like it might be us…”

   proof ↗⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round to Windmill
   Make—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

   Windmill✓fullprobed9/10

   Windmill provides official self-hosting docs with Docker Compose for small setups and Helm/Kubernetes for production, corroborated by independent community reports of successful self-hosting via Caprover and Podman+Docker Compose. Missing for 10: more recent/large-scale production self-host case studies beyond community hobbyist reports.

   • [claimed-docs] “For small setups, use Docker and Docker Compose on a single instance. For larger and production use-cases, use our Helm chart to deploy on K…”
   • [community] “Yesterday I submitted a one-click install to Caprover, never been so easy to self host. It's really fast on Oracle's free 4vCPU, 24GB RAM ma…”
   • [community] “Pretty easy to selfhost as I deployed it on a Alpine Linux server using Podman + Docker Compose binary. Really nice piece of software.”
   • [probe] “PROBE llms.txt: HTTP 200 at https://www.windmill.dev/llms.txt # Windmill > Windmill is an open-source and self-hostable workflow engine and…”

   proof ↗⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round to Windmill
   Make—none0/10

   The evidence pack contains no mention of data residency, regional data storage, or EU/US hosting options for Make; all citations concern MCP server, webhooks, API auth, and error handling. No evidence for data residency capability.

   ⚑ flag

   Windmill~partialcommunity4/10

   Windmill is self-hostable via Docker/Docker Compose or Kubernetes Helm chart, which implicitly lets a user control where their data resides by deploying in any region/infrastructure of their choosing. However, there is no explicit documentation of a region-selection feature for Windmill's hosted/cloud offering or any formal data-residency guarantee. Missing for 10: explicit cloud region selection options, formal data residency/compliance statements, and independent confirmation of self-hosted deployments meeting residency requirements.

   • [claimed-docs] “For small setups, use Docker and Docker Compose on a single instance. For larger and production use-cases, use our Helm chart to deploy on K…”
   • [community] “Yesterday I submitted a one-click install to Caprover, never been so easy to self host. It's really fast on Oracle's free 4vCPU, 24GB RAM ma…”
   • [community] “Pretty easy to selfhost as I deployed it on a Alpine Linux server using Podman + Docker Compose binary. Really nice piece of software.”

   proof ↗⚑ flag

2. ai-native userPrevent my data from being used to train AI models

   weight 3 · round drawn
   Make—none0/10

   The evidence pack contains no mention of AI training data opt-out, data usage policies for AI model training, or any privacy controls addressing this specific concern; all evidence covers API auth, MCP server, webhooks, and error handling instead.

   ⚑ flag

   Windmill—none0/10

   No evidence in the pack addresses AI-training data usage or opt-out policies for Windmill; the product is a workflow/automation engine and could plausibly publish a data-usage/privacy policy, but none is documented here.

   ⚑ flag

3. ai-native userControl data retention and deletion

   weight 2 · round drawn
   Make—none0/10

   The evidence pack contains no documentation of data retention policies, data deletion requests, or privacy/data lifecycle controls for AI-native users; only tangential scenario-management references (e.g., 'delete a scenario') exist, which do not address retention or deletion of underlying data/logs.

   • [claimed-docs] “schedule a scenario docid 8rwfo krohjlepg4qhx3 clone a scenario docid\ c8f35kwpicyg1az5vlgehdelete a scenario”

   proof ↗⚑ flag

   Windmill—none0/10

   The evidence pack covers encryption of variables, roles/permissions, and token scoping, but there is no mention of data retention policies, data deletion controls, or export/purge capabilities for an AI-native user's data. No documentation or community evidence addresses this axis.

   ⚑ flag

4. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   Make—none0/10

   No evidence pack items mention telemetry, usage tracking, analytics opt-out, or privacy settings; the pack covers API, MCP server, webhooks, and scenario docs only.

   ⚑ flag

   Windmill—none0/10

   No evidence in the pack mentions telemetry, usage tracking, or an opt-out mechanism for Windmill; the docs focus on features, self-hosting, and permissions but never address telemetry data collection. Since Windmill is self-hostable software where telemetry opt-out is a plausible and fair question, absence of evidence yields 'none' rather than 'na'. missing for 10: any mention of telemetry collection, opt-out settings/env vars, or privacy documentation addressing usage tracking.

   ⚑ flag

Durability

1. developerThrottle or queue workflow executions to respect downstream rate limits

   weight 1 · round to Make
   Make~partialclaimed4/10

   Make's webhook documentation shows a queuing mechanism where incoming webhook data can be scheduled to be processed periodically in batches rather than immediately, which functions as a basic queue (make-docs-6, make-docs-16, make-docs-21). However, there is no documented general-purpose throttling/rate-limit control for scenario executions against downstream APIs, and no explicit concurrency or rate-limit configuration feature is evidenced. Missing for 10: explicit rate-limit/throttle settings for scenario modules calling downstream APIs, documentation of concurrency controls, and any independent/hands-on confirmation that queuing reliably respects downstream limits.

   • [claimed-docs] “If you don't want to run your immediately after a webhook receives data, you can schedule your to process all webhook requests periodically”
   • [claimed-docs] “you can schedule your to process all webhook requests periodically”
   • [claimed-docs] “the whole queue is then processed every time your schedule criteria are met”
   • [claimed-docs] “custom webhooks allow you to create a url to which you can send any data”
   • [claimed-docs] “webhooks create a url that you can call from an external app or service, or from another use webhooks to trigger the execution”

   proof ↗⚑ flag

   Windmill—none0/10

   The evidence pack shows retries, scheduling, and branching (windmill-docs-13, windmill-docs-18, windmill-docs-12) but contains no mention of concurrency limits, execution throttling, or queuing mechanisms to respect downstream rate limits. Missing for 10: any documentation of concurrency-limit settings, rate-limit-aware queuing, or throttling controls on flows/scripts.

   • [claimed-docs] “If defined, upon error this step will be retried with a delay and a maximum number of attempts as defined below.”
   • [claimed-docs] “A Schedule consists of a Script or Flow, its arguments, a CRON expression that controls the execution frequency and optional Error and Recov…”

   proof ↗⚑ flag

2. developerRun long-lived workflows that wait for days and survive worker or platform restarts without losing state

   weight 2 · round to Windmill
   Make—none0/10

   Make is a scenario-based automation platform; while it supports scheduling and webhook queuing, there is no evidence of durable long-running workflow execution primitives (e.g., multi-day waits with guaranteed state persistence across worker/platform restarts) comparable to durable-execution frameworks. Evidence only covers webhooks, scheduling, error handling, and MCP server — none address long-lived stateful workflow durability.

   • [claimed-docs] “custom webhooks allow you to create a url to which you can send any data”
   • [claimed-docs] “If you don't want to run your immediately after a webhook receives data, you can schedule your to process all webhook requests periodically”
   • [claimed-docs] “the whole queue is then processed every time your schedule criteria are met”
   • [claimed-docs] “this page helps users navigate errors, diagnose and resolve issues in by providing detailed information on common errors and warnings, error…”

   proof ↗⚑ flag

   Windmill~partialclaimed5/10

   Windmill's docs describe 'workflows as code' with built-in checkpointing, parallelism, and fault tolerance, plus retry/error-handling mechanisms (windmill-docs-5, windmill-docs-13, windmill-docs-19), which imply support for durable, resumable flows. However, there is no explicit documentation or independent confirmation of long-duration (multi-day) sleeps/waits or of state surviving worker/platform restarts specifically. Missing for 10: explicit docs on sleep/suspend semantics for days-long waits, and hands-on/community evidence confirming state survives worker or platform restarts.

   • [claimed-docs] “Workflows as code let you define orchestration logic directly in TypeScript or Python, using familiar language constructs like functions, co…”
   • [claimed-docs] “If defined, upon error this step will be retried with a delay and a maximum number of attempts as defined below.”
   • [claimed-docs] “A script or flow can return a top-level `wm_failure` field (string) to retag the run as a failure even though the runtime did not raise an e…”
   • [claimed-docs] “A Schedule consists of a Script or Flow, its arguments, a CRON expression that controls the execution frequency and optional Error and Recov…”

   proof ↗⚑ flag

Error handling

1. developerConfigure automatic retries with backoff for failed steps or activities

   weight 3 · round to Windmill
   Make~partialclaimed3/10

   Make's docs repeatedly reference a dedicated 'error handling' system with 'error handlers' for diagnosing and resolving scenario failures, implying some built-in mechanism for handling failed steps, but none of the evidence explicitly mentions configurable retry counts or backoff intervals. missing for 10: explicit documentation of a retry directive with configurable attempts/backoff, and any hands-on confirmation of this behavior.

   • [claimed-docs] “this page helps users navigate errors, diagnose and resolve issues in by providing detailed information on common errors and warnings, error…”
   • [claimed-docs] “diagnose and resolve issues in by providing detailed information on common errors and warnings, error handlers”
   • [claimed-docs] “detailed information on common errors and warnings, error handlers, and how to use them”
   • [claimed-docs] “detailed information on common errors and warnings, error handlers, and how to use them in your”

   proof ↗⚑ flag

   Windmill✓fullclaimed9/10

   Windmill's docs explicitly document a retries feature for flow steps with configurable delay and maximum attempts on error, directly matching the story, and error/recovery handlers extend this to scheduled flows. Missing for 10: independent hands-on confirmation of retry/backoff behavior in production use beyond docs.

   • [claimed-docs] “If defined, upon error this step will be retried with a delay and a maximum number of attempts as defined below.”
   • [claimed-docs] “A Schedule consists of a Script or Flow, its arguments, a CRON expression that controls the execution frequency and optional Error and Recov…”

   proof ↗⚑ flag

2. ops userDefine dedicated error-handling paths or error workflows and get notified when a run fails

   weight 2 · round to Windmill
   Make~partialclaimed5/10

   Make's help docs confirm a dedicated error-handling system with 'error handlers' for diagnosing and resolving failed runs, which is the core mechanism ops users would use to define error-handling paths. However, the evidence never explicitly describes configurable failure notifications (e.g., email/Slack alerts on scenario failure) or how error routes are wired into scenarios beyond generic mentions. Missing for 10: explicit notification-on-failure documentation, concrete error-handler route configuration details, and independent/hands-on confirmation.

   • [claimed-docs] “this page helps users navigate errors, diagnose and resolve issues in by providing detailed information on common errors and warnings, error…”
   • [claimed-docs] “diagnose and resolve issues in by providing detailed information on common errors and warnings, error handlers”
   • [claimed-docs] “detailed information on common errors and warnings, error handlers, and how to use them”
   • [claimed-docs] “detailed information on common errors and warnings, error handlers, and how to use them in your”

   proof ↗⚑ flag

   Windmill~partialcommunity7/10

   Windmill docs describe dedicated failure-handling mechanisms: retries with configurable delay/attempts (windmill-docs-13), conditional branch execution (windmill-docs-12), custom failure tagging via wm_failure (windmill-docs-19), and explicit 'Error and Recovery Handlers to deal with failed scheduled executions' (windmill-docs-18), which together support building dedicated error paths and being alerted on failure. However, the evidence doesn't detail concrete notification channels (email/Slack/webhook alerts) beyond the handler concept, and error-handler support is documented mainly in the scheduling context rather than as a general flow-level feature; a community question (windmill-comm-8) about error handling/branching is asked but not answered by the evidence pack. missing for 10: explicit documentation of flow-level (not just schedule-level) dedicated error workflows, and concrete notification-channel integration for failure alerts.

   • [claimed-docs] “If defined, upon error this step will be retried with a delay and a maximum number of attempts as defined below.”
   • [claimed-docs] “Branches allow to split the execution of the flow based on a condition. ... Branch all: all the branches will be executed.”
   • [claimed-docs] “A Schedule consists of a Script or Flow, its arguments, a CRON expression that controls the execution frequency and optional Error and Recov…”
   • [claimed-docs] “A script or flow can return a top-level `wm_failure` field (string) to retag the run as a failure even though the runtime did not raise an e…”
   • [community] “This looks really cool. Is there error handling? For example, if the email sending fails for some reason can I either select the flow to con…”

   proof ↗⚑ flag

Observability

1. ops userInspect past execution logs and re-run a failed execution, resuming from the failing step

   weight 2 · round to Make
   Make~partialclaimed4/10

   Make's docs confirm scenario execution history for inspecting past runs and dedicated error-handling documentation, which supports viewing logs and diagnosing failures, but none of the evidence describes a feature to re-run a failed execution and resume specifically from the failing step. Missing for 10: explicit documentation of a 'rerun/resume from failed step' capability, and any hands-on confirmation that partial re-execution (vs. full restart) is supported.

   • [claimed-docs] “scenario history”
   • [claimed-docs] “this page helps users navigate errors, diagnose and resolve issues in by providing detailed information on common errors and warnings, error…”
   • [claimed-docs] “diagnose and resolve issues in by providing detailed information on common errors and warnings, error handlers”
   • [claimed-docs] “detailed information on common errors and warnings, error handlers, and how to use them”
   • [claimed-docs] “detailed information on common errors and warnings, error handlers, and how to use them in your”

   proof ↗⚑ flag

   Windmill~partialclaimed3/10

   Windmill's docs describe automatic per-step retries, error handling (wm_failure tagging), and schedule error/recovery handlers, which support reliability around failures, but no evidence explicitly describes an ops user inspecting past execution logs and manually re-running a failed execution resuming from the specific failing step. missing for 10: explicit documentation of log-based re-run/resume-from-step UI or API, and confirmation this is a distinct manual operation rather than automated retry.

   • [claimed-docs] “If defined, upon error this step will be retried with a delay and a maximum number of attempts as defined below.”
   • [claimed-docs] “A Schedule consists of a Script or Flow, its arguments, a CRON expression that controls the execution frequency and optional Error and Recov…”
   • [claimed-docs] “A script or flow can return a top-level `wm_failure` field (string) to retag the run as a failure even though the runtime did not raise an e…”

   proof ↗⚑ flag

Schedules

1. ops userRun workflows on cron-style schedules with timezone control

   weight 2 · round to Windmill
   Make~partialclaimed3/10

   Evidence confirms Make has a scenario scheduling feature ("schedule a scenario") and periodic webhook queue processing, but no documentation snippet describes cron-style expressions or explicit timezone configuration options. missing for 10: cron/interval expression syntax details, explicit timezone selection UI/API evidence, independent confirmation of scheduling behavior.

   • [claimed-docs] “schedule a scenario docid 8rwfo krohjlepg4qhx3 clone a scenario docid\ c8f35kwpicyg1az5vlgehdelete a scenario”
   • [claimed-docs] “If you don't want to run your immediately after a webhook receives data, you can schedule your to process all webhook requests periodically”
   • [claimed-docs] “you can schedule your to process all webhook requests periodically”
   • [claimed-docs] “the whole queue is then processed every time your schedule criteria are met”

   proof ↗⚑ flag

   Windmill✓fullclaimed8/10

   Windmill's docs explicitly describe Schedules as a CRON expression tied to a Script or Flow with error/recovery handlers, and schedules are listed among supported trigger types alongside webhooks, HTTP routes, etc. Timezone control is a standard part of Windmill's schedule UI per its core concept documentation, though the evidence pack doesn't explicitly quote a timezone field or independent hands-on confirmation. Missing for 10: explicit citation confirming timezone selector UI, independent/community verification of scheduling reliability.

   • [claimed-docs] “A Schedule consists of a Script or Flow, its arguments, a CRON expression that controls the execution frequency and optional Error and Recov…”
   • [claimed-docs] “How do I trigger scripts and flows in Windmill? Use schedules, webhooks, emails, HTTP routes, websockets, Kafka, Postgres, NATS, SQS, MQTT, …”
   • [claimed-docs] “A script or flow can return a top-level `wm_failure` field (string) to retag the run as a failure even though the runtime did not raise an e…”

   proof ↗⚑ flag

Triggers

1. ops userTrigger workflows from events in connected apps (new record, message, email, form submission)

   weight 3 · round to Make
   Make✓fullclaimed7/10

   Make's docs describe webhooks that let external apps or events trigger scenario execution (make-docs-5,15,25) plus scheduling to batch-process trigger events (make-docs-6,16,21), which is the mechanism ops users use to fire workflows off new records, messages, forms, etc. from connected apps/modules (make-docs-10 lists Modules/Webhooks as core app components). Missing for 10: explicit named examples of app-specific instant triggers (e.g., new Gmail email, new Typeform submission) and independent/hands-on confirmation beyond vendor docs.

   • [claimed-docs] “custom webhooks allow you to create a url to which you can send any data”
   • [claimed-docs] “webhooks create a url that you can call from an external app or service, or from another use webhooks to trigger the execution”
   • [claimed-docs] “webhooks create a url that you can call from an external app or service, or from another use webhooks to trigger the execution of”
   • [claimed-docs] “If you don't want to run your immediately after a webhook receives data, you can schedule your to process all webhook requests periodically”
   • [claimed-docs] “you can schedule your to process all webhook requests periodically”
   • [claimed-docs] “the whole queue is then processed every time your schedule criteria are met”
   • [claimed-docs] “App components * [Base] * [Connections] * [Webhooks] * [Modules] * [Remote Procedure Calls]”

   proof ↗⚑ flag

   Windmill~partialclaimed6/10

   Windmill documents generic trigger mechanisms (webhooks, email, HTTP routes, Postgres, Kafka, MQTT, etc.) that can be wired to fire on external events like new records, emails, or form submissions [windmill-docs-15][windmill-docs-16]. However, there's no evidence of pre-built native connectors for specific SaaS apps (e.g., Salesforce, Gmail, Slack) that an ops user could configure without engineering setup — these are generic infra-level hooks requiring technical wiring, not app-specific integrations. missing for 10: native app connector library/marketplace, no-code trigger setup UI for specific third-party apps, evidence of non-technical ops users successfully configuring these triggers.

   • [claimed-docs] “How do I trigger scripts and flows in Windmill? Use schedules, webhooks, emails, HTTP routes, websockets, Kafka, Postgres, NATS, SQS, MQTT, …”
   • [claimed-docs] “Flows additionally expose version-pinned webhook endpoints that target a specific flow version by its numeric version ID instead of the late…”
   • [claimed-docs] “A Schedule consists of a Script or Flow, its arguments, a CRON expression that controls the execution frequency and optional Error and Recov…”

   proof ↗⚑ flag

2. developerExpose a custom webhook URL that receives external HTTP requests and starts a workflow run with the payload

   weight 3 · round drawn
   Make✓fullclaimed8/10

   Make's custom webhooks feature lets developers create a URL that receives external HTTP requests and triggers scenario (workflow) execution with the received payload, with options for immediate or scheduled/batched processing of the queue. missing for 10: independent/hands-on corroboration beyond first-party docs, and no detail on payload parsing/validation specifics.

   • [claimed-docs] “custom webhooks allow you to create a url to which you can send any data”
   • [claimed-docs] “If you don't want to run your immediately after a webhook receives data, you can schedule your to process all webhook requests periodically”
   • [claimed-docs] “webhooks create a url that you can call from an external app or service, or from another use webhooks to trigger the execution”
   • [claimed-docs] “the whole queue is then processed every time your schedule criteria are met”
   • [claimed-docs] “webhooks create a url that you can call from an external app or service, or from another use webhooks to trigger the execution of”

   proof ↗⚑ flag

   Windmill✓fullclaimed8/10

   Windmill's docs explicitly list webhooks as a trigger type that starts scripts/flows with the request payload, and even describe version-pinned webhook endpoints for flows, confirming custom webhook URLs can kick off workflow runs. missing for 10: no independent/hands-on confirmation of webhook payload handling or setup walkthrough beyond docs references.

   • [claimed-docs] “How do I trigger scripts and flows in Windmill? Use schedules, webhooks, emails, HTTP routes, websockets, Kafka, Postgres, NATS, SQS, MQTT, …”
   • [claimed-docs] “Flows additionally expose version-pinned webhook endpoints that target a specific flow version by its numeric version ID instead of the late…”

   proof ↗⚑ flag

Builder

1. ops userBranch a workflow with conditions, filters, and parallel paths that merge back together

   weight 2 · round to Windmill
   Make—none0/10

   The evidence pack contains no mention of routers, filters, conditional branching, or parallel path merging in Make scenarios — it only covers API auth, MCP server, webhooks, error handling, and scenario management pages. Missing for 10: any documentation of router/filter modules, branch conditions, or parallel-path merge behavior.

   ⚑ flag

   Windmill✓fullcommunity7/10

   Windmill's docs explicitly describe flow Branches that split execution based on conditions and a 'branch all' mode that runs all branches in parallel, directly matching conditional and parallel-path branching in a low-code flow builder. Merging paths back together is implied by the flow's linear continuation after a branch block but is not explicitly documented, and 'filters' as a distinct branching primitive isn't named. missing for 10: explicit doc/example of branches merging back into a single downstream step, and confirmation of filter-based branching syntax.

   • [claimed-docs] “Branches allow to split the execution of the flow based on a condition. ... Branch all: all the branches will be executed.”
   • [claimed-docs] “An orchestrator for assembling these functions into efficient, low-latency flows, using either a low-code builder or YAML”
   • [community] “This looks really cool. Is there error handling? For example, if the email sending fails for some reason can I either select the flow to con…”

   proof ↗⚑ flag

2. ops userBuild multi-step workflows in a visual editor without writing code

   weight 3 · round to Windmill
   Make~partialclaimed4/10

   Evidence confirms Make's core building blocks—scenarios, modules, connections, webhooks, functions, and error handlers—implying a scenario-based workflow model, but nothing in the pack explicitly describes a visual drag-and-drop editor or no-code experience. Missing for 10: explicit description of the visual canvas/editor UI, no-code claims, and evidence of building multi-step workflows without code.

   • [claimed-docs] “clone a scenario”
   • [claimed-docs] “scenario history”
   • [claimed-docs] “schedule a scenario docid 8rwfo krohjlepg4qhx3 clone a scenario docid\ c8f35kwpicyg1az5vlgehdelete a scenario”
   • [claimed-docs] “transform and format data using our range of functions”
   • [claimed-docs] “this page helps users navigate errors, diagnose and resolve issues in by providing detailed information on common errors and warnings, error…”
   • [claimed-docs] “webhooks create a url that you can call from an external app or service, or from another use webhooks to trigger the execution”

   proof ↗⚑ flag

   Windmill~partialcommunity6/10

   Windmill's docs confirm a visual/low-code flow builder (drag-and-drop orchestration, branches, retries, scheduling) that lets users assemble flows without writing orchestration logic (windmill-docs-2, windmill-docs-12, windmill-docs-13, windmill-docs-18, windmill-comm-8). However, Windmill is fundamentally code-centric — individual flow steps are typically scripts in TypeScript/Python/etc, and the docs repeatedly describe it as 'low-code' rather than no-code, with community feedback confirming it's aimed at developers writing scripts (windmill-docs-1, windmill-comm-3). Missing for 10: evidence of pure no-code step types (e.g. built-in connectors/actions requiring zero code), and hands-on testimony from a non-technical ops user successfully building a flow without touching code.

   • [claimed-docs] “An orchestrator for assembling these functions into efficient, low-latency flows, using either a low-code builder or YAML”
   • [claimed-docs] “Branches allow to split the execution of the flow based on a condition. ... Branch all: all the branches will be executed.”
   • [claimed-docs] “If defined, upon error this step will be retried with a delay and a maximum number of attempts as defined below.”
   • [claimed-docs] “A Schedule consists of a Script or Flow, its arguments, a CRON expression that controls the execution frequency and optional Error and Recov…”
   • [claimed-docs] “It supports coding in TypeScript, Python, Go, PHP, Bash, C#, SQL, Rust, Ruby and R, or any Docker image”
   • [community] “Was hoping it allowed my runtime. I have a bunch of scripts in NodeJS, Python 2, C#, etc. I don't care to rewrite them and/or write scripts …”
   • [community] “This looks really cool. Is there error handling? For example, if the email sending fails for some reason can I either select the flow to con…”

   proof ↗⚑ flag

Composition

1. developerCompose reusable sub-workflows or modules that other workflows call

   weight 2 · round to Windmill
   Make~partialclaimed4/10

   Evidence shows webhooks can be used to trigger one scenario's execution from another scenario or external app (make-docs-15, make-docs-25), which offers a rudimentary way to compose workflows, and custom apps can define reusable 'Modules' and 'Remote Procedure Calls' (make-docs-10). However there is no explicit documentation of a native 'call another scenario/sub-workflow' module or reusable workflow-as-module composition pattern within the visual builder itself. Missing for 10: dedicated sub-scenario/sub-workflow invocation feature, parameter passing between parent/child scenarios, and independent/hands-on confirmation of this composability pattern.

   • [claimed-docs] “webhooks create a url that you can call from an external app or service, or from another use webhooks to trigger the execution”
   • [claimed-docs] “webhooks create a url that you can call from an external app or service, or from another use webhooks to trigger the execution of”
   • [claimed-docs] “App components * [Base] * [Connections] * [Webhooks] * [Modules] * [Remote Procedure Calls]”
   • [claimed-docs] “custom webhooks allow you to create a url to which you can send any data”

   proof ↗⚑ flag

   Windmill~partialclaimed6/10

   Windmill's docs and README confirm scripts/functions can be composed together into flows (windmill-gh-1, windmill-docs-2) and workflows-as-code lets you call functions/loops within flows (windmill-docs-5), implying reusable modules, but there is no explicit documentation shown for one flow invoking another flow as a sub-workflow/module across multiple parent workflows. missing for 10: explicit doc/example of a flow calling another flow as a reusable subflow, and independent confirmation of this pattern working in practice.

   • [github] “Scripts are turned into sharable UIs automatically, and can be composed together into flows or used into richer apps built with low-code.”
   • [claimed-docs] “An orchestrator for assembling these functions into efficient, low-latency flows, using either a low-code builder or YAML”
   • [claimed-docs] “Workflows as code let you define orchestration logic directly in TypeScript or Python, using familiar language constructs like functions, co…”

   proof ↗⚑ flag

Testing

1. developerTest a workflow with sample or pinned data and inspect each step's input and output before going live

   weight 2 · round drawn
   Make—none0/10

   The evidence pack covers Make's API, MCP server, webhooks, and error-handling docs, but contains no documentation of running a scenario with sample/pinned data or inspecting per-step input/output before publishing. 'Scenario history' and 'error-handling' entries are the closest topics but don't describe a test-run/inspect-bundle workflow. missing for 10: docs on 'Run once' test execution, pinned/sample data configuration, per-module input/output bundle inspection.

   • [claimed-docs] “scenario history”
   • [claimed-docs] “this page helps users navigate errors, diagnose and resolve issues in by providing detailed information on common errors and warnings, error…”
   • [claimed-docs] “diagnose and resolve issues in by providing detailed information on common errors and warnings, error handlers”

   proof ↗⚑ flag

   Windmill—none0/10

   The evidence describes Windmill's flow builder, branching, retries, and error handling, but nothing in the pack documents a test-run mode with sample/pinned data or step-by-step input/output inspection before deployment. A community question (windmill-comm-5) about non-destructive testing/local runs goes unanswered in the evidence. Missing for 10: documentation of a 'test flow' or step-by-step debug/preview feature, evidence of pinning/sample input data per step, and confirmation that outputs of each step can be inspected pre-deployment.

   • [claimed-docs] “An orchestrator for assembling these functions into efficient, low-latency flows, using either a low-code builder or YAML”
   • [claimed-docs] “Branches allow to split the execution of the flow based on a condition. ... Branch all: all the branches will be executed.”
   • [claimed-docs] “If defined, upon error this step will be retried with a delay and a maximum number of attempts as defined below.”
   • [community] “My first questions with any new tool of this kind: Can I run/validate branch versions non-destructively as part of pre-merge checks? Can I r…”

   proof ↗⚑ flag