Infrastructure as Code arenaBuyer checklist
Every requirement we judge infrastructure as code products against, as a ready-to-send RFP checklist — with each item's priority, why it matters, and how the top-ranked products score on it today.
54 requirements · 12 themes · verdicts for 4 products · updated 2026-09-16 · priorities mirror the story weights our scoring uses (methodology)
Show the markdown export
# Infrastructure as Code — buyer checklist (RFP) Derived from ProductArena's evidence-graded user-story taxonomy for Infrastructure as Code: 54 judged requirements. Priorities mirror story weights (3 = must-have, 2 = should-have, 1 = nice-to-have). ## Agenticness - [ ] **[must-have]** Plug MCP servers into this product so it can use their tools - [ ] **[must-have]** Connect an agent via an official MCP server - [ ] **[must-have]** Drive the product through a documented public API - [ ] **[must-have]** Delegate tasks to a built-in AI assistant inside the product - [ ] **[should-have]** Point an agent at llms.txt or agent-oriented docs - [ ] **[should-have]** Run the product headlessly / in CI for automation - [ ] **[should-have]** Use an official CLI - [ ] **[should-have]** Issue scoped/least-privilege API credentials for an agent - [ ] **[should-have]** Build against official SDKs - [ ] **[should-have]** Subscribe to events via webhooks - [ ] **[should-have]** Get AI-generated insights and suggestions from my data inside the product - [ ] **[should-have]** Set up automations that run autonomously in the background - [ ] **[should-have]** Operate the product with natural-language commands - [ ] **[should-have]** Explore an interactive API reference with runnable examples - [ ] **[should-have]** Download a machine-readable API spec (OpenAPI or equivalent) - [ ] **[should-have]** Rely on versioned APIs with a documented deprecation policy - [ ] **[nice-to-have]** Test against a sandbox environment without touching production data ## Automation depth - [ ] **[must-have]** Define rules that trigger actions automatically on events - [ ] **[should-have]** Perform bulk operations across many items at once - [ ] **[should-have]** Schedule recurring jobs or workflows - [ ] **[nice-to-have]** Version, review, and roll back my automations ## Import migration - [ ] **[must-have]** Import existing cloud resources under management and generate matching configuration code - [ ] **[should-have]** Migrate an existing Terraform-format codebase and its state into this tool ## Licensing governance - [ ] **[must-have]** Rely on an open license and open governance so the tool cannot be relicensed out from under my company - [ ] **[should-have]** Rely on documented compatibility promises and upgrade guides between releases ## Openness - [ ] **[must-have]** Export all of my data in open formats and leave - [ ] **[must-have]** Self-host the core product - [ ] **[should-have]** Do everything through the API that I can do in the UI - [ ] **[should-have]** Read the product's source under an open license ## Plan apply - [ ] **[must-have]** Get machine-readable plan output (JSON) that an agent can parse to reason about a proposed change - [ ] **[must-have]** Have an agent author an infrastructure change, run a plan headlessly, and present the diff for my approval - [ ] **[must-have]** Run plan and apply non-interactively in CI using saved plan artifacts and approval flags - [ ] **[must-have]** Preview exactly what will change — creates, updates, and destroys — before applying - [ ] **[should-have]** Drive deployments programmatically from my own application code rather than only through the CLI - [ ] **[should-have]** Tear down an entire environment cleanly with a destroy operation - [ ] **[nice-to-have]** Target or exclude specific resources in a plan or apply ## Policy as code - [ ] **[must-have]** Enforce policy-as-code checks that block non-compliant infrastructure changes before apply - [ ] **[should-have]** Let an agent plan and apply with least-privilege credentials and review gates so it cannot make unapproved changes ## Privacy posture - [ ] **[must-have]** Prevent my data from being used to train AI models - [ ] **[should-have]** Choose where my data is stored (region/residency) - [ ] **[should-have]** Control data retention and deletion - [ ] **[should-have]** Opt out of telemetry and usage tracking ## Providers modules - [ ] **[must-have]** Consume and publish reusable modules or components from a public registry - [ ] **[must-have]** Manage resources across all major clouds and SaaS providers through a broad provider ecosystem - [ ] **[should-have]** Generate infrastructure code from natural language using AI assistance built into the toolchain - [ ] **[should-have]** Define infrastructure in a general-purpose programming language with types, loops, and IDE support ## Secrets config - [ ] **[must-have]** Pass secrets and sensitive configuration into deployments without exposing them in code or logs - [ ] **[should-have]** Manage per-environment configuration (dev, staging, prod) as separate stacks or workspaces ## State management - [ ] **[must-have]** Detect drift between my declared configuration and the actual cloud resources - [ ] **[must-have]** Store state in a remote backend with locking so concurrent runs cannot corrupt it - [ ] **[should-have]** Encrypt state at rest so credentials and sensitive values are not readable in plaintext state files - [ ] **[should-have]** Safely inspect and modify state — moving, removing, or renaming resources — when refactoring ## Testing validation - [ ] **[should-have]** Write automated tests for my infrastructure code and run them without touching production - [ ] **[nice-to-have]** Validate and auto-format my configuration before planning --- Source: https://ultrametric.ai/productarena/arena/infra-as-code (evidence-graded verdicts for 4 products) · methodology: https://ultrametric.ai/productarena/methodology
Chips show the top 4 ranked products' current verdict on each requirement — ✓ full · ~ partial · ! disputed · — none · n/a not applicable.
Agenticness — how well agents can access and operate the productAgenticness· 17 items
How well agents can access and operate the product
- must-have
ai-native userPlug MCP servers into this product so it can use their tools
Core requirement — weighs 3× in arena scoring · no product fully delivers this yet
- must-have
ai-native userConnect an agent via an official MCP server
Core requirement — weighs 3× in arena scoring · 2 of 3 products fully deliver this today
- must-have
ai-native userDrive the product through a documented public API
Core requirement — weighs 3× in arena scoring · 1 of 4 products fully deliver this today
- must-have
ai-native userDelegate tasks to a built-in AI assistant inside the product
Core requirement — weighs 3× in arena scoring · 1 of 2 products fully deliver this today
- should-have
ai-native userPoint an agent at llms.txt or agent-oriented docs
Important, not disqualifying — weighs 2× in arena scoring · 1 of 4 products fully deliver this today
- should-have
ai-native userRun the product headlessly / in CI for automation
Important, not disqualifying — weighs 2× in arena scoring · 3 of 4 products fully deliver this today
- should-have
ai-native userUse an official CLI
Important, not disqualifying — weighs 2× in arena scoring · 3 of 4 products fully deliver this today
- should-have
ai-native userIssue scoped/least-privilege API credentials for an agent
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
- should-have
ai-native userBuild against official SDKs
Important, not disqualifying — weighs 2× in arena scoring · 1 of 4 products fully deliver this today
- should-have
ai-native userSubscribe to events via webhooks
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
- should-have
ai-native userGet AI-generated insights and suggestions from my data inside the product
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
- should-have
ai-native userSet up automations that run autonomously in the background
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
- should-have
ai-native userOperate the product with natural-language commands
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
- should-have
ai-native userExplore an interactive API reference with runnable examples
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
- should-have
ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
- should-have
ai-native userRely on versioned APIs with a documented deprecation policy
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
- nice-to-have
ai-native userTest against a sandbox environment without touching production data
Differentiator, not a dealbreaker — weighs 1× in arena scoring · no product fully delivers this yet
Automation depth — how much of the product can run unattendedAutomation depth· 4 items
How much of the product can run unattended
- must-have
ai-native userDefine rules that trigger actions automatically on events
Core requirement — weighs 3× in arena scoring · no product fully delivers this yet
- should-have
ai-native userPerform bulk operations across many items at once
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
- should-have
ai-native userSchedule recurring jobs or workflows
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
- nice-to-have
ai-native userVersion, review, and roll back my automations
Differentiator, not a dealbreaker — weighs 1× in arena scoring · no product fully delivers this yet
Import migration — stories about import migration in this arenaImport migration· 2 items
Stories about import migration in this arena
- must-have
platform-engineerImport existing cloud resources under management and generate matching configuration code
Core requirement — weighs 3× in arena scoring · 1 of 4 products fully deliver this today
- should-have
platform-engineerMigrate an existing Terraform-format codebase and its state into this tool
Important, not disqualifying — weighs 2× in arena scoring · 1 of 3 products fully deliver this today
Licensing governance — stories about licensing governance in this arenaLicensing governance· 2 items
Stories about licensing governance in this arena
- must-have
devops-leadRely on an open license and open governance so the tool cannot be relicensed out from under my company
Core requirement — weighs 3× in arena scoring · 1 of 4 products fully deliver this today
- should-have
developerRely on documented compatibility promises and upgrade guides between releases
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
Openness — open source, data portability, and self-hosting storiesOpenness· 4 items
Open source, data portability, and self-hosting stories
- must-have
ai-native userExport all of my data in open formats and leave
Core requirement — weighs 3× in arena scoring · no product fully delivers this yet
- must-have
ai-native userSelf-host the core product
Core requirement — weighs 3× in arena scoring · all 4 products fully deliver this today
- should-have
ai-native userDo everything through the API that I can do in the UI
Important, not disqualifying — weighs 2× in arena scoring · 1 of 3 products fully deliver this today
- should-have
ai-native userRead the product's source under an open license
Important, not disqualifying — weighs 2× in arena scoring · 1 of 4 products fully deliver this today
Plan apply — the plan/apply loop — previewing infrastructure changes and applying them safelyPlan apply· 7 items
The plan/apply loop — previewing infrastructure changes and applying them safely
- must-have
ai-native userGet machine-readable plan output (JSON) that an agent can parse to reason about a proposed change
Core requirement — weighs 3× in arena scoring · 3 of 4 products fully deliver this today
- must-have
ai-native userHave an agent author an infrastructure change, run a plan headlessly, and present the diff for my approval
Core requirement — weighs 3× in arena scoring · 2 of 4 products fully deliver this today
- must-have
devops-leadRun plan and apply non-interactively in CI using saved plan artifacts and approval flags
Core requirement — weighs 3× in arena scoring · 2 of 4 products fully deliver this today
- must-have
platform-engineerPreview exactly what will change — creates, updates, and destroys — before applying
Core requirement — weighs 3× in arena scoring · 3 of 4 products fully deliver this today
- should-have
developerDrive deployments programmatically from my own application code rather than only through the CLI
Important, not disqualifying — weighs 2× in arena scoring · 1 of 4 products fully deliver this today
- should-have
developerTear down an entire environment cleanly with a destroy operation
Important, not disqualifying — weighs 2× in arena scoring · 2 of 4 products fully deliver this today
- nice-to-have
platform-engineerTarget or exclude specific resources in a plan or apply
Differentiator, not a dealbreaker — weighs 1× in arena scoring · 1 of 4 products fully deliver this today
Policy as code — stories about policy as code in this arenaPolicy as code· 2 items
Stories about policy as code in this arena
- must-have
devops-leadEnforce policy-as-code checks that block non-compliant infrastructure changes before apply
Core requirement — weighs 3× in arena scoring · 1 of 4 products fully deliver this today
- should-have
ai-native userLet an agent plan and apply with least-privilege credentials and review gates so it cannot make unapproved changes
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
Privacy posture — data-handling and privacy storiesPrivacy posture· 4 items
Data-handling and privacy stories
- must-have
ai-native userPrevent my data from being used to train AI models
Core requirement — weighs 3× in arena scoring
- should-have
ai-native userChoose where my data is stored (region/residency)
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
- should-have
ai-native userControl data retention and deletion
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
- should-have
ai-native userOpt out of telemetry and usage tracking
Important, not disqualifying — weighs 2× in arena scoring · no product fully delivers this yet
Providers modules — stories about providers modules in this arenaProviders modules· 4 items
Stories about providers modules in this arena
- must-have
developerConsume and publish reusable modules or components from a public registry
Core requirement — weighs 3× in arena scoring · 1 of 4 products fully deliver this today
- must-have
developerManage resources across all major clouds and SaaS providers through a broad provider ecosystem
Core requirement — weighs 3× in arena scoring · 3 of 4 products fully deliver this today
- should-have
ai-native userGenerate infrastructure code from natural language using AI assistance built into the toolchain
Important, not disqualifying — weighs 2× in arena scoring · 1 of 3 products fully deliver this today
- should-have
developerDefine infrastructure in a general-purpose programming language with types, loops, and IDE support
Important, not disqualifying — weighs 2× in arena scoring · 1 of 4 products fully deliver this today
Secrets config — stories about secrets config in this arenaSecrets config· 2 items
Stories about secrets config in this arena
- must-have
developerPass secrets and sensitive configuration into deployments without exposing them in code or logs
Core requirement — weighs 3× in arena scoring · no product fully delivers this yet
- should-have
developerManage per-environment configuration (dev, staging, prod) as separate stacks or workspaces
Important, not disqualifying — weighs 2× in arena scoring · 1 of 4 products fully deliver this today
State management — stories about state management in this arenaState management· 4 items
Stories about state management in this arena
- must-have
platform-engineerDetect drift between my declared configuration and the actual cloud resources
Core requirement — weighs 3× in arena scoring · 3 of 4 products fully deliver this today
- must-have
platform-engineerStore state in a remote backend with locking so concurrent runs cannot corrupt it
Core requirement — weighs 3× in arena scoring · 1 of 3 products fully deliver this today
- should-have
devops-leadEncrypt state at rest so credentials and sensitive values are not readable in plaintext state files
Important, not disqualifying — weighs 2× in arena scoring · 2 of 4 products fully deliver this today
- should-have
platform-engineerSafely inspect and modify state — moving, removing, or renaming resources — when refactoring
Important, not disqualifying — weighs 2× in arena scoring · 1 of 4 products fully deliver this today
Testing validation — stories about testing validation in this arenaTesting validation· 2 items
Stories about testing validation in this arena
- should-have
developerWrite automated tests for my infrastructure code and run them without touching production
Important, not disqualifying — weighs 2× in arena scoring · 2 of 4 products fully deliver this today
- nice-to-have
developerValidate and auto-format my configuration before planning
Differentiator, not a dealbreaker — weighs 1× in arena scoring · no product fully delivers this yet
Full evidence behind every verdict lives on the arena page and each product page — chips above deep-link straight to the judged story.