Skip to content
ProductArena

Infrastructure as Code Arena

Infrastructure as Code — procurement report

ProductArena · rankings as of 2026-09-16 · evidence as of 2026-09-15 · 4 products · 54 judged requirements · 216 judged cells

Methodology: Every product is judged against a shared taxonomy of user stories using cited evidence — hands-on probes > repository code > independent community sources > vendor claims — never opinion. Full writeup: https://ultrametric.ai/productarena/methodology

Leaderboard

#ProductPA ScoreCoverage scoreApplicable cellsConfidence
1Pulumi34.247.253/54A
2OpenTofu30.241.543/54A
3Terraform26.833.451/54B
4Crossplane22.518.346/54C

PA Score = agent-readiness blend (see methodology). Coverage score = weighted share of judged requirements met. Confidence = how much of the score rests on tested vs claimed evidence (A–D).

Uncertainty note

The current #1/#2 gap in this arena is not close enough to qualify for the multi-judge uncertainty pass (or the pass has not covered it yet) — no extra caveat applies beyond the per-product confidence grades above.

Buyer checklist (RFP)

The arena's 54 judged user stories as requirements, grouped by theme. Priorities mirror the story weights our scoring uses (3 = must-have, 2 = should-have, 1 = nice-to-have). Interactive version with per-requirement verdicts for the top products: /arena/infra-as-code/checklist

Agenticness — how well agents can access and operate the productAgenticness

How well agents can access and operate the product

  • ai-native userPlug MCP servers into this product so it can use their toolsmust-have
  • ai-native userConnect an agent via an official MCP servermust-have
  • ai-native userDrive the product through a documented public APImust-have
  • ai-native userDelegate tasks to a built-in AI assistant inside the productmust-have
  • ai-native userPoint an agent at llms.txt or agent-oriented docsshould-have
  • ai-native userRun the product headlessly / in CI for automationshould-have
  • ai-native userUse an official CLIshould-have
  • ai-native userIssue scoped/least-privilege API credentials for an agentshould-have
  • ai-native userBuild against official SDKsshould-have
  • ai-native userSubscribe to events via webhooksshould-have
  • ai-native userGet AI-generated insights and suggestions from my data inside the productshould-have
  • ai-native userSet up automations that run autonomously in the backgroundshould-have
  • ai-native userOperate the product with natural-language commandsshould-have
  • ai-native userExplore an interactive API reference with runnable examplesshould-have
  • ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)should-have
  • ai-native userRely on versioned APIs with a documented deprecation policyshould-have
  • ai-native userTest against a sandbox environment without touching production datanice-to-have

Automation depth — how much of the product can run unattendedAutomation depth

How much of the product can run unattended

  • ai-native userDefine rules that trigger actions automatically on eventsmust-have
  • ai-native userPerform bulk operations across many items at onceshould-have
  • ai-native userSchedule recurring jobs or workflowsshould-have
  • ai-native userVersion, review, and roll back my automationsnice-to-have

Import migration — stories about import migration in this arenaImport migration

Stories about import migration in this arena

  • platform-engineerImport existing cloud resources under management and generate matching configuration codemust-have
  • platform-engineerMigrate an existing Terraform-format codebase and its state into this toolshould-have

Licensing governance — stories about licensing governance in this arenaLicensing governance

Stories about licensing governance in this arena

  • devops-leadRely on an open license and open governance so the tool cannot be relicensed out from under my companymust-have
  • developerRely on documented compatibility promises and upgrade guides between releasesshould-have

Openness — open source, data portability, and self-hosting storiesOpenness

Open source, data portability, and self-hosting stories

  • ai-native userExport all of my data in open formats and leavemust-have
  • ai-native userSelf-host the core productmust-have
  • ai-native userDo everything through the API that I can do in the UIshould-have
  • ai-native userRead the product's source under an open licenseshould-have

Plan apply — the plan/apply loop — previewing infrastructure changes and applying them safelyPlan apply

The plan/apply loop — previewing infrastructure changes and applying them safely

  • ai-native userGet machine-readable plan output (JSON) that an agent can parse to reason about a proposed changemust-have
  • ai-native userHave an agent author an infrastructure change, run a plan headlessly, and present the diff for my approvalmust-have
  • devops-leadRun plan and apply non-interactively in CI using saved plan artifacts and approval flagsmust-have
  • platform-engineerPreview exactly what will change — creates, updates, and destroys — before applyingmust-have
  • developerDrive deployments programmatically from my own application code rather than only through the CLIshould-have
  • developerTear down an entire environment cleanly with a destroy operationshould-have
  • platform-engineerTarget or exclude specific resources in a plan or applynice-to-have

Policy as code — stories about policy as code in this arenaPolicy as code

Stories about policy as code in this arena

  • devops-leadEnforce policy-as-code checks that block non-compliant infrastructure changes before applymust-have
  • ai-native userLet an agent plan and apply with least-privilege credentials and review gates so it cannot make unapproved changesshould-have

Privacy posture — data-handling and privacy storiesPrivacy posture

Data-handling and privacy stories

  • ai-native userPrevent my data from being used to train AI modelsmust-have
  • ai-native userChoose where my data is stored (region/residency)should-have
  • ai-native userControl data retention and deletionshould-have
  • ai-native userOpt out of telemetry and usage trackingshould-have

Providers modules — stories about providers modules in this arenaProviders modules

Stories about providers modules in this arena

  • developerConsume and publish reusable modules or components from a public registrymust-have
  • developerManage resources across all major clouds and SaaS providers through a broad provider ecosystemmust-have
  • ai-native userGenerate infrastructure code from natural language using AI assistance built into the toolchainshould-have
  • developerDefine infrastructure in a general-purpose programming language with types, loops, and IDE supportshould-have

Secrets config — stories about secrets config in this arenaSecrets config

Stories about secrets config in this arena

  • developerPass secrets and sensitive configuration into deployments without exposing them in code or logsmust-have
  • developerManage per-environment configuration (dev, staging, prod) as separate stacks or workspacesshould-have

State management — stories about state management in this arenaState management

Stories about state management in this arena

  • platform-engineerDetect drift between my declared configuration and the actual cloud resourcesmust-have
  • platform-engineerStore state in a remote backend with locking so concurrent runs cannot corrupt itmust-have
  • devops-leadEncrypt state at rest so credentials and sensitive values are not readable in plaintext state filesshould-have
  • platform-engineerSafely inspect and modify state — moving, removing, or renaming resources — when refactoringshould-have

Testing validation — stories about testing validation in this arenaTesting validation

Stories about testing validation in this arena

  • developerWrite automated tests for my infrastructure code and run them without touching productionshould-have
  • developerValidate and auto-format my configuration before planningnice-to-have

Appendix: recorded probes

Hands-on probe recordings — transcripts/videos a human can replay, the strongest evidence tier. Watch them at https://ultrametric.ai/productarena/proofs

  • Crossplanecrossplane version --clientterminal · recorded 2026-09-04 · exit 0
  • OpenTofutofu versionterminal · recorded 2026-09-04 · exit 0
  • OpenTofutofu plan -helpterminal · recorded 2026-09-04 · exit 0
  • Pulumipulumi versionterminal · recorded 2026-09-04 · exit 0
  • Pulumipulumi preview --helpterminal · recorded 2026-09-04 · exit 0
  • Terraformterraform versionterminal · recorded 2026-09-04 · exit 0
  • Terraformterraform plan -helpterminal · recorded 2026-09-04 · exit 0

Cite as: ProductArena by Ultrametric Inc, Infrastructure as Code arena, rankings as of 2026-09-16 — https://ultrametric.ai/productarena/arena/infra-as-code

License: © 2026 Ultrametric Inc. Brief quotation of individual verdicts, scores, or evidence excerpts is permitted with attribution to "ProductArena by Ultrametric Inc (ultrametric.ai/productarena)", as is use of the data to evaluate, contest, or contribute corrections. Bulk copying, redistribution, or use to build competing datasets requires prior written permission (see DATA-LICENSE in the repository).

No liability: rankings, verdicts, and scores are research outputs derived from the cited evidence at a point in time, provided "as is", without warranties. Ultrametric Inc accepts no responsibility for procurement, purchasing, or other decisions made in reliance on them — verify against the cited evidence before acting (https://ultrametric.ai/productarena/terms).