Vast.ai vs Paperspace

Ide

1. developerOpen Jupyter or connect my IDE (VS Code/Cursor) to the instance in one step

   weight 1 · round to Vast.ai
   Vast.ai~partialclaimed5/10

   Vast.ai documents secure SSH access to instances (key-based auth, run commands, transfer files) and one-click template launches that could include Jupyter environments, which together enable IDE remote-connection workflows (e.g., VS Code Remote-SSH). However, the evidence never explicitly names Jupyter notebook access or a documented one-step VS Code/Cursor connection flow. Missing for 10: explicit Jupyter launch/access docs, explicit VS Code/Cursor remote-connect guide or extension support, and any hands-on confirmation of a true 'one step' connect experience.

   • [claimed-docs] “Vast.ai instances are configured to accept keys only - Password authentication is disabled for improved security.”
   • [claimed-docs] “Log in securely, Run commands remotely, Transfer files without exposing your data”
   • [claimed-docs] “SSH (Secure Shell) is a protocol for safely connecting to remote servers. It encrypts your connection so you can: Log in securely, Run comma…”
   • [claimed-docs] “Launch prebuilt or custom templates with one click.”
   • [claimed-docs] “A template is how Vast helps you launch an instance, setting up your rented machine with whatever software and formatting you need.”
   • [claimed-docs] “Build a template - Package your environment so any GPU can run it with one click.”

   proof ↗⚑ flag

   Paperspace!disputedcontradicted4/10

   Paperspace ships a native web-based Jupyter Notebooks product and documents SSH access with 'bring your own key, full root access' to Machines, which in principle supports VS Code Remote-SSH style connections, but there is no documentation of a genuine one-step IDE handshake (e.g., no VS Code/Cursor-specific integration, devcontainer, or remote-SSH config generator). A first-hand community report directly contradicts the 'easy SSH' claim, saying getting SSH access to Paperspace Core VMs was 'an uphill battle' with a 'confusing GUI' instead. missing for 10: explicit VS Code/Cursor one-click connect docs, evidence of an SSH config/remote extension workflow, and independent confirmation that SSH setup is actually fast/frictionless.

   • [claimed-docs] “Notebooks are a web-based Jupyter IDE with shared persistent storage for long-term development and inter-notebook collaboration, backed by a…”
   • [claimed-docs] “You can connect to your Linux or Windows-based machine using the Paperspace console or desktop app using SSH connection.”
   • [claimed-docs] “Bring your SSH key and connect directly to your VM with full root access.”
   • [claimed-docs] “Root access, connect with SSH Bring your SSH key and connect directly to your VM with full root access.”
   • [community] “Their DL virtual servers (Core) are horrible - very slow internet, takes forever to copy datasets in. Getting SSH access is an uphill battle…”

   proof ↗⚑ flag

Networking

1. developerExpose ports to serve applications from my instance and connect instances over private networking

   weight 1 · round to Vast.ai
   Vast.ai✓fullclaimed7/10

   Docs explicitly describe exposing ports (search filter `direct_port_count>=1`, SSH access) and creating private overlay networks so instances on different machines can share a virtual LAN for multi-node workloads, directly matching both halves of the story. Missing for 10: independent/hands-on confirmation of port-forwarding for arbitrary web apps beyond SSH, and more detail on overlay network setup/limitations.

   • [claimed-docs] “Vast now supports creating *overlay* networks for instances, allowing client instances on different machines on the same physical LAN to sha…”
   • [claimed-docs] “Vast now supports creating overlay networks for instances, allowing client instances on different machines on the same physical LAN to share…”
   • [claimed-docs] “vastai search offers 'gpu_name=RTX_4090 num_gpus=1 verified=true direct_port_count>=1 rentable=true' -o 'dlperf_usd-'”
   • [claimed-docs] “Vast.ai instances are configured to accept keys only - Password authentication is disabled for improved security.”
   • [claimed-docs] “SSH (Secure Shell) is a protocol for safely connecting to remote servers. It encrypts your connection so you can: Log in securely, Run comma…”

   proof ↗⚑ flag

   Paperspace~partialclaimed5/10

   Docs confirm private networking between machines (shared drives across a private network in paperspace-docs-4) and an option to enable a 'private network' or 'public IP address' when creating a machine (paperspace-docs-25), plus SSH/root access to instances (paperspace-docs-12/19/21). However, there is no explicit documentation of exposing arbitrary application ports or firewall/port-forwarding configuration for serving apps beyond SSH; 'Deployments' (containers-as-a-service, paperspace-docs-8) hints at serving models but isn't tied to port exposure or private networking specifics. missing for 10: explicit port-exposure/firewall configuration docs, hands-on confirmation of connecting instances over private network for app traffic, and any independent corroboration of these networking features working as described.

   • [claimed-docs] “Shared drives provide storage that is accessible from multiple machines in a private network.”
   • [claimed-docs] “When you create a new machine, you choose your machine type, operating system or custom template, disk size, region, authentication, startin…”
   • [claimed-docs] “Deployments are containers-as-a-service that let you run container images and serve machine learning models.”
   • [claimed-docs] “Bring your SSH key and connect directly to your VM with full root access.”
   • [claimed-docs] “Root access, connect with SSH Bring your SSH key and connect directly to your VM with full root access.”

   proof ↗⚑ flag

Ssh

1. developerSSH into my GPU instance with my own keys and get root-level control of the environment

   weight 3 · round to Vast.ai
   Vast.ai~partialclaimed6/10

   Docs confirm SSH access is key-based only (password auth disabled) and lets you log in securely, run commands remotely, and transfer files, which implies developer control of the instance. However, the evidence pack never explicitly confirms root-level privileges or documents the workflow for adding a user's own SSH key. Missing for 10: explicit documentation of root/sudo access inside instances, and details on uploading/managing your own SSH keys.

   • [claimed-docs] “Vast.ai instances are configured to accept keys only - Password authentication is disabled for improved security.”
   • [claimed-docs] “Log in securely, Run commands remotely, Transfer files without exposing your data”
   • [claimed-docs] “SSH (Secure Shell) is a protocol for safely connecting to remote servers. It encrypts your connection so you can: Log in securely, Run comma…”

   proof ↗⚑ flag

   Paperspace!disputedcontradicted5/10

   Paperspace's marketing explicitly promises 'bring your own SSH key' with full root access to the VM, and DigitalOcean docs describe SSH-based connection to machines, but a hands-on community report states that SSH access on Paperspace's Core VMs was 'an uphill battle' with a 'pointless virtual console and confusing GUI' instead of easy SSH, contradicting the frictionless claim. missing for 10: independent confirmation that SSH access works smoothly as advertised, and no rebuttal to the community complaint about SSH being hard to obtain.

   • [claimed-docs] “Bring your SSH key and connect directly to your VM with full root access.”
   • [claimed-docs] “Root access, connect with SSH Bring your SSH key and connect directly to your VM with full root access.”
   • [claimed-docs] “Root access, connect with SSH... Bring your SSH key and connect directly to your VM with full root access.”
   • [claimed-docs] “You can connect to your Linux or Windows-based machine using the Paperspace console or desktop app using SSH connection.”
   • [community] “Their DL virtual servers (Core) are horrible - very slow internet, takes forever to copy datasets in. Getting SSH access is an uphill battle…”

   proof ↗⚑ flag

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round to Vast.ai
   Vast.ai✓fullprobed9/10

   Vast.ai publishes a live llms.txt at docs.vast.ai/llms.txt (confirmed via probe returning HTTP 200 with structured doc links) and additionally ships an explicit agent-oriented guide/skill ('vastai agent skill') documenting how AI coding assistants can drive the platform directly. This directly satisfies pointing an agent at llms.txt or agent-oriented docs. Missing for 10: no independent/community corroboration of agents actually consuming llms.txt in practice.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.vast.ai/llms.txt # Vast.ai Documentation: Affordable GPU Cloud Marketplace - [Welcome](https://doc…”
   • [claimed-docs] “The `vastai` agent skill lets AI coding assistants drive Vast.ai on your behalf — create instances, deploy endpoints, manage SSH keys, check…”
   • [claimed-docs] “The vastai agent skill lets AI coding assistants drive Vast.ai on your behalf — create instances, deploy endpoints, manage SSH keys, check b…”
   • [claimed-docs] “The vastai agent skill lets AI coding assistants drive Vast.ai on your behalf — create instances, deploy endpoints, manage SSH keys, check b…”

   proof ↗⚑ flag

   Paperspace~partialprobed5/10

   The Paperspace docs live under docs.digitalocean.com, and that domain does serve a working llms.txt (HTTP 200) with a general description of DigitalOcean's documentation corpus, giving agents a discoverable entry point. However, a direct markdown/agent-friendly version of the Paperspace-specific docs page 404s, and there's no evidence the llms.txt specifically indexes or highlights Paperspace content vs. the broader DigitalOcean product catalog. Missing for 10: Paperspace-specific agent-readable docs (e.g. .md endpoint), confirmation llms.txt references Paperspace pages, and any documentation stating this is an intentional agent-onboarding feature.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.digitalocean.com/llms.txt # DigitalOcean Documentation > Comprehensive tutorials, references, exam…”
   • [probe] “PROBE docs-md: HTTP 404 at https://docs.digitalocean.com/products/paperspace/.md”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round to Vast.ai
   Vast.ai✓fullprobed8/10

   Vast.ai offers a full CLI and REST API for scriptable, non-interactive control of the entire instance lifecycle (search, create, destroy), confirmed by both docs and hands-on runtime probes running keylessly. Scoped API keys are explicitly recommended for CI/CD and shared tooling, and the CLI was verified to run headlessly via uvx with no login. Missing for 10: no explicit CI pipeline example (e.g., GitHub Actions workflow) or third-party case study of running it in an automated CI pipeline.

   • [claimed-docs] “The Vast.ai CLI gives you command-line access to the entire platform, authentication, GPU search, instance lifecycle, templates, volumes, se…”
   • [claimed-docs] “You can also create scoped keys with limited permissions using `vastai create api-key`, useful for CI/CD or shared tooling.”
   • [claimed-docs] “For shared tooling, CI/CD, or automation, create a scoped key with only the permissions that workload needs.”
   • [claimed-docs] “Automate with the CLI or SDK ... `vastai` from the shell or Python. Same operations, different syntax.”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Vast.ai CLI (pypi `vastai`) installed via uvx and ran keylessly — `vastai --help` prints t…”
   • [probe] “PROBE runtime (recorded 2026-09-05): `vastai search offers 'gpu_name=RTX_4090 num_gpus=1' -o dph` searched the LIVE GPU marketplace with NO …”
   • [probe] “PROBE runtime (recorded 2026-09-05): a bare keyless GET to https://console.vast.ai/api/v0/bundles/ returned live marketplace offers as JSON …”

   proof ↗⚑ flag

   Paperspace~partialprobed6/10

   Paperspace exposes a REST API, JS SDK, and a working CLI (pspace) that can create/manage machines, deployments, and templates non-interactively, and Workflows offers pipeline-style automation — enough to script headless usage in CI. However there's no first-party CI-integration guide (e.g. GitHub Actions), no documented service-account/non-interactive auth flow for CI runners, and community feedback flags friction around SSH/API access reliability and product quality, so full CI-native support isn't clearly evidenced. missing for 10: documented CI/CD integration examples, non-interactive/service-account auth for automation, independent confirmation of reliable headless workflows in CI pipelines.

   • [claimed-docs] “API keys let you interact with Paperspace through the Core RESTful API, the Core JavaScript SDK, and Gradient command-line utility (CLI).”
   • [claimed-docs] “Programmatically manage Paperspace resources using conventional HTTP requests.”
   • [claimed-docs] “Install and use the new Paperspace Command Line Interface.”
   • [claimed-docs] “Workflows automate machine learning tasks, combining GPU instances with an expressive syntax to generate production-ready machine learning p…”
   • [probe] “official CLI documented at https://docs.digitalocean.com/reference/paperspace/pspace/”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official installer (`curl -fsSL https://paperspace.com/install.sh | sh`) installed the pspace CLI i…”
   • [community] “Their DL virtual servers (Core) are horrible - very slow internet, takes forever to copy datasets in. Getting SSH access is an uphill battle…”

   proof ↗⚑ flag

3. ai-native userConnect an agent via an official MCP server

   weight 3 · round drawn
   Vast.ai—none0/10

   Vast.ai documents a CLI, REST API, Python SDK, and an 'agent skill' file for coding assistants, but no evidence anywhere describes an official MCP server or MCP integration. Since Vast.ai is a platform/service (not itself an agent), the MCP-server axis applies, and its absence from the evidence pack means the story is unmet.

   • [claimed-docs] “The `vastai` agent skill lets AI coding assistants drive Vast.ai on your behalf — create instances, deploy endpoints, manage SSH keys, check…”
   • [claimed-docs] “The vastai agent skill lets AI coding assistants drive Vast.ai on your behalf — create instances, deploy endpoints, manage SSH keys, check b…”
   • [claimed-docs] “The vastai agent skill lets AI coding assistants drive Vast.ai on your behalf — create instances, deploy endpoints, manage SSH keys, check b…”
   • [claimed-docs] “The Vast.ai CLI gives you command-line access to the entire platform, authentication, GPU search, instance lifecycle, templates, volumes, se…”
   • [claimed-docs] “The Vast.ai REST API gives you programmatic control over the entire platform. It's the foundation that the CLI and SDK are built on.”

   proof ↗⚑ flag

   Paperspace—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

4. ai-native userUse an official CLI

   weight 2 · round to Vast.ai
   Vast.ai✓fullprobed9/10

   Vast.ai ships an official CLI (pypi `vastai`) with full documentation of the entire platform surface (search, instances, templates, volumes, serverless), and runtime probes confirm it installs and works keylessly for real marketplace queries. Missing for 10: independent third-party reviews specifically praising/critiquing the CLI's UX beyond vendor docs and probes.

   • [claimed-docs] “The Vast.ai CLI gives you command-line access to the entire platform, authentication, GPU search, instance lifecycle, templates, volumes, se…”
   • [claimed-docs] “vastai search offers 'gpu_name=RTX_4090 num_gpus=1 verified=true direct_port_count>=1 rentable=true' -o 'dlperf_usd-'”
   • [claimed-docs] “Automate with the CLI or SDK ... `vastai` from the shell or Python. Same operations, different syntax.”
   • [probe] “official CLI documented at https://docs.vast.ai/cli/hello-world”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Vast.ai CLI (pypi `vastai`) installed via uvx and ran keylessly — `vastai --help` prints t…”
   • [probe] “PROBE runtime (recorded 2026-09-05): `vastai search offers 'gpu_name=RTX_4090 num_gpus=1' -o dph` searched the LIVE GPU marketplace with NO …”

   proof ↗⚑ flag

   Paperspace✓fullprobed8/10

   Paperspace ships an official CLI (pspace), documented in DigitalOcean docs and independently verified via a runtime probe showing successful install and functional commands for machines, templates, autoscaling, and deployments. missing for 10: independent third-party review specifically praising CLI agentic workflows, and no evidence of AI-native scripting/automation examples beyond basic resource management.

   • [claimed-docs] “Install and use the new Paperspace Command Line Interface.”
   • [claimed-docs] “API keys let you interact with Paperspace through the Core RESTful API, the Core JavaScript SDK, and Gradient command-line utility (CLI).”
   • [probe] “official CLI documented at https://docs.digitalocean.com/reference/paperspace/pspace/”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official installer (`curl -fsSL https://paperspace.com/install.sh | sh`) installed the pspace CLI i…”

   proof ↗⚑ flag

5. ai-native userDrive the product through a documented public API

   weight 3 · round to Vast.ai
   Vast.ai✓fullprobed9/10

   Vast.ai publishes a documented REST API (with live OpenAPI spec) and CLI/SDK built on top of it, explicitly positioned for programmatic/agentic use, including scoped API keys for automation and a dedicated 'agent skill' for AI coding assistants. Runtime probes confirm the CLI and REST endpoints work keylessly for core operations like search/create/destroy instances, corroborating the documentation with hands-on evidence. Missing for 10: independent third-party developer reports specifically about API robustness/rate limits beyond docs and probes.

   • [claimed-docs] “The Vast.ai REST API gives you programmatic control over the entire platform. It's the foundation that the CLI and SDK are built on.”
   • [claimed-docs] “The Vast.ai REST API gives you programmatic control over the entire platform.”
   • [claimed-docs] “The Vast.ai CLI gives you command-line access to the entire platform, authentication, GPU search, instance lifecycle, templates, volumes, se…”
   • [claimed-docs] “The `vastai` agent skill lets AI coding assistants drive Vast.ai on your behalf — create instances, deploy endpoints, manage SSH keys, check…”
   • [claimed-docs] “The vastai agent skill lets AI coding assistants drive Vast.ai on your behalf — create instances, deploy endpoints, manage SSH keys, check b…”
   • [claimed-docs] “create a scoped key with only the permissions that workload needs”
   • [claimed-docs] “For shared tooling, CI/CD, or automation, create a scoped key with only the permissions that workload needs.”
   • [probe] “PROBE openapi: HTTP 200 at https://docs.vast.ai/api/openapi.json — contains "openapi" key”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Vast.ai CLI (pypi `vastai`) installed via uvx and ran keylessly — `vastai --help` prints t…”
   • [probe] “PROBE runtime (recorded 2026-09-05): `vastai search offers 'gpu_name=RTX_4090 num_gpus=1' -o dph` searched the LIVE GPU marketplace with NO …”
   • [probe] “PROBE runtime (recorded 2026-09-05): a bare keyless GET to https://console.vast.ai/api/v0/bundles/ returned live marketplace offers as JSON …”

   proof ↗⚑ flag

   Paperspace✓fullprobed8/10

   Paperspace documents a Core RESTful API, JavaScript SDK, and CLI for programmatically managing machines, notebooks, deployments, and workflows, and a runtime probe confirms the official pspace CLI installs and works with commands for machine/deployment management. Missing for 10: no public OpenAPI/swagger spec found, no independent third-party corroboration of API robustness beyond docs.

   • [claimed-docs] “API keys let you interact with Paperspace through the Core RESTful API, the Core JavaScript SDK, and Gradient command-line utility (CLI).”
   • [claimed-docs] “Programmatically manage Paperspace resources using conventional HTTP requests.”
   • [claimed-docs] “Install and use the new Paperspace Command Line Interface.”
   • [probe] “official CLI documented at https://docs.digitalocean.com/reference/paperspace/pspace/”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official installer (`curl -fsSL https://paperspace.com/install.sh | sh`) installed the pspace CLI i…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.digitalocean.com/openapi.json, https://docs.digitalocean.com/swagger.json, https://docs…”

   proof ↗⚑ flag

6. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round to Vast.ai
   Vast.ai✓fullclaimed8/10

   Docs explicitly describe creating scoped API keys with limited permissions for workloads/CI-CD/shared tooling via `vastai create api-key`, directly matching least-privilege agent credentialing, and this is reinforced by a dedicated api-keys reference page and the agent-skill docs describing agent-driven usage. Missing for 10: independent/hands-on verification of scope enforcement (e.g., testing that a scoped key actually blocks restricted actions) and detail on the granularity of available scopes.

   • [claimed-docs] “You can also create scoped keys with limited permissions using `vastai create api-key`, useful for CI/CD or shared tooling.”
   • [claimed-docs] “create a scoped key with only the permissions that workload needs”
   • [claimed-docs] “For shared tooling, CI/CD, or automation, create a scoped key with only the permissions that workload needs.”
   • [claimed-docs] “The `vastai` agent skill lets AI coding assistants drive Vast.ai on your behalf — create instances, deploy endpoints, manage SSH keys, check…”
   • [claimed-docs] “The vastai agent skill lets AI coding assistants drive Vast.ai on your behalf — create instances, deploy endpoints, manage SSH keys, check b…”

   proof ↗⚑ flag

   Paperspace—none0/10

   Paperspace offers API keys for programmatic access, but evidence shows no scoped or least-privilege permission model — keys appear to be account-level, not fine-grained or agent-specific credentials.

   • [claimed-docs] “API keys let you interact with Paperspace through the Core RESTful API, the Core JavaScript SDK, and Gradient command-line utility (CLI).”
   • [claimed-docs] “Programmatically manage Paperspace resources using conventional HTTP requests.”

   proof ↗⚑ flag

7. ai-native userBuild against official SDKs

   weight 2 · round to Vast.ai
   Vast.ai✓fullprobed9/10

   Vast.ai documents an official Python SDK (built alongside CLI) and REST API with OpenAPI spec, a documented migration path from the legacy vastai_sdk package, and runtime-probed confirmation that the CLI/SDK surface works live against the marketplace API. Documentation explicitly frames CLI/SDK as 'same operations, different syntax' with Python usage examples for serverless workflows, satisfying an AI-native builder's need for official SDK access. Missing for 10: independent (non-vendor) hands-on reports specifically validating the Python SDK's completeness/stability beyond CLI parity.

   • [claimed-docs] “The Vast.ai REST API gives you programmatic control over the entire platform. It's the foundation that the CLI and SDK are built on.”
   • [claimed-docs] “Automate with the CLI or SDK ... `vastai` from the shell or Python. Same operations, different syntax.”
   • [claimed-docs] “Interacting with Vast Serverless is made easy through a powerful python SDK.”
   • [github] “Migrating from vastai-sdk? The old import still works: from vastai_sdk import VastAI”
   • [probe] “PROBE openapi: HTTP 200 at https://docs.vast.ai/api/openapi.json — contains "openapi" key”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Vast.ai CLI (pypi `vastai`) installed via uvx and ran keylessly — `vastai --help` prints t…”
   • [probe] “PROBE runtime (recorded 2026-09-05): a bare keyless GET to https://console.vast.ai/api/v0/bundles/ returned live marketplace offers as JSON …”

   proof ↗⚑ flag

   Paperspace~partialprobed6/10

   Docs confirm an official Core RESTful API and Core JavaScript SDK plus a documented CLI (pspace) verified via runtime probe, giving developers concrete official interfaces to build against. However, evidence only names one language SDK (JavaScript), with no mention of Python or other SDKs, and no independent corroboration of SDK quality or ecosystem breadth. missing for 10: evidence of additional language SDKs (e.g. Python), independent developer corroboration of SDK reliability, and a public SDK repo/changelog.

   • [claimed-docs] “API keys let you interact with Paperspace through the Core RESTful API, the Core JavaScript SDK, and Gradient command-line utility (CLI).”
   • [claimed-docs] “Programmatically manage Paperspace resources using conventional HTTP requests.”
   • [claimed-docs] “Install and use the new Paperspace Command Line Interface.”
   • [probe] “official CLI documented at https://docs.digitalocean.com/reference/paperspace/pspace/”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official installer (`curl -fsSL https://paperspace.com/install.sh | sh`) installed the pspace CLI i…”

   proof ↗⚑ flag

8. ai-native userSubscribe to events via webhooks

   weight 2 · round drawn
   Vast.ai—none0/10

   No evidence of a webhook subscription mechanism anywhere in the docs, CLI, API reference, or probes; Vast.ai exposes REST API, CLI, and SDK for polling/imperative control but nothing describing event-driven push notifications or webhook callbacks. missing for 10: any documentation of webhook registration/endpoints, event types, or delivery/retry semantics.

   • [claimed-docs] “The Vast.ai REST API gives you programmatic control over the entire platform. It's the foundation that the CLI and SDK are built on.”
   • [claimed-docs] “The Vast.ai CLI gives you command-line access to the entire platform, authentication, GPU search, instance lifecycle, templates, volumes, se…”
   • [probe] “PROBE openapi: HTTP 200 at https://docs.vast.ai/api/openapi.json — contains "openapi" key”

   proof ↗⚑ flag

   Paperspace—none0/10

   No evidence of any webhook or event-subscription mechanism in Paperspace's API, CLI, or docs; only REST API, CLI, and SDK access are documented.

   • [claimed-docs] “Programmatically manage Paperspace resources using conventional HTTP requests.”
   • [claimed-docs] “Install and use the new Paperspace Command Line Interface.”
   • [probe] “official CLI documented at https://docs.digitalocean.com/reference/paperspace/pspace/”

   proof ↗⚑ flag

Agentic features

1. ai-native userSet up automations that run autonomously in the background

   weight 2 · round to Vast.ai
   Vast.ai~partialprobed5/10

   Vast.ai exposes a CLI/SDK/REST API, scoped API keys for CI/CD, and an 'agent skill' letting AI coding assistants drive the platform (create instances, deploy endpoints, manage keys) — all usable to script automations, and Serverless endpoints let workloads run without manual GPU management. However, there is no evidence of a built-in scheduler, cron/trigger system, or persistent background job orchestration that runs autonomously without an external driver; automation depends on the user's own agent/script staying alive. Missing for 10: native scheduling/trigger mechanism, evidence of long-running autonomous background jobs, and independent confirmation of the agent-skill working unattended in production.

   • [claimed-docs] “The `vastai` agent skill lets AI coding assistants drive Vast.ai on your behalf — create instances, deploy endpoints, manage SSH keys, check…”
   • [claimed-docs] “The vastai agent skill lets AI coding assistants drive Vast.ai on your behalf — create instances, deploy endpoints, manage SSH keys, check b…”
   • [claimed-docs] “The vastai agent skill lets AI coding assistants drive Vast.ai on your behalf — create instances, deploy endpoints, manage SSH keys, check b…”
   • [claimed-docs] “You can also create scoped keys with limited permissions using `vastai create api-key`, useful for CI/CD or shared tooling.”
   • [claimed-docs] “create a scoped key with only the permissions that workload needs”
   • [claimed-docs] “For shared tooling, CI/CD, or automation, create a scoped key with only the permissions that workload needs.”
   • [claimed-docs] “Vast Serverless is an AI infrastructure platform that lets you run compute-intensive workloads without managing GPUs, paying for execution r…”
   • [claimed-docs] “Automate with the CLI or SDK ... `vastai` from the shell or Python. Same operations, different syntax.”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Vast.ai CLI (pypi `vastai`) installed via uvx and ran keylessly — `vastai --help` prints t…”

   proof ↗⚑ flag

   Paperspace~partialclaimed4/10

   Paperspace's 'Workflows' feature is documented as automating ML tasks/pipelines, and 'Deployments' run containerized models continuously, both of which could constitute background automation, plus API/CLI access enables scripting external automations. However there's no documentation of triggers, schedules, or persistent autonomous execution akin to cron/agentic loops, and no community evidence confirming this works well in practice. Missing for 10: scheduling/trigger mechanisms, evidence of long-running unattended execution, and independent confirmation that Workflows/Deployments actually operate autonomously without manual intervention.

   • [claimed-docs] “Workflows automate machine learning tasks, combining GPU instances with an expressive syntax to generate production-ready machine learning p…”
   • [claimed-docs] “Deployments are containers-as-a-service that let you run container images and serve machine learning models.”
   • [claimed-docs] “API keys let you interact with Paperspace through the Core RESTful API, the Core JavaScript SDK, and Gradient command-line utility (CLI).”
   • [claimed-docs] “Install and use the new Paperspace Command Line Interface.”

   proof ↗⚑ flag

2. ai-native userOperate the product with natural-language commands

   weight 2 · round to Vast.ai
   Vast.ai✓fullprobed7/10

   Vast.ai documents a dedicated 'vastai agent skill' explicitly designed so AI coding assistants can drive the platform on the user's behalf (create instances, deploy endpoints, manage keys, check balances) via natural-language-style interaction, backed by a full CLI/REST API surface. Missing for 10: independent or hands-on verification of the natural-language agent-skill flow itself (probes only confirm raw CLI/REST access, not NL interpretation), and no third-party account of an AI agent successfully using it end-to-end.

   • [claimed-docs] “The `vastai` agent skill lets AI coding assistants drive Vast.ai on your behalf — create instances, deploy endpoints, manage SSH keys, check…”
   • [claimed-docs] “The vastai agent skill lets AI coding assistants drive Vast.ai on your behalf — create instances, deploy endpoints, manage SSH keys, check b…”
   • [claimed-docs] “The vastai agent skill lets AI coding assistants drive Vast.ai on your behalf — create instances, deploy endpoints, manage SSH keys, check b…”
   • [claimed-docs] “The Vast.ai CLI gives you command-line access to the entire platform, authentication, GPU search, instance lifecycle, templates, volumes, se…”
   • [claimed-docs] “Automate with the CLI or SDK ... `vastai` from the shell or Python. Same operations, different syntax.”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Vast.ai CLI (pypi `vastai`) installed via uvx and ran keylessly — `vastai --help` prints t…”

   proof ↗⚑ flag

   Paperspace—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round to Vast.ai
   Vast.ai~partialprobed4/10

   Vast.ai publishes a machine-readable OpenAPI spec and API-reference docs, and the CLI docs include copy-pasteable, runnable example commands that were verified to work live (search offers, create instance) without auth — showing some 'runnable example' quality. However there is no evidence of an interactive API console (e.g., embedded 'try it' Swagger-UI style playground) on the docs site itself. Missing for 10: an interactive in-browser API explorer/console, explicit 'try it now' runnable request execution within the API reference pages, and independent confirmation of such interactivity.

   • [claimed-docs] “The Vast.ai REST API gives you programmatic control over the entire platform. It's the foundation that the CLI and SDK are built on.”
   • [claimed-docs] “The Vast.ai REST API gives you programmatic control over the entire platform.”
   • [probe] “PROBE openapi: HTTP 200 at https://docs.vast.ai/api/openapi.json — contains "openapi" key”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Vast.ai CLI (pypi `vastai`) installed via uvx and ran keylessly — `vastai --help` prints t…”
   • [probe] “PROBE runtime (recorded 2026-09-05): `vastai search offers 'gpu_name=RTX_4090 num_gpus=1' -o dph` searched the LIVE GPU marketplace with NO …”

   proof ↗⚑ flag

   Paperspace—none0/10

   No evidence of an interactive API reference with runnable examples; probes show no OpenAPI/swagger spec found (404s) and no interactive docs playground mentioned, only static docs and CLI/API mentions.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.digitalocean.com/openapi.json, https://docs.digitalocean.com/swagger.json, https://docs…”
   • [probe] “PROBE docs-md: HTTP 404 at https://docs.digitalocean.com/products/paperspace/.md”
   • [claimed-docs] “Programmatically manage Paperspace resources using conventional HTTP requests.”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round to Vast.ai
   Vast.ai✓fullprobed9/10

   Vast.ai publishes a live OpenAPI spec at docs.vast.ai/api/openapi.json (verified HTTP 200 with an 'openapi' key), backed by a full REST API reference and CLI/SDK built on it. Missing for 10: no independent third-party confirmation that the spec is comprehensive/kept in sync beyond the probe check.

   • [probe] “PROBE openapi: HTTP 200 at https://docs.vast.ai/api/openapi.json — contains "openapi" key”
   • [claimed-docs] “The Vast.ai REST API gives you programmatic control over the entire platform. It's the foundation that the CLI and SDK are built on.”
   • [claimed-docs] “The Vast.ai REST API gives you programmatic control over the entire platform.”
   • [probe] “PROBE runtime (recorded 2026-09-05): a bare keyless GET to https://console.vast.ai/api/v0/bundles/ returned live marketplace offers as JSON …”

   proof ↗⚑ flag

   Paperspace—none0/10

   Paperspace/DigitalOcean documents a Core RESTful API and CLI, but explicit probes for an OpenAPI/Swagger spec at all standard locations returned 404, and no machine-readable spec is referenced anywhere in the docs.

   • [claimed-docs] “Programmatically manage Paperspace resources using conventional HTTP requests.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.digitalocean.com/openapi.json, https://docs.digitalocean.com/swagger.json, https://docs…”

   proof ↗⚑ flag

3. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   Vast.ai—none0/10

   Evidence shows Vast.ai has a REST API, OpenAPI spec, and CLI/SDK, but there is no documentation of API versioning scheme or a deprecation policy anywhere in the pack; the only related signal is a backward-compatible import shim for the old SDK (vast-ai-gh-2), which is not a documented policy.

   • [claimed-docs] “The Vast.ai REST API gives you programmatic control over the entire platform. It's the foundation that the CLI and SDK are built on.”
   • [probe] “PROBE openapi: HTTP 200 at https://docs.vast.ai/api/openapi.json — contains "openapi" key”
   • [github] “Migrating from vastai-sdk? The old import still works: from vastai_sdk import VastAI”

   proof ↗⚑ flag

   Paperspace—none0/10

   Evidence shows Paperspace has a REST API, CLI, and SDK, but nothing documents API versioning schemes or a deprecation policy anywhere in the docs or community sources.

   • [claimed-docs] “API keys let you interact with Paperspace through the Core RESTful API, the Core JavaScript SDK, and Gradient command-line utility (CLI).”
   • [claimed-docs] “Programmatically manage Paperspace resources using conventional HTTP requests.”
   • [claimed-docs] “Install and use the new Paperspace Command Line Interface.”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round to Vast.ai
   Vast.ai~partialprobed6/10

   The CLI and REST API expose full programmatic control (search, create, destroy, manage keys) and are explicitly positioned for automation/scripting from shell or Python, which lets an AI-native user loop over many offers/instances (e.g., filtered searches returning many GPU offers at once). However, there is no documented native batch/bulk endpoint (e.g., one call to create or destroy N instances simultaneously) — only single-item CRUD operations that must be scripted in a loop. Missing for 10: explicit bulk/batch API endpoints or CLI flags for multi-item operations, and community/hands-on confirmation of bulk usage at scale.

   • [claimed-docs] “The Vast.ai CLI gives you command-line access to the entire platform, authentication, GPU search, instance lifecycle, templates, volumes, se…”
   • [claimed-docs] “The Vast.ai REST API gives you programmatic control over the entire platform. It's the foundation that the CLI and SDK are built on.”
   • [claimed-docs] “Automate with the CLI or SDK ... `vastai` from the shell or Python. Same operations, different syntax.”
   • [github] “vast.search_offers(query='gpu_name=RTX_4090 num_gpus>=4')”
   • [claimed-docs] “vastai search offers 'gpu_name=RTX_4090 num_gpus=1 verified=true direct_port_count>=1 rentable=true' -o 'dlperf_usd-'”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Vast.ai CLI (pypi `vastai`) installed via uvx and ran keylessly — `vastai --help` prints t…”
   • [probe] “PROBE runtime (recorded 2026-09-05): `vastai search offers 'gpu_name=RTX_4090 num_gpus=1' -o dph` searched the LIVE GPU marketplace with NO …”

   proof ↗⚑ flag

   Paperspace—none0/10

   Paperspace exposes an API, CLI (pspace), and Core JS SDK for programmatic resource management, which theoretically could be scripted for bulk actions, but no evidence documents any actual bulk/batch operation feature (e.g., multi-machine create/delete/update in one call, batch endpoints, or CLI loop/apply commands) across many items at once.

   • [claimed-docs] “API keys let you interact with Paperspace through the Core RESTful API, the Core JavaScript SDK, and Gradient command-line utility (CLI).”
   • [claimed-docs] “Programmatically manage Paperspace resources using conventional HTTP requests.”
   • [claimed-docs] “Install and use the new Paperspace Command Line Interface.”
   • [probe] “official CLI documented at https://docs.digitalocean.com/reference/paperspace/pspace/”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official installer (`curl -fsSL https://paperspace.com/install.sh | sh`) installed the pspace CLI i…”

   proof ↗⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round to Vast.ai
   Vast.ai~partialclaimed4/10

   Vast.ai documents 'autobilling', an explicit event-triggered rule (balance drops low → auto top-up from saved card), which is genuine automation-on-event. Beyond that, evidence only shows generic CLI/SDK/API scripting and agent-skill hooks for manual automation, not a broader rules/trigger engine (no webhooks, alerts, or conditional actions for instance events like preemption, cost thresholds, or health failures). Missing for 10: a general-purpose event/trigger system (webhooks, alerting, conditional rules) beyond the single billing use case, and independent confirmation of its reliability.

   • [claimed-docs] “Use the autobilling feature to have the system automatically top up your account using a saved credit card when it runs low.”
   • [claimed-docs] “Autobilling keeps your balance positive to protect your instances from interruption or deletion.”
   • [claimed-docs] “Automate with the CLI or SDK ... `vastai` from the shell or Python. Same operations, different syntax.”
   • [claimed-docs] “For shared tooling, CI/CD, or automation, create a scoped key with only the permissions that workload needs.”

   proof ↗⚑ flag

   Paperspace—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

3. ai-native userSchedule recurring jobs or workflows

   weight 2 · round drawn
   Vast.ai—none0/10

   Vast.ai's evidence covers CLI/API/SDK for instance and serverless management but contains no mention of a built-in scheduler, cron-like recurring job feature, or workflow orchestration; users would have to bring their own external scheduler to hit the API/CLI repeatedly, which isn't documented as a first-party capability.

   ⚑ flag

   Paperspace—none0/10

   Paperspace docs mention on-demand Workflows for ML pipelines and auto-shutdown after inactivity, but there is no evidence of a scheduler, cron-like trigger, or recurring/periodic job execution capability. Absence of evidence for this applicable capability yields none.

   • [claimed-docs] “Workflows automate machine learning tasks, combining GPU instances with an expressive syntax to generate production-ready machine learning p…”
   • [claimed-docs] “The auto-shutdown feature places your machine in a stop state after a predefined period of inactivity, which can range from one hour to one …”

   proof ↗⚑ flag

Availability

1. ml engineerSee real-time GPU availability by type and region before I try to provision, instead of discovering stockouts by failure

   weight 2 · round to Vast.ai
   Vast.ai✓fullprobed8/10

   Vast.ai's search/offers API and CLI let engineers query live marketplace inventory (GPU type, count, verified status, price) before renting, and this was independently confirmed via a keyless live probe against both the CLI and the public REST bundles endpoint returning real-time offers with reliability data. This directly satisfies seeing availability before provisioning rather than hitting a stockout on create. missing for 10: explicit region/geolocation filter documentation and evidence of real-time refresh/alerting for capacity changes.

   • [claimed-docs] “vastai search offers 'gpu_name=RTX_4090 num_gpus=1 verified=true direct_port_count>=1 rentable=true' -o 'dlperf_usd-'”
   • [claimed-docs] “The search page is the main portal for finding good machines and creating instances on them.”
   • [github] “vast.search_offers(query='gpu_name=RTX_4090 num_gpus>=4')”
   • [probe] “PROBE runtime (recorded 2026-09-05): `vastai search offers 'gpu_name=RTX_4090 num_gpus=1' -o dph` searched the LIVE GPU marketplace with NO …”
   • [probe] “PROBE runtime (recorded 2026-09-05): a bare keyless GET to https://console.vast.ai/api/v0/bundles/ returned live marketplace offers as JSON …”

   proof ↗⚑ flag

   Paperspace—none0/10

   No evidence of any real-time GPU availability dashboard, API endpoint, or CLI command showing stock/quota by type and region; docs only describe choosing machine type/region at creation time and needing approval for high-end GPUs, with no visibility into availability before provisioning.

   • [claimed-docs] “There are several machine types grouped by CPU, GPU, and multi-GPU. If you choose any high-end machines, such as NVIDIA H100, you need to re…”
   • [claimed-docs] “When you create a new machine, you choose your machine type, operating system or custom template, disk size, region, authentication, startin…”

   proof ↗⚑ flag

Hardware

1. ml engineerChoose from current-generation datacenter GPUs (H100/H200/B200 class) as well as cheaper previous-generation options

   weight 2 · round to Paperspace
   Vast.ai~partialprobed5/10

   Vast.ai's docs and CLI clearly support searching/filtering the marketplace by GPU model (e.g., `gpu_name=RTX_4090`) and offer tiered pricing (on-demand, reserved, interruptible) that would let an ML engineer pick pricier or cheaper GPU options, and 'Secure Cloud (Datacenter)' verification implies access to datacenter-grade hardware. However, none of the evidence explicitly confirms availability of current-generation H100/H200/B200-class GPUs alongside older cards — all concrete examples in the pack use RTX_4090 (a consumer card), not datacenter-tier flagship GPUs. Missing for 10: explicit documentation or listing showing H100/H200/B200 GPUs are actually rentable on the platform alongside cheaper previous-gen options.

   • [github] “vast.search_offers(query='gpu_name=RTX_4090 num_gpus>=4')”
   • [claimed-docs] “vastai search offers 'gpu_name=RTX_4090 num_gpus=1 verified=true direct_port_count>=1 rentable=true' -o 'dlperf_usd-'”
   • [claimed-docs] “Secure Cloud (Datacenter): These machines are verified and confirmed to be in a certified datacenter that meets our datacenter criteria.”
   • [claimed-docs] “Vast verifies that these machines are in a datacenter with a TIER 2/3 rating or ISO 27001. These offer cards have a blue label and are recom…”
   • [claimed-docs] “Reserved instances allow you to get significant discounts (up to 50%) by pre-paying for GPU time. You can convert any on-demand instance to …”
   • [claimed-docs] “Interruptible instances use a bidding system: **Lowest cost** (often 50%+ cheaper than on-demand)”
   • [probe] “PROBE runtime (recorded 2026-09-05): `vastai search offers 'gpu_name=RTX_4090 num_gpus=1' -o dph` searched the LIVE GPU marketplace with NO …”

   proof ↗⚑ flag

   Paperspace~partialclaimed6/10

   Docs confirm access to current-gen H100 (behind an approval gate) and previous-gen A100 GPUs, plus NVLink support and 'largest GPU catalog' claims, supporting a range of GPU tiers and cost/performance tradeoffs (docs-11, docs-17, docs-20, docs-26). However, there's no explicit mention of H200/B200-class GPUs or a clear listing of older, cheaper GPU tiers (e.g., T4/P100/V100) to fully match the story's specificity. missing for 10: explicit H200/B200 listing, explicit older/cheaper GPU tier catalog, independent benchmarking/corroboration of availability.

   • [claimed-docs] “Choose from the largest GPU catalog in the world. Leverage the latest NVIDIA GPUs including Ampere A100s with up to 8 GPUs.”
   • [claimed-docs] “There are several machine types grouped by CPU, GPU, and multi-GPU. If you choose any high-end machines, such as NVIDIA H100, you need to re…”
   • [claimed-docs] “Enhance data transfer speeds and scalability between NVIDIA GPUs for high-performance computing tasks by enabling NVLink.”
   • [claimed-docs] “Easily change instance types anytime so you always have access to the mix of cost and performance. Cancel anytime.”
   • [claimed-docs] “Leverage the latest NVIDIA GPUs including Ampere A100s with up to 8 GPUs.”

   proof ↗⚑ flag

Quotas

1. platform engineerSee documented quotas and instance limits and raise them through a defined process

   weight 2 · round to Paperspace
   Vast.ai—none0/10

   The evidence pack covers billing, autobilling, reserved/interruptible instance pricing, API keys, and CLI/SDK usage, but contains no documentation of account quotas, per-user instance limits, or a defined process to request limit increases. This axis is applicable to any capacity-oriented GPU marketplace, so absence of evidence yields 'none'. Missing for 10: documented quota/limit tables, a support/ticket process for raising limits, any mention of default instance caps or how to request more capacity.

   ⚑ flag

   Paperspace~partialclaimed3/10

   Docs mention that high-end GPU machines (e.g., H100) require an approval request, implying some quota/limit gating exists, but there is no dedicated documentation of quota tiers, numeric limits, or a defined escalation/support process for raising them beyond that single mention. Missing for 10: a documented quotas/limits reference page, explicit instance-count or resource caps, and a clear support ticket/process for requesting increases.

   • [claimed-docs] “There are several machine types grouped by CPU, GPU, and multi-GPU. If you choose any high-end machines, such as NVIDIA H100, you need to re…”

   proof ↗⚑ flag

Clusters

1. ml engineerProvision a multi-node GPU cluster with fast interconnect for distributed training without a sales cycle

   weight 3 · round to Vast.ai
   Vast.ai!disputedcontradicted5/10

   Docs describe self-service, no-sales-cycle provisioning (CLI/API/search offers for multi-GPU nodes) and an overlay-network feature specifically for multi-node NCCL/PyTorch training, which addresses the 'no sales cycle' and 'multi-node' parts of the story. However, independent hands-on reports describe bandwidth as 'lousy' and 'all over the place compared to what's listed in the console,' directly undercutting the 'fast interconnect' claim needed for distributed training performance. Missing for 10: documented interconnect specs (e.g. InfiniBand/NVLink bandwidth guarantees), benchmarked multi-node throughput, and independent confirmation that overlay networking delivers low-latency performance at scale.

   • [claimed-docs] “Vast now supports creating *overlay* networks for instances, allowing client instances on different machines on the same physical LAN to sha…”
   • [claimed-docs] “Vast now supports creating overlay networks for instances, allowing client instances on different machines on the same physical LAN to share…”
   • [github] “vast.search_offers(query='gpu_name=RTX_4090 num_gpus>=4')”
   • [claimed-docs] “search for a GPU, rent it, wait for it to boot, connect to it, copy data, and clean up”
   • [community] “Prices for compute were competitive, but most instances had lousy bandwidth. You don't need bandwidth to run bitcoin miners, apparently. The…”
   • [community] “As a client (renting GPUs) I've found it to be pretty unreliable. I've tried renting GPUs a handful of times and hosts, bandwidth, and perfo…”

   proof ↗⚑ flag

   Paperspace—none0/10

   Evidence shows Paperspace provisions single machines with up to 8 GPUs and NVLink for intra-node scaling, but nothing describes provisioning a multi-node cluster with fast cross-node interconnect (e.g., InfiniBand/RDMA) for distributed training, nor any self-service multi-node cluster workflow.

   • [claimed-docs] “Choose from the largest GPU catalog in the world. Leverage the latest NVIDIA GPUs including Ampere A100s with up to 8 GPUs.”
   • [claimed-docs] “Enhance data transfer speeds and scalability between NVIDIA GPUs for high-performance computing tasks by enabling NVLink.”
   • [claimed-docs] “Leverage the latest NVIDIA GPUs including Ampere A100s with up to 8 GPUs.”
   • [claimed-docs] “There are several machine types grouped by CPU, GPU, and multi-GPU. If you choose any high-end machines, such as NVIDIA H100, you need to re…”

   proof ↗⚑ flag

Orchestration

1. platform engineerSchedule jobs on managed Slurm or Kubernetes instead of building my own scheduler on raw nodes

   weight 2 · round drawn
   Vast.ai—none0/10

   Evidence shows Vast.ai provides raw GPU instance rental, templates, overlay networking for multi-node training, and CLI/API automation, but nothing about a managed Slurm or Kubernetes scheduling service — users would need to build their own scheduler on top of raw nodes. missing for 10: managed Slurm service, managed Kubernetes service, any orchestration layer beyond raw instance provisioning.

   • [claimed-docs] “Vast now supports creating *overlay* networks for instances, allowing client instances on different machines on the same physical LAN to sha…”
   • [claimed-docs] “Vast now supports creating overlay networks for instances, allowing client instances on different machines on the same physical LAN to share…”
   • [claimed-docs] “A template is how Vast helps you launch an instance, setting up your rented machine with whatever software and formatting you need.”
   • [claimed-docs] “The Vast.ai CLI gives you command-line access to the entire platform, authentication, GPU search, instance lifecycle, templates, volumes, se…”

   proof ↗⚑ flag

   Paperspace—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round to Vast.ai
   Vast.ai✓fullprobed9/10

   Vast.ai's REST API is explicitly documented as the foundation underlying both the CLI and SDK, giving full programmatic control over the entire platform (search, instance lifecycle, templates, volumes, serverless endpoints), and this is corroborated by a live OpenAPI spec and runtime probes showing the CLI/API perform the same search/rent/manage operations available in the UI, including a dedicated agent skill for AI assistants to drive the platform. Minor gap: no explicit UI-vs-API feature parity audit exists confirming literally every UI-only setting (e.g., some billing/account UI screens) has an API equivalent. Missing for 10: an explicit parity matrix or independent confirmation that 100% of UI actions (not just core lifecycle) are API-exposed.

   • [claimed-docs] “The Vast.ai CLI gives you command-line access to the entire platform, authentication, GPU search, instance lifecycle, templates, volumes, se…”
   • [claimed-docs] “The Vast.ai REST API gives you programmatic control over the entire platform. It's the foundation that the CLI and SDK are built on.”
   • [claimed-docs] “The Vast.ai REST API gives you programmatic control over the entire platform.”
   • [claimed-docs] “Automate with the CLI or SDK ... `vastai` from the shell or Python. Same operations, different syntax.”
   • [claimed-docs] “The vastai agent skill lets AI coding assistants drive Vast.ai on your behalf — create instances, deploy endpoints, manage SSH keys, check b…”
   • [probe] “PROBE openapi: HTTP 200 at https://docs.vast.ai/api/openapi.json — contains "openapi" key”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Vast.ai CLI (pypi `vastai`) installed via uvx and ran keylessly — `vastai --help` prints t…”
   • [probe] “PROBE runtime (recorded 2026-09-05): `vastai search offers 'gpu_name=RTX_4090 num_gpus=1' -o dph` searched the LIVE GPU marketplace with NO …”
   • [probe] “PROBE runtime (recorded 2026-09-05): a bare keyless GET to https://console.vast.ai/api/v0/bundles/ returned live marketplace offers as JSON …”

   proof ↗⚑ flag

   Paperspace~partialprobed6/10

   Paperspace ships both an API and a first-party pspace CLI that support machine creation, custom templates, autoscaling groups, and deployments (paperspace-docs-1, paperspace-docs-6, paperspace-docs-23/24, paperspace-probe-4, paperspace-probe-rt-1), giving strong API/CLI parity for core resource lifecycle tasks. However, some UI-only features like SSH connection setup are described as done via console/desktop app (paperspace-docs-2), no public OpenAPI/Swagger spec was found (paperspace-probe-3), and there's no explicit evidence that shared drives, auto-shutdown, or notebook management are fully API-controllable. Missing for 10: OpenAPI/spec confirmation of full endpoint coverage, explicit API parity for shared drives/auto-shutdown/notebooks, and independent hands-on confirmation that API-driven workflows match UI capability without gaps.

   • [claimed-docs] “You can create a Linux machine using the Paperspace console, Paperspace API, or Paperspace CLI.”
   • [claimed-docs] “API keys let you interact with Paperspace through the Core RESTful API, the Core JavaScript SDK, and Gradient command-line utility (CLI).”
   • [claimed-docs] “Programmatically manage Paperspace resources using conventional HTTP requests.”
   • [claimed-docs] “Install and use the new Paperspace Command Line Interface.”
   • [probe] “official CLI documented at https://docs.digitalocean.com/reference/paperspace/pspace/”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official installer (`curl -fsSL https://paperspace.com/install.sh | sh`) installed the pspace CLI i…”
   • [claimed-docs] “You can connect to your Linux or Windows-based machine using the Paperspace console or desktop app using SSH connection.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.digitalocean.com/openapi.json, https://docs.digitalocean.com/swagger.json, https://docs…”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round to Paperspace
   Vast.ai~partialclaimed3/10

   Vast.ai instances run standard Linux with SSH access explicitly described as supporting file transfer ('Transfer files without exposing your data') and volumes are persistent, detachable storage, implying no proprietary lock-in on user data — but there is no explicit documentation of a bulk data-export feature, open-format guarantee, or account-closure data portability process. missing for 10: explicit data-export/portability documentation, statement on open formats, and evidence of full account data extraction on leaving.

   • [claimed-docs] “Log in securely, Run commands remotely, Transfer files without exposing your data”
   • [claimed-docs] “SSH (Secure Shell) is a protocol for safely connecting to remote servers. It encrypts your connection so you can: Log in securely, Run comma…”
   • [claimed-docs] “Volumes provide persistent storage that survives instance destruction and can be reattached to new instances.”

   proof ↗⚑ flag

   Paperspace~partialprobed4/10

   Paperspace machines use standard SSH/root access and shared drives, and API/CLI access lets users programmatically pull down VM configs, custom templates, and data via SSH/SCP, which provides some data portability. However there is no documented one-click 'export all data' or bulk data-export feature, no explicit open-format guarantee for notebooks/deployments/workflows metadata, and no evidence of a full account data export or migration tool. missing for 10: dedicated bulk data export/download feature, explicit open-format export guarantees for notebooks/deployments/workflows, documented account-level data portability or migration tooling, independent confirmation of successful full data export.

   • [claimed-docs] “You can connect to your Linux or Windows-based machine using the Paperspace console or desktop app using SSH connection.”
   • [claimed-docs] “Shared drives provide storage that is accessible from multiple machines in a private network.”
   • [claimed-docs] “API keys let you interact with Paperspace through the Core RESTful API, the Core JavaScript SDK, and Gradient command-line utility (CLI).”
   • [claimed-docs] “Programmatically manage Paperspace resources using conventional HTTP requests.”
   • [claimed-docs] “Install and use the new Paperspace Command Line Interface.”
   • [probe] “official CLI documented at https://docs.digitalocean.com/reference/paperspace/pspace/”

   proof ↗⚑ flag

Billing

1. platform engineerPull usage and billing breakdowns programmatically to attribute GPU spend by team or workload

   weight 2 · round to Vast.ai
   Vast.ai~partialclaimed4/10

   Vast.ai's REST API and CLI provide general programmatic access to the platform, including account balance checks and billing pages (autobilling, credit top-up), and scoped API keys can be created per workload/team for CI/CD use, which could support some cost attribution. However there is no explicit evidence of a dedicated usage/billing breakdown endpoint, cost tagging, or per-team/workload spend reporting API. missing for 10: dedicated billing/usage export API, cost tagging or labels for spend attribution, documented reports/analytics endpoint for team-level breakdowns.

   • [claimed-docs] “The Vast.ai REST API gives you programmatic control over the entire platform. It's the foundation that the CLI and SDK are built on.”
   • [claimed-docs] “Use the autobilling feature to have the system automatically top up your account using a saved credit card when it runs low.”
   • [claimed-docs] “create a scoped key with only the permissions that workload needs”
   • [claimed-docs] “For shared tooling, CI/CD, or automation, create a scoped key with only the permissions that workload needs.”
   • [claimed-docs] “Autobilling keeps your balance positive to protect your instances from interruption or deletion.”

   proof ↗⚑ flag

   Paperspace—none0/10

   No evidence of any billing/usage export or cost-attribution API, report, or CLI command — the API/CLI docs cover compute resources (machines, notebooks, deployments) but nothing about programmatic billing or GPU spend breakdown by team/workload.

   ⚑ flag

2. ml engineerI am billed at per-second or per-minute granularity and only while my instance is actually running

   weight 3 · round drawn
   Vast.ai—none0/10

   Evidence shows Vast.ai quotes prices as hourly rates (dph/dph_total) and discusses reserved/interruptible pricing models and autobilling, but nowhere does the evidence pack confirm sub-hour (per-second or per-minute) billing granularity or explicitly state billing stops precisely when an instance is not running.

   ⚑ flag

   Paperspace—none0/10

   No evidence in the pack states per-second or per-minute billing granularity; the only granularity mentioned is an 'hourly rate' (paperspace-comm-8) and a flat monthly subscription tier with fixed hours (paperspace-comm-5), neither of which confirms sub-minute billing precision while running.

   • [community] “I use a Paperspace VM + Parsec for personal ML projects. An hourly rate on a standard VM w/GPU is better than buying a local machine, and yo…”
   • [community] “I use Stable Diffusion with Paperspace's Pro tier ($9/mo) for up to 6 hours of non per-usage GPU time, avoiding worry about electricity cost…”

   proof ↗⚑ flag

Discovery

1. ai agentQuery the GPU catalog with live pricing and availability from a public or documented endpoint before committing any spend

   weight 2 · round to Vast.ai
   Vast.ai✓fullprobed9/10

   Vast.ai provides both a documented public REST endpoint (bundles/offers) and CLI (`vastai search offers`) that return live GPU pricing, specs, and availability without requiring an account or API key, verified via runtime probes. This directly matches the story of an agent querying the catalog before committing spend; missing for 10: independent third-party corroboration beyond vendor docs/probes and no explicit rate-limit/auth-tier documentation for heavy automated querying.

   • [probe] “PROBE runtime (recorded 2026-09-05): `vastai search offers 'gpu_name=RTX_4090 num_gpus=1' -o dph` searched the LIVE GPU marketplace with NO …”
   • [probe] “PROBE runtime (recorded 2026-09-05): a bare keyless GET to https://console.vast.ai/api/v0/bundles/ returned live marketplace offers as JSON …”
   • [claimed-docs] “vastai search offers 'gpu_name=RTX_4090 num_gpus=1 verified=true direct_port_count>=1 rentable=true' -o 'dlperf_usd-'”
   • [github] “vast.search_offers(query='gpu_name=RTX_4090 num_gpus>=4')”
   • [probe] “PROBE openapi: HTTP 200 at https://docs.vast.ai/api/openapi.json — contains "openapi" key”

   proof ↗⚑ flag

   Paperspace—none0/10

   While Paperspace documents machine types, GPU options, and an API/CLI for creating machines, there is no evidence of a documented endpoint or CLI command that returns live pricing or availability data for the GPU catalog before provisioning—only marketing claims like 'save up to 70%' and 'largest GPU catalog' with no queryable pricing API surfaced.

   • [claimed-docs] “Choose from the largest GPU catalog in the world. Leverage the latest NVIDIA GPUs including Ampere A100s with up to 8 GPUs.”
   • [claimed-docs] “There are several machine types grouped by CPU, GPU, and multi-GPU. If you choose any high-end machines, such as NVIDIA H100, you need to re…”
   • [claimed-docs] “Programmatically manage Paperspace resources using conventional HTTP requests.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.digitalocean.com/openapi.json, https://docs.digitalocean.com/swagger.json, https://docs…”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official installer (`curl -fsSL https://paperspace.com/install.sh | sh`) installed the pspace CLI i…”

   proof ↗⚑ flag

Pricing

1. platform engineerLock in reserved or committed-use discounts for sustained GPU capacity

   weight 1 · round to Vast.ai
   Vast.ai✓fullclaimed7/10

   Vast.ai documents Reserved Instances that let users pre-pay for GPU time to lock in up to 50% discounts, convertible from any on-demand instance at any time, directly matching the committed-use discount story. Missing for 10: no independent/community corroboration of reserved-instance pricing behavior, no detail on commitment duration/terms flexibility, and no evidence of longer-term (e.g. multi-month/annual) contractual reserved capacity beyond simple pre-payment conversion.

   • [claimed-docs] “Reserved instances allow you to get significant discounts (up to 50%) by pre-paying for GPU time. You can convert any on-demand instance to …”
   • [claimed-docs] “You can convert any on-demand instance to a reserved instance at any time.”
   • [claimed-docs] “You can convert an on-demand instance into a reserved instance with a lower hourly rate by pre-paying.”
   • [claimed-docs] “Reserved instances allow you to get significant discounts (up to 50%) by pre-paying for GPU time.”

   proof ↗⚑ flag

   Paperspace—none0/10

   No evidence of reserved instances, committed-use contracts, or sustained-use discounts; only pay-as-you-go pricing claims like 'save up to 70%' and 'cancel anytime' are documented, which implies on-demand rather than committed pricing.

   • [claimed-docs] “Save up to 70% on compute costs”
   • [claimed-docs] “Easily change instance types anytime so you always have access to the mix of cost and performance. Cancel anytime.”

   proof ↗⚑ flag

2. ml engineerSee the published per-GPU-hour price for every GPU type on a public pricing page without talking to sales

   weight 3 · round to Vast.ai
   Vast.ai~partialprobed7/10

   The marketplace search page and public REST/CLI endpoints let anyone see live per-GPU-hour prices without an account or contacting sales (vast-ai-docs-41, vast-ai-probe-rt-2, vast-ai-probe-rt-3), which covers the core of the story. However, there's no dedicated static 'pricing page' listing every GPU type's rate at a glance—pricing is discovered via a dynamic search/filter interface, and a community user explicitly complained about the number of clicks needed to find even an approximate price (vast-ai-comm-3). Missing for 10: a single canonical pricing table/page enumerating all GPU types' rates, and stronger independent corroboration that pricing is easy to find without friction.

   • [claimed-docs] “The search page is the main portal for finding good machines and creating instances on them.”
   • [probe] “PROBE runtime (recorded 2026-09-05): `vastai search offers 'gpu_name=RTX_4090 num_gpus=1' -o dph` searched the LIVE GPU marketplace with NO …”
   • [probe] “PROBE runtime (recorded 2026-09-05): a bare keyless GET to https://console.vast.ai/api/v0/bundles/ returned live marketplace offers as JSON …”
   • [community] “As a potential user, I'm disliking how many clicks it takes to find even an approximate price. Prices should be easiest to find.”
   • [claimed-docs] “Vast typically delivers more competitive pricing than traditional cloud providers. The marketplace model creates natural price competition”

   proof ↗⚑ flag

   Paperspace—none0/10

   The evidence pack contains marketing claims like 'save up to 70% on compute costs' but no actual published per-GPU-hour pricing table or pricing page content is shown; no citation demonstrates a public price list for each GPU type.

   • [claimed-docs] “Save up to 70% on compute costs”
   • [claimed-docs] “Save up to 70% on compute costs Spend significantly less on your GPU compute compared to the major public clouds or buying your own servers.”
   • [claimed-docs] “Save up to 70% on compute costs. Spend significantly less on your GPU compute compared to the major public clouds or buying your own servers…”

   proof ↗⚑ flag

Spot

1. ml engineerRent spot or interruptible GPU capacity at a deep discount with clearly documented preemption semantics

   weight 3 · round to Vast.ai
   Vast.ai~partialcommunity6/10

   Vast.ai clearly documents an interruptible/bidding instance type offering 50%+ discounts vs on-demand (vast-ai-docs-7/17/22/35/44), which satisfies the 'deep discount' part of the story. However, the evidence pack gives only the pricing angle and doesn't detail concrete preemption mechanics (notice period, how bidding/eviction actually triggers, restart behavior) — missing for 10: explicit preemption trigger/notice documentation, and independent confirmation that discount/preemption behavior matches docs (community notes reliability variance, e.g. vast-ai-comm-6, but this isn't a concrete dispute of interruptible semantics specifically).

   • [claimed-docs] “Interruptible instances use a bidding system: **Lowest cost** (often 50%+ cheaper than on-demand)”
   • [claimed-docs] “Lowest cost (often 50%+ cheaper than on-demand)”
   • [claimed-docs] “Interruptible instances use a bidding system: Lowest cost (often 50%+ cheaper than on-demand)”
   • [claimed-docs] “Interruptible instances use a bidding system... Lowest cost (often 50%+ cheaper than on-demand)”
   • [claimed-docs] “Vast.ai offers three instance types with different priority levels and pricing models to match your workload requirements and budget.”
   • [community] “As a client (renting GPUs) I've found it to be pretty unreliable. I've tried renting GPUs a handful of times and hosts, bandwidth, and perfo…”

   proof ↗⚑ flag

   Paperspace—none0/10

   No evidence anywhere in the pack mentions spot, preemptible, or interruptible instances, discount pricing tiers, or preemption semantics; only standard on-demand GPU rentals and general 'save up to 70%' marketing are documented.

   ⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round to Paperspace
   Vast.ai—none0/10

   The evidence pack shows Vast.ai lets users search/filter GPU offers by attributes like GPU model and reliability, and it distinguishes 'Secure Cloud' verified datacenters, but nowhere documents an explicit region/country selection or data-residency guarantee for where data is stored. A community report even flags a case where a host's advertised location (US) was misrepresented (actually China), undermining confidence that location can be reliably chosen or verified.

   • [claimed-docs] “Secure Cloud (Datacenter): These machines are verified and confirmed to be in a certified datacenter that meets our datacenter criteria.”
   • [claimed-docs] “Vast verifies that these machines are in a datacenter with a TIER 2/3 rating or ISO 27001. These offer cards have a blue label and are recom…”
   • [community] “Discussion flagged a Chinese host on vast.ai masquerading as a US host, raising concerns about where rented GPU compute is actually located.”

   proof ↗⚑ flag

   Paperspace~partialclaimed4/10

   Docs confirm you select a 'region' when creating a machine ([paperspace-docs-25]), showing basic region choice for compute placement, but there is no evidence of dedicated data residency controls, storage region selection, compliance certifications (e.g., GDPR, SOC2 data residency), or documentation on where notebooks/deployments data is stored. missing for 10: explicit data residency/compliance documentation, storage-specific region selection, list of available regions, independent confirmation of enforcement.

   • [claimed-docs] “When you create a new machine, you choose your machine type, operating system or custom template, disk size, region, authentication, startin…”

   proof ↗⚑ flag

2. ai-native userControl data retention and deletion

   weight 2 · round to Vast.ai
   Vast.ai~partialprobed3/10

   Evidence shows users can destroy instances and volumes (which otherwise persist across instance destruction) via CLI/API, giving basic control over deletion of compute resources and data, but there is no explicit documentation of data retention policies, data-deletion guarantees, or privacy controls beyond simply terminating instances/volumes. missing for 10: explicit data retention/deletion policy docs, guarantees on backend data purge after deletion, account-level data deletion controls, independent verification of deletion behavior.

   • [claimed-docs] “Volumes provide persistent storage that survives instance destruction and can be reattached to new instances.”
   • [claimed-docs] “search for a GPU, rent it, wait for it to boot, connect to it, copy data, and clean up”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Vast.ai CLI (pypi `vastai`) installed via uvx and ran keylessly — `vastai --help` prints t…”

   proof ↗⚑ flag

   Paperspace—none0/10

   No evidence pack items address data retention policies, deletion controls, or privacy/data lifecycle management for Paperspace resources like notebooks, storage, or machine data; the docs cover creation, connection, and compute features but nothing on retention/deletion control.

   ⚑ flag

3. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   Vast.ai—none0/10

   No evidence in the pack mentions telemetry, usage tracking, analytics opt-out, or privacy settings for Vast.ai's own platform/CLI; the docs focus on billing, instances, SSH, and API/CLI usage. missing for 10: any documentation of telemetry collection, an opt-out mechanism, or a privacy policy addressing usage tracking.

   ⚑ flag

   Paperspace—none0/10

   No evidence pack items mention telemetry, usage tracking, analytics opt-out, or privacy controls for Paperspace; the docs cover machine creation, CLI, and pricing but never address data collection or opt-out mechanisms.

   ⚑ flag

Agent ops

1. ai agentProvision a GPU, monitor it, run a workload, and tear it down end to end through documented APIs, CLI, or MCP without a human in the console

   weight 3 · round to Vast.ai
   Vast.ai✓fullprobed8/10

   Vast.ai documents a full REST API, CLI, and Python SDK covering the entire instance lifecycle (search offers, create/rent, connect, destroy), confirmed by runtime probes showing the CLI and API working keylessly for search and an OpenAPI spec being live. It also ships a dedicated 'vastai agent skill' explicitly for AI coding assistants to create instances, deploy endpoints, and manage keys without human console interaction, plus scoped API keys for automation/CI. Missing for 10: no independent hands-on evidence of a full agent-driven create→monitor→run→teardown cycle end-to-end (only search was probed live), and no first-party MCP server confirmed beyond the 'agent skill' framing.

   • [claimed-docs] “The Vast.ai CLI gives you command-line access to the entire platform, authentication, GPU search, instance lifecycle, templates, volumes, se…”
   • [claimed-docs] “The Vast.ai REST API gives you programmatic control over the entire platform. It's the foundation that the CLI and SDK are built on.”
   • [claimed-docs] “The `vastai` agent skill lets AI coding assistants drive Vast.ai on your behalf — create instances, deploy endpoints, manage SSH keys, check…”
   • [claimed-docs] “The vastai agent skill lets AI coding assistants drive Vast.ai on your behalf — create instances, deploy endpoints, manage SSH keys, check b…”
   • [claimed-docs] “For shared tooling, CI/CD, or automation, create a scoped key with only the permissions that workload needs.”
   • [claimed-docs] “search for a GPU, rent it, wait for it to boot, connect to it, copy data, and clean up”
   • [probe] “PROBE openapi: HTTP 200 at https://docs.vast.ai/api/openapi.json — contains "openapi" key”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Vast.ai CLI (pypi `vastai`) installed via uvx and ran keylessly — `vastai --help` prints t…”
   • [probe] “PROBE runtime (recorded 2026-09-05): `vastai search offers 'gpu_name=RTX_4090 num_gpus=1' -o dph` searched the LIVE GPU marketplace with NO …”
   • [probe] “PROBE runtime (recorded 2026-09-05): a bare keyless GET to https://console.vast.ai/api/v0/bundles/ returned live marketplace offers as JSON …”

   proof ↗⚑ flag

   Paperspace~partialprobed5/10

   Paperspace documents API keys, a Core RESTful API, and an official CLI (pspace) for creating/managing machines, custom templates, and auto-shutdown, and a runtime probe confirms the CLI installs and lists machine/deployment commands — enough to plausibly script provisioning and teardown without a human in the console. However there is no MCP server, no documented end-to-end 'create→monitor→run workload→teardown' workflow example, and community reports describe friction getting SSH/API access working reliably on Core VMs, undercutting a clean automated experience. Missing for 10: an MCP server, explicit workload-run/monitoring API docs, and independent confirmation of a full agent-driven lifecycle without console fallback.

   • [claimed-docs] “You can create a Linux machine using the Paperspace console, Paperspace API, or Paperspace CLI.”
   • [claimed-docs] “API keys let you interact with Paperspace through the Core RESTful API, the Core JavaScript SDK, and Gradient command-line utility (CLI).”
   • [claimed-docs] “Programmatically manage Paperspace resources using conventional HTTP requests.”
   • [claimed-docs] “Install and use the new Paperspace Command Line Interface.”
   • [claimed-docs] “The auto-shutdown feature places your machine in a stop state after a predefined period of inactivity, which can range from one hour to one …”
   • [probe] “official CLI documented at https://docs.digitalocean.com/reference/paperspace/pspace/”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official installer (`curl -fsSL https://paperspace.com/install.sh | sh`) installed the pspace CLI i…”
   • [community] “Their DL virtual servers (Core) are horrible - very slow internet, takes forever to copy datasets in. Getting SSH access is an uphill battle…”

   proof ↗⚑ flag

Manage

1. ml engineerSet auto-shutdown timers or spend limits so a forgotten instance can't silently run up a huge bill

   weight 2 · round to Paperspace
   Vast.ai—none0/10

   The evidence pack documents autobilling (auto top-up to keep instances running), reserved/interruptible instance pricing, and scoped API keys, but none of these are auto-shutdown timers or spend caps — autobilling in fact works against this story by automatically refilling balance to prevent instance interruption rather than capping spend. No mention of idle-timeout, max-runtime, or budget-limit features exists anywhere in the pack.

   • [claimed-docs] “Use the autobilling feature to have the system automatically top up your account using a saved credit card when it runs low.”
   • [claimed-docs] “Autobilling keeps your balance positive to protect your instances from interruption or deletion.”
   • [claimed-docs] “Reserved instances allow you to get significant discounts (up to 50%) by pre-paying for GPU time. You can convert any on-demand instance to …”
   • [claimed-docs] “create a scoped key with only the permissions that workload needs”

   proof ↗⚑ flag

   Paperspace~partialclaimed6/10

   Paperspace documents an auto-shutdown feature that stops a machine after inactivity (from 1 hour to 1 week), directly addressing the 'forgotten instance' risk, and this is corroborated by official docs. However, there is no evidence of configurable spend/budget limits or billing caps, which is the other half of the story. missing for 10: documented spend-limit/budget-cap feature, independent hands-on confirmation that auto-shutdown reliably prevents runaway billing.

   • [claimed-docs] “The auto-shutdown feature places your machine in a stop state after a predefined period of inactivity, which can range from one hour to one …”

   proof ↗⚑ flag

2. developerStart, stop, restart, and terminate instances programmatically and keep paying only for what is running

   weight 2 · round to Vast.ai
   Vast.ai✓fullprobed8/10

   Vast.ai's CLI and REST API provide documented commands/endpoints for the full instance lifecycle (search, create, destroy) confirmed by runtime probes, plus billing docs (autobilling, per-hour pricing, reserved/interruptible discounts) demonstrate pay-only-for-what-runs. Community feedback corroborates cost savings but also notes reliability variance, which is a secondary concern rather than a lifecycle-control failure. Missing for 10: explicit documented 'stop'/'restart' CLI subcommands beyond create/destroy in the evidence, and independent hands-on confirmation of stop/restart specifically (only create/destroy verified at runtime).

   • [claimed-docs] “The Vast.ai CLI gives you command-line access to the entire platform, authentication, GPU search, instance lifecycle, templates, volumes, se…”
   • [claimed-docs] “The Vast.ai REST API gives you programmatic control over the entire platform. It's the foundation that the CLI and SDK are built on.”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Vast.ai CLI (pypi `vastai`) installed via uvx and ran keylessly — `vastai --help` prints t…”
   • [probe] “PROBE runtime (recorded 2026-09-05): `vastai search offers 'gpu_name=RTX_4090 num_gpus=1' -o dph` searched the LIVE GPU marketplace with NO …”
   • [probe] “PROBE runtime (recorded 2026-09-05): a bare keyless GET to https://console.vast.ai/api/v0/bundles/ returned live marketplace offers as JSON …”
   • [claimed-docs] “Use the autobilling feature to have the system automatically top up your account using a saved credit card when it runs low.”
   • [claimed-docs] “Autobilling keeps your balance positive to protect your instances from interruption or deletion.”
   • [claimed-docs] “Reserved instances allow you to get significant discounts (up to 50%) by pre-paying for GPU time. You can convert any on-demand instance to …”
   • [claimed-docs] “Interruptible instances use a bidding system: **Lowest cost** (often 50%+ cheaper than on-demand)”
   • [community] “Absolutely incredible how cheap the compute power is on this platform. Vast has saved me tens of thousands of dollars running ML experiments…”

   proof ↗⚑ flag

   Paperspace~partialprobed5/10

   Evidence confirms programmatic creation of machines via API/CLI/console and an auto-shutdown feature that stops machines after inactivity to control costs, and the pspace CLI clearly manages 'machine' resources. However, no direct documentation shows explicit start/stop/restart/terminate commands or billing-only-when-running semantics beyond auto-shutdown. missing for 10: explicit API/CLI start, stop, restart, terminate operations, confirmation that billing pauses immediately on stop, independent verification of lifecycle commands working.

   • [claimed-docs] “You can create a Linux machine using the Paperspace console, Paperspace API, or Paperspace CLI.”
   • [claimed-docs] “The auto-shutdown feature places your machine in a stop state after a predefined period of inactivity, which can range from one hour to one …”
   • [claimed-docs] “API keys let you interact with Paperspace through the Core RESTful API, the Core JavaScript SDK, and Gradient command-line utility (CLI).”
   • [claimed-docs] “Programmatically manage Paperspace resources using conventional HTTP requests.”
   • [claimed-docs] “Install and use the new Paperspace Command Line Interface.”
   • [probe] “official CLI documented at https://docs.digitalocean.com/reference/paperspace/pspace/”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official installer (`curl -fsSL https://paperspace.com/install.sh | sh`) installed the pspace CLI i…”

   proof ↗⚑ flag

Provision

1. developerProvision an on-demand GPU instance from the console or API and be running code on it within minutes

   weight 3 · round to Vast.ai
   Vast.ai✓fullprobed7/10

   Docs and runtime probes show the full quickstart flow — search offers via CLI/API keylessly, one-click templates, instance creation, SSH access with password auth disabled, and REST/CLI parity for automation — matching the 'search, rent, boot, connect' workflow described in vast-ai-docs-34. Community reviews largely corroborate ease and cost savings, though some flag friction/hoops versus AWS/GCP and inconsistent reliability, which tempers but doesn't contradict the core provisioning claim. Missing for 10: an independent hands-on timing test confirming actual boot-to-running-code within minutes, and resolution of community complaints about setup friction/reliability variance.

   • [claimed-docs] “This Quickstart will guide you through setting up your Vast.ai account and running your first instance in just a few steps.”
   • [claimed-docs] “search for a GPU, rent it, wait for it to boot, connect to it, copy data, and clean up”
   • [claimed-docs] “The search page is the main portal for finding good machines and creating instances on them.”
   • [claimed-docs] “Launch prebuilt or custom templates with one click.”
   • [claimed-docs] “The Vast.ai CLI gives you command-line access to the entire platform, authentication, GPU search, instance lifecycle, templates, volumes, se…”
   • [claimed-docs] “The Vast.ai REST API gives you programmatic control over the entire platform. It's the foundation that the CLI and SDK are built on.”
   • [claimed-docs] “SSH (Secure Shell) is a protocol for safely connecting to remote servers. It encrypts your connection so you can: Log in securely, Run comma…”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official Vast.ai CLI (pypi `vastai`) installed via uvx and ran keylessly — `vastai --help` prints t…”
   • [probe] “PROBE runtime (recorded 2026-09-05): `vastai search offers 'gpu_name=RTX_4090 num_gpus=1' -o dph` searched the LIVE GPU marketplace with NO …”
   • [probe] “PROBE runtime (recorded 2026-09-05): a bare keyless GET to https://console.vast.ai/api/v0/bundles/ returned live marketplace offers as JSON …”
   • [community] “Absolutely incredible how cheap the compute power is on this platform. Vast has saved me tens of thousands of dollars running ML experiments…”
   • [community] “While they've done a pretty good job considering with their software stack it's still kind of a pain to use with significant limitations and…”

   proof ↗⚑ flag

   Paperspace!disputedcontradicted5/10

   Docs and marketing show provisioning via console/API/CLI, SSH root access, and ML-ready templates enabling fast startup (paperspace-docs-1,6,10,12,13,24), and a runtime probe confirms the CLI installs and lists machine-management commands (paperspace-probe-rt-1). However, a hands-on community report directly contradicts the 'connect and run in minutes' claim: 'Getting SSH access is an uphill battle; instead there's a pointless virtual console... very slow internet, takes forever to copy datasets in' (paperspace-comm-7), and another user calls the notebook experience 'really bad' (paperspace-comm-6). Missing for 10: independent verified timing benchmarks of provisioning-to-running-code, resolution of the SSH-access friction reported by users, and more recent hands-on corroboration.

   • [claimed-docs] “You can create a Linux machine using the Paperspace console, Paperspace API, or Paperspace CLI.”
   • [claimed-docs] “API keys let you interact with Paperspace through the Core RESTful API, the Core JavaScript SDK, and Gradient command-line utility (CLI).”
   • [claimed-docs] “Go from signup to training a model in seconds”
   • [claimed-docs] “Bring your SSH key and connect directly to your VM with full root access.”
   • [claimed-docs] “Choose "ML in a Box" template that comes preinstalled with all the major ML frameworks and CUDA® drivers.”
   • [claimed-docs] “Install and use the new Paperspace Command Line Interface.”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official installer (`curl -fsSL https://paperspace.com/install.sh | sh`) installed the pspace CLI i…”
   • [community] “Their DL virtual servers (Core) are horrible - very slow internet, takes forever to copy datasets in. Getting SSH access is an uphill battle…”
   • [community] “PaperSpace product recently has been really bad. Gradient Notebooks are a worse version of Google Colab, useless for serious DL product buil…”

   proof ↗⚑ flag

Serverless

1. developerDeploy code to autoscaling serverless GPU workers that scale to zero, instead of managing always-on instances

   weight 2 · round to Vast.ai
   Vast.ai~partialclaimed6/10

   Vast.ai explicitly ships 'Vast Serverless', described as letting users run compute-intensive workloads 'without managing GPUs, paying for execution rather than GPU rental time,' with a Python SDK and CLI support for deploying endpoints — matching the pay-per-execution serverless model a developer would want instead of always-on instances. However, the evidence never explicitly confirms autoscaling behavior or scale-to-zero semantics, nor gives hands-on/independent corroboration of how workers scale under load. Missing for 10: explicit autoscaling/scale-to-zero documentation, independent or hands-on validation of serverless endpoint behavior, and details on cold-start/latency tradeoffs.

   • [claimed-docs] “Vast Serverless is an AI infrastructure platform that lets you run compute-intensive workloads without managing GPUs, paying for execution r…”
   • [claimed-docs] “Interacting with Vast Serverless is made easy through a powerful python SDK.”
   • [claimed-docs] “The vastai agent skill lets AI coding assistants drive Vast.ai on your behalf — create instances, deploy endpoints, manage SSH keys, check b…”
   • [claimed-docs] “The `vastai` agent skill lets AI coding assistants drive Vast.ai on your behalf — create instances, deploy endpoints, manage SSH keys, check…”

   proof ↗⚑ flag

   Paperspace—none0/10

   Paperspace's evidence describes VM-based 'Machines' with auto-shutdown after inactivity (docs-5) and container 'Deployments'/autoscaling-groups (docs-8, probe-rt-1), but nothing confirms true serverless GPU workers that scale to zero per-request and back up automatically without idle billing — the model described is always-provisioned instances that stop, not ephemeral serverless invocation.

   • [claimed-docs] “The auto-shutdown feature places your machine in a stop state after a predefined period of inactivity, which can range from one hour to one …”
   • [claimed-docs] “Deployments are containers-as-a-service that let you run container images and serve machine learning models.”
   • [probe] “PROBE runtime (recorded 2026-09-05): the official installer (`curl -fsSL https://paperspace.com/install.sh | sh`) installed the pspace CLI i…”

   proof ↗⚑ flag

Data movement

1. developerMove data in and out efficiently — S3-compatible endpoints, cloud-storage sync, or documented transfer tooling

   weight 2 · round to Vast.ai
   Vast.ai~partialcommunity5/10

   Vast.ai documents SSH-based secure file transfer to/from instances and a CLI workflow that includes 'copy data' as a step, plus persistent Volumes for data that survives instance destruction. However there is no evidence of S3-compatible endpoints or native cloud-storage (S3/GCS/Azure Blob) sync tooling, and a community comment notes persistent-disk workflows are still cumbersome over time. Missing for 10: S3-compatible storage endpoint, built-in cloud-storage sync/mirroring tooling, independent hands-on validation of transfer throughput/performance.

   • [claimed-docs] “Log in securely, Run commands remotely, Transfer files without exposing your data”
   • [claimed-docs] “SSH (Secure Shell) is a protocol for safely connecting to remote servers. It encrypts your connection so you can: Log in securely, Run comma…”
   • [claimed-docs] “search for a GPU, rent it, wait for it to boot, connect to it, copy data, and clean up”
   • [claimed-docs] “Volumes provide persistent storage that survives instance destruction and can be reattached to new instances.”
   • [community] “Love vast.ai. Only wish is for it to be easier to work with persistent disks over time. I only use it for one off expensive jobs now.”

   proof ↗⚑ flag

   Paperspace—none0/10

   No evidence of S3-compatible endpoints, cloud-storage sync, or dedicated data-transfer tooling; only generic 'shared drives' (docs-4) and CLI/API for resource management are documented, and community reports (paperspace-comm-7) describe data transfer to Core VMs as slow and cumbersome, further undermining any efficiency claim. missing for 10: S3-compatible endpoint, documented sync/transfer tool, benchmarked transfer speeds, and any first-party guidance on moving data in/out.

   • [claimed-docs] “Shared drives provide storage that is accessible from multiple machines in a private network.”
   • [claimed-docs] “Machines are Linux and Windows virtual machines with persistent storage, GPU options, and free unlimited bandwidth.”
   • [community] “Their DL virtual servers (Core) are horrible - very slow internet, takes forever to copy datasets in. Getting SSH access is an uphill battle…”

   proof ↗⚑ flag

Storage

1. developerAttach persistent network storage that survives instance teardown, so datasets and checkpoints outlive any single GPU rental

   weight 3 · round drawn
   Vast.ai~partialcommunity6/10

   Vast.ai docs explicitly describe 'Volumes' as persistent storage that survives instance destruction and can be reattached to new instances, directly matching the story. However, a community user notes wishing it were 'easier to work with persistent disks over time' and now avoids it for anything but one-off jobs, indicating real friction in practice. Missing for 10: independent hands-on verification of volume reattachment reliability, docs detailing size/performance limits, and broader community corroboration beyond a single anecdotal complaint.

   • [claimed-docs] “Volumes provide persistent storage that survives instance destruction and can be reattached to new instances.”
   • [community] “Love vast.ai. Only wish is for it to be easier to work with persistent disks over time. I only use it for one off expensive jobs now.”

   proof ↗⚑ flag

   Paperspace~partialcommunity6/10

   Docs describe persistent storage attached to machines (paperspace-docs-16) and separate 'shared drives' accessible from multiple machines in a private network (paperspace-docs-4), plus persistent notebook storage (paperspace-docs-7), which together imply data can outlive a single GPU instance. However, no evidence explicitly confirms shared drives survive full instance deletion/teardown, nor documents attach/detach workflow, pricing, or size limits, and one community comment complains about slow data transfer into Paperspace VMs (paperspace-comm-7). Missing for 10: explicit lifecycle documentation proving storage persists after instance termination, attach/detach mechanics, and independent hands-on confirmation of durability across teardown.

   • [claimed-docs] “Shared drives provide storage that is accessible from multiple machines in a private network.”
   • [claimed-docs] “Machines are Linux and Windows virtual machines with persistent storage, GPU options, and free unlimited bandwidth.”
   • [claimed-docs] “Notebooks are a web-based Jupyter IDE with shared persistent storage for long-term development and inter-notebook collaboration, backed by a…”
   • [community] “Their DL virtual servers (Core) are horrible - very slow internet, takes forever to copy datasets in. Getting SSH access is an uphill battle…”

   proof ↗⚑ flag

Images

1. developerRun my own Docker image or custom machine template with my exact environment

   weight 2 · round to Vast.ai
   Vast.ai✓fullclaimed8/10

   Vast.ai's template system explicitly lets developers package their own Docker environment ('Package your environment so any GPU can run it with one click') and launch prebuilt or custom templates with one click, backed by CLI/API/SDK automation for creating and managing instances from these templates. Documentation confirms templates configure the rented machine with whatever software/formatting is needed, directly matching the story. missing for 10: no independent/hands-on confirmation of building and running a fully custom Docker image end-to-end (community evidence focuses on pricing/reliability, not template customization).

   • [claimed-docs] “Build a template - Package your environment so any GPU can run it with one click.”
   • [claimed-docs] “A template is how Vast helps you launch an instance, setting up your rented machine with whatever software and formatting you need.”
   • [claimed-docs] “Launch prebuilt or custom templates with one click.”
   • [claimed-docs] “PyTorch, vLLM, ComfyUI, agents, and more. Real deployments you can copy.”
   • [claimed-docs] “Automate with the CLI or SDK ... `vastai` from the shell or Python. Same operations, different syntax.”

   proof ↗⚑ flag

   Paperspace✓fullcommunity7/10

   Paperspace lets users create custom machine templates (paperspace-docs-3, paperspace-docs-25) and run container images via Deployments (paperspace-docs-8), plus SSH root access to VMs for custom setups (paperspace-docs-12, paperspace-docs-19). Community reports (paperspace-comm-7) note friction getting reliable SSH/console access on Core VMs, which tempers confidence. Missing for 10: independent hands-on confirmation that custom Docker images run smoothly end-to-end, and clearer detail on template versioning/sharing.

   • [claimed-docs] “Custom templates are templates of your machines and their configurations. You can use these custom templates to create new machines with you…”
   • [claimed-docs] “Deployments are containers-as-a-service that let you run container images and serve machine learning models.”
   • [claimed-docs] “When you create a new machine, you choose your machine type, operating system or custom template, disk size, region, authentication, startin…”
   • [claimed-docs] “Bring your SSH key and connect directly to your VM with full root access.”
   • [claimed-docs] “Root access, connect with SSH Bring your SSH key and connect directly to your VM with full root access.”
   • [community] “Their DL virtual servers (Core) are horrible - very slow internet, takes forever to copy datasets in. Getting SSH access is an uphill battle…”

   proof ↗⚑ flag

Templates

1. developerLaunch from pre-built ML templates (PyTorch, CUDA, vLLM, ComfyUI) instead of assembling an environment from scratch

   weight 2 · round to Vast.ai
   Vast.ai✓fullclaimed8/10

   Vast.ai's docs explicitly describe templates as a first-class launch mechanism ('Launch prebuilt or custom templates with one click', 'A template is how Vast helps you launch an instance') and directly name PyTorch, vLLM, and ComfyUI as available prebuilt templates ('PyTorch, vLLM, ComfyUI, agents, and more. Real deployments you can copy'). CUDA support is implied through GPU offer filtering and template packaging but not explicitly named as its own template. missing for 10: independent/hands-on confirmation of launching a specific named template (e.g., a community post describing a ComfyUI or vLLM template launch), and explicit CUDA-template documentation rather than inferred CUDA support.

   • [claimed-docs] “Launch prebuilt or custom templates with one click.”
   • [claimed-docs] “PyTorch, vLLM, ComfyUI, agents, and more. Real deployments you can copy.”
   • [claimed-docs] “A template is how Vast helps you launch an instance, setting up your rented machine with whatever software and formatting you need.”
   • [claimed-docs] “Build a template - Package your environment so any GPU can run it with one click.”

   proof ↗⚑ flag

   Paperspace~partialcommunity5/10

   Paperspace docs confirm a pre-built "ML in a Box" template with major ML frameworks and CUDA drivers preinstalled, plus custom-template creation for reuse, supporting the general concept of launching from ready-made ML environments. However, no evidence names specific frameworks like PyTorch, vLLM, or ComfyUI templates, and community comments describe friction (slow setup, SDK upgrade walls, GUI confusion) that undercuts a seamless 'launch instantly' experience. missing for 10: explicit PyTorch/vLLM/ComfyUI-named templates, hands-on confirmation of frictionless template launch, independent corroboration beyond marketing docs.

   • [claimed-docs] “Choose "ML in a Box" template that comes preinstalled with all the major ML frameworks and CUDA® drivers.”
   • [claimed-docs] “Custom templates are templates of your machines and their configurations. You can use these custom templates to create new machines with you…”
   • [claimed-docs] “When you create a new machine, you choose your machine type, operating system or custom template, disk size, region, authentication, startin…”
   • [community] “Paperspace gives free access to Graphcore IPU nodes (4 IPUs each), theoretically more throughput than Colab T4. But porting Stable Diffusion…”
   • [community] “Their DL virtual servers (Core) are horrible - very slow internet, takes forever to copy datasets in. Getting SSH access is an uphill battle…”

   proof ↗⚑ flag

Compliance

1. platform engineerVerify the provider's security and compliance posture (SOC 2, data handling, datacenter tiers) before putting proprietary models on it

   weight 2 · round to Vast.ai
   Vast.ai!disputedcontradicted4/10

   Vast.ai docs claim some infrastructure verification — 'Secure Cloud' machines are checked for TIER 2/3 datacenter rating or ISO 27001 certification (vast-ai-docs-14, vast-ai-docs-28) — but there is no mention of SOC 2 attestation, data-handling/privacy policies, or compliance documentation anywhere in the pack. Community evidence directly undercuts trust in these verifications: a host was found masquerading as US-based while actually located in China (vast-ai-comm-10), and another thread questions whether running consumer GPUs at scale even complies with Nvidia's own terms (vast-ai-comm-9) — concrete contradictions of the 'verified' security posture platform engineers would need to rely on. Missing for 10: SOC 2 or equivalent third-party compliance attestation, explicit data-handling/retention policy, and independent audit corroborating the datacenter tier claims.

   • [claimed-docs] “Secure Cloud (Datacenter): These machines are verified and confirmed to be in a certified datacenter that meets our datacenter criteria.”
   • [claimed-docs] “Vast verifies that these machines are in a datacenter with a TIER 2/3 rating or ISO 27001. These offer cards have a blue label and are recom…”
   • [community] “Running a fleet of RTX3090's on demand does not sound at all compliant with what little I know of NVidia's terms of use. There's surprisingl…”
   • [community] “Discussion flagged a Chinese host on vast.ai masquerading as a US host, raising concerns about where rented GPU compute is actually located.”

   proof ↗⚑ flag

   Paperspace—none0/10

   No evidence pack items mention SOC 2 compliance, certifications, data handling policies, or datacenter tier information; the evidence is entirely product feature docs and community complaints about performance/support, none of which address compliance posture. missing for 10: SOC 2/ISO certifications, data handling/privacy documentation, datacenter tier specs, any compliance attestations.

   ⚑ flag

Governance

1. platform engineerManage team members with roles and scoped API keys so credentials and spend stay controlled

   weight 1 · round to Vast.ai
   Vast.ai~partialclaimed4/10

   Vast.ai documents scoped API keys with limited permissions for CI/CD and shared tooling use cases (create api-key, per-workload scoping), which addresses the credential-control half of the story. However, there is no evidence of team/organization member management, role assignment, or multi-user account governance — the evidence pack only covers individual scoped keys, not team-based RBAC. Missing for 10: team member invitation/management, role definitions (admin/viewer/etc.), org-level spend controls tied to roles, and any audit trail for team activity.

   • [claimed-docs] “You can also create scoped keys with limited permissions using `vastai create api-key`, useful for CI/CD or shared tooling.”
   • [claimed-docs] “create a scoped key with only the permissions that workload needs”
   • [claimed-docs] “For shared tooling, CI/CD, or automation, create a scoped key with only the permissions that workload needs.”

   proof ↗⚑ flag

   Paperspace—none0/10

   Evidence confirms API keys exist for programmatic access (paperspace-docs-6, paperspace-docs-23) but there is no mention of team member management, role assignment, scoped/permissioned API keys, or spend-control governance features anywhere in the pack.

   • [claimed-docs] “API keys let you interact with Paperspace through the Core RESTful API, the Core JavaScript SDK, and Gradient command-line utility (CLI).”
   • [claimed-docs] “Programmatically manage Paperspace resources using conventional HTTP requests.”

   proof ↗⚑ flag