Together AI vs Baseten

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round drawn
   Together AI✓fullprobed9/10

   Together AI publishes a working llms.txt (HTTP 200) with a documentation index, per-page .md docs, and even a live MCP server for docs lookup, all independently confirmed via probes plus first-party docs on agent-skills. missing for 10: no independent third-party report of an agent actually consuming llms.txt successfully in practice.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.together.ai/llms.txt # Together AI docs > Documentation for the Together AI platform for inference…”
   • [probe] “PROBE docs-md: HTTP 200 at https://docs.together.ai/intro.md > ## Documentation Index > Fetch the complete documentation index at: https://d…”
   • [probe] “PROBE mcp-endpoint (2026-09-04): POST initialize to https://docs.together.ai/mcp answered HTTP 200 with a JSON-RPC/MCP response (event: mess…”
   • [claimed-docs] “Make your AI coding agent Together-AI-aware with ready-made skills for code generation and an MCP server for live docs lookup.”
   • [claimed-docs] “Docs MCP server: Gives your agent live access to this documentation site so it can look up current information without leaving your editor.”

   proof ↗⚑ flag

   Baseten✓fullprobed9/10

   Baseten hosts a verified llms.txt (HTTP 200, indexing docs) and an agent-setup page directing agents to install skills/MCP servers to search the docs, confirmed live via a working MCP endpoint. missing for 10: independent third-party confirmation of an agent successfully using llms.txt end-to-end beyond Baseten's own probes.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.baseten.co/llms.txt # Baseten - [Baseten overview](https://docs.baseten.co/overview.md): Run hoste…”
   • [probe] “PROBE docs-md: HTTP 200 at https://docs.baseten.co/overview.md > ## Documentation Index > Fetch the complete documentation index at: https:/…”
   • [claimed-docs] “Install the Baseten skill and MCP servers so your coding agent can manage your Baseten workspace and search these docs.”
   • [probe] “official MCP server documented at https://docs.baseten.co/agent-setup”
   • [probe] “PROBE mcp-endpoint (2026-09-04): POST initialize to https://docs.baseten.co/mcp answered HTTP 200 with a JSON-RPC/MCP response (event: messa…”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round drawn
   Together AI✓fullprobed7/10

   Together AI is an API/SDK-first inference platform with OpenAI-compatible REST endpoints, official server-side SDKs (TypeScript/Python), batch inference, streaming via SSE, and structured/function-calling outputs — all natively usable in headless CI/automation pipelines without any UI dependency. Live probes confirm the API and status endpoints are reachable programmatically. missing for 10: no explicit CI/CD example or GitHub Actions template in evidence, no documented CLI exit-code/scripting conventions, and no independent case study of a CI pipeline using Together AI headlessly.

   • [claimed-docs] “you can point it at models hosted on Together with two changes: the API key and base URL”
   • [claimed-docs] “Call 100+ open-source models with per-token pricing and no provisioning latency.”
   • [claimed-docs] “Run asynchronous batch workloads at up to 50% lower cost.”
   • [github] “We provide support for streaming responses using Server Sent Events (SSE).”
   • [github] “This library provides convenient access to the Together REST API from server-side TypeScript or JavaScript.”
   • [probe] “PROBE models-endpoint (2026-09-04): GET https://api.together.xyz/v1/models without a key returned HTTP 401 (Missing API key) — the OpenAI-st…”
   • [claimed-docs] “Call chat, image, audio, embedding, and more through one API.”

   proof ↗⚑ flag

   Baseten✓fullclaimed7/10

   Baseten provides a CLI (truss push, Baseten CLI/Switch), config.yaml-driven deployment, OpenAI-compatible REST APIs with async/webhook/polling support, and API-key-based auth/rate limiting, all of which support scripted, non-interactive CI/automation workflows. missing for 10: explicit CI/CD pipeline examples (e.g. GitHub Actions integration), independent hands-on confirmation of headless CLI use in automation, and documented non-interactive auth flow for CI secrets.

   • [github] “You write a `config.yaml` that specifies the model, the hardware, and the engine, then `uvx truss push` builds a TensorRT-optimized containe…”
   • [github] “Deploying a model to Baseten via Truss turns a Hugging Face model into a production-ready API endpoint. You write a `config.yaml` that speci…”
   • [claimed-docs] “Async inference returns a request ID quickly and completes later through webhook or polling, which suits batch work, long documents”
   • [claimed-docs] “Async inference returns a request ID quickly and completes later through webhook or polling, which suits batch work, long documents, or any …”
   • [claimed-docs] “Call hosted models through an OpenAI-compatible API, deploy your own models on dedicated infrastructure”
   • [claimed-docs] “x-ratelimit-remaining-requests: Reports how many requests remain before you reach the request rate limit.”
   • [claimed-docs] “You can also set a workspace budget and query usage by API key or model.”
   • [claimed-docs] “a `config.yaml` can define the model, hardware, and inference engine without custom serving code”

   proof ↗⚑ flag

3. ai-native userPlug MCP servers into this product so it can use their tools

   weight 3 · round drawn
   Together AI—none0/10

   The evidence only shows Together AI publishing its own MCP server (docs lookup) for coding agents to consume — i.e., Together acting as an MCP *server*, not as a client that lets users plug external MCP servers into it. There's no documentation or probe showing Together's platform/API can connect to and use tools from user-supplied MCP servers, despite having function/tool-calling APIs. missing for 10: any evidence of MCP-client support (configuring/connecting external MCP servers within Together's product), integration of MCP tool results into its agentic function-calling flow.

   • [claimed-docs] “Make your AI coding agent Together-AI-aware with ready-made skills for code generation and an MCP server for live docs lookup.”
   • [claimed-docs] “12 domain-specific skills that load on demand and teach your agent how to write correct Together AI code (right model IDs, SDK patterns, bes…”
   • [claimed-docs] “Gives your agent live access to this documentation site so it can look up current information without leaving your editor.”
   • [claimed-docs] “Docs MCP server: Gives your agent live access to this documentation site so it can look up current information without leaving your editor.”
   • [probe] “official MCP server documented at https://docs.together.ai/docs/agent-skills”
   • [probe] “PROBE mcp-endpoint (2026-09-04): POST initialize to https://docs.together.ai/mcp answered HTTP 200 with a JSON-RPC/MCP response (event: mess…”
   • [claimed-docs] “To build agent loops, chain tool calls inside one response (multi-step), and conversations that thread tools across many turns (multi-turn).”
   • [claimed-docs] “Multi-step function calling chains sequential function calls within one conversation turn.”

   proof ↗⚑ flag

   Baseten—none0/10

   Baseten's evidence only shows it exposing its own MCP server so external coding agents (Claude Code, Codex, Pi) can call Baseten's workspace tools — the reverse relationship. There is no evidence that Baseten itself can consume/plug in third-party MCP servers to gain access to their tools during inference or agent workflows.

   • [claimed-docs] “Install the Baseten skill and MCP servers so your coding agent can manage your Baseten workspace and search these docs.”
   • [claimed-docs] “Connect Claude Code, Codex CLI, or Pi with Baseten Switch.”
   • [claimed-docs] “Use [Baseten Switch](/reference/cli/baseten-switch) to route requests from Claude Code or Codex CLI to Model APIs.”
   • [probe] “PROBE mcp-endpoint (2026-09-04): POST initialize to https://docs.baseten.co/mcp answered HTTP 200 with a JSON-RPC/MCP response (event: messa…”
   • [probe] “official MCP server documented at https://docs.baseten.co/agent-setup”

   proof ↗⚑ flag

4. ai-native userConnect an agent via an official MCP server

   weight 3 · round drawn
   Together AI✓fullprobed8/10

   Together AI documents an official Docs MCP server that gives agents live access to documentation, and a live probe confirms the MCP endpoint at docs.together.ai/mcp responds correctly to an initialize call with proper protocol capabilities. This is corroborated by both first-party docs and independent hands-on verification. Missing for 10: broader coverage beyond docs lookup (e.g., an MCP server exposing inference/model-management actions rather than just documentation retrieval), and no community discussion confirming real-world agent integration experiences.

   • [claimed-docs] “Make your AI coding agent Together-AI-aware with ready-made skills for code generation and an MCP server for live docs lookup.”
   • [claimed-docs] “12 domain-specific skills that load on demand and teach your agent how to write correct Together AI code (right model IDs, SDK patterns, bes…”
   • [claimed-docs] “Gives your agent live access to this documentation site so it can look up current information without leaving your editor.”
   • [claimed-docs] “Docs MCP server: Gives your agent live access to this documentation site so it can look up current information without leaving your editor.”
   • [probe] “official MCP server documented at https://docs.together.ai/docs/agent-skills”
   • [probe] “PROBE mcp-endpoint (2026-09-04): POST initialize to https://docs.together.ai/mcp answered HTTP 200 with a JSON-RPC/MCP response (event: mess…”

   proof ↗⚑ flag

   Baseten✓fullprobed8/10

   Baseten documents and hosts an official MCP server (docs.baseten.co/agent-setup, docs.baseten.co/mcp) that lets coding agents manage the Baseten workspace and search docs, and a live probe confirms the endpoint responds to MCP initialize with valid JSON-RPC capabilities. missing for 10: independent/community hands-on validation of the MCP server beyond the vendor's own probe, and more detail on the full tool surface exposed.

   • [claimed-docs] “Install the Baseten skill and MCP servers so your coding agent can manage your Baseten workspace and search these docs.”
   • [probe] “official MCP server documented at https://docs.baseten.co/agent-setup”
   • [probe] “PROBE mcp-endpoint (2026-09-04): POST initialize to https://docs.baseten.co/mcp answered HTTP 200 with a JSON-RPC/MCP response (event: messa…”

   proof ↗⚑ flag

5. ai-native userUse an official CLI

   weight 2 · round to Baseten
   Together AI~partialclaimed5/10

   Docs mention an official CLI (the 'tg' command, e.g. 'tg beta endpoints deploy...') and explicitly state fine-tuning jobs can be launched 'through the API/SDK, or with the CLI', confirming a real CLI exists for agentic/dev workflows. However, evidence lacks a dedicated CLI reference page, install instructions, full command coverage, or independent hands-on corroboration. Missing for 10: dedicated CLI documentation/installation guide, comprehensive command reference, independent user verification of CLI usage.

   • [claimed-docs] “tg beta endpoints deploy google/gemma-4-E4B-it --endpoint my-endpoint”
   • [claimed-docs] “You can launch a fine-tuning job from the console, through the API/SDK, or with the CLI”

   proof ↗⚑ flag

   Baseten✓fullclaimed7/10

   Baseten ships official CLIs: Truss (`uvx truss push` to build/deploy models from a config.yaml, per baseten-gh-2/6) and Baseten Switch (a CLI referenced at /reference/cli/baseten-switch for routing coding-agent traffic, baseten-docs-34), plus one-command checkpoint deployment (baseten-docs-29). missing for 10: independent/hands-on verification of CLI behavior, and a full CLI reference/command list beyond scattered doc mentions.

   • [github] “You write a `config.yaml` that specifies the model, the hardware, and the engine, then `uvx truss push` builds a TensorRT-optimized containe…”
   • [github] “Deploying a model to Baseten via Truss turns a Hugging Face model into a production-ready API endpoint. You write a `config.yaml` that speci…”
   • [claimed-docs] “Deploy any synced checkpoint with one CLI command”
   • [claimed-docs] “Use [Baseten Switch](/reference/cli/baseten-switch) to route requests from Claude Code or Codex CLI to Model APIs.”
   • [claimed-docs] “Connect Claude Code, Codex CLI, or Pi with Baseten Switch.”

   proof ↗⚑ flag

6. ai-native userDrive the product through a documented public API

   weight 3 · round to Together AI
   Together AI✓fullprobed9/10

   Together AI exposes an OpenAI-compatible REST API with official SDKs (TypeScript/JS), documented endpoints for chat, batch, fine-tuning, dedicated deployment, and a live, publicly reachable models endpoint confirmed by probe; docs are extensive and public. Missing for 10: a discoverable formal OpenAPI/swagger spec (probe found all candidate paths 404).

   • [claimed-docs] “you can point it at models hosted on Together with two changes: the API key and base URL”
   • [claimed-docs] “Call 100+ open-source models with per-token pricing and no provisioning latency.”
   • [github] “This library provides convenient access to the Together REST API from server-side TypeScript or JavaScript.”
   • [probe] “PROBE models-endpoint (2026-09-04): GET https://api.together.xyz/v1/models without a key returned HTTP 401 (Missing API key) — the OpenAI-st…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.together.ai/openapi.json, https://docs.together.ai/swagger.json, https://docs.together.…”

   proof ↗⚑ flag

   Baseten✓fullprobed8/10

   Baseten exposes an OpenAI/Anthropic-compatible inference API (docs-21, docs-30), a documented CLI/Truss deployment workflow, training APIs (docs-32, docs-36), and usage/rate-limit endpoints, all backed by extensive public docs and a live, probe-verified /v1/models endpoint and MCP server. Missing for 10: a discoverable OpenAPI/swagger spec (probe-3 shows 404s across candidate paths) and independent third-party API usage reports beyond community anecdotes.

   • [claimed-docs] “They support the OpenAI Chat Completions API and the Anthropic Messages API in beta, so you can use familiar client SDKs.”
   • [claimed-docs] “Call hosted models through an OpenAI-compatible API, deploy your own models on dedicated infrastructure”
   • [claimed-docs] “Async inference returns a request ID quickly and completes later through webhook or polling, which suits batch work, long documents”
   • [claimed-docs] “Fine-tuning or running RL on a supported base model: Loops provisions a dedicated trainer and paired sampler, and each training step is an A…”
   • [claimed-docs] “Model APIs bill by token and enforce request and token rate limits. You can also set a workspace budget and query usage by API key or model.”
   • [probe] “PROBE models-endpoint (2026-09-04): GET https://inference.baseten.co/v1/models without a key returned HTTP 401 (No Authorization header prov…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.baseten.co/openapi.json, https://docs.baseten.co/swagger.json, https://docs.baseten.co/…”
   • [github] “You write a `config.yaml` that specifies the model, the hardware, and the engine, then `uvx truss push` builds a TensorRT-optimized containe…”

   proof ↗⚑ flag

7. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round drawn
   Together AI—none0/10

   No evidence of scoped or least-privilege API key issuance (e.g., per-project keys, role/permission scoping, or restricted-scope tokens for agents); docs only mention a single API key used for authentication, with no mention of scoping controls.

   • [claimed-docs] “you can point it at models hosted on Together with two changes: the API key and base URL”

   proof ↗⚑ flag

   Baseten—none0/10

   Baseten documents API keys, usage monitoring by API key, and workspace budgets, but no evidence describes scoped/least-privilege credential issuance (e.g., role-based permissions, restricted-scope keys, or per-agent credential minting). Monitoring usage by key is not the same as issuing scoped credentials.

   • [claimed-docs] “Model APIs bill by token and enforce request and token rate limits. You can also set a workspace budget and query usage by API key or model.”
   • [claimed-docs] “To monitor token and request consumption by API key or model, see Usage.”
   • [claimed-docs] “You can also set a workspace budget and query usage by API key or model.”
   • [claimed-docs] “To raise a Basic account's limits, request email verification. You can also use that form to move to Pro or Enterprise.”

   proof ↗⚑ flag

8. ai-native userBuild against official SDKs

   weight 2 · round to Together AI
   Together AI✓fullclaimed8/10

   Together AI provides an official TypeScript/JavaScript SDK (github repo) and OpenAI-compatible API/SDK usage, with docs covering chat, structured outputs, function calling, streaming, fine-tuning, and dedicated deployment all launchable via API/SDK. This directly supports AI-native, agentic development against official SDKs, including agent loop patterns and tool calling. Missing for 10: independent hands-on developer reviews specifically praising SDK quality/completeness, and no official Python SDK repo cited in evidence pack.

   • [github] “We provide support for streaming responses using Server Sent Events (SSE).”
   • [github] “This library provides convenient access to the Together REST API from server-side TypeScript or JavaScript.”
   • [claimed-docs] “you can point it at models hosted on Together with two changes: the API key and base URL”
   • [claimed-docs] “Function calling (also called tool calling) lets LLMs respond with structured function names and arguments that you can execute in your appl…”
   • [claimed-docs] “To build agent loops, chain tool calls inside one response (multi-step), and conversations that thread tools across many turns (multi-turn).”
   • [claimed-docs] “You can launch a fine-tuning job from the console, through the API/SDK, or with the CLI”
   • [claimed-docs] “Supported models can return JSON that conforms to any schema you supply, so you can read the output directly in code without retries or frag…”

   proof ↗⚑ flag

   Baseten✓fullclaimed7/10

   Baseten ships the official open-source Truss SDK/CLI for packaging and deploying models (github.com/basetenlabs/truss) and also exposes OpenAI-compatible and Anthropic Messages-compatible endpoints so developers can use those official client SDKs (and LangChain) directly against Baseten (baseten-docs-21, baseten-docs-33, baseten-gh-1/2/5/6). This gives AI-native users multiple first-party and compatible SDK paths for building agentic workflows. Missing for 10: a dedicated first-party 'Baseten Python client' distinct from Truss, and independent/hands-on corroboration of SDK reliability beyond docs and repo README.

   • [github] “Write once, run anywhere: Package model code, weights, and dependencies with a model server that behaves the same in development and product…”
   • [github] “You write a `config.yaml` that specifies the model, the hardware, and the engine, then `uvx truss push` builds a TensorRT-optimized containe…”
   • [github] “Truss lets you serve models with the Baseten Inference Stack as well as deploy models from any open-source framework: vLLM, SGLang, TensorRT…”
   • [github] “Deploying a model to Baseten via Truss turns a Hugging Face model into a production-ready API endpoint. You write a `config.yaml` that speci…”
   • [claimed-docs] “They support the OpenAI Chat Completions API and the Anthropic Messages API in beta, so you can use familiar client SDKs.”
   • [claimed-docs] “Because Baseten exposes an OpenAI-compatible endpoint, you can use LangChain's `ChatOpenAI` with `with_structured_output` by pointing `base_…”
   • [claimed-docs] “Call hosted models through an OpenAI-compatible API, deploy your own models on dedicated infrastructure”

   proof ↗⚑ flag

9. ai-native userSubscribe to events via webhooks

   weight 2 · round to Baseten
   Together AI—none0/10

   No evidence of webhook subscription support anywhere in the docs, SDKs, or probes; Together AI offers SSE streaming and a docs MCP server, but nothing about webhook event subscriptions for async notifications (e.g., fine-tune job completion, batch job status).

   ⚑ flag

   Baseten~partialclaimed4/10

   Docs mention that async inference results can be delivered via webhook callback, showing Baseten supports webhook-based notification for job completion, but there is no dedicated webhook subscription system (event types, signing, retry, subscription management) described anywhere in the evidence. Missing for 10: a general event-subscription/webhook API or docs page (event catalog, signature verification, delivery guarantees), coverage beyond async-inference completion notifications, and any independent confirmation of webhook reliability.

   • [claimed-docs] “Async inference returns a request ID quickly and completes later through webhook or polling, which suits batch work, long documents”
   • [claimed-docs] “Async inference returns a request ID quickly and completes later through webhook or polling, which suits batch work, long documents, or any …”
   • [claimed-docs] “Async inference returns a request ID quickly and completes later through webhook or polling”

   proof ↗⚑ flag

Agentic features

1. ai-native userSet up automations that run autonomously in the background

   weight 2 · round drawn
   Together AI—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

   Baseten—none0/10

   Baseten's evidence covers async inference (deferred single requests via webhook/polling), model deployment, and training, but there is no evidence of scheduling, triggers, or autonomous multi-step automations running in the background — async inference is single-request deferral, not an automation/workflow engine.

   ⚑ flag

2. ai-native userDelegate tasks to a built-in AI assistant inside the product

   weight 3 · round drawn
   Together AI—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

   Baseten—none0/10

   Baseten's evidence shows MCP servers and a Baseten 'skill' that let external coding agents (Claude Code, Codex, etc.) manage a Baseten workspace — this is the reverse of a built-in in-product assistant; nothing in the evidence describes a first-party AI assistant living inside the Baseten UI/dashboard that a user can delegate platform tasks to.

   • [claimed-docs] “Install the Baseten skill and MCP servers so your coding agent can manage your Baseten workspace and search these docs.”
   • [claimed-docs] “Connect Claude Code, Codex CLI, or Pi with Baseten Switch.”
   • [claimed-docs] “Use [Baseten Switch](/reference/cli/baseten-switch) to route requests from Claude Code or Codex CLI to Model APIs.”
   • [probe] “official MCP server documented at https://docs.baseten.co/agent-setup”
   • [probe] “PROBE mcp-endpoint (2026-09-04): POST initialize to https://docs.baseten.co/mcp answered HTTP 200 with a JSON-RPC/MCP response (event: messa…”

   proof ↗⚑ flag

3. ai-native userOperate the product with natural-language commands

   weight 2 · round to Baseten
   Together AI—none0/10

   Together AI is an inference/training API platform operated via REST API, SDKs, CLI, and console — the evidence shows structured commands (API calls, CLI syntax like 'tg beta endpoints deploy...') rather than any natural-language command interface for operating the platform itself. The agent-skills/MCP docs only help external coding agents write correct Together AI code, not let a user operate Together AI's own product surface via natural language.

   ⚑ flag

   Baseten~partialprobed6/10

   Baseten ships an official skill and MCP server so a coding agent can 'manage your Baseten workspace and search these docs' via natural language, and a probe confirms a live, reachable MCP endpoint responding to JSON-RPC calls — this is genuine natural-language operability of the product through an agent intermediary. However the docs give only a high-level description of what workspace actions are exposed, with no detailed command catalog or independent hands-on verification of breadth/reliability. Missing for 10: a documented list of specific natural-language-operable actions (deploy, scale, monitor, etc.), and independent/hands-on evidence confirming the MCP tools work reliably end-to-end.

   • [claimed-docs] “Install the Baseten skill and MCP servers so your coding agent can manage your Baseten workspace and search these docs.”
   • [probe] “official MCP server documented at https://docs.baseten.co/agent-setup”
   • [probe] “PROBE mcp-endpoint (2026-09-04): POST initialize to https://docs.baseten.co/mcp answered HTTP 200 with a JSON-RPC/MCP response (event: messa…”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   Together AI—none0/10

   The evidence pack shows extensive prose documentation (guides, overview pages) but no reference to an interactive API reference or runnable/try-it-now code examples; a direct probe for OpenAPI/Swagger specs at standard paths returned 404 for all candidates, indicating no interactive spec-driven reference was found. No community or hands-on evidence confirms a runnable API console exists.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.together.ai/openapi.json, https://docs.together.ai/swagger.json, https://docs.together.…”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.together.ai/llms.txt # Together AI docs > Documentation for the Together AI platform for inference…”
   • [probe] “PROBE docs-md: HTTP 200 at https://docs.together.ai/intro.md > ## Documentation Index > Fetch the complete documentation index at: https://d…”

   proof ↗⚑ flag

   Baseten—none0/10

   Evidence shows extensive prose documentation (structured outputs, function calling, streaming, pricing) but no interactive API reference or runnable-example playground; a direct probe for an OpenAPI/Swagger spec (which typically powers interactive references) returned 404 on all candidate paths.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.baseten.co/openapi.json, https://docs.baseten.co/swagger.json, https://docs.baseten.co/…”
   • [claimed-docs] “Structured outputs let you generate text that conforms to specific JSON schemas, providing reliable data extraction and controlled text gene…”
   • [claimed-docs] “Function calling* (also called *tool calling*) lets a model choose a tool and produce its arguments from a user request.”
   • [claimed-docs] “Return model output token by token as it is generated.”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   Together AI—none0/10

   Active probing for standard OpenAPI/swagger spec locations (openapi.json, swagger.json, etc.) all returned 404, and no docs page in the pack references a downloadable OpenAPI/machine-readable spec for the Together API.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.together.ai/openapi.json, https://docs.together.ai/swagger.json, https://docs.together.…”

   proof ↗⚑ flag

   Baseten—none0/10

   A direct probe for a machine-readable API spec (openapi.json, swagger.json, and related paths) returned 404 on all candidates, and no docs page claims to publish an OpenAPI/Swagger spec — only that the API is OpenAI/Anthropic-compatible in shape, which is not the same as Baseten publishing its own downloadable spec.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.baseten.co/openapi.json, https://docs.baseten.co/swagger.json, https://docs.baseten.co/…”
   • [claimed-docs] “They support the OpenAI Chat Completions API and the Anthropic Messages API in beta, so you can use familiar client SDKs.”
   • [claimed-docs] “Call hosted models through an OpenAI-compatible API, deploy your own models on dedicated infrastructure”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round to Baseten
   Together AI—none0/10

   No evidence of a dedicated sandbox/staging environment; the only related mention is a test account with trial credit on the same production API, not an isolated sandbox for testing without touching production data.

   • [community] “the price is pretty low... 4B MODEL, PRICE 1K TOKENS: $0.0001. register with an email, test account has $25 credit, python API as well, good…”

   proof ↗⚑ flag

   Baseten~partialclaimed4/10

   Docs mention that deployments 'support stable environments for development, staging, and production' (baseten-docs-20) and that Truss models 'behave the same in development and production' (baseten-gh-1), implying separate non-prod environments exist. However there is no detailed documentation of how sandbox/staging is isolated from production data, no guidance on seeding test data, and no independent confirmation of this workflow. Missing for 10: explicit sandbox/staging setup guide, data-isolation guarantees, and hands-on/community corroboration of safe non-prod testing.

   • [claimed-docs] “They also support stable environments for development, staging, and production.”
   • [github] “Write once, run anywhere: Package model code, weights, and dependencies with a model server that behaves the same in development and product…”

   proof ↗⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round to Baseten
   Together AI—none0/10

   The evidence shows a documented, OpenAI-compatible REST API (with a /v1/ endpoint) but contains no mention of a versioning scheme or a documented deprecation policy for API changes. Absence of evidence for this applicable capability means it cannot be credited as delivered.

   ⚑ flag

   Baseten~partialprobed3/10

   Baseten documents a deprecation policy for Model APIs, instructing users to migrate to dedicated deployments when weights are deprecated (baseten-docs-12), showing some commitment to managing API lifecycle. However, there is no evidence of explicit API versioning (e.g., version numbers, changelogs) and probes for an OpenAPI/swagger spec returned 404s, suggesting no formal versioned API contract is published. Missing for 10: explicit API version scheme, published changelog, machine-readable API spec, and independent confirmation of deprecation timelines.

   • [claimed-docs] “Migrate to a dedicated deployment with the deprecated model weights. Contact us for assistance.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.baseten.co/openapi.json, https://docs.baseten.co/swagger.json, https://docs.baseten.co/…”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round to Together AI
   Together AI✓fullclaimed7/10

   Together AI offers an asynchronous Batch API explicitly designed for bulk workloads across many items at once, at up to 50% lower cost than synchronous calls, alongside dedicated endpoints and fine-tuning pipelines that can process large datasets end-to-end. missing for 10: no documented batch size/throughput limits, no independent/hands-on report validating batch job behavior at scale, and no detail on input format flexibility for large item counts.

   • [claimed-docs] “Run asynchronous batch workloads at up to 50% lower cost.”
   • [claimed-docs] “Together AI handles the full lifecycle: data upload, training, hosting, and inference on a dedicated endpoint.”
   • [claimed-docs] “Fine-tuning tailors a pretrained model to a smaller, targeted dataset so it performs better on a specific task or domain. Together AI handle…”

   proof ↗⚑ flag

   Baseten~partialclaimed5/10

   Baseten's async inference is explicitly described as suited to 'batch work, long documents' where a request ID is returned and completion happens later via webhook or polling, which supports processing many items without holding open connections. However, there is no documented batch/bulk API endpoint for submitting many items in a single call, nor bulk management operations (e.g., bulk model/deployment updates) across the platform. Missing for 10: explicit multi-item batch submission API, bulk resource management (models/deployments/keys) in one operation, and independent evidence of throughput at scale for bulk workloads.

   • [claimed-docs] “Async inference returns a request ID quickly and completes later through webhook or polling, which suits batch work, long documents”
   • [claimed-docs] “Async inference returns a request ID quickly and completes later through webhook or polling, which suits batch work, long documents, or any …”
   • [claimed-docs] “Async inference returns a request ID quickly and completes later through webhook or polling”

   proof ↗⚑ flag

2. ai-native userSchedule recurring jobs or workflows

   weight 2 · round drawn
   Together AI—none0/10

   Together AI documents async batch workloads, fine-tuning jobs, and dedicated endpoints, but nowhere in the evidence is there a scheduler, cron-like trigger, or recurring-job/workflow orchestration mechanism; jobs appear to be triggered manually via API/CLI/console rather than on a recurring schedule.

   • [claimed-docs] “Run asynchronous batch workloads at up to 50% lower cost.”
   • [claimed-docs] “You can launch a fine-tuning job from the console, through the API/SDK, or with the CLI”
   • [claimed-docs] “Fine-tuning tailors a pretrained model to a smaller, targeted dataset so it performs better on a specific task or domain. Together AI handle…”

   proof ↗⚑ flag

   Baseten—none0/10

   Baseten's docs describe async inference (webhook/polling), training jobs, and deployments, but nothing in the evidence pack mentions a scheduler, cron-like trigger, or recurring/automated job orchestration feature. Since Baseten already runs jobs (training, inference) a scheduling capability is a fair ask, but no evidence shows it exists.

   ⚑ flag

Batch

1. ml-engineerSubmit asynchronous batch inference jobs at a documented discount versus real-time pricing

   weight 2 · round to Together AI
   Together AI✓fullclaimed8/10

   Together AI has documented batch inference at up to 50% lower cost than real-time serverless pricing, directly matching the story's requirement for asynchronous jobs at a documented discount [together-ai-docs-3]. missing for 10: independent/hands-on corroboration of actual batch job submission and discount realization beyond the docs page, and more detail on submission workflow/API specifics.

   • [claimed-docs] “Run asynchronous batch workloads at up to 50% lower cost.”

   proof ↗⚑ flag

   Baseten~partialclaimed4/10

   Baseten documents asynchronous inference (request ID returned quickly, completion via webhook/polling) explicitly for batch work and long documents, so the submission mechanism is real. However, the only documented pricing discount is for KV-cached input tokens, not for async/batch requests specifically — there is no evidence of a documented discount rate for async vs real-time inference pricing. missing for 10: a documented async/batch discount rate or pricing tier distinct from real-time pricing, first-party pricing page showing the discount percentage.

   • [claimed-docs] “Async inference returns a request ID quickly and completes later through webhook or polling, which suits batch work, long documents”
   • [claimed-docs] “Async inference returns a request ID quickly and completes later through webhook or polling, which suits batch work, long documents, or any …”
   • [claimed-docs] “Async inference returns a request ID quickly and completes later through webhook or polling”
   • [claimed-docs] “Cached input tokens are prompt tokens served from the KV cache at a discounted rate. Caching is automatic and requires no request flags.”
   • [claimed-docs] “Model APIs bill by token and enforce request and token rate limits. You can also set a workspace budget and query usage by API key or model.”

   proof ↗⚑ flag

Dedicated

1. ml-engineerDeploy a model on dedicated GPU capacity with autoscaling so my traffic is isolated from the shared serverless pool

   weight 2 · round to Together AI
   Together AI✓fullclaimed8/10

   Docs explicitly describe Dedicated Model Inference (DMI) for serving models on reserved/isolated hardware distinct from the shared serverless pool, with no hard rate limits, and a dedicated scaling doc showing configurable autoscaling via replica limits, plus deployment of fine-tuned or custom models via CLI. This directly matches the story's requirements of dedicated GPU capacity, autoscaling, and isolation from shared serverless traffic. Missing for 10: independent/hands-on verification of autoscaling behavior in production and concrete GPU-type/SLA details beyond docs.

   • [claimed-docs] “Dedicated model inference (DMI) lets you serve a model on reserved hardware, providing several advantages over serverless models”
   • [claimed-docs] “Configure your deployment to scale automatically by setting limits on how many replicas it can run.”
   • [claimed-docs] “Fine-tuned models: Deploy a model you fine-tuned from a supported base model.”
   • [claimed-docs] “tg beta endpoints deploy google/gemma-4-E4B-it --endpoint my-endpoint”
   • [claimed-docs] “Dedicated model inference (DMI) lets you serve a model on reserved hardware... Better performance... No hard rate limits”

   proof ↗⚑ flag

   Baseten~partialclaimed5/10

   Baseten clearly supports deploying models on dedicated GPUs/dedicated infrastructure (baseten-docs-2, baseten-docs-17, baseten-docs-30) with production-stable environments (baseten-docs-20) and resilient multi-cluster deployment (baseten-docs-15), which is distinct from the shared Model APIs pool (baseten-docs-1). However, no evidence explicitly describes autoscaling behavior for these dedicated deployments (scale-to-zero, concurrency-based scaling, etc.), so the isolation-plus-autoscaling combination is only partially substantiated. Missing for 10: explicit autoscaling documentation for dedicated deployments, and independent/hands-on confirmation of traffic isolation from the shared pool.

   • [claimed-docs] “Deploy an open-source, fine-tuned, or custom model on dedicated GPUs.”
   • [claimed-docs] “Deploy an open-source, fine-tuned, or custom model on dedicated infrastructure.”
   • [claimed-docs] “Call hosted models through an OpenAI-compatible API, deploy your own models on dedicated infrastructure”
   • [claimed-docs] “They also support stable environments for development, staging, and production.”
   • [claimed-docs] “Deployments run active-active across clusters and clouds. If a region or provider loses capacity, MCM reroutes and reprovisions workloads.”
   • [claimed-docs] “Use Model APIs to call supported language models without deploying them.”

   proof ↗⚑ flag

Fine tune

1. ml-engineerFine-tune a supported base model on my own data and serve the result on the same platform

   weight 2 · round to Together AI
   Together AI✓fullclaimed9/10

   Docs clearly describe fine-tuning (LoRA or full) on supported base models via console/API/CLI, with Together AI handling the full lifecycle including data upload, training, hosting, and inference on a dedicated endpoint, and dedicated endpoints explicitly support deploying fine-tuned models. This is a first-party documented end-to-end workflow with concrete CLI examples. Missing for 10: independent/hands-on confirmation from a third party that a fine-tune-then-serve workflow works as described.

   • [claimed-docs] “Fine-tune models on your own data and deploy them for inference.”
   • [claimed-docs] “LoRA: Trains a small set of adapter weights on top of the frozen base model... Full fine-tuning: Updates every weight in the base model.”
   • [claimed-docs] “You can launch a fine-tuning job from the console, through the API/SDK, or with the CLI”
   • [claimed-docs] “Fine-tuned models: Deploy a model you fine-tuned from a supported base model.”
   • [claimed-docs] “Together AI handles the full lifecycle: data upload, training, hosting, and inference on a dedicated endpoint.”
   • [claimed-docs] “Fine-tuning tailors a pretrained model to a smaller, targeted dataset so it performs better on a specific task or domain. Together AI handle…”

   proof ↗⚑ flag

   Baseten✓fullclaimed8/10

   Baseten documents Training Jobs/Loops for fine-tuning supported base models (Axolotl/TRL/custom loops), automatic checkpoint syncing, and one-CLI-command deployment of any synced checkpoint to production inference on the same platform, matching the story end-to-end. missing for 10: independent/hands-on third-party confirmation of the fine-tune-to-serve workflow (community evidence only covers unrelated Chains/embedding topics) and explicit list of which base models are 'supported' for fine-tuning.

   • [claimed-docs] “Fine-tune with Loops or run your own training code with Training Jobs.”
   • [claimed-docs] “Baseten trains models on managed GPUs and deploys the resulting checkpoints to production inference on the same platform.”
   • [claimed-docs] “Baseten provisions GPUs through MCM, runs your training container, and syncs checkpoints to storage as the job progresses.”
   • [claimed-docs] “Your Axolotl config, TRL script, or custom loop runs unchanged in a container. Baseten provisions the GPUs, syncs checkpoints as your job sa…”
   • [claimed-docs] “Deploy any synced checkpoint with one CLI command”
   • [claimed-docs] “Fine-tuning or running RL on a supported base model: Loops provisions a dedicated trainer and paired sampler, and each training step is an A…”
   • [claimed-docs] “Loops provisions a dedicated trainer and paired sampler, and each training step is an API call from a Python loop you write.”

   proof ↗⚑ flag

2. ml-engineerUpload and serve my own custom model weights or LoRA adapters

   weight 2 · round to Baseten
   Together AI~partialclaimed7/10

   Together AI's docs clearly support fine-tuning (LoRA or full) and deploying the resulting fine-tuned models to dedicated inference endpoints, covering the LoRA-adapter serving use case end-to-end (data upload, training, hosting, inference). However, there's no evidence of uploading externally-trained/custom weights not produced via Together's own fine-tuning pipeline (e.g., bringing your own checkpoint from another framework). missing for 10: evidence of uploading externally-trained custom weights/checkpoints (not produced by Together's fine-tuning), and independent/hands-on confirmation of this workflow.

   • [claimed-docs] “LoRA: Trains a small set of adapter weights on top of the frozen base model... Full fine-tuning: Updates every weight in the base model.”
   • [claimed-docs] “Fine-tuned models: Deploy a model you fine-tuned from a supported base model.”
   • [claimed-docs] “Together AI handles the full lifecycle: data upload, training, hosting, and inference on a dedicated endpoint.”
   • [claimed-docs] “Fine-tuning tailors a pretrained model to a smaller, targeted dataset so it performs better on a specific task or domain. Together AI handle…”
   • [claimed-docs] “You can launch a fine-tuning job from the console, through the API/SDK, or with the CLI”

   proof ↗⚑ flag

   Baseten✓fullclaimed8/10

   Baseten documents deploying custom/fine-tuned/open-source models via Truss (config.yaml + weights, supports transformers/diffusers/vLLM/etc.), and its Training product syncs LoRA/checkpoint artifacts that can be deployed as production endpoints with one CLI command, directly covering custom weights and LoRA adapter serving. Missing for 10: explicit LoRA-adapter-specific hot-swapping/multi-adapter serving docs and independent hands-on confirmation beyond vendor docs/GitHub README.

   • [claimed-docs] “Deploy an open-source, fine-tuned, or custom model on dedicated GPUs.”
   • [claimed-docs] “Deploy an open-source, fine-tuned, or custom model on dedicated infrastructure.”
   • [github] “Write once, run anywhere: Package model code, weights, and dependencies with a model server that behaves the same in development and product…”
   • [github] “You write a `config.yaml` that specifies the model, the hardware, and the engine, then `uvx truss push` builds a TensorRT-optimized containe…”
   • [github] “Support for all Python frameworks: From `transformers` and `diffusers` to PyTorch and TensorFlow to vLLM, SGLang, and TensorRT-LLM, Truss su…”
   • [claimed-docs] “Your Axolotl config, TRL script, or custom loop runs unchanged in a container. Baseten provisions the GPUs, syncs checkpoints as your job sa…”
   • [claimed-docs] “Deploy any synced checkpoint with one CLI command”
   • [claimed-docs] “Baseten trains models on managed GPUs and deploys the resulting checkpoints to production inference on the same platform.”

   proof ↗⚑ flag

Catalog

1. ml-engineerGet newly released open-weight models on the platform quickly after their public release

   weight 2 · round drawn
   Together AI—none0/10

   The evidence pack shows Together AI hosts 100+ open-source models and supports fine-tuning/dedicated deployment, but there is no evidence—first-party or independent—about the speed or process of adding newly released open-weight models to the catalog. missing for 10: any documentation or community report on time-to-availability for new model releases, changelog/release-notes evidence, or comparison of catalog freshness versus competitors.

   ⚑ flag

   Baseten—none0/10

   The evidence describes Baseten's general capabilities (Model APIs, Truss deployment, training) but contains no claims or evidence about how quickly newly released open-weight models are added to the hosted Model APIs catalog after their public release. Missing for 10: any documentation or track record on model-catalog update cadence, day-1 support claims, or examples of specific new model releases being onboarded quickly.

   ⚑ flag

2. developerRely on a documented deprecation policy with advance notice before a hosted model is removed

   weight 1 · round to Baseten
   Together AI—none0/10

   No evidence in the pack mentions a deprecation policy, sunset schedule, or advance-notice commitment for retiring hosted models; docs cover serverless/dedicated inference, fine-tuning, and agent tooling but nothing about model lifecycle/deprecation notices.

   ⚑ flag

   Baseten~partialclaimed4/10

   Baseten has a dedicated deprecation docs page instructing users to migrate to a dedicated deployment with deprecated model weights, showing some documented deprecation handling exists, but the evidence pack shows no explicit advance-notice period, timeline, or notification process. Missing for 10: concrete advance-notice window/SLA, notification channel details, and independent confirmation that notice is actually given before removal.

   • [claimed-docs] “Migrate to a dedicated deployment with the deprecated model weights. Contact us for assistance.”

   proof ↗⚑ flag

3. ai-native userHave an agent enumerate the live model catalog programmatically via a documented GET /v1/models-style endpoint

   weight 2 · round drawn
   Together AI✓fullprobed8/10

   Together AI exposes an OpenAI-compatible /v1/models endpoint confirmed live via probe (returns 401 without key, meaning it exists and speaks JSON, per together-ai-probe-rt-1), and docs confirm OpenAI-compatibility for base URL/API key swaps (together-ai-docs-1) enabling programmatic catalog enumeration. missing for 10: a dedicated docs page explicitly documenting the /v1/models endpoint schema/response format, and independent hands-on confirmation of a successful authenticated call listing the full catalog.

   • [probe] “PROBE models-endpoint (2026-09-04): GET https://api.together.xyz/v1/models without a key returned HTTP 401 (Missing API key) — the OpenAI-st…”
   • [claimed-docs] “you can point it at models hosted on Together with two changes: the API key and base URL”

   proof ↗⚑ flag

   Baseten✓fullprobed8/10

   A live probe confirms the OpenAI-style GET /v1/models endpoint exists at https://inference.baseten.co/v1/models and returns proper JSON auth-error responses when unauthenticated, consistent with Baseten's documented OpenAI-compatible Chat Completions API (baseten-docs-21, baseten-docs-30). This lets an agent enumerate the model catalog programmatically once authenticated. Missing for 10: explicit first-party documentation of the /v1/models endpoint's response schema/fields and an authenticated hands-on example showing the returned catalog contents.

   • [probe] “PROBE models-endpoint (2026-09-04): GET https://inference.baseten.co/v1/models without a key returned HTTP 401 (No Authorization header prov…”
   • [claimed-docs] “They support the OpenAI Chat Completions API and the Anthropic Messages API in beta, so you can use familiar client SDKs.”
   • [claimed-docs] “Call hosted models through an OpenAI-compatible API, deploy your own models on dedicated infrastructure”

   proof ↗⚑ flag

4. developerChoose among a broad catalog of current open-weight model families (Llama, Qwen, DeepSeek, GPT-OSS and peers) on shared serverless endpoints

   weight 3 · round to Together AI
   Together AI✓fullprobed7/10

   Docs confirm serverless endpoints serving '100+ open-source models' with per-token pricing, no provisioning latency, and OpenAI-compatible API for easy model switching, and community evidence corroborates real-world use of Llama-family models on the platform. However, the evidence pack never explicitly enumerates Qwen, DeepSeek, or GPT-OSS by name, so full breadth across all cited model families isn't directly confirmed. Missing for 10: explicit documentation naming Qwen/DeepSeek/GPT-OSS as catalog entries, and independent verification of catalog breadth beyond the generic '100+ models' claim.

   • [claimed-docs] “Call 100+ open-source models with per-token pricing and no provisioning latency.”
   • [claimed-docs] “you can point it at models hosted on Together with two changes: the API key and base URL”
   • [claimed-docs] “you can prototype on serverless and move to reserved hardware later without changing your application code”
   • [community] “Colleague and I were discussing need for a service like this. Their pricing is much cheaper than self-hosting Llama-2-70B on AWS ($20.36/hr)…”
   • [probe] “PROBE models-endpoint (2026-09-04): GET https://api.together.xyz/v1/models without a key returned HTTP 401 (Missing API key) — the OpenAI-st…”

   proof ↗⚑ flag

   Baseten~partialprobed4/10

   Baseten's docs confirm a serverless 'Model APIs' offering for calling supported language models without deploying them, with OpenAI/Anthropic-compatible endpoints, implying a hosted catalog exists — but no evidence enumerates specific open-weight families like Llama, Qwen, DeepSeek, or GPT-OSS, and the live models endpoint requires an API key to inspect. missing for 10: explicit list of supported model families, breadth/currency of catalog, independent confirmation of specific models available on shared serverless endpoints.

   • [claimed-docs] “Use Model APIs to call supported language models without deploying them.”
   • [claimed-docs] “They support the OpenAI Chat Completions API and the Anthropic Messages API in beta, so you can use familiar client SDKs.”
   • [claimed-docs] “Call hosted models through an OpenAI-compatible API, deploy your own models on dedicated infrastructure”
   • [probe] “PROBE models-endpoint (2026-09-04): GET https://inference.baseten.co/v1/models without a key returned HTTP 401 (No Authorization header prov…”

   proof ↗⚑ flag

Modalities

1. developerGenerate embeddings (and rerank results) for retrieval pipelines without a second vendor

   weight 1 · round drawn
   Together AI~partialclaimed5/10

   Docs confirm a unified API that includes embeddings alongside chat/image/audio (together-ai-docs-21), supporting retrieval pipelines without a separate embeddings vendor. However, no evidence in the pack documents a dedicated rerank endpoint or reranking models, so the reranking half of the story is unsupported. Missing for 10: dedicated rerank API/model documentation, embeddings model list/dimensions, and independent confirmation of embeddings quality/usage in RAG pipelines.

   • [claimed-docs] “Call chat, image, audio, embedding, and more through one API.”

   proof ↗⚑ flag

   Baseten~partialcommunity5/10

   Baseten's general-purpose model deployment (Truss, custom model hosting, transformers/diffusers/vLLM support) can host embedding or reranking models on dedicated infrastructure, and a community report confirms real-world 'Baseten.co embedding workloads' being run on the platform. However, there is no first-party documentation of a dedicated embeddings or rerank API/product page, no mention of specific embedding model catalog entries, or built-in reranking features analogous to competitors' turnkey embedding endpoints. missing for 10: dedicated embeddings/rerank API docs, named embedding/rerank models in the catalog, first-party guidance on building RAG retrieval pipelines end-to-end.

   • [community] “Developer notes that with Baseten.co embedding workloads, the client (not server) becomes the bottleneck due to Python's GIL, prompting them…”
   • [claimed-docs] “Deploy an open-source, fine-tuned, or custom model on dedicated GPUs.”
   • [claimed-docs] “Deploy an open-source, fine-tuned, or custom model on dedicated infrastructure.”
   • [github] “Support for all Python frameworks: From `transformers` and `diffusers` to PyTorch and TensorFlow to vLLM, SGLang, and TensorRT-LLM, Truss su…”
   • [github] “Truss lets you serve models with the Baseten Inference Stack as well as deploy models from any open-source framework: vLLM, SGLang, TensorRT…”

   proof ↗⚑ flag

2. developerCall vision, audio, or image-generation models beyond text chat on the same platform

   weight 1 · round to Together AI
   Together AI✓fullclaimed8/10

   Together AI's docs explicitly state you can 'Call chat, image, audio, embedding, and more through one API' on the serverless platform, indicating multimodal model support beyond text chat on the same platform [together-ai-docs-21]. This is corroborated by the broader serverless/dedicated inference docs describing a unified API/base URL approach [together-ai-docs-1, together-ai-docs-2]. Missing for 10: no independent hands-on report or example code specifically calling vision/audio/image-generation endpoints, and no detailed model list confirming breadth of non-text modalities.

   • [claimed-docs] “Call chat, image, audio, embedding, and more through one API.”
   • [claimed-docs] “Call 100+ open-source models with per-token pricing and no provisioning latency.”
   • [claimed-docs] “you can point it at models hosted on Together with two changes: the API key and base URL”

   proof ↗⚑ flag

   Baseten~partialclaimed5/10

   Baseten's Truss framework explicitly supports 'diffusers' (used for image-generation models) and any open-source framework, and the platform lets you deploy 'open-source, fine-tuned, or custom' models on dedicated GPUs, implying vision/audio/image models can be hosted alongside text. However, the hosted 'Model APIs' product is explicitly scoped to 'supported language models' (baseten-docs-1), and no evidence names specific vision, audio, or image-generation models or endpoints being offered as first-class hosted APIs — multimodal support relies on self-deploying via Truss rather than a turnkey catalog like text chat models get. missing for 10: explicit vision/audio/image-generation model catalog or API examples, first-party documentation naming specific multimodal models supported out-of-the-box.

   • [claimed-docs] “Use Model APIs to call supported language models without deploying them.”
   • [claimed-docs] “Deploy an open-source, fine-tuned, or custom model on dedicated infrastructure.”
   • [github] “Support for all Python frameworks: From `transformers` and `diffusers` to PyTorch and TensorFlow to vLLM, SGLang, and TensorRT-LLM, Truss su…”
   • [github] “Truss lets you serve models with the Baseten Inference Stack as well as deploy models from any open-source framework: vLLM, SGLang, TensorRT…”
   • [claimed-docs] “Use a Python model class or a custom Docker server when you need custom preprocessing, postprocessing, dependencies, or server behavior.”

   proof ↗⚑ flag

Compat

1. ai-native userPlug the provider into coding agents and agent frameworks through documented, first-party integration guides

   weight 2 · round drawn
   Together AI✓fullprobed9/10

   Together AI documents first-party agent-skills and an MCP server specifically for coding agent integration (docs-agent-skills, docs-12/19/20/24/29/30), confirmed live via probe (probe-4, probe-rt-3), plus OpenAI-compatible base URL swap for framework compatibility (docs-1) and function/tool-calling docs for agentic workflows (docs-8/9). Missing for 10: independent hands-on reports specifically validating the agent-skills/MCP integration with popular frameworks like LangChain or Claude Code.

   • [claimed-docs] “you can point it at models hosted on Together with two changes: the API key and base URL”
   • [claimed-docs] “Make your AI coding agent Together-AI-aware with ready-made skills for code generation and an MCP server for live docs lookup.”
   • [claimed-docs] “12 domain-specific skills that load on demand and teach your agent how to write correct Together AI code (right model IDs, SDK patterns, bes…”
   • [claimed-docs] “Gives your agent live access to this documentation site so it can look up current information without leaving your editor.”
   • [claimed-docs] “Docs MCP server: Gives your agent live access to this documentation site so it can look up current information without leaving your editor.”
   • [claimed-docs] “Function calling (also called tool calling) lets LLMs respond with structured function names and arguments that you can execute in your appl…”
   • [claimed-docs] “To build agent loops, chain tool calls inside one response (multi-step), and conversations that thread tools across many turns (multi-turn).”
   • [probe] “official MCP server documented at https://docs.together.ai/docs/agent-skills”
   • [probe] “PROBE mcp-endpoint (2026-09-04): POST initialize to https://docs.together.ai/mcp answered HTTP 200 with a JSON-RPC/MCP response (event: mess…”

   proof ↗⚑ flag

   Baseten✓fullprobed9/10

   Baseten has dedicated first-party docs for connecting coding agents (Claude Code, Codex CLI, Pi) via Baseten Switch, plus a documented MCP server/skill for agent workspace management, and OpenAI/Anthropic-compatible endpoints enabling framework integrations like LangChain. This is corroborated by a live probe confirming the MCP endpoint responds correctly. Missing for 10: independent hands-on validation of the coding-agent connection flow beyond docs/probes.

   • [claimed-docs] “To point a coding agent at Model APIs, see [Coding agents](/inference/model-apis/coding-agents).”
   • [claimed-docs] “Install the Baseten skill and MCP servers so your coding agent can manage your Baseten workspace and search these docs.”
   • [claimed-docs] “Switch can also install Pi's direct Baseten provider, compare Baseten spend with estimated costs from Anthropic or OpenAI, and route request…”
   • [claimed-docs] “Connect Claude Code, Codex CLI, or Pi with Baseten Switch.”
   • [claimed-docs] “Use [Baseten Switch](/reference/cli/baseten-switch) to route requests from Claude Code or Codex CLI to Model APIs.”
   • [claimed-docs] “Because Baseten exposes an OpenAI-compatible endpoint, you can use LangChain's `ChatOpenAI` with `with_structured_output` by pointing `base_…”
   • [probe] “official MCP server documented at https://docs.baseten.co/agent-setup”
   • [probe] “PROBE mcp-endpoint (2026-09-04): POST initialize to https://docs.baseten.co/mcp answered HTTP 200 with a JSON-RPC/MCP response (event: messa…”

   proof ↗⚑ flag

2. ai-native userHave an agent switch to or away from this provider mid-workflow because it speaks the standard chat-completions API without provider-specific code changes

   weight 3 · round drawn
   Together AI✓fullprobed8/10

   Docs explicitly state OpenAI-compatibility requires only changing API key and base URL, meaning agents/tools written against the standard chat-completions API can point at Together with no provider-specific code changes; the live /v1/models endpoint confirms OpenAI-style REST shape. This directly supports switching to/away from Together mid-workflow with minimal friction. Missing for 10: independent third-party confirmation of drop-in swap behavior (e.g., a framework's changelog showing zero-code-change provider swap) and explicit mention of streaming/tool-call parity with OpenAI's spec beyond docs.

   • [claimed-docs] “you can point it at models hosted on Together with two changes: the API key and base URL”
   • [claimed-docs] “you can prototype on serverless and move to reserved hardware later without changing your application code”
   • [probe] “PROBE models-endpoint (2026-09-04): GET https://api.together.xyz/v1/models without a key returned HTTP 401 (Missing API key) — the OpenAI-st…”

   proof ↗⚑ flag

   Baseten✓fullclaimed8/10

   Baseten documents OpenAI Chat Completions API compatibility for Model APIs, explicitly enabling drop-in use with familiar client SDKs (e.g., LangChain's ChatOpenAI via base_url swap), and even documents a Switch tool that can route requests between Baseten and other providers like Anthropic/OpenAI mid-workflow. This directly supports agent-driven provider switching without provider-specific code changes. Missing for 10: independent/hands-on confirmation of seamless mid-workflow switching beyond vendor docs, and no explicit statement of full drop-in parity with OpenAI's complete API surface.

   • [claimed-docs] “They support the OpenAI Chat Completions API and the Anthropic Messages API in beta, so you can use familiar client SDKs.”
   • [claimed-docs] “Call hosted models through an OpenAI-compatible API, deploy your own models on dedicated infrastructure”
   • [claimed-docs] “Because Baseten exposes an OpenAI-compatible endpoint, you can use LangChain's `ChatOpenAI` with `with_structured_output` by pointing `base_…”
   • [claimed-docs] “Switch can also install Pi's direct Baseten provider, compare Baseten spend with estimated costs from Anthropic or OpenAI, and route request…”
   • [claimed-docs] “Use [Baseten Switch](/reference/cli/baseten-switch) to route requests from Claude Code or Codex CLI to Model APIs.”

   proof ↗⚑ flag

3. developerPoint an existing OpenAI SDK client at the provider by changing only the base URL and API key

   weight 3 · round to Baseten
   Together AI✓fullprobed8/10

   Docs explicitly state you can point an existing client at Together by changing only the API key and base URL, and the API is OpenAI-compatible with a live /v1/models endpoint confirmed via probe. missing for 10: independent developer testimonial specifically confirming drop-in OpenAI SDK compatibility (community evidence covers pricing/UX but not this specific mechanic), and no explicit code snippet shown in evidence.

   • [claimed-docs] “you can point it at models hosted on Together with two changes: the API key and base URL”
   • [probe] “PROBE models-endpoint (2026-09-04): GET https://api.together.xyz/v1/models without a key returned HTTP 401 (Missing API key) — the OpenAI-st…”
   • [claimed-docs] “you can prototype on serverless and move to reserved hardware later without changing your application code”

   proof ↗⚑ flag

   Baseten✓fullprobed9/10

   Baseten explicitly documents an OpenAI Chat Completions-compatible endpoint for its Model APIs, and shows real-world usage where only base_url and API key need to change (e.g. LangChain's ChatOpenAI pointed at Baseten). A live probe confirms an OpenAI-style /v1/models endpoint responding at inference.baseten.co. Missing for 10: no independent third-party hands-on report explicitly confirming a drop-in swap with the official OpenAI SDK client itself (only LangChain example and docs claims).

   • [claimed-docs] “They support the OpenAI Chat Completions API and the Anthropic Messages API in beta, so you can use familiar client SDKs.”
   • [claimed-docs] “Call hosted models through an OpenAI-compatible API, deploy your own models on dedicated infrastructure”
   • [claimed-docs] “Because Baseten exposes an OpenAI-compatible endpoint, you can use LangChain's `ChatOpenAI` with `with_structured_output` by pointing `base_…”
   • [probe] “PROBE models-endpoint (2026-09-04): GET https://inference.baseten.co/v1/models without a key returned HTTP 401 (No Authorization header prov…”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round to Together AI
   Together AI~partialprobed7/10

   Docs show broad API/CLI/SDK coverage mirroring UI actions—fine-tuning can be launched 'from the console, through the API/SDK, or with the CLI', dedicated endpoints can be deployed via CLI (tg beta endpoints deploy), and inference, batch, and cluster provisioning are all API-accessible—suggesting strong UI/API parity for core workflows. However, there's no explicit documentation enumerating full parity (e.g., billing, account/org management, usage dashboards) or confirming every UI-only feature has an API equivalent, and no OpenAPI spec was found (probe shows all openapi.json paths 404), making completeness hard to verify. Missing for 10: an explicit parity statement or API coverage for account/billing/admin functions, and a published OpenAPI spec confirming full API surface.

   • [claimed-docs] “You can launch a fine-tuning job from the console, through the API/SDK, or with the CLI”
   • [claimed-docs] “tg beta endpoints deploy google/gemma-4-E4B-it --endpoint my-endpoint”
   • [claimed-docs] “Together AI handles the full lifecycle: data upload, training, hosting, and inference on a dedicated endpoint.”
   • [claimed-docs] “Spin up H100 and B200 clusters with attached storage for training or large batch jobs.”
   • [claimed-docs] “Run asynchronous batch workloads at up to 50% lower cost.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.together.ai/openapi.json, https://docs.together.ai/swagger.json, https://docs.together.…”

   proof ↗⚑ flag

   Baseten~partialprobed6/10

   Baseten's core workflows — deploying models (Truss/config.yaml/CLI), running inference (OpenAI/Anthropic-compatible endpoints, streaming, async, structured outputs, function calling), and training (Loops API calls, Training Jobs CLI deploy) — are all API/CLI-driven, and usage/budget can be queried by API key, suggesting strong API-UI parity for the platform's main functions. However, there is no explicit documentation asserting full UI/API parity, no discoverable OpenAPI/reference spec (probe found only 404s), and no evidence that workspace/account-management UI features (team, billing settings) are exposed via API. Missing for 10: an explicit parity statement or full API reference, confirmation of workspace/admin functions being API-accessible, and independent verification of parity beyond inference/training/deploy workflows.

   • [github] “You write a `config.yaml` that specifies the model, the hardware, and the engine, then `uvx truss push` builds a TensorRT-optimized containe…”
   • [claimed-docs] “Your Axolotl config, TRL script, or custom loop runs unchanged in a container. Baseten provisions the GPUs, syncs checkpoints as your job sa…”
   • [claimed-docs] “Deploy any synced checkpoint with one CLI command”
   • [claimed-docs] “Loops provisions a dedicated trainer and paired sampler, and each training step is an API call from a Python loop you write.”
   • [claimed-docs] “Async inference returns a request ID quickly and completes later through webhook or polling, which suits batch work, long documents”
   • [claimed-docs] “Structured outputs let you generate text that conforms to specific JSON schemas, providing reliable data extraction and controlled text gene…”
   • [claimed-docs] “Function calling* (also called *tool calling*) lets a model choose a tool and produce its arguments from a user request.”
   • [claimed-docs] “You can also set a workspace budget and query usage by API key or model.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.baseten.co/openapi.json, https://docs.baseten.co/swagger.json, https://docs.baseten.co/…”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round to Baseten
   Together AI—none0/10

   The evidence pack covers inference, fine-tuning, dedicated endpoints, and agent tooling, but contains no mention of a data export feature, downloadable account data, or a documented way to retrieve fine-tuning datasets/model weights in open formats and leave the platform. This is a fair question for a hosted AI platform that stores prompts, fine-tuning data, and trained models, so absence of evidence means 'none' rather than 'na'.

   ⚑ flag

   Baseten~partialclaimed4/10

   Baseten's tooling leans open: models are packaged via the open-source Truss format (config.yaml, no proprietary container lock-in), and training checkpoints are synced to external storage as the job runs, so a user's model weights and training artifacts aren't trapped in a proprietary format. However there is no explicit documentation of exporting workspace data, logs, usage/billing history, or deployment configs in bulk, and no dedicated 'account export' or data-portability feature. missing for 10: explicit full-account/data export tooling, documentation of exporting logs/usage/config metadata, and any user testimonial of migrating away with all data intact.

   • [github] “Write once, run anywhere: Package model code, weights, and dependencies with a model server that behaves the same in development and product…”
   • [github] “You write a `config.yaml` that specifies the model, the hardware, and the engine, then `uvx truss push` builds a TensorRT-optimized containe…”
   • [claimed-docs] “Your Axolotl config, TRL script, or custom loop runs unchanged in a container. Baseten provisions the GPUs, syncs checkpoints as your job sa…”
   • [claimed-docs] “Deploy any synced checkpoint with one CLI command”
   • [claimed-docs] “Baseten provisions GPUs through MCM, runs your training container, and syncs checkpoints to storage as the job progresses.”

   proof ↗⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round to Baseten
   Together AI—none0/10

   The evidence pack covers Together AI's hosted inference, fine-tuning, and dedicated endpoint features, and its client SDKs (e.g., together-typescript) are open on GitHub, but there is no evidence that Together AI's core platform/product source is released under an open license—only that it serves open-source models to customers.

   ⚑ flag

   Baseten~partialclaimed3/10

   Baseten's core platform (inference service, dashboard, orchestration/MCM) is proprietary SaaS with no evidence of an open license; only the Truss packaging library is open-source on GitHub, which is a peripheral client tool rather than the product itself. missing for 10: no open license for the actual Baseten platform/service source, no repository for the core inference or orchestration engine, only a tangential SDK is open.

   • [github] “Write once, run anywhere: Package model code, weights, and dependencies with a model server that behaves the same in development and product…”
   • [github] “You write a `config.yaml` that specifies the model, the hardware, and the engine, then `uvx truss push` builds a TensorRT-optimized containe…”
   • [github] “Fast developer loop: Iterate with live reload, skip Docker and Kubernetes configuration, and use a batteries-included serving environment.”
   • [github] “Support for all Python frameworks: From `transformers` and `diffusers` to PyTorch and TensorFlow to vLLM, SGLang, and TensorRT-LLM, Truss su…”
   • [github] “Truss lets you serve models with the Baseten Inference Stack as well as deploy models from any open-source framework: vLLM, SGLang, TensorRT…”
   • [github] “Deploying a model to Baseten via Truss turns a Hugging Face model into a production-ready API endpoint. You write a `config.yaml` that speci…”

   proof ↗⚑ flag

Limits

1. developerRead documented rate limits and how they scale across usage tiers before I hit them in production

   weight 2 · round drawn
   Together AI~partialclaimed6/10

   Together AI documents a rate-limit model explicitly (docs-14): dynamic per-organization, per-model limits based on live model capacity and recent successful usage, and notes dedicated endpoints have no hard rate limits (docs-31). This gives developers some visibility into how limits work before hitting them, but there's no documented tiered structure (e.g., spend-based tiers with specific numeric thresholds) showing how limits scale as usage/spend grows over time. Missing for 10: explicit usage-tier table or thresholds, numeric rate-limit values per tier, guidance on how to request/track tier upgrades.

   • [claimed-docs] “Together uses dynamic rate limits instead of fixed thresholds. Each organization has a dynamic rate per model that adjusts based on: The mod…”
   • [claimed-docs] “Dedicated model inference (DMI) lets you serve a model on reserved hardware... Better performance... No hard rate limits”

   proof ↗⚑ flag

   Baseten~partialclaimed6/10

   Baseten has a dedicated pricing-and-limits doc covering request/token rate limits, rate-limit response headers, tier upgrades (Basic→Pro/Enterprise via email verification), workspace budgets, and usage monitoring by API key/model. However, it lacks a detailed table of exact numeric limits per tier and doesn't cover rate limits for dedicated-deployment (non-Model-APIs) inference. missing for 10: explicit numeric limit tables per tier, independent/community corroboration of documented limits, coverage of dedicated deployment rate limits (not just Model APIs).

   • [claimed-docs] “Model APIs bill by token and enforce request and token rate limits. You can also set a workspace budget and query usage by API key or model.”
   • [claimed-docs] “To monitor token and request consumption by API key or model, see Usage.”
   • [claimed-docs] “To raise a Basic account's limits, request email verification. You can also use that form to move to Pro or Enterprise.”
   • [claimed-docs] “x-ratelimit-remaining-requests: Reports how many requests remain before you reach the request rate limit.”
   • [claimed-docs] “You can also set a workspace budget and query usage by API key or model.”

   proof ↗⚑ flag

Pricing

1. founderSet spending caps or budget alerts so a runaway workload cannot generate an unbounded bill

   weight 1 · round to Baseten
   Together AI—none0/10

   No evidence of spending caps, budget alerts, or usage-based cutoffs; docs mention rate limits (dynamic, based on capacity/usage) and pricing structures but nothing about setting a spend ceiling or receiving alerts to prevent runaway bills.

   ⚑ flag

   Baseten~partialclaimed5/10

   Baseten docs mention setting a workspace budget and monitoring usage by API key/model, plus rate limits (request/token) that cap throughput, which supports pricing-limits awareness (baseten-docs-10, -23, -37, -38). However, there's no evidence of automated spending caps that hard-stop billing or proactive budget alerts/notifications — the workspace budget appears to be a settable figure without documented enforcement or alerting mechanics. Missing for 10: explicit alerting/notification system, enforcement behavior when budget is exceeded, and independent confirmation that budgets actually cap spend rather than just track it.

   • [claimed-docs] “Model APIs bill by token and enforce request and token rate limits. You can also set a workspace budget and query usage by API key or model.”
   • [claimed-docs] “To monitor token and request consumption by API key or model, see Usage.”
   • [claimed-docs] “x-ratelimit-remaining-requests: Reports how many requests remain before you reach the request rate limit.”
   • [claimed-docs] “You can also set a workspace budget and query usage by API key or model.”

   proof ↗⚑ flag

2. founderSee public per-token prices for every hosted model without talking to sales

   weight 3 · round to Together AI
   Together AI~partialprobed6/10

   Docs advertise "per-token pricing" for 100+ serverless models and community evidence confirms self-serve signup with visible per-token cost (e.g. "$0.0001/1K tokens", $25 trial credit) with no sales contact required. However, no evidence pack item directly shows a public pricing page enumerating rates for every hosted model, and the live models API requires an API key to enumerate the catalog. Missing for 10: a documented public pricing page/table covering all models, independent confirmation that all model prices (not just examples) are visible pre-signup.

   • [claimed-docs] “Call 100+ open-source models with per-token pricing and no provisioning latency.”
   • [community] “the price is pretty low... 4B MODEL, PRICE 1K TOKENS: $0.0001. register with an email, test account has $25 credit, python API as well, good…”
   • [community] “I've explored most of the inference as a service platforms, theirs is particularly nice. (Clear/simple UI, good pricing, good speed.)”
   • [probe] “PROBE models-endpoint (2026-09-04): GET https://api.together.xyz/v1/models without a key returned HTTP 401 (Missing API key) — the OpenAI-st…”

   proof ↗⚑ flag

   Baseten~partialprobed3/10

   Docs confirm Model APIs are billed per token and that usage/rate limits can be queried without contacting sales, but no evidence shows an actual public price list with per-model per-token rates; the models endpoint even requires an API key to enumerate the catalog. missing for 10: a public pricing/rate-card page listing token prices per model, independent confirmation that pricing is visible without account creation or sales contact.

   • [claimed-docs] “Model APIs bill by token and enforce request and token rate limits. You can also set a workspace budget and query usage by API key or model.”
   • [claimed-docs] “To raise a Basic account's limits, request email verification. You can also use that form to move to Pro or Enterprise.”
   • [probe] “PROBE models-endpoint (2026-09-04): GET https://inference.baseten.co/v1/models without a key returned HTTP 401 (No Authorization header prov…”

   proof ↗⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round drawn
   Together AI—none0/10

   No evidence in the pack mentions data residency, regional storage options, or geographic control over where data/models are hosted; only privacy policy criticism unrelated to residency is present. Missing for 10: any documentation of region selection, data residency guarantees, or geo-specific hosting options.

   ⚑ flag

   Baseten—none0/10

   No evidence in the pack mentions region selection, data residency options, or multi-region storage controls for user data; deployments are described as active-active across clusters/clouds for reliability (baseten-docs-15), not as a user-selectable residency control. Missing for 10: any mention of region/residency selection, data storage location controls, or compliance-driven data locality options.

   • [claimed-docs] “Deployments run active-active across clusters and clouds. If a region or provider loses capacity, MCM reroutes and reprovisions workloads.”

   proof ↗⚑ flag

2. ai-native userPrevent my data from being used to train AI models

   weight 3 · round drawn
   Together AI—none0/10

   No first-party documentation describes an opt-out, data-retention control, or 'no training on your data' policy; the only relevant evidence is a community report describing Together AI's privacy policy as broad/vague, allowing data use 'for other purposes,' which points toward the opposite of a training-opt-out guarantee.

   • [community] “Together AI's privacy policy collects broad personal data including prompts, IP addresses, and contact info, with vague language allowing us…”
   • [community] “Colleague and I were discussing need for a service like this. Their pricing is much cheaper than self-hosting Llama-2-70B on AWS ($20.36/hr)…”

   proof ↗⚑ flag

   Baseten—none0/10

   No evidence pack content addresses data usage for training, opt-out controls, or any privacy/data-retention policy commitments; the axis is applicable (Baseten processes customer data/prompts and could plausibly offer such guarantees) but nothing in the docs, GitHub, or community evidence confirms it.

   ⚑ flag

3. ai-native userControl data retention and deletion

   weight 2 · round drawn
   Together AI—none0/10

   No evidence pack item documents user-facing data retention or deletion controls (e.g., API/dashboard settings to delete stored prompts or data). The only relevant community evidence (together-ai-comm-1) actually raises concerns that Together AI's privacy policy allows broad data collection with vague retention/usage language, which is the opposite of demonstrating retention/deletion controls.

   • [community] “Together AI's privacy policy collects broad personal data including prompts, IP addresses, and contact info, with vague language allowing us…”

   proof ↗⚑ flag

   Baseten—none0/10

   The evidence pack covers deployment, training, inference APIs, pricing, and coding-agent integration, but no documentation addresses data retention policies, deletion of inputs/outputs, or user control over stored data/logs. This is a fair axis for an AI inference/hosting platform (buyers routinely ask about data handling), so it applies, but there is no evidence it is delivered.

   ⚑ flag

4. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   Together AI—none0/10

   No documentation or feature reference describes a telemetry/usage-tracking opt-out mechanism; the only related evidence is a community critique noting Together AI's privacy policy broadly collects prompts, IPs, and contact info with vague 'other purposes' language and no opt-out mentioned, reinforcing the absence of this capability.

   • [community] “Together AI's privacy policy collects broad personal data including prompts, IP addresses, and contact info, with vague language allowing us…”
   • [community] “Colleague and I were discussing need for a service like this. Their pricing is much cheaper than self-hosting Llama-2-70B on AWS ($20.36/hr)…”

   proof ↗⚑ flag

   Baseten—none0/10

   No evidence in the pack addresses telemetry/usage-tracking opt-out settings for Baseten's platform, CLI, or SDK; the evidence only covers inference features, training, and pricing/usage monitoring for billing purposes, not privacy telemetry controls.

   ⚑ flag

Reliability

1. founderGet a stated availability SLA on paid or enterprise tiers

   weight 1 · round drawn
   Together AI—none0/10

   Evidence shows a public status page and various product docs, but nothing states an actual uptime/availability SLA percentage or enterprise contractual guarantee. Absence of evidence for this applicable capability yields none.

   • [probe] “PROBE status-page (2026-09-04): https://status.together.ai returns HTTP 200 and renders a public service-status page (page body includes "op…”

   proof ↗⚑ flag

   Baseten—none0/10

   Baseten documents Pro/Enterprise tiers, active-active multi-cloud deployments, and a public status page, but no evidence pack item states an actual SLA (uptime percentage, response-time guarantee, or contractual commitment) for any paid or enterprise tier.

   • [claimed-docs] “To raise a Basic account's limits, request email verification. You can also use that form to move to Pro or Enterprise.”
   • [claimed-docs] “Deployments run active-active across clusters and clouds. If a region or provider loses capacity, MCM reroutes and reprovisions workloads.”
   • [probe] “PROBE status-page (2026-09-04): https://status.baseten.co returns HTTP 200 and renders a public service-status page (page body includes "ope…”

   proof ↗⚑ flag

2. founderCheck a public status page with incident history before betting production traffic on the platform

   weight 2 · round to Together AI
   Together AI✓fullprobed8/10

   A live public status page at status.together.ai returns HTTP 200 and shows service status ('operational'), confirming a real status dashboard exists that a founder could check before committing production traffic. Missing for 10: no evidence of detailed incident history/postmortems or uptime SLA history visible in the pack, so depth of incident history can't be fully confirmed.

   • [probe] “PROBE status-page (2026-09-04): https://status.together.ai returns HTTP 200 and renders a public service-status page (page body includes "op…”

   proof ↗⚑ flag

   Baseten~partialprobed6/10

   A live probe confirms Baseten operates a public status page at status.baseten.co that returns HTTP 200 and shows current operational status, which is the core reliability-status signal a founder would check before committing production traffic. However, the evidence pack does not confirm the page includes a visible incident history/archive log, only that it renders current status. missing for 10: explicit confirmation that the status page displays past incident history/timeline, not just live status.

   • [probe] “PROBE status-page (2026-09-04): https://status.baseten.co returns HTTP 200 and renders a public service-status page (page body includes "ope…”

   proof ↗⚑ flag

Benchmarks

1. ml-engineerSee published tokens-per-second or latency numbers, benchmarks, or load-testing guides backing the provider's speed claims

   weight 2 · round drawn
   Together AI—none0/10

   The evidence pack contains only vague qualitative claims ('no provisioning latency', 'Better performance' for dedicated endpoints) and subjective community remarks ('good speed') but no published tokens-per-second figures, latency benchmarks, or load-testing methodology/guides that would let an ml-engineer verify speed claims.

   • [claimed-docs] “Call 100+ open-source models with per-token pricing and no provisioning latency.”
   • [claimed-docs] “Dedicated model inference (DMI) lets you serve a model on reserved hardware... Better performance... No hard rate limits”
   • [community] “I've explored most of the inference as a service platforms, theirs is particularly nice. (Clear/simple UI, good pricing, good speed.)”

   proof ↗⚑ flag

   Baseten—none0/10

   The evidence pack covers Baseten's product features (Model APIs, Truss, training, coding-agent integrations) but contains no published benchmark numbers, tokens-per-second figures, latency claims, or load-testing guides; the only performance-adjacent comment is a third-party observation about client-side GIL bottlenecks, which is unrelated to Baseten's own speed claims.

   ⚑ flag

Serving

1. developerServe latency-sensitive workloads with fast time-to-first-token and high-throughput generation

   weight 3 · round to Baseten
   Together AI!disputedcontradicted5/10

   Together AI's docs claim strong speed characteristics — 'no provisioning latency' for serverless, DMI offering 'better performance' and 'no hard rate limits', dynamic rate limits based on live capacity, streaming SSE support, and automatic prompt caching to speed repeated prefills — all directly supporting fast TTFT and high throughput [together-ai-docs-2][together-ai-docs-31][together-ai-docs-14][together-ai-gh-1][together-ai-docs-13]. However, an independent hands-on comparison explicitly disputes this, stating Together AI was 'at the bottom of the pile' for large-scale deployment speed/pricing versus competitors like Cerebras, DeepInfra, and Hyperbolic [together-ai-comm-5], while another community post gives a positive but vaguer 'good speed' impression [together-ai-comm-4] — a genuine mixed/contradicted signal rather than mere skepticism. Missing for 10: independent benchmarks (TTFT/tokens-per-second numbers) confirming or refuting the vendor's latency claims, and resolution of the conflicting community reports.

   • [claimed-docs] “Call 100+ open-source models with per-token pricing and no provisioning latency.”
   • [claimed-docs] “Dedicated model inference (DMI) lets you serve a model on reserved hardware... Better performance... No hard rate limits”
   • [claimed-docs] “Together uses dynamic rate limits instead of fixed thresholds. Each organization has a dynamic rate per model that adjusts based on: The mod…”
   • [claimed-docs] “Automatic: There is no header, parameter, or account toggle to enable it. Send the same prompt prefix again and any portion that's still war…”
   • [github] “We provide support for streaming responses using Server Sent Events (SSE).”
   • [community] “I've explored most of the inference as a service platforms, theirs is particularly nice. (Clear/simple UI, good pricing, good speed.)”
   • [community] “'Together AI has quickly grown into the leading hosting platform for AI' - Lol. Definitely not. I went through pricing and large scale deplo…”

   proof ↗⚑ flag

   Baseten✓fullcommunity8/10

   Baseten's docs and GitHub materials show explicit architecture for latency-sensitive, high-throughput serving: token-by-token streaming (baseten-docs-8, baseten-docs-26), dedicated GPU deployment with TensorRT-LLM/vLLM/SGLang optimized builds via Truss (baseten-gh-2, baseten-gh-4, baseten-gh-6), automatic KV-cache discounting for repeated prompt tokens (baseten-docs-11), and active-active multi-cluster routing to avoid capacity-related slowdowns (baseten-docs-15). Async inference is also offered for non-latency-critical batch work, showing the platform differentiates paths for latency vs throughput use cases (baseten-docs-9, baseten-docs-27). missing for 10: published quantitative TTFT/throughput benchmarks or independent third-party latency comparisons; one community note (baseten-comm-3) suggests client-side bottlenecks can arise in high-throughput client code, an important but non-contradicting caveat.

   • [claimed-docs] “Return model output token by token as it is generated.”
   • [claimed-docs] “Streaming sends tokens as they are generated over server-sent events, which suits long generations and UIs where partial output beats a blan…”
   • [claimed-docs] “Cached input tokens are prompt tokens served from the KV cache at a discounted rate. Caching is automatic and requires no request flags.”
   • [claimed-docs] “Deployments run active-active across clusters and clouds. If a region or provider loses capacity, MCM reroutes and reprovisions workloads.”
   • [github] “You write a `config.yaml` that specifies the model, the hardware, and the engine, then `uvx truss push` builds a TensorRT-optimized containe…”
   • [github] “Support for all Python frameworks: From `transformers` and `diffusers` to PyTorch and TensorFlow to vLLM, SGLang, and TensorRT-LLM, Truss su…”
   • [github] “Deploying a model to Baseten via Truss turns a Hugging Face model into a production-ready API endpoint. You write a `config.yaml` that speci…”
   • [claimed-docs] “Async inference returns a request ID quickly and completes later through webhook or polling, which suits batch work, long documents”
   • [claimed-docs] “Async inference returns a request ID quickly and completes later through webhook or polling, which suits batch work, long documents, or any …”
   • [community] “Developer notes that with Baseten.co embedding workloads, the client (not server) becomes the bottleneck due to Python's GIL, prompting them…”

   proof ↗⚑ flag

2. ml-engineerBenefit from prompt/prefix caching that reduces latency or cost on repeated context

   weight 1 · round to Together AI
   Together AI✓fullclaimed8/10

   Together AI documents automatic prompt/prefix caching for serverless chat models, explicitly stating cached input tokens are billed at a steep discount with no toggle needed, directly reducing cost on repeated context. missing for 10: independent/hands-on benchmark confirming latency reduction, and clarity on which models/latency gains apply beyond the cost-discount framing.

   • [claimed-docs] “Automatic: There is no header, parameter, or account toggle to enable it. Send the same prompt prefix again and any portion that's still war…”
   • [claimed-docs] “Send the same prompt prefix again and any portion that's still warm in the shared cache is billed at the cached rate.”
   • [claimed-docs] “Select serverless chat models bill cached input tokens at a steep discount... Automatic: There is no header, parameter, or account toggle to…”

   proof ↗⚑ flag

   Baseten✓fullclaimed7/10

   Docs explicitly describe automatic prompt caching: cached input tokens are served from KV cache at a discounted rate with no request flags needed, directly addressing latency/cost reduction on repeated context. missing for 10: no quantified latency/cost benchmarks, no independent/community corroboration of caching behavior in practice.

   • [claimed-docs] “Cached input tokens are prompt tokens served from the KV cache at a discounted rate. Caching is automatic and requires no request flags.”
   • [claimed-docs] “Model APIs bill by token and enforce request and token rate limits. You can also set a workspace budget and query usage by API key or model.”

   proof ↗⚑ flag

3. developerStream completions token by token over SSE for responsive user experiences

   weight 3 · round drawn
   Together AI✓fullclaimed8/10

   Together AI's SDKs explicitly document SSE-based streaming support ('We provide support for streaming responses using Server Sent Events (SSE)'), and the API is OpenAI-compatible, implying standard streaming chat completion endpoints. Missing for 10: no independent hands-on latency benchmark or explicit token-by-token example snippet in the evidence pack.

   • [github] “We provide support for streaming responses using Server Sent Events (SSE).”
   • [github] “This library provides convenient access to the Together REST API from server-side TypeScript or JavaScript.”
   • [claimed-docs] “you can point it at models hosted on Together with two changes: the API key and base URL”

   proof ↗⚑ flag

   Baseten✓fullclaimed8/10

   Baseten's docs explicitly describe SSE token-by-token streaming ('Return model output token by token as it is generated' / 'Streaming sends tokens as they are generated over server-sent events') as a first-class inference mode alongside async, with OpenAI/Anthropic-compatible client SDK support for easy integration. Missing for 10: no independent/hands-on benchmark or community confirmation of streaming latency/responsiveness in practice.

   • [claimed-docs] “Return model output token by token as it is generated.”
   • [claimed-docs] “Streaming sends tokens as they are generated over server-sent events, which suits long generations and UIs where partial output beats a blan…”
   • [claimed-docs] “They support the OpenAI Chat Completions API and the Anthropic Messages API in beta, so you can use familiar client SDKs.”

   proof ↗⚑ flag

Structured

1. developerEnforce structured outputs against a JSON schema (or grammar) so model responses parse reliably

   weight 3 · round drawn
   Together AI✓fullclaimed8/10

   Together AI's docs explicitly describe structured outputs where supported models return JSON conforming to a supplied schema, avoiding retries/fragile parsing, alongside function/tool calling and multi-step agentic tool call chaining. This directly matches the story's request for JSON-schema-enforced structured outputs; missing for 10: no independent/hands-on validation of schema-conformance reliability or grammar-based constraints beyond JSON schema.

   • [claimed-docs] “Supported models can return JSON that conforms to any schema you supply, so you can read the output directly in code without retries or frag…”
   • [claimed-docs] “Function calling (also called tool calling) lets LLMs respond with structured function names and arguments that you can execute in your appl…”
   • [claimed-docs] “To build agent loops, chain tool calls inside one response (multi-step), and conversations that thread tools across many turns (multi-turn).”
   • [claimed-docs] “Multi-step function calling chains sequential function calls within one conversation turn.”

   proof ↗⚑ flag

   Baseten✓fullclaimed8/10

   Baseten has dedicated first-party docs on structured outputs enforcing JSON schemas for reliable data extraction, plus separate function/tool calling support and OpenAI-compatible client integration (e.g., LangChain with_structured_output) confirming reliable parsing workflows. Missing for 10: independent/hands-on validation of schema-enforcement reliability and no mention of grammar-based (e.g., regex/CFG) constraints beyond JSON schema.

   • [claimed-docs] “Structured outputs let you generate text that conforms to specific JSON schemas, providing reliable data extraction and controlled text gene…”
   • [claimed-docs] “Function calling* (also called *tool calling*) lets a model choose a tool and produce its arguments from a user request.”
   • [claimed-docs] “Function calling (also called tool calling) lets a model choose a tool and produce its arguments from a user request.”
   • [claimed-docs] “Because Baseten exposes an OpenAI-compatible endpoint, you can use LangChain's `ChatOpenAI` with `with_structured_output` by pointing `base_…”
   • [claimed-docs] “They support the OpenAI Chat Completions API and the Anthropic Messages API in beta, so you can use familiar client SDKs.”

   proof ↗⚑ flag

Tools

1. ai-native userRely on faithful function/tool calling — including parallel and multi-step tool use — so agent loops run on open models without breaking

   weight 3 · round to Together AI
   Together AI~partialclaimed6/10

   Together AI documents native function/tool calling with explicit support for multi-step (chained) and multi-turn agentic tool use, positioned as the foundation for agent loops (together-ai-docs-8,9,15,16). However, there is no independent/hands-on evidence confirming reliability of parallel tool calls or that it avoids breakage across open models in practice — only first-party docs. missing for 10: independent benchmarks or community reports on parallel tool-call accuracy, explicit parallel tool-calling documentation, and hands-on verification that agent loops don't break on open models.

   • [claimed-docs] “Function calling (also called tool calling) lets LLMs respond with structured function names and arguments that you can execute in your appl…”
   • [claimed-docs] “To build agent loops, chain tool calls inside one response (multi-step), and conversations that thread tools across many turns (multi-turn).”
   • [claimed-docs] “Function calling (also called *tool calling*) lets LLMs respond with structured function names and arguments that you can execute in your ap…”
   • [claimed-docs] “Multi-step function calling chains sequential function calls within one conversation turn.”

   proof ↗⚑ flag

   Baseten~partialclaimed4/10

   Baseten docs confirm function/tool-calling support ("lets a model choose a tool and produce its arguments from a user request") and structured outputs for reliable JSON generation, but the pack never mentions parallel tool calls, multi-step agent loops, or any reliability testing on open models. Missing for 10: explicit parallel tool-call support docs, multi-step agent loop examples, and independent/hands-on evidence of reliability across open models.

   • [claimed-docs] “Function calling* (also called *tool calling*) lets a model choose a tool and produce its arguments from a user request.”
   • [claimed-docs] “Function calling (also called tool calling) lets a model choose a tool and produce its arguments from a user request.”
   • [claimed-docs] “Structured outputs let you generate text that conforms to specific JSON schemas, providing reliable data extraction and controlled text gene…”
   • [claimed-docs] “Because Baseten exposes an OpenAI-compatible endpoint, you can use LangChain's `ChatOpenAI` with `with_structured_output` by pointing `base_…”

   proof ↗⚑ flag