How Arize Phoenix’s scores are calculated

Point an agent at llms.txt or agent-oriented docsweight 2

2 (weight) × 9 (quality) × 1.0 (full) = 18.0 of 20 max

• [probe] https://arize.com/llms.txt“PROBE llms.txt: HTTP 200 at https://arize.com/llms.txt # Arize — AI & Agent Engineering > Arize is the AI engineering platform for teams building and improving AI agents and”
• [probe] https://arize.com/docs/phoenix.md“PROBE docs-md: HTTP 200 at https://arize.com/docs/phoenix.md > ## Documentation Index > Fetch the complete documentation index at: https://arizeai-433a7140.mintlify.site/llms.txt >”

Run the product headlessly / in CI for automationweight 2

2 (weight) × 7 (quality) × 0.6 (partial) = 8.4 of 20 max

• [claimed-docs] https://arize.com/phoenix/“$ uvx arize-phoenix serve”
• [claimed-docs] https://arize.com/docs/phoenix/self-hosting“Pin to a specific version (e.g., `arizephoenix/phoenix:version-8.0.0`) for production deployments.”
• [claimed-docs] https://arize.com/docs/phoenix/datasets-and-experiments/overview-datasets“Dataset Evaluators serve as test cases that automatically score outputs when running experiments—forming an evaluation harness similar to a unit test suite.”
• [claimed-docs] https://arize.com/docs/phoenix/datasets-and-experiments/overview-datasets“Dataset Evaluators serve as test cases that automatically score outputs when running experiments”
• [claimed-docs] https://arize.com/docs/phoenix/evaluation/llm-evals“This means you can run thousands of evaluations without writing any retry or concurrency logic yourself.”
• [claimed-docs] https://arize.com/docs/phoenix/sdk-api-reference“Phoenix’s Python SDK is modular by design, allowing you to install only what you need.”
• [claimed-docs] https://arize.com/docs/phoenix/self-hosting“Phoenix is **free to self-host** with no feature limitations. Your data stays entirely within your infrastructure—nothing is sent to Arize and can be fully air-gapped.”

Plug MCP servers into this product so it can use their toolsweight 3

n/a — not applicable to this product: excluded from numerator and denominator

• [claimed-docs] https://arize.com/docs/phoenix/integrations/remote-mcp“Point any MCP-compatible client (Claude Code, Cursor, VS Code, and others) at your Phoenix instance’s `/mcp` endpoint and it can search, query, and operate on your projects, traces, datasets, experiments, prompts, and annotations”
• [probe] https://arize.com/docs/phoenix/integrations/remote-mcp“official MCP server documented at https://arize.com/docs/phoenix/integrations/remote-mcp”

Connect an agent via an official MCP serverweight 3

3 (weight) × 8 (quality) × 1.0 (full) = 24.0 of 30 max

• [claimed-docs] https://arize.com/docs/phoenix/integrations/remote-mcp“Point any MCP-compatible client (Claude Code, Cursor, VS Code, and others) at your Phoenix instance’s `/mcp` endpoint and it can search, query, and operate on your projects, traces, datasets, experiments, prompts, and annotations”
• [probe] https://arize.com/docs/phoenix/integrations/remote-mcp“official MCP server documented at https://arize.com/docs/phoenix/integrations/remote-mcp”

Use an official CLIweight 2

2 (weight) × 6 (quality) × 0.6 (partial) = 7.2 of 20 max

• [claimed-docs] https://arize.com/docs/phoenix/tracing/llm-traces“Let your coding agent set up tracing for you: start Phoenix, then run npx -y @arizeai/phoenix-cli setup from your app's root directory.”
• [claimed-docs] https://arize.com/phoenix/“$ uvx arize-phoenix serve”
• [claimed-docs] https://arize.com/docs/phoenix/tracing/llm-traces“Let your coding agent set up tracing for you: start Phoenix, then run `npx -y @arizeai/phoenix-cli setup` from your app's root directory.”
• [claimed-docs] https://arize.com/docs/phoenix/tracing/llm-traces“start Phoenix, then run `npx -y @arizeai/phoenix-cli setup` from your app’s root directory”

Drive the product through a documented public APIweight 3

3 (weight) × 7 (quality) × 1.0 (full) = 21.0 of 30 max

• [claimed-docs] https://arize.com/docs/phoenix/sdk-api-reference“Zero-config tracing — Enable auto_instrument=True to automatically trace AI libraries”
• [claimed-docs] https://arize.com/docs/phoenix/sdk-api-reference“Tracing decorators — @tracer.chain, @tracer.tool, and more for manual instrumentation”
• [claimed-docs] https://arize.com/docs/phoenix/sdk-api-reference“Spans — Query and analyze traces with powerful filtering capabilities”
• [claimed-docs] https://arize.com/docs/phoenix/sdk-api-reference“Phoenix’s Python SDK is modular by design, allowing you to install only what you need.”
• [claimed-docs] https://arize.com/docs/phoenix/prompt-engineering/overview-prompts“Phoenix offers client SDKs to keep your prompts in sync across different applications and environments.”
• [claimed-docs] https://arize.com/docs/phoenix/integrations/remote-mcp“Point any MCP-compatible client (Claude Code, Cursor, VS Code, and others) at your Phoenix instance’s `/mcp` endpoint and it can search, query, and operate on your projects, traces, datasets, experiments, prompts, and annotations”
• [probe] https://arize.com/openapi.json“PROBE openapi: all candidate paths 404 (https://arize.com/openapi.json, https://arize.com/swagger.json, https://arize.com/api/openapi.json, https://arize.com/.well-known/openapi.json)”
• [probe] https://arize.com/docs/phoenix/integrations/remote-mcp“official MCP server documented at https://arize.com/docs/phoenix/integrations/remote-mcp”
• [claimed-docs] https://arize.com/phoenix/“$ uvx arize-phoenix serve”

Issue scoped/least-privilege API credentials for an agentweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Build against official SDKsweight 2

2 (weight) × 8 (quality) × 1.0 (full) = 16.0 of 20 max

• [claimed-docs] https://arize.com/docs/phoenix/sdk-api-reference“Zero-config tracing — Enable auto_instrument=True to automatically trace AI libraries”
• [claimed-docs] https://arize.com/docs/phoenix/sdk-api-reference“Tracing decorators — @tracer.chain, @tracer.tool, and more for manual instrumentation”
• [claimed-docs] https://arize.com/docs/phoenix/sdk-api-reference“Spans — Query and analyze traces with powerful filtering capabilities”
• [claimed-docs] https://arize.com/docs/phoenix/sdk-api-reference“Phoenix’s Python SDK is modular by design, allowing you to install only what you need.”
• [claimed-docs] https://arize.com/docs/phoenix/prompt-engineering/overview-prompts“Phoenix offers client SDKs to keep your prompts in sync across different applications and environments.”
• [claimed-docs] https://arize.com/docs/phoenix/prompt-engineering/overview-prompts“Prompts in Code - Phoenix offers client SDKs to keep your prompts in sync across different applications and environments.”
• [claimed-docs] https://arize.com/docs/phoenix“Prompts in Code — Sync prompts across environments via SDK”
• [github] https://github.com/Arize-ai/phoenix“Datasets - Create versioned datasets of examples for experimen”

Subscribe to events via webhooksweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Explore an interactive API reference with runnable examplesweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

• [claimed-docs] https://arize.com/docs/phoenix/sdk-api-reference“Zero-config tracing — Enable auto_instrument=True to automatically trace AI libraries”
• [claimed-docs] https://arize.com/docs/phoenix/sdk-api-reference“Tracing decorators — @tracer.chain, @tracer.tool, and more for manual instrumentation”
• [claimed-docs] https://arize.com/docs/phoenix/sdk-api-reference“Spans — Query and analyze traces with powerful filtering capabilities”
• [probe] https://arize.com/openapi.json“PROBE openapi: all candidate paths 404 (https://arize.com/openapi.json, https://arize.com/swagger.json, https://arize.com/api/openapi.json, https://arize.com/.well-known/openapi.json)”

Download a machine-readable API spec (OpenAPI or equivalent)weight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

• [probe] https://arize.com/openapi.json“PROBE openapi: all candidate paths 404 (https://arize.com/openapi.json, https://arize.com/swagger.json, https://arize.com/api/openapi.json, https://arize.com/.well-known/openapi.json)”

Test against a sandbox environment without touching production dataweight 1

1 (weight) × 5 (quality) × 0.6 (partial) = 3.0 of 10 max

• [claimed-docs] https://arize.com/docs/phoenix/datasets-and-experiments/overview-datasets“Datasets allow you to collect data from production, staging, evaluations, and even manually.”
• [claimed-docs] https://arize.com/docs/phoenix/self-hosting“Phoenix is free to self-host with no feature limitations. Your data stays entirely within your infrastructure—nothing is sent to Arize and can be fully air-gapped.”
• [claimed-docs] https://arize.com/docs/phoenix/self-hosting“Phoenix is **free to self-host** with no feature limitations. Your data stays entirely within your infrastructure—nothing is sent to Arize and can be fully air-gapped.”
• [claimed-docs] https://arize.com/docs/phoenix/tracing/llm-traces“Projects Organize traces into separate projects for different applications”
• [claimed-docs] https://arize.com/docs/phoenix/tracing/llm-traces“Organize traces into separate projects for different applications”

Rely on versioned APIs with a documented deprecation policyweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

• [claimed-docs] https://arize.com/docs/phoenix/self-hosting“Pin to a specific version (e.g., `arizephoenix/phoenix:version-8.0.0`) for production deployments.”
• [probe] https://arize.com/openapi.json“PROBE openapi: all candidate paths 404 (https://arize.com/openapi.json, https://arize.com/swagger.json, https://arize.com/api/openapi.json, https://arize.com/.well-known/openapi.json)”

Do everything through the API that I can do in the UIweight 2

2 (weight) × 5 (quality) × 0.6 (partial) = 6.0 of 20 max

• [claimed-docs] https://arize.com/docs/phoenix“Human annotations — Attach ground truth labels directly in the UI”
• [claimed-docs] https://arize.com/docs/phoenix/evaluation/llm-evals“Configure evaluators in the Phoenix UI and attach them to your datasets. Phoenix scores experiment results automatically — no code required.”
• [claimed-docs] https://arize.com/docs/phoenix/sdk-api-reference“Zero-config tracing — Enable auto_instrument=True to automatically trace AI libraries”
• [claimed-docs] https://arize.com/docs/phoenix/sdk-api-reference“Tracing decorators — @tracer.chain, @tracer.tool, and more for manual instrumentation”
• [claimed-docs] https://arize.com/docs/phoenix/sdk-api-reference“Phoenix’s Python SDK is modular by design, allowing you to install only what you need.”
• [probe] https://arize.com/openapi.json“PROBE openapi: all candidate paths 404 (https://arize.com/openapi.json, https://arize.com/swagger.json, https://arize.com/api/openapi.json, https://arize.com/.well-known/openapi.json)”
• [claimed-docs] https://arize.com/docs/phoenix/integrations/remote-mcp“Point any MCP-compatible client (Claude Code, Cursor, VS Code, and others) at your Phoenix instance’s `/mcp` endpoint and it can search, query, and operate on your projects, traces, datasets, experiments, prompts, and annotations”

Export all of my data in open formats and leaveweight 3

3 (weight) × 6 (quality) × 0.6 (partial) = 10.8 of 30 max

• [claimed-docs] https://arize.com/docs/phoenix/self-hosting“Phoenix is free to self-host with no feature limitations. Your data stays entirely within your infrastructure—nothing is sent to Arize and can be fully air-gapped.”
• [claimed-docs] https://arize.com/docs/phoenix/self-hosting“Phoenix is **free to self-host** with no feature limitations. Your data stays entirely within your infrastructure—nothing is sent to Arize and can be fully air-gapped.”
• [claimed-docs] https://arize.com/docs/phoenix“Phoenix is built by Arize AI and the open-source community. It is built on top of OpenTelemetry and is powered by OpenInference instrumentation.”
• [claimed-docs] https://arize.com/docs/phoenix“Phoenix accepts traces over OpenTelemetry (OTLP) and provides auto-instrumentation for popular frameworks (LlamaIndex, LangChain, DSPy, Mastra, Vercel AI SDK)”
• [github] https://github.com/Arize-ai/phoenix“Datasets - Create versioned datasets of examples for experimen”

Read the product's source under an open licenseweight 2

2 (weight) × 5 (quality) × 0.6 (partial) = 6.0 of 20 max

• [claimed-docs] https://arize.com/docs/phoenix“Phoenix is built by Arize AI and the open-source community. It is built on top of OpenTelemetry and is powered by OpenInference instrumentation.”
• [github] https://github.com/Arize-ai/phoenix“Datasets - Create versioned datasets of examples for experimen”

Self-host the core productweight 3

3 (weight) × 9 (quality) × 1.0 (full) = 27.0 of 30 max

• [claimed-docs] https://arize.com/docs/phoenix/self-hosting“Phoenix is free to self-host with no feature limitations. Your data stays entirely within your infrastructure—nothing is sent to Arize and can be fully air-gapped.”
• [claimed-docs] https://arize.com/docs/phoenix/self-hosting“Phoenix is **free to self-host** with no feature limitations. Your data stays entirely within your infrastructure—nothing is sent to Arize and can be fully air-gapped.”
• [claimed-docs] https://arize.com/phoenix/“$ uvx arize-phoenix serve”
• [claimed-docs] https://arize.com/docs/phoenix/self-hosting“Pin to a specific version (e.g., `arizephoenix/phoenix:version-8.0.0`) for production deployments.”

Get AI-generated insights and suggestions from my data inside the productweight 2

2 (weight) × 5 (quality) × 0.6 (partial) = 6.0 of 20 max

• [claimed-docs] https://arize.com/docs/phoenix/evaluation/llm-evals“Built-in Explanations — all Phoenix LLM evaluations return explanations by default for better results and richer signals”
• [claimed-docs] https://arize.com/docs/phoenix/evaluation/llm-evals“Every evaluation execution captures the input data, the exact prompts sent to the judge LLM, the model's full reasoning, the final scores, and execution timing.”
• [claimed-docs] https://arize.com/docs/phoenix/evaluation/llm-evals“Use Phoenix's trace viewer to explore evaluation traces, identify systematic biases, and continuously improve evaluator performance.”
• [claimed-docs] https://arize.com/docs/phoenix/evaluation/llm-evals“Phoenix supports both deterministic code-based evaluators (exact match, regex, custom heuristics) and LLM-as-a-judge evaluators”

Set up automations that run autonomously in the backgroundweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Delegate tasks to a built-in AI assistant inside the productweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

• [claimed-docs] https://arize.com/docs/phoenix/integrations/remote-mcp“Point any MCP-compatible client (Claude Code, Cursor, VS Code, and others) at your Phoenix instance’s `/mcp` endpoint and it can search, query, and operate on your projects, traces, datasets, experiments, prompts, and annotations”
• [claimed-docs] https://arize.com/docs/phoenix/tracing/llm-traces“Let your coding agent set up tracing for you: start Phoenix, then run `npx -y @arizeai/phoenix-cli setup` from your app's root directory.”
• [claimed-docs] https://arize.com/docs/phoenix/tracing/llm-traces“Let your coding agent set up tracing for you: start Phoenix, then run npx -y @arizeai/phoenix-cli setup from your app's root directory.”

Operate the product with natural-language commandsweight 2

2 (weight) × 7 (quality) × 1.0 (full) = 14.0 of 20 max

• [claimed-docs] https://arize.com/docs/phoenix/integrations/remote-mcp“Point any MCP-compatible client (Claude Code, Cursor, VS Code, and others) at your Phoenix instance’s `/mcp` endpoint and it can search, query, and operate on your projects, traces, datasets, experiments, prompts, and annotations”
• [probe] https://arize.com/docs/phoenix/integrations/remote-mcp“official MCP server documented at https://arize.com/docs/phoenix/integrations/remote-mcp”
• [claimed-docs] https://arize.com/docs/phoenix/tracing/llm-traces“Let your coding agent set up tracing for you: start Phoenix, then run npx -y @arizeai/phoenix-cli setup from your app's root directory.”

Perform bulk operations across many items at onceweight 2

2 (weight) × 5 (quality) × 0.6 (partial) = 6.0 of 20 max

• [claimed-docs] https://arize.com/docs/phoenix/evaluation/llm-evals“This means you can run thousands of evaluations without writing any retry or concurrency logic yourself.”
• [claimed-docs] https://arize.com/docs/phoenix/datasets-and-experiments/overview-datasets“Datasets allow you to collect data from production, staging, evaluations, and even manually.”
• [claimed-docs] https://arize.com/docs/phoenix“You can group traces into datasets, rerun them through different versions of your application, and compare evaluation results”
• [claimed-docs] https://arize.com/docs/phoenix/integrations/remote-mcp“Point any MCP-compatible client (Claude Code, Cursor, VS Code, and others) at your Phoenix instance’s `/mcp` endpoint and it can search, query, and operate on your projects, traces, datasets, experiments, prompts, and annotations”

Define rules that trigger actions automatically on eventsweight 3

3 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 30 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Schedule recurring jobs or workflowsweight 2

2 (weight) × 0 (quality) × 0.0 (none) = 0.0 of 20 max

no evidence cited — the verdict rests on absence of evidence, re-checked on refresh

Version, review, and roll back my automationsweight 1

1 (weight) × 5 (quality) × 0.6 (partial) = 3.0 of 10 max

• [claimed-docs] https://arize.com/docs/phoenix“Prompt Management — Version, store, and deploy prompts”
• [claimed-docs] https://arize.com/docs/phoenix“Phoenix helps you iterate on prompts using real examples from your application. You can version prompts, test prompt variants across datasets, and replay calls to see how changes affect outputs before rolling them out.”
• [claimed-docs] https://arize.com/docs/phoenix“Prompts in Code — Sync prompts across environments via SDK”
• [claimed-docs] https://arize.com/docs/phoenix/prompt-engineering/overview-prompts“Phoenix offers client SDKs to keep your prompts in sync across different applications and environments.”
• [claimed-docs] https://arize.com/docs/phoenix“You can group traces into datasets, rerun them through different versions of your application, and compare evaluation results”