Consensus vs FutureHouse Platform

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round to Consensus
   Consensus✓fullprobed8/10

   A live probe confirms Consensus serves an llms.txt file at its root (HTTP 200) with a structured summary of the product, directly enabling agents to be pointed at agent-oriented docs. Missing for 10: broader agent-oriented doc formats (e.g. .md endpoints) return 404, and no independent third-party confirmation of llms.txt usage exists.

   • [probe] “PROBE llms.txt: HTTP 200 at https://consensus.app/llms.txt # Consensus > Consensus is an AI-powered scientific search engine that finds, ra…”
   • [probe] “PROBE docs-md: HTTP 404 at https://consensus.app/home/resources/how-consensus-works/.md”

   proof ↗⚑ flag

   FutureHouse Platform✓fullprobed7/10

   A working llms.txt file is directly confirmed via probe at futurehouse.gitbook.io/llms.txt returning HTTP 200 with structured agent-oriented documentation links, satisfying the core ask of pointing an agent at llms.txt. missing for 10: markdown-per-page fallback (docs-md probe returned 404), no OpenAPI/machine-readable schema confirmation, and no independent third-party corroboration of agents actually consuming this file successfully.

   • [probe] “PROBE llms.txt: HTTP 200 at https://futurehouse.gitbook.io/llms.txt # Edison Scientific Documentation ## Home - [Overview](https://docs.ed…”
   • [probe] “PROBE docs-md: HTTP 200 at https://futurehouse.gitbook.io/futurehouse-cookbook.md # Page Not Found The URL `futurehouse-cookbook` does not …”
   • [probe] “PROBE openapi: all candidate paths 404 (https://futurehouse.gitbook.io/openapi.json, https://futurehouse.gitbook.io/swagger.json, https://fu…”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round to FutureHouse Platform
   Consensus~partialclaimed4/10

   Consensus offers an API for integrating its search into custom workflows and running automated searches (consensus-docs-1, consensus-docs-15), which implies some programmatic/headless usability. However, there is no explicit documentation of CI integration, headless execution modes, CLI tooling, or automation pipeline examples. Missing for 10: CI/CD integration examples, headless mode documentation, CLI or SDK for automation, and independent evidence of running in automated pipelines.

   • [claimed-docs] “Connect the Consensus API within your project to seamlessly integrate up-to-date peer-reviewed citations into your own custom workflow.”
   • [claimed-docs] “Save your team hours of manual discovery research and run automated searches with our API to quickly and easily find the most relevant and r…”

   proof ↗⚑ flag

   FutureHouse Platform✓fullclaimed7/10

   The Edison client is a pip-installable Python SDK (`edison-client`) authenticated via API key, with a scriptable interface (`client.run_tasks_until_done(task_data)`) that requires no UI, making it straightforward to invoke headlessly or from CI pipelines. Missing for 10: explicit CI/automation examples (e.g., GitHub Actions workflow), documentation of exit codes/async polling patterns for pipelines, and independent confirmation of real-world CI usage.

   • [claimed-docs] “uv pip install edison-client”
   • [claimed-docs] “Authentication is done by providing an API key, which can be obtained directly from your profile page in the Edison platform”
   • [claimed-docs] “task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…”
   • [claimed-docs] “task_response = client.run_tasks_until_done(task_data)”

   proof ↗⚑ flag

3. ai-native userConnect an agent via an official MCP server

   weight 3 · round drawn
   Consensus—none0/10

   Consensus is not an agent product itself, so the MCP-server axis applies as an ecosystem/API capability, but evidence only shows a REST API and llms.txt file — no mention of an official MCP server for connecting agents. missing for 10: any documented MCP server endpoint, MCP spec compliance, or third-party confirmation of MCP support.

   • [claimed-docs] “Connect the Consensus API within your project to seamlessly integrate up-to-date peer-reviewed citations into your own custom workflow.”
   • [claimed-docs] “Save your team hours of manual discovery research and run automated searches with our API to quickly and easily find the most relevant and r…”
   • [probe] “PROBE llms.txt: HTTP 200 at https://consensus.app/llms.txt # Consensus > Consensus is an AI-powered scientific search engine that finds, ra…”

   proof ↗⚑ flag

   FutureHouse Platform—none0/10

   The evidence pack only shows a Python client (edison-client) for calling FutureHouse agents via API key, plus probes confirming no OpenAPI/MCP-related endpoints were found; there is no mention of an official MCP server for connecting agents.

   • [claimed-docs] “uv pip install edison-client”
   • [probe] “PROBE openapi: all candidate paths 404 (https://futurehouse.gitbook.io/openapi.json, https://futurehouse.gitbook.io/swagger.json, https://fu…”

   proof ↗⚑ flag

4. ai-native userUse an official CLI

   weight 2 · round drawn
   Consensus—none0/10

   The evidence pack documents a REST API and an MCP server (consensus-docs-16) but no official command-line interface is mentioned anywhere in the docs or probes. Missing for 10: any mention of a CLI tool, CLI installation instructions, or CLI command reference.

   • [claimed-docs] “a REST API and an MCP server expose the same retrieval and synthesis surface that powers the web app”

   proof ↗⚑ flag

   FutureHouse Platform—none0/10

   Evidence shows only a Python client library (edison-client, installed via pip, used programmatically with client.run_tasks_until_done) rather than a command-line interface; no CLI tool, command syntax, or terminal usage is documented anywhere in the pack.

   • [claimed-docs] “uv pip install edison-client”
   • [claimed-docs] “task_response = client.run_tasks_until_done(task_data)”

   proof ↗⚑ flag

5. ai-native userDrive the product through a documented public API

   weight 3 · round to FutureHouse Platform
   Consensus~partialprobed5/10

   Consensus advertises a documented API for integrating citations and running automated searches into custom workflows, and its site provides an llms.txt for AI-agent discovery, showing basic public-API and agent-friendliness. However, the evidence pack only shows marketing/landing pages, not actual API reference documentation, authentication, endpoints, or example requests/responses, and there's no independent or hands-on corroboration that the API works as described. Missing for 10: full API reference/spec details, code/SDK examples, and independent verification of API usage.

   • [claimed-docs] “Connect the Consensus API within your project to seamlessly integrate up-to-date peer-reviewed citations into your own custom workflow.”
   • [claimed-docs] “Save your team hours of manual discovery research and run automated searches with our API to quickly and easily find the most relevant and r…”
   • [probe] “PROBE llms.txt: HTTP 200 at https://consensus.app/llms.txt # Consensus > Consensus is an AI-powered scientific search engine that finds, ra…”

   proof ↗⚑ flag

   FutureHouse Platform✓fullprobed7/10

   Docs describe an installable Python client (edison-client) with API-key/token auth and a documented workflow (client.run_tasks_until_done) to submit tasks like literature review, matching a genuine public API for driving the platform programmatically. Missing for 10: no OpenAPI/swagger spec found (404s), and no independent/hands-on corroboration beyond first-party docs.

   • [claimed-docs] “uv pip install edison-client”
   • [claimed-docs] “Authentication is done by providing an API key, which can be obtained directly from your profile page in the Edison platform”
   • [claimed-docs] “Under "API Tokens", click "Create New Token."”
   • [claimed-docs] “task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…”
   • [claimed-docs] “task_response = client.run_tasks_until_done(task_data)”
   • [probe] “PROBE openapi: all candidate paths 404 (https://futurehouse.gitbook.io/openapi.json, https://futurehouse.gitbook.io/swagger.json, https://fu…”

   proof ↗⚑ flag

6. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round drawn
   Consensus—none0/10

   Evidence shows an API and MCP server exist, but there is no mention of scoped or least-privilege API keys, permission scopes, or credential management for agents — just generic API access. missing for 10: scoped/least-privilege credential issuance, API key permission controls, agent-specific auth documentation.

   • [claimed-docs] “Connect the Consensus API within your project to seamlessly integrate up-to-date peer-reviewed citations into your own custom workflow.”
   • [claimed-docs] “Save your team hours of manual discovery research and run automated searches with our API to quickly and easily find the most relevant and r…”
   • [claimed-docs] “a REST API and an MCP server expose the same retrieval and synthesis surface that powers the web app”

   proof ↗⚑ flag

   FutureHouse Platform—none0/10

   Evidence shows only a single, account-wide API token creation flow with no mention of scopes, permissions, or least-privilege controls for agents; no evidence of scoped or restricted credential issuance.

   • [claimed-docs] “Authentication is done by providing an API key, which can be obtained directly from your profile page in the Edison platform”
   • [claimed-docs] “Under "API Tokens", click "Create New Token."”
   • [claimed-docs] “Under "API Tokens," click "Create New Token."”

   proof ↗⚑ flag

7. ai-native userBuild against official SDKs

   weight 2 · round to FutureHouse Platform
   Consensus—none0/10

   Consensus documents an API for integration (consensus-docs-1, consensus-docs-15) but no evidence pack item mentions official SDKs (Python, JS, etc.) or client libraries for AI-native development — only the raw API and llms.txt discovery file are shown.

   • [claimed-docs] “Connect the Consensus API within your project to seamlessly integrate up-to-date peer-reviewed citations into your own custom workflow.”
   • [claimed-docs] “Save your team hours of manual discovery research and run automated searches with our API to quickly and easily find the most relevant and r…”
   • [probe] “PROBE llms.txt: HTTP 200 at https://consensus.app/llms.txt # Consensus > Consensus is an AI-powered scientific search engine that finds, ra…”

   proof ↗⚑ flag

   FutureHouse Platform~partialprobed6/10

   There is a documented official Python client (edison-client) with install, auth via API key, and task submission/run examples, showing a real SDK path for programmatic/agentic use. However, no OpenAPI spec was found (all probed paths 404), and the docs page itself returned 404 on a direct fetch, suggesting incomplete or unstable documentation coverage. Missing for 10: verified OpenAPI/REST spec, broader SDK language coverage, independent developer corroboration of SDK reliability.

   • [claimed-docs] “uv pip install edison-client”
   • [claimed-docs] “Authentication is done by providing an API key, which can be obtained directly from your profile page in the Edison platform”
   • [claimed-docs] “Under "API Tokens", click "Create New Token."”
   • [claimed-docs] “task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…”
   • [claimed-docs] “task_response = client.run_tasks_until_done(task_data)”
   • [probe] “PROBE openapi: all candidate paths 404 (https://futurehouse.gitbook.io/openapi.json, https://futurehouse.gitbook.io/swagger.json, https://fu…”
   • [probe] “PROBE docs-md: HTTP 200 at https://futurehouse.gitbook.io/futurehouse-cookbook.md # Page Not Found The URL `futurehouse-cookbook` does not …”

   proof ↗⚑ flag

8. ai-native userSubscribe to events via webhooks

   weight 2 · round drawn
   Consensus—none0/10

   No evidence anywhere in the pack mentions webhooks or event subscriptions; Consensus's API/MCP surface is described only as REST retrieval/synthesis, not event-driven push notifications.

   ⚑ flag

   FutureHouse Platform—none0/10

   No mention of webhooks, event subscriptions, or callback mechanisms anywhere in the docs; the client is a polling/run-tasks style API and OpenAPI probe returned 404s, giving no evidence of webhook support.

   • [claimed-docs] “task_response = client.run_tasks_until_done(task_data)”
   • [probe] “PROBE openapi: all candidate paths 404 (https://futurehouse.gitbook.io/openapi.json, https://futurehouse.gitbook.io/swagger.json, https://fu…”

   proof ↗⚑ flag

Agentic features

1. ai-native userGet AI-generated insights and suggestions from my data inside the product

   weight 2 · round to Consensus
   Consensus✓fullclaimed8/10

   Consensus generates AI-driven synthesis, summaries, the Consensus Meter, PICO extraction, and literature review synthesis directly from the papers in its corpus/library, with citations tracing insights back to sources. This is core native functionality (not a bolt-on), covering search, synthesis, and structured insight generation. Missing for 10: independent/hands-on third-party verification of insight quality beyond vendor docs.

   • [claimed-docs] “It searches over 200 million academic papers and uses language models to help you find, understand, and synthesize the literature faster.”
   • [claimed-docs] “Every response includes citations, so you can trace each insight back to the original source.”
   • [claimed-docs] “The Consensus Meter is a visual aggregator that, for a yes/no/possibly question, classifies each relevant paper as supporting, refuting, or …”
   • [claimed-docs] “The Consensus Meter is a visual aggregator that, for a yes/no/possibly question, classifies each relevant paper as supporting, refuting, or …”
   • [claimed-docs] “Extracted population, intervention, comparator, and outcome (PICO) where applicable”
   • [claimed-docs] “Consensus is an AI-powered research engine built to speed up literature reviews. Search, screen, extract, and synthesize evidence faster—whi…”
   • [claimed-docs] “The Consensus Library brings your entire research library into one searchable, AI-powered workspace.”

   proof ↗⚑ flag

   FutureHouse Platform✓fullprobed7/10

   The platform's core agents (Kosmos, Precedent, Molecules) are documented as autonomously generating insights, hypotheses, and cited reports directly from user-supplied data and literature, matching the story closely (e.g., Kosmos 'autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report'). Missing for 10: independent/hands-on corroboration beyond vendor docs, and some doc links returned 404 in probes, weakening confidence in documentation completeness.

   • [claimed-docs] “Kosmos autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report.”
   • [claimed-docs] “Given a research objective and one or more datasets, Kosmos autonomously reads literature, writes and executes analysis code, generates hypo…”
   • [claimed-docs] “You can ask it a complex scientific question and get a high-accuracy, cited response, or task it with a deep literature review synthesizing …”
   • [claimed-docs] “It turns raw data into detailed analyses, statistical results, and publication-ready figures.”
   • [claimed-docs] “Precedent determines whether a research idea has been tried before. It searches across fields to assess novelty and identify gaps”
   • [claimed-docs] “Every conclusion is fully auditable. You can trace any finding back to the specific code or literature passage that produced it.”
   • [probe] “PROBE docs-md: HTTP 200 at https://futurehouse.gitbook.io/futurehouse-cookbook.md # Page Not Found The URL `futurehouse-cookbook` does not …”

   proof ↗⚑ flag

2. ai-native userSet up automations that run autonomously in the background

   weight 2 · round drawn
   Consensus—none0/10

   Consensus is a research search/synthesis engine with an API and MCP server for on-demand retrieval, but there is no evidence of scheduled or event-triggered automations that run autonomously in the background without user invocation. Missing for 10: any scheduling/trigger mechanism, background job execution, or autonomous recurring workflow capability.

   ⚑ flag

   FutureHouse Platform—none0/10

   The evidence describes an API client for submitting tasks/queries and autonomous agents that complete a single task end-to-end, but there is no mention of scheduling, triggers, or background automations that run without explicit invocation. missing for 10: scheduling/cron mechanism, event-driven triggers, persistent background job management, and any docs describing recurring or unattended automation setup.

   • [claimed-docs] “task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…”
   • [claimed-docs] “task_response = client.run_tasks_until_done(task_data)”
   • [claimed-docs] “Kosmos autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report.”

   proof ↗⚑ flag

3. ai-native userDelegate tasks to a built-in AI assistant inside the product

   weight 3 · round to FutureHouse Platform
   Consensus✓fullprobed7/10

   Consensus ships a built-in "Research Agent" that chains citation crawling, DOI lookup, author search and similar-paper discovery on top of its search engine, and its core AI assistant performs search, screen, extract, and synthesize workflows with cited answers — this is essentially delegating research tasks to an in-product AI assistant. missing for 10: independent/hands-on validation of the agent's autonomy and reliability, and more detail on the scope/limits of delegable tasks beyond literature discovery.

   • [claimed-docs] “Citation crawling, DOI lookup, author search, similar papers, and more - chained together on top of the worlds best academic search engine.”
   • [claimed-docs] “Search, screen, extract, and synthesize evidence faster—while keeping full transparency and scholarly rigor.”
   • [claimed-docs] “Consensus is an AI-powered research engine built to speed up literature reviews. Search, screen, extract, and synthesize evidence faster—whi…”
   • [claimed-docs] “It searches over 200 million academic papers and uses language models to help you find, understand, and synthesize the literature faster.”
   • [claimed-docs] “Every response includes citations, so you can trace each insight back to the original source.”
   • [probe] “PROBE llms.txt: HTTP 200 at https://consensus.app/llms.txt # Consensus > Consensus is an AI-powered scientific search engine that finds, ra…”

   proof ↗⚑ flag

   FutureHouse Platform✓fullclaimed8/10

   The platform ships built-in autonomous agents (Kosmos, Precedent, Molecules, literature review agent) that users can delegate research tasks to via API/client, with docs describing task submission and autonomous execution producing cited reports. Missing for 10: independent/hands-on corroboration beyond vendor docs, and clearer in-product (non-API) delegation UI evidence.

   • [claimed-docs] “task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…”
   • [claimed-docs] “Kosmos autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report.”
   • [claimed-docs] “You can ask it a complex scientific question and get a high-accuracy, cited response, or task it with a deep literature review synthesizing …”
   • [claimed-docs] “Precedent determines whether a research idea has been tried before. It searches across fields to assess novelty and identify gaps”
   • [claimed-docs] “Molecules is a chemistry-focused agent for molecular design and analysis.”
   • [claimed-docs] “task_response = client.run_tasks_until_done(task_data)”
   • [claimed-docs] “Given a research objective and one or more datasets, Kosmos autonomously reads literature, writes and executes analysis code, generates hypo…”

   proof ↗⚑ flag

4. ai-native userOperate the product with natural-language commands

   weight 2 · round to Consensus
   Consensus✓fullclaimed7/10

   Consensus's core interaction model is natural-language research queries (search, synthesize, Consensus Meter for yes/no questions) rather than rigid query syntax, and it exposes this same NL-driven retrieval/synthesis surface via an MCP server and REST API for programmatic/agentic use. Missing for 10: independent hands-on evidence of natural-language command execution quality, and no detailed example transcripts showing complex multi-step NL commands being interpreted.

   • [claimed-docs] “It searches over 200 million academic papers and uses language models to help you find, understand, and synthesize the literature faster.”
   • [claimed-docs] “a REST API and an MCP server expose the same retrieval and synthesis surface that powers the web app”
   • [claimed-docs] “The Consensus Meter is a visual aggregator that, for a yes/no/possibly question, classifies each relevant paper as supporting, refuting, or …”
   • [claimed-docs] “The Consensus Meter is a visual aggregator that, for a yes/no/possibly question, classifies each relevant paper as supporting, refuting, or …”
   • [claimed-docs] “Think of Consensus as an AI-native alternative to Google Scholar with a more-refined corpus.”

   proof ↗⚑ flag

   FutureHouse Platform~partialclaimed7/10

   The platform's core interaction model is natural-language queries (e.g., the query string "Which neglected diseases had a treatment developed by artificial intelligence?") passed to autonomous agents like Kosmos, Precedent, and Molecules, and docs explicitly say users can 'ask it a complex scientific question' or 'task it with a deep literature review.' However, the demonstrated flow is via a Python client (task_data dict + run_tasks_until_done), not a conversational chat UI, so it's NL-as-input-parameter rather than full conversational natural-language operation of the whole platform (e.g., no evidence of multi-turn dialogue or a chat interface). Missing for 10: evidence of a conversational/chat UI, multi-turn NL interaction, or NL control over platform features beyond the initial query.

   • [claimed-docs] “task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…”
   • [claimed-docs] “Kosmos autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report.”
   • [claimed-docs] “You can ask it a complex scientific question and get a high-accuracy, cited response, or task it with a deep literature review synthesizing …”
   • [claimed-docs] “Given a research objective and one or more datasets, Kosmos autonomously reads literature, writes and executes analysis code, generates hypo…”
   • [claimed-docs] “task it with a deep literature review synthesizing conflicting evidence across hundreds of papers”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   Consensus—none0/10

   Evidence confirms Consensus offers a REST API and MCP server (consensus-docs-1, consensus-docs-15, consensus-docs-16), but there is no mention of an interactive API reference, sandbox, or runnable code examples anywhere in the pack.

   • [claimed-docs] “Connect the Consensus API within your project to seamlessly integrate up-to-date peer-reviewed citations into your own custom workflow.”
   • [claimed-docs] “Save your team hours of manual discovery research and run automated searches with our API to quickly and easily find the most relevant and r…”
   • [claimed-docs] “a REST API and an MCP server expose the same retrieval and synthesis surface that powers the web app”

   proof ↗⚑ flag

   FutureHouse Platform—none0/10

   Evidence shows only a quickstart guide with basic client code snippets, not an interactive API reference with runnable examples; probes for OpenAPI/swagger specs and doc endpoints all returned 404s, indicating no interactive reference exists.

   • [probe] “PROBE openapi: all candidate paths 404 (https://futurehouse.gitbook.io/openapi.json, https://futurehouse.gitbook.io/swagger.json, https://fu…”
   • [probe] “PROBE docs-md: HTTP 200 at https://futurehouse.gitbook.io/futurehouse-cookbook.md # Page Not Found The URL `futurehouse-cookbook` does not …”
   • [claimed-docs] “uv pip install edison-client”
   • [claimed-docs] “task_response = client.run_tasks_until_done(task_data)”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   Consensus—none0/10

   Consensus documents a REST API and MCP server (consensus-docs-15, consensus-docs-16) but no evidence pack item mentions an OpenAPI spec, Swagger file, or any downloadable machine-readable API schema; the llms.txt probe returns a plain-text description, not an API spec. missing for 10: OpenAPI/Swagger file, machine-readable schema download link, independent confirmation of spec availability.

   • [claimed-docs] “Save your team hours of manual discovery research and run automated searches with our API to quickly and easily find the most relevant and r…”
   • [claimed-docs] “a REST API and an MCP server expose the same retrieval and synthesis surface that powers the web app”
   • [probe] “PROBE llms.txt: HTTP 200 at https://consensus.app/llms.txt # Consensus > Consensus is an AI-powered scientific search engine that finds, ra…”

   proof ↗⚑ flag

   FutureHouse Platform—none0/10

   Probes for OpenAPI/swagger spec paths all returned 404, and no docs reference a machine-readable API spec; only a Python client library and manual REST auth flow are documented. Missing for 10: any OpenAPI/Swagger endpoint or downloadable spec file, any documentation mentioning API schema export.

   • [probe] “PROBE openapi: all candidate paths 404 (https://futurehouse.gitbook.io/openapi.json, https://futurehouse.gitbook.io/swagger.json, https://fu…”

   proof ↗⚑ flag

3. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   Consensus—none0/10

   There's an API and MCP server mentioned, but no evidence of API versioning scheme or a documented deprecation policy anywhere in the pack. missing for 10: versioning scheme documentation, deprecation policy, changelog/migration guides.

   • [claimed-docs] “a REST API and an MCP server expose the same retrieval and synthesis surface that powers the web app”
   • [claimed-docs] “Save your team hours of manual discovery research and run automated searches with our API to quickly and easily find the most relevant and r…”

   proof ↗⚑ flag

   FutureHouse Platform—none0/10

   No evidence of API versioning scheme, changelog, or deprecation policy anywhere in the docs; OpenAPI spec probes all 404 and no versioning documentation is present. missing for 10: versioned API endpoints, changelog/release notes, documented deprecation policy or timelines.

   • [probe] “PROBE openapi: all candidate paths 404 (https://futurehouse.gitbook.io/openapi.json, https://futurehouse.gitbook.io/swagger.json, https://fu…”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round to Consensus
   Consensus~partialclaimed6/10

   Docs show bulk-style capabilities: one-click import of thousands of papers into a library, an API/MCP server for automated bulk searches, and Deep Searches across many studies — supporting bulk operations for an AI-native/automation persona. missing for 10: independent/hands-on verification of bulk API throughput or rate limits, explicit batch-processing endpoints (e.g., bulk extract/export across many items in one call), and any third-party confirmation of scale performance.

   • [claimed-docs] “Import thousands of papers in one click - then search, find gaps, and put your collection to work.”
   • [claimed-docs] “Turn your library into a research engine. Import thousands of papers in one click - then search, find gaps, and put your collection to work.”
   • [claimed-docs] “Save your team hours of manual discovery research and run automated searches with our API to quickly and easily find the most relevant and r…”
   • [claimed-docs] “a REST API and an MCP server expose the same retrieval and synthesis surface that powers the web app”
   • [claimed-docs] “Deep Searches (more comprehensive Lit Reviews across many studies)”
   • [claimed-docs] “Import from Zotero ... or import from BibTex, PDF, or RIS”

   proof ↗⚑ flag

   FutureHouse Platform—none0/10

   The docs show only single-task submission via task_data/run_tasks_until_done, with no mention of batch endpoints, bulk task lists, or parallel multi-item processing APIs; missing for 10: batch/bulk submission API, documented loop or list-based task queuing, evidence of concurrent multi-item execution.

   • [claimed-docs] “task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…”
   • [claimed-docs] “task_response = client.run_tasks_until_done(task_data)”

   proof ↗⚑ flag

2. ai-native userSchedule recurring jobs or workflows

   weight 2 · round drawn
   Consensus—none0/10

   No evidence in the pack mentions scheduling, recurring jobs, alerts, or automated re-running of searches/workflows over time; the API and MCP server are described as on-demand retrieval/synthesis interfaces, not schedulable automation. missing for 10: any scheduling/cron feature, recurring alert or saved-search re-run capability, or workflow automation trigger.

   • [claimed-docs] “Save your team hours of manual discovery research and run automated searches with our API to quickly and easily find the most relevant and r…”
   • [claimed-docs] “a REST API and an MCP server expose the same retrieval and synthesis surface that powers the web app”

   proof ↗⚑ flag

   FutureHouse Platform—none0/10

   Evidence covers running one-off tasks via the client (run_tasks_until_done) but there is no mention of scheduling, cron-like recurrence, or persistent workflow automation anywhere in the docs or probes.

   • [claimed-docs] “task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…”
   • [claimed-docs] “task_response = client.run_tasks_until_done(task_data)”

   proof ↗⚑ flag

Sharing

1. analystShare a research session or report with collaborators who can view or build on it

   weight 2 · round drawn
   Consensus—none0/10

   No evidence pack items mention sharing sessions, reports, collaborators, team accounts, or collaborative viewing/editing features—only individual research, library import, and API/agent capabilities are documented. missing for 10: any mention of sharing links, collaborator invites, team workspaces, or comment/build-on functionality.

   ⚑ flag

   FutureHouse Platform—none0/10

   No evidence in the pack mentions sharing sessions, reports, or collaborative access controls; documentation focuses on API usage, agents, and task execution, not collaboration features. Missing for 10: any mention of sharing/exporting reports, collaborator invites, or team/workspace features.

   ⚑ flag

Alerts

1. researcherSet up standing searches or alerts that surface new relevant sources as they appear

   weight 1 · round drawn
   Consensus—none0/10

   No evidence of standing searches, saved-search alerts, or notification features when new relevant papers appear; the evidence pack covers search, library import, citation graph, API/MCP retrieval, and literature review synthesis but nothing about recurring/alert-based monitoring of new sources.

   ⚑ flag

   FutureHouse Platform—none0/10

   The evidence describes on-demand literature search, review, and analysis agents (Kosmos, Precedent, etc.) invoked via API calls, but nothing describes standing searches, saved alerts, or recurring notifications when new relevant sources appear.

   ⚑ flag

Corpus

1. researcherUpload my own PDFs or corpus and have the agent research over them

   weight 2 · round to Consensus
   Consensus~partialclaimed6/10

   Consensus's Library feature explicitly supports importing PDFs, BibTeX, RIS, and Zotero corpora and turns them into a 'searchable, AI-powered workspace' for finding gaps and using the collection, which matches the story's upload+research intent. However, evidence doesn't detail how deeply the AI synthesis/agent features (Meter, PICO extraction, literature review synthesis) operate specifically over a user's uploaded corpus versus the general 200M-paper index. Missing for 10: explicit documentation of agent-style synthesis/Q&A running directly over an uploaded private corpus, and independent/hands-on confirmation of this workflow.

   • [claimed-docs] “Import thousands of papers in one click - then search, find gaps, and put your collection to work.”
   • [claimed-docs] “Turn your library into a research engine. Import thousands of papers in one click - then search, find gaps, and put your collection to work.”
   • [claimed-docs] “Import from Zotero ... or import from BibTex, PDF, or RIS”
   • [claimed-docs] “The Consensus Library brings your entire research library into one searchable, AI-powered workspace.”
   • [claimed-docs] “Reference managers are great at saving papers — not so great at helping you use them. The Consensus Library brings your entire research libr…”

   proof ↗⚑ flag

   FutureHouse Platform—none0/10

   The docs describe FutureHouse's agents (Kosmos, literature review, Precedent) searching a fixed 175M+ paper corpus and analyzing 'datasets' for experiments, but nowhere describe a mechanism for a researcher to upload their own PDFs or personal corpus for the agent to search/reason over. Without evidence of a file-upload or custom-corpus ingestion feature, this capability is not shown.

   • [claimed-docs] “Given a research objective and one or more datasets, Kosmos autonomously reads literature, writes and executes analysis code, generates hypo…”
   • [claimed-docs] “It accesses 175M+ papers, trials, and patents with native understanding of citation graphs, journal quality, and clinical trial data.”
   • [claimed-docs] “specializes in processing complex experimental data, including flow cytometry, RNA-seq, and other biological datasets”
   • [claimed-docs] “task it with a deep literature review synthesizing conflicting evidence across hundreds of papers”

   proof ↗⚑ flag

Reviews

1. researcherRun a systematic screening and extraction workflow across many papers with consistent criteria

   weight 2 · round to Consensus
   Consensus~partialclaimed6/10

   Consensus offers literature-review features (search, screen, extract, synthesize per docs-8/12), library import at scale, PICO extraction, and filters by study type/year/discipline that support systematic screening with consistent criteria. However, there is no evidence of documented inter-rater reliability, exportable screening decision logs, or PRISMA-style workflow tracking that a systematic review would require. missing for 10: evidence of structured screening criteria configuration/audit trail, PRISMA-compliant workflow support, independent validation of extraction consistency across large paper sets.

   • [claimed-docs] “Search, screen, extract, and synthesize evidence faster—while keeping full transparency and scholarly rigor.”
   • [claimed-docs] “Consensus is an AI-powered research engine built to speed up literature reviews. Search, screen, extract, and synthesize evidence faster—whi…”
   • [claimed-docs] “Turn your library into a research engine. Import thousands of papers in one click - then search, find gaps, and put your collection to work.”
   • [claimed-docs] “Filters allow narrowing by study type (RCT, meta-analysis, systematic review, observational), publication year, journal, open-access status,…”
   • [claimed-docs] “Extracted population, intervention, comparator, and outcome (PICO) where applicable”
   • [claimed-docs] “Deep Searches (more comprehensive Lit Reviews across many studies)”

   proof ↗⚑ flag

   FutureHouse Platform~partialprobed5/10

   The API client supports programmatic task submission (e.g., JobNames.LITERATURE) and batch execution via run_tasks_until_done, and the docs describe a literature agent that can synthesize evidence across hundreds of papers with citations, which could be scripted for repeated, consistent queries. However, there is no explicit documented workflow for structured screening/extraction with defined inclusion/exclusion criteria across a large corpus (e.g., systematic review style tables), and the cookbook page itself returns 404 on the probe, reducing confidence in the full pipeline. missing for 10: explicit systematic-review/extraction workflow docs, example of consistent-criteria batch screening, working cookbook page confirming the API details.

   • [claimed-docs] “task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…”
   • [claimed-docs] “task_response = client.run_tasks_until_done(task_data)”
   • [claimed-docs] “task it with a deep literature review synthesizing conflicting evidence across hundreds of papers”
   • [claimed-docs] “You can ask it a complex scientific question and get a high-accuracy, cited response, or task it with a deep literature review synthesizing …”
   • [probe] “PROBE docs-md: HTTP 200 at https://futurehouse.gitbook.io/futurehouse-cookbook.md # Page Not Found The URL `futurehouse-cookbook` does not …”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round to Consensus
   Consensus~partialclaimed6/10

   The API/MCP server is documented to expose 'the same retrieval and synthesis surface that powers the web app' (consensus-docs-16), and supports automated search (consensus-docs-15), suggesting broad parity for core search/synthesis. However, UI-specific workflows like Library import/reference management (Zotero/BibTeX/RIS import), Citation Graph, and Consensus Meter visualizations are not explicitly confirmed as API-accessible endpoints. Missing for 10: explicit API documentation confirming library management, citation graph, and meter features are callable via API, plus independent/hands-on verification of claimed parity.

   • [claimed-docs] “a REST API and an MCP server expose the same retrieval and synthesis surface that powers the web app”
   • [claimed-docs] “Save your team hours of manual discovery research and run automated searches with our API to quickly and easily find the most relevant and r…”
   • [claimed-docs] “The Consensus Library brings your entire research library into one searchable, AI-powered workspace.”
   • [claimed-docs] “Import from Zotero ... or import from BibTex, PDF, or RIS”
   • [claimed-docs] “The Consensus Citation Graph turns a single seed paper into a complete map of the work that built it, the work it inspired, and the studies …”
   • [claimed-docs] “The Consensus Meter is a visual aggregator that, for a yes/no/possibly question, classifies each relevant paper as supporting, refuting, or …”

   proof ↗⚑ flag

   FutureHouse Platform~partialprobed5/10

   The edison-client library and quickstart docs show that core agent tasks (e.g., JobNames.LITERATURE) can be invoked via API with authentication and task submission (docs-1–4,12), and the cookbook describes multiple agents (Kosmos, Precedent, Molecules) that presumably map to API job types, suggesting broad functional overlap with the UI. However, there is no explicit documentation stating full feature parity between UI and API, no OpenAPI/swagger spec discoverable (probe-3 shows 404s across all candidate paths), and no confirmation that all UI-exposed capabilities (e.g., account management, dataset uploads, report viewing) are also API-accessible. missing for 10: explicit parity statement, discoverable OpenAPI schema, evidence covering non-task UI features (billing, dataset management, report exports) via API.

   • [claimed-docs] “task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…”
   • [claimed-docs] “task_response = client.run_tasks_until_done(task_data)”
   • [claimed-docs] “Kosmos autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report.”
   • [claimed-docs] “Precedent determines whether a research idea has been tried before. It searches across fields to assess novelty and identify gaps”
   • [claimed-docs] “Molecules is a chemistry-focused agent for molecular design and analysis.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://futurehouse.gitbook.io/openapi.json, https://futurehouse.gitbook.io/swagger.json, https://fu…”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round drawn
   Consensus—none0/10

   Evidence shows only import capabilities (Zotero, BibTeX, PDF, RIS) into the Consensus Library, with no mention of exporting a user's library, annotations, or account data back out in open formats. Data portability/export is a fair axis for a reference-manager-style product, but no evidence supports it.

   • [claimed-docs] “Import from Zotero ... or import from BibTex, PDF, or RIS”
   • [claimed-docs] “Import from Zotero”
   • [claimed-docs] “The Consensus Library brings your entire research library into one searchable, AI-powered workspace.”
   • [claimed-docs] “Turn your library into a research engine. Import thousands of papers in one click - then search, find gaps, and put your collection to work.”

   proof ↗⚑ flag

   FutureHouse Platform—none0/10

   No evidence of any data export capability, open-format export, or account portability/deletion feature; documentation covers agent/task usage and API access but nothing about exporting user data or leaving with it.

   ⚑ flag

Pricing

1. researcherTry the product meaningfully on a free tier or trial

   weight 1 · round to FutureHouse Platform
   Consensus—none0/10

   The evidence pack references a pricing page (consensus-docs-17) but only quotes a single line about 'Deep Searches' feature tiering; there is no description of a free tier, trial period, usage caps, or sign-up-free access that a researcher could evaluate. No first-party or independent evidence confirms Consensus offers a meaningful free/trial experience.

   • [claimed-docs] “Deep Searches (more comprehensive Lit Reviews across many studies)”

   proof ↗⚑ flag

   FutureHouse Platform✓fullclaimed7/10

   Docs explicitly state Edison maintains a generous free tier for academics, with paid plans only needed for higher rate limits/additional features, and the quickstart shows a full API key + client workflow to run real tasks (literature review job). Missing for 10: independent/hands-on confirmation of free tier limits and no clarity on exact quota/duration restrictions.

   • [claimed-docs] “Edison maintains a generous free tier for academics. Researchers who need higher rate limits or additional features can subscribe to paid pl…”
   • [claimed-docs] “Researchers who need higher rate limits or additional features can subscribe to paid plans.”
   • [claimed-docs] “Authentication is done by providing an API key, which can be obtained directly from your profile page in the Edison platform”
   • [claimed-docs] “Under "API Tokens", click "Create New Token."”
   • [claimed-docs] “task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…”
   • [claimed-docs] “task_response = client.run_tasks_until_done(task_data)”

   proof ↗⚑ flag

2. researcherUnderstand plan pricing and usage limits before committing

   weight 2 · round to FutureHouse Platform
   Consensus—none0/10

   No evidence pack items mention pricing plans, tiers, free/paid limits, or usage quotas — the pack is entirely about product features (citation graph, library, API capabilities). Absence of any pricing/limits documentation for an applicable axis yields none.

   ⚑ flag

   FutureHouse Platform~partialclaimed4/10

   Docs confirm a free tier for academics and paid plans for higher rate limits/features, but no concrete pricing figures, tier breakdowns, or specific usage limit numbers are provided anywhere in the pack. missing for 10: actual price points, detailed rate-limit thresholds, feature comparison table across tiers, billing/usage dashboard evidence.

   • [claimed-docs] “Researchers who need higher rate limits or additional features can subscribe to paid plans.”
   • [claimed-docs] “Edison maintains a generous free tier for academics. Researchers who need higher rate limits or additional features can subscribe to paid pl…”

   proof ↗⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round drawn
   Consensus—none0/10

   No evidence in the pack mentions data residency, regional storage options, or any data-location controls for Consensus; all evidence concerns search, citation, and library features. Missing for 10: any mention of region selection, data residency policy, or storage location controls.

   ⚑ flag

   FutureHouse Platform—none0/10

   No evidence in the pack addresses data residency, regional storage options, or geographic control over where data is processed/stored; documentation only covers API usage, agents, and pricing tiers.

   ⚑ flag

2. ai-native userPrevent my data from being used to train AI models

   weight 3 · round drawn
   Consensus—none0/10

   No evidence pack item addresses data-training opt-out, privacy controls, or AI training data policies for Consensus; all citations concern search, library, and API features. missing for 10: any privacy policy statement, opt-out mechanism, or data usage/training disclosure.

   ⚑ flag

   FutureHouse Platform—none0/10

   No evidence in the pack addresses data usage for AI training opt-out, data privacy controls, or any training-data policy; the docs focus entirely on product features and API usage.

   ⚑ flag

3. ai-native userControl data retention and deletion

   weight 2 · round drawn
   Consensus—none0/10

   No evidence in the pack addresses data retention policies, deletion controls, or privacy settings for user data/library content; all citations focus on search, citation, library, and API features. Missing for 10: any documentation on data retention windows, user-initiated deletion, export/erasure workflows, or privacy policy specifics.

   ⚑ flag

   FutureHouse Platform—none0/10

   No evidence in the pack addresses data retention policies, deletion controls, or privacy/data lifecycle management; documentation focuses on client setup, agents, and use cases only.

   ⚑ flag

4. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   Consensus—none0/10

   No evidence in the pack mentions telemetry, usage tracking, or any opt-out/privacy settings for Consensus; all citations are about search, citation, and library features unrelated to telemetry controls.

   ⚑ flag

   FutureHouse Platform—none0/10

   No evidence in the pack mentions telemetry, usage tracking, analytics collection, or an opt-out mechanism anywhere in the FutureHouse docs or probes; the documentation focuses on client setup and agent features only.

   ⚑ flag

Reports

1. researcherExport results to common formats, including documents, spreadsheets, and reference-manager files

   weight 1 · round drawn
   Consensus—none0/10

   Evidence only documents importing papers into Consensus (from Zotero, BibTeX, PDF, RIS) but contains no mention of exporting results to documents, spreadsheets, or reference-manager formats. Missing for 10: any export-to-Word/PDF, export-to-CSV/spreadsheet, or export-to-Zotero/EndNote/BibTeX functionality.

   • [claimed-docs] “Import from Zotero ... or import from BibTex, PDF, or RIS”
   • [claimed-docs] “Reference managers are great at saving papers — not so great at helping you use them. The Consensus Library brings your entire research libr…”
   • [claimed-docs] “Import from Zotero”

   proof ↗⚑ flag

   FutureHouse Platform—none0/10

   The evidence describes cited reports and analyses produced by Kosmos/agents but nowhere mentions exporting to document, spreadsheet, or reference-manager formats (e.g., PDF/DOCX, CSV/XLSX, BibTeX/RIS); the docs-cookbook page itself is even reported as not found in the probe.

   • [claimed-docs] “Kosmos autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report.”
   • [claimed-docs] “Every conclusion is fully auditable. You can trace any finding back to the specific code or literature passage that produced it.”
   • [claimed-docs] “It turns raw data into detailed analyses, statistical results, and publication-ready figures.”
   • [probe] “PROBE docs-md: HTTP 200 at https://futurehouse.gitbook.io/futurehouse-cookbook.md # Page Not Found The URL `futurehouse-cookbook` does not …”

   proof ↗⚑ flag

2. analystGet a structured report with sections, tables, and a summary that I can share with stakeholders

   weight 3 · round to FutureHouse Platform
   Consensus~partialclaimed5/10

   Consensus offers 'Literature Review' and 'Deep Search' features that synthesize evidence across papers, extract structured fields like PICO, and provide citations—suggesting output with some structure and sourcing suitable for sharing. However, there's no explicit evidence of a polished 'report' format with distinct sections, tables, and an executive summary designed for stakeholder sharing (e.g., export to PDF/Word, formatted report templates). Missing for 10: explicit documentation of report formatting/export (sections, tables, summary), evidence of stakeholder-sharing features like PDF export or presentation-ready output, and independent confirmation of report quality.

   • [claimed-docs] “Search, screen, extract, and synthesize evidence faster—while keeping full transparency and scholarly rigor.”
   • [claimed-docs] “Consensus is an AI-powered research engine built to speed up literature reviews. Search, screen, extract, and synthesize evidence faster—whi…”
   • [claimed-docs] “Deep Searches (more comprehensive Lit Reviews across many studies)”
   • [claimed-docs] “Extracted population, intervention, comparator, and outcome (PICO) where applicable”
   • [claimed-docs] “The Consensus Meter is a visual aggregator that, for a yes/no/possibly question, classifies each relevant paper as supporting, refuting, or …”

   proof ↗⚑ flag

   FutureHouse Platform~partialclaimed6/10

   Docs describe Kosmos/agents producing a 'comprehensive cited report' with statistical results, publication-ready figures, and traceable citations, which implies structured, shareable output, but there's no explicit mention of report sections/tables format or export/sharing mechanisms for stakeholders. missing for 10: explicit description of report structure (sections/tables), export or sharing functionality, and independent confirmation of report format quality.

   • [claimed-docs] “Kosmos autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report.”
   • [claimed-docs] “Every conclusion is fully auditable. You can trace any finding back to the specific code or literature passage that produced it.”
   • [claimed-docs] “It turns raw data into detailed analyses, statistical results, and publication-ready figures.”
   • [claimed-docs] “Given a research objective and one or more datasets, Kosmos autonomously reads literature, writes and executes analysis code, generates hypo…”
   • [claimed-docs] “You can trace any finding back to the specific code or literature passage that produced it.”

   proof ↗⚑ flag

Agent runs

1. researcherPose a research question and get an autonomous multi-step investigation, not just a single-pass summary

   weight 3 · round to FutureHouse Platform
   Consensus~partialclaimed5/10

   Consensus advertises a 'Research Agent' that chains citation crawling, DOI lookup, author search, and similar-papers search on top of its search engine, plus a literature-review feature that searches, screens, extracts, and synthesizes evidence — both suggesting multi-step, not single-pass, investigation. However, evidence is limited to marketing feature pages with no walkthrough, example transcript, or independent corroboration of true autonomous multi-step reasoning over a posed question. Missing for 10: a documented end-to-end example of the agent autonomously chaining steps for a specific question, independent/hands-on verification, and detail on how far it goes without user intervention.

   • [claimed-docs] “Citation crawling, DOI lookup, author search, similar papers, and more - chained together on top of the worlds best academic search engine.”
   • [claimed-docs] “Search, screen, extract, and synthesize evidence faster—while keeping full transparency and scholarly rigor.”
   • [claimed-docs] “Consensus is an AI-powered research engine built to speed up literature reviews. Search, screen, extract, and synthesize evidence faster—whi…”

   proof ↗⚑ flag

   FutureHouse Platform✓fullclaimed8/10

   Docs describe Kosmos and other agents autonomously reading literature, writing/executing analysis code, generating hypotheses, and producing cited multi-step reports (not single-pass summaries), with auditable traceability of findings and multi-agent capabilities (Precedent, Molecules, literature review across hundreds of papers). This directly matches the story of posing a question and getting an autonomous multi-step investigation. Missing for 10: independent/hands-on verification of the multi-step autonomy claims (probe shows docs page 404'd, reducing corroboration strength) and no third-party case study confirming real-world multi-step behavior.

   • [claimed-docs] “Kosmos autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report.”
   • [claimed-docs] “Given a research objective and one or more datasets, Kosmos autonomously reads literature, writes and executes analysis code, generates hypo…”
   • [claimed-docs] “Every conclusion is fully auditable. You can trace any finding back to the specific code or literature passage that produced it.”
   • [claimed-docs] “You can ask it a complex scientific question and get a high-accuracy, cited response, or task it with a deep literature review synthesizing …”
   • [claimed-docs] “task it with a deep literature review synthesizing conflicting evidence across hundreds of papers”
   • [claimed-docs] “Precedent determines whether a research idea has been tried before. It searches across fields to assess novelty and identify gaps”

   proof ↗⚑ flag

2. analystStart a long research job that keeps working unattended and notifies me when the result is ready

   weight 2 · round to FutureHouse Platform
   Consensus—none0/10

   Evidence shows Deep Searches/Lit Reviews and a research agent chaining searches, but there is no mention of async job submission, background/unattended execution, or notification when a long-running job completes.

   ⚑ flag

   FutureHouse Platform~partialclaimed5/10

   The client SDK exposes a task submission and `run_tasks_until_done` pattern (docs-4, docs-12) plus descriptions of Kosmos autonomously running long, multi-step research (literature review, code execution, hypothesis generation) unattended (docs-5, docs-13, docs-19), supporting the 'long unattended job' half of the story. However there is no evidence of any notification mechanism (email, webhook, push) when a job completes — the only documented pattern is a client-side polling call that blocks until done, not an async notify-on-completion flow. Missing for 10: explicit notification/webhook/email-on-completion feature, evidence of true async fire-and-forget job submission separate from polling.

   • [claimed-docs] “task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…”
   • [claimed-docs] “task_response = client.run_tasks_until_done(task_data)”
   • [claimed-docs] “Kosmos autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report.”
   • [claimed-docs] “Given a research objective and one or more datasets, Kosmos autonomously reads literature, writes and executes analysis code, generates hypo…”
   • [claimed-docs] “task it with a deep literature review synthesizing conflicting evidence across hundreds of papers”

   proof ↗⚑ flag

3. researcherSteer the depth, effort, and scope of a research run before or while it executes

   weight 1 · round drawn
   Consensus—none0/10

   The evidence pack describes search, citation graph, library, and research-agent features but nowhere mentions controls for adjusting depth, effort, or scope of a research run before or during execution — no parameters, modes, or configuration options are documented.

   ⚑ flag

   FutureHouse Platform—none0/10

   Evidence shows job submission with a query and job name, plus a mid-run polling helper (run_tasks_until_done), but no documented parameters or controls for adjusting depth, effort level, or scope of a run before or during execution.

   • [claimed-docs] “task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…”
   • [claimed-docs] “task_response = client.run_tasks_until_done(task_data)”

   proof ↗⚑ flag

Citations

1. researcherSee citations for every substantive claim so I can verify it against the underlying source

   weight 3 · round to Consensus
   Consensus✓fullclaimed8/10

   Consensus documents that every AI-generated response includes citations tracing back to the original source paper, and features like the Consensus Meter classify individual papers (supporting/refuting) with traceable provenance, directly matching the researcher's need to verify claims against sources. Missing for 10: independent/hands-on verification of citation accuracy and completeness beyond vendor docs.

   • [claimed-docs] “Every response includes citations, so you can trace each insight back to the original source.”
   • [claimed-docs] “The Consensus Meter is a visual aggregator that, for a yes/no/possibly question, classifies each relevant paper as supporting, refuting, or …”
   • [claimed-docs] “The Consensus Meter is a visual aggregator that, for a yes/no/possibly question, classifies each relevant paper as supporting, refuting, or …”
   • [claimed-docs] “It searches over 200 million academic papers and uses language models to help you find, understand, and synthesize the literature faster.”

   proof ↗⚑ flag

   FutureHouse Platform✓fullclaimed7/10

   Docs state that outputs are cited and auditable, with claims traceable back to specific literature passages or code, and that literature review/answer agents produce cited responses. This is first-party documentation only, with no independent/hands-on corroboration of citation accuracy or completeness. missing for 10: independent verification of citation accuracy, example of an actual cited report output, detail on citation format/traceability mechanism.

   • [claimed-docs] “Every conclusion is fully auditable. You can trace any finding back to the specific code or literature passage that produced it.”
   • [claimed-docs] “You can trace any finding back to the specific code or literature passage that produced it.”
   • [claimed-docs] “You can ask it a complex scientific question and get a high-accuracy, cited response, or task it with a deep literature review synthesizing …”
   • [claimed-docs] “Kosmos autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report.”
   • [claimed-docs] “Given a research objective and one or more datasets, Kosmos autonomously reads literature, writes and executes analysis code, generates hypo…”

   proof ↗⚑ flag

Corpus

1. researcherSearch scholarly literature and primary sources, not just the open web

   weight 2 · round to Consensus
   Consensus✓fullprobed9/10

   Consensus is explicitly built as a scholarly-search engine over 200M+ academic papers, including full-text and paywalled content, positioned as an AI-native alternative to Google Scholar, with citation tracing back to original sources. Missing for 10: independent third-party verification of corpus quality/coverage beyond vendor claims.

   • [claimed-docs] “Consensus analyzes the full text, including paywalled papers from major publishers, so you can find the most relevant papers.”
   • [claimed-docs] “Think of Consensus as an AI-native alternative to Google Scholar with a more-refined corpus.”
   • [claimed-docs] “It searches over 200 million academic papers and uses language models to help you find, understand, and synthesize the literature faster.”
   • [claimed-docs] “Every response includes citations, so you can trace each insight back to the original source.”
   • [probe] “PROBE llms.txt: HTTP 200 at https://consensus.app/llms.txt # Consensus > Consensus is an AI-powered scientific search engine that finds, ra…”

   proof ↗⚑ flag

   FutureHouse Platform✓fullclaimed8/10

   Docs describe agents (LITERATURE task, Kosmos, Precedent) that explicitly search scholarly literature, trials, and patents, accessing '175M+ papers, trials, and patents' with citation graph and journal quality awareness, and produce cited reports auditable back to specific literature passages. This directly matches the researcher story of searching scholarly/primary sources beyond the open web. Missing for 10: independent/hands-on corroboration of search coverage and source quality beyond vendor docs, and clarity on which specific databases/indices are used.

   • [claimed-docs] “It accesses 175M+ papers, trials, and patents with native understanding of citation graphs, journal quality, and clinical trial data.”
   • [claimed-docs] “Precedent determines whether a research idea has been tried before. It searches across fields to assess novelty and identify gaps”
   • [claimed-docs] “Precedent determines whether a research idea has been tried before. It searches across fields to assess novelty and identify gaps, helping y…”
   • [claimed-docs] “Kosmos autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report.”
   • [claimed-docs] “Every conclusion is fully auditable. You can trace any finding back to the specific code or literature passage that produced it.”
   • [claimed-docs] “task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…”

   proof ↗⚑ flag

Synthesis

1. analystSee where sources agree and disagree instead of a single unqualified answer

   weight 2 · round to Consensus
   Consensus✓fullclaimed8/10

   The Consensus Meter explicitly classifies each relevant paper as supporting, refuting, or mixed/inconclusive on a given question and displays the distribution, directly surfacing agreement/disagreement across sources rather than a single answer, and every response includes citations back to originals. Missing for 10: independent/hands-on corroboration of the Meter's accuracy and no worked example showing disagreement handling in practice.

   • [claimed-docs] “The Consensus Meter is a visual aggregator that, for a yes/no/possibly question, classifies each relevant paper as supporting, refuting, or …”
   • [claimed-docs] “The Consensus Meter is a visual aggregator that, for a yes/no/possibly question, classifies each relevant paper as supporting, refuting, or …”
   • [claimed-docs] “Every response includes citations, so you can trace each insight back to the original source.”

   proof ↗⚑ flag

   FutureHouse Platform~partialclaimed5/10

   Docs mention tasking the platform with 'a deep literature review synthesizing conflicting evidence across hundreds of papers' and full auditability back to source passages, suggesting some capability to surface disagreement across sources, but there is no dedicated UI/feature shown for explicitly contrasting agreeing vs disagreeing sources, no example output demonstrating this, and no independent/hands-on verification. missing for 10: concrete example output showing conflicting evidence highlighted, dedicated UI feature for source agreement/disagreement, independent corroboration beyond vendor docs.

   • [claimed-docs] “You can ask it a complex scientific question and get a high-accuracy, cited response, or task it with a deep literature review synthesizing …”
   • [claimed-docs] “task it with a deep literature review synthesizing conflicting evidence across hundreds of papers”
   • [claimed-docs] “Every conclusion is fully auditable. You can trace any finding back to the specific code or literature passage that produced it.”
   • [claimed-docs] “You can trace any finding back to the specific code or literature passage that produced it.”

   proof ↗⚑ flag