FutureHouse Platform vs Undermind

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round to FutureHouse Platform
   FutureHouse Platform✓fullprobed7/10

   A working llms.txt file is directly confirmed via probe at futurehouse.gitbook.io/llms.txt returning HTTP 200 with structured agent-oriented documentation links, satisfying the core ask of pointing an agent at llms.txt. missing for 10: markdown-per-page fallback (docs-md probe returned 404), no OpenAPI/machine-readable schema confirmation, and no independent third-party corroboration of agents actually consuming this file successfully.

   • [probe] “PROBE llms.txt: HTTP 200 at https://futurehouse.gitbook.io/llms.txt # Edison Scientific Documentation ## Home - [Overview](https://docs.ed…”
   • [probe] “PROBE docs-md: HTTP 200 at https://futurehouse.gitbook.io/futurehouse-cookbook.md # Page Not Found The URL `futurehouse-cookbook` does not …”
   • [probe] “PROBE openapi: all candidate paths 404 (https://futurehouse.gitbook.io/openapi.json, https://futurehouse.gitbook.io/swagger.json, https://fu…”

   proof ↗⚑ flag

   Undermind—none0/10

   Direct probes show no llms.txt, no docs.md, and no openapi spec (all 404), meaning there is no agent-consumable documentation file for a generic AI agent to fetch. The only agent-oriented artifact is the MCP server page itself, which is a different integration mechanism, not llms.txt-style docs.

   • [probe] “PROBE llms.txt: HTTP 404 at https://undermind.ai/llms.txt”
   • [probe] “PROBE docs-md: HTTP 404 at https://undermind.ai/mcp.md”
   • [probe] “PROBE openapi: all candidate paths 404 (https://undermind.ai/openapi.json, https://undermind.ai/swagger.json, https://undermind.ai/api/opena…”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round to FutureHouse Platform
   FutureHouse Platform✓fullclaimed7/10

   The Edison client is a pip-installable Python SDK (`edison-client`) authenticated via API key, with a scriptable interface (`client.run_tasks_until_done(task_data)`) that requires no UI, making it straightforward to invoke headlessly or from CI pipelines. Missing for 10: explicit CI/automation examples (e.g., GitHub Actions workflow), documentation of exit codes/async polling patterns for pipelines, and independent confirmation of real-world CI usage.

   • [claimed-docs] “uv pip install edison-client”
   • [claimed-docs] “Authentication is done by providing an API key, which can be obtained directly from your profile page in the Edison platform”
   • [claimed-docs] “task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…”
   • [claimed-docs] “task_response = client.run_tasks_until_done(task_data)”

   proof ↗⚑ flag

   Undermind~partialprobed2/10

   Undermind's enterprise page claims 'programmatic queries via API' and integration into 'other tools and workflows,' hinting at automatable/headless use, but there is no CLI, no published API reference, and probes for OpenAPI/API docs all returned 404s. The only concretely documented automation path is the MCP server, which is designed for interactive agent clients (Cursor, VS Code, Claude) rather than headless CI pipelines. Missing for 10: documented API/CLI with auth and examples, evidence of CI/automation usage, and confirmation the enterprise API is publicly accessible rather than bespoke.

   • [claimed-docs] “Integrate Undermind's deep literature research capabilities directly into your other tools and workflows.”
   • [claimed-docs] “Programmatic queries via API”
   • [probe] “PROBE openapi: all candidate paths 404 (https://undermind.ai/openapi.json, https://undermind.ai/swagger.json, https://undermind.ai/api/opena…”

   proof ↗⚑ flag

3. ai-native userConnect an agent via an official MCP server

   weight 3 · round to Undermind
   FutureHouse Platform—none0/10

   The evidence pack only shows a Python client (edison-client) for calling FutureHouse agents via API key, plus probes confirming no OpenAPI/MCP-related endpoints were found; there is no mention of an official MCP server for connecting agents.

   • [claimed-docs] “uv pip install edison-client”
   • [probe] “PROBE openapi: all candidate paths 404 (https://futurehouse.gitbook.io/openapi.json, https://futurehouse.gitbook.io/swagger.json, https://fu…”

   proof ↗⚑ flag

   Undermind✓fullprobed8/10

   Undermind publishes an official hosted MCP server (mcp.undermind.ai/mcp) with documented setup instructions for Cursor, VS Code, Claude, and any MCP-compatible client, plus specific tool capabilities (literature review, PDF Q&A, notes, curation). This is first-party documentation with concrete transport/endpoint details, confirmed by probe evidence of the docs page existing. Missing for 10: independent/hands-on community confirmation specifically of MCP connectivity (community evidence only covers the web search product, not MCP usage) and no public API schema (openapi probes 404).

   • [claimed-docs] “This adds Undermind directly to Cursor”
   • [claimed-docs] “Undermind works with any MCP-compatible client. The protocol's recommended way for a new client to identify itself is a Client ID Metadata D…”
   • [claimed-docs] “This adds Undermind to VS Code as an MCP server.”
   • [claimed-docs] “claude mcp add --transport http undermind https://mcp.undermind.ai/mcp”
   • [claimed-docs] “Point the client at `https://mcp.undermind.ai/mcp`”
   • [claimed-docs] “Undermind works with any MCP-compatible client.”
   • [probe] “official MCP server documented at https://undermind.ai/mcp”

   proof ↗⚑ flag

4. ai-native userUse an official CLI

   weight 2 · round drawn
   FutureHouse Platform—none0/10

   Evidence shows only a Python client library (edison-client, installed via pip, used programmatically with client.run_tasks_until_done) rather than a command-line interface; no CLI tool, command syntax, or terminal usage is documented anywhere in the pack.

   • [claimed-docs] “uv pip install edison-client”
   • [claimed-docs] “task_response = client.run_tasks_until_done(task_data)”

   proof ↗⚑ flag

   Undermind—none0/10

   Evidence shows an MCP server, API access, and web/ChatGPT app integrations, but there is no mention of an official CLI tool for Undermind anywhere in the docs or probes; llms.txt, docs-md, and openapi probes all 404, and no CLI is documented.

   • [claimed-docs] “claude mcp add --transport http undermind https://mcp.undermind.ai/mcp”
   • [probe] “PROBE llms.txt: HTTP 404 at https://undermind.ai/llms.txt”
   • [probe] “PROBE docs-md: HTTP 404 at https://undermind.ai/mcp.md”
   • [probe] “PROBE openapi: all candidate paths 404 (https://undermind.ai/openapi.json, https://undermind.ai/swagger.json, https://undermind.ai/api/opena…”

   proof ↗⚑ flag

5. ai-native userDrive the product through a documented public API

   weight 3 · round to FutureHouse Platform
   FutureHouse Platform✓fullprobed7/10

   Docs describe an installable Python client (edison-client) with API-key/token auth and a documented workflow (client.run_tasks_until_done) to submit tasks like literature review, matching a genuine public API for driving the platform programmatically. Missing for 10: no OpenAPI/swagger spec found (404s), and no independent/hands-on corroboration beyond first-party docs.

   • [claimed-docs] “uv pip install edison-client”
   • [claimed-docs] “Authentication is done by providing an API key, which can be obtained directly from your profile page in the Edison platform”
   • [claimed-docs] “Under "API Tokens", click "Create New Token."”
   • [claimed-docs] “task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…”
   • [claimed-docs] “task_response = client.run_tasks_until_done(task_data)”
   • [probe] “PROBE openapi: all candidate paths 404 (https://futurehouse.gitbook.io/openapi.json, https://futurehouse.gitbook.io/swagger.json, https://fu…”

   proof ↗⚑ flag

   Undermind~partialprobed5/10

   Undermind documents a public MCP server (mcp.undermind.ai) with clear tool definitions for search, PDF Q&A, and workspace notes, which does let an AI agent drive the product programmatically (undermind-docs-3,4,5,13,14,15,19). However, a general documented REST/OpenAPI-style public API is only vaguely alluded to ('Programmatic queries via API' under Enterprise) and probes for llms.txt, docs.md, and openapi/swagger specs all return 404s, indicating no broadly published API reference. Missing for 10: a public OpenAPI/Swagger spec or REST API docs, and confirmation that the enterprise API is self-serve rather than sales-gated.

   • [claimed-docs] “Undermind works with any MCP-compatible client. The protocol's recommended way for a new client to identify itself is a Client ID Metadata D…”
   • [claimed-docs] “Runs a deep literature review from an open-ended research goal and produces a ranked list of papers with a written synthesis. Plans its own …”
   • [claimed-docs] “This adds Undermind to VS Code as an MCP server.”
   • [claimed-docs] “claude mcp add --transport http undermind https://mcp.undermind.ai/mcp”
   • [claimed-docs] “Point the client at `https://mcp.undermind.ai/mcp`”
   • [claimed-docs] “Programmatic queries via API”
   • [probe] “PROBE llms.txt: HTTP 404 at https://undermind.ai/llms.txt”
   • [probe] “PROBE docs-md: HTTP 404 at https://undermind.ai/mcp.md”
   • [probe] “PROBE openapi: all candidate paths 404 (https://undermind.ai/openapi.json, https://undermind.ai/swagger.json, https://undermind.ai/api/opena…”
   • [probe] “official MCP server documented at https://undermind.ai/mcp”

   proof ↗⚑ flag

6. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round drawn
   FutureHouse Platform—none0/10

   Evidence shows only a single, account-wide API token creation flow with no mention of scopes, permissions, or least-privilege controls for agents; no evidence of scoped or restricted credential issuance.

   • [claimed-docs] “Authentication is done by providing an API key, which can be obtained directly from your profile page in the Edison platform”
   • [claimed-docs] “Under "API Tokens", click "Create New Token."”
   • [claimed-docs] “Under "API Tokens," click "Create New Token."”

   proof ↗⚑ flag

   Undermind—none0/10

   No evidence of scoped/least-privilege API credential issuance for agents; the API is only mentioned generically ('Programmatic queries via API') with no docs on credential scoping, permissions, or key management, and OpenAPI probes returned 404s.

   • [claimed-docs] “Programmatic queries via API”
   • [probe] “PROBE openapi: all candidate paths 404 (https://undermind.ai/openapi.json, https://undermind.ai/swagger.json, https://undermind.ai/api/opena…”

   proof ↗⚑ flag

7. ai-native userBuild against official SDKs

   weight 2 · round to FutureHouse Platform
   FutureHouse Platform~partialprobed6/10

   There is a documented official Python client (edison-client) with install, auth via API key, and task submission/run examples, showing a real SDK path for programmatic/agentic use. However, no OpenAPI spec was found (all probed paths 404), and the docs page itself returned 404 on a direct fetch, suggesting incomplete or unstable documentation coverage. Missing for 10: verified OpenAPI/REST spec, broader SDK language coverage, independent developer corroboration of SDK reliability.

   • [claimed-docs] “uv pip install edison-client”
   • [claimed-docs] “Authentication is done by providing an API key, which can be obtained directly from your profile page in the Edison platform”
   • [claimed-docs] “Under "API Tokens", click "Create New Token."”
   • [claimed-docs] “task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…”
   • [claimed-docs] “task_response = client.run_tasks_until_done(task_data)”
   • [probe] “PROBE openapi: all candidate paths 404 (https://futurehouse.gitbook.io/openapi.json, https://futurehouse.gitbook.io/swagger.json, https://fu…”
   • [probe] “PROBE docs-md: HTTP 200 at https://futurehouse.gitbook.io/futurehouse-cookbook.md # Page Not Found The URL `futurehouse-cookbook` does not …”

   proof ↗⚑ flag

   Undermind—none0/10

   Evidence only mentions a vague 'Programmatic queries via API' for enterprise customers and an MCP server, but no official SDKs (client libraries, language bindings) are documented; probes for OpenAPI specs and docs (llms.txt, mcp.md, openapi.json) all return 404, indicating no public developer SDK resources exist.

   • [claimed-docs] “Programmatic queries via API”
   • [probe] “PROBE llms.txt: HTTP 404 at https://undermind.ai/llms.txt”
   • [probe] “PROBE docs-md: HTTP 404 at https://undermind.ai/mcp.md”
   • [probe] “PROBE openapi: all candidate paths 404 (https://undermind.ai/openapi.json, https://undermind.ai/swagger.json, https://undermind.ai/api/opena…”

   proof ↗⚑ flag

8. ai-native userSubscribe to events via webhooks

   weight 2 · round drawn
   FutureHouse Platform—none0/10

   No mention of webhooks, event subscriptions, or callback mechanisms anywhere in the docs; the client is a polling/run-tasks style API and OpenAPI probe returned 404s, giving no evidence of webhook support.

   • [claimed-docs] “task_response = client.run_tasks_until_done(task_data)”
   • [probe] “PROBE openapi: all candidate paths 404 (https://futurehouse.gitbook.io/openapi.json, https://futurehouse.gitbook.io/swagger.json, https://fu…”

   proof ↗⚑ flag

   Undermind—none0/10

   The evidence mentions a notification feature for new papers (undermind-docs-9) but nothing indicates this is delivered via webhooks or any programmatic subscription mechanism; no webhook API, endpoint, or docs are present, and probes for API/openapi specs all 404.

   • [claimed-docs] “Get notified whenever relevant papers are published.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://undermind.ai/openapi.json, https://undermind.ai/swagger.json, https://undermind.ai/api/opena…”

   proof ↗⚑ flag

Agentic features

1. ai-native userGet AI-generated insights and suggestions from my data inside the product

   weight 2 · round to Undermind
   FutureHouse Platform✓fullprobed7/10

   The platform's core agents (Kosmos, Precedent, Molecules) are documented as autonomously generating insights, hypotheses, and cited reports directly from user-supplied data and literature, matching the story closely (e.g., Kosmos 'autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report'). Missing for 10: independent/hands-on corroboration beyond vendor docs, and some doc links returned 404 in probes, weakening confidence in documentation completeness.

   • [claimed-docs] “Kosmos autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report.”
   • [claimed-docs] “Given a research objective and one or more datasets, Kosmos autonomously reads literature, writes and executes analysis code, generates hypo…”
   • [claimed-docs] “You can ask it a complex scientific question and get a high-accuracy, cited response, or task it with a deep literature review synthesizing …”
   • [claimed-docs] “It turns raw data into detailed analyses, statistical results, and publication-ready figures.”
   • [claimed-docs] “Precedent determines whether a research idea has been tried before. It searches across fields to assess novelty and identify gaps”
   • [claimed-docs] “Every conclusion is fully auditable. You can trace any finding back to the specific code or literature passage that produced it.”
   • [probe] “PROBE docs-md: HTTP 200 at https://futurehouse.gitbook.io/futurehouse-cookbook.md # Page Not Found The URL `futurehouse-cookbook` does not …”

   proof ↗⚑ flag

   Undermind✓fullcommunity8/10

   Undermind's core capability is AI-generated synthesis and insight extraction from literature data: it runs deep research plans, produces ranked papers with written synthesis, answers cross-paper questions from PDFs, traces citations, and proactively notifies users of new relevant papers—all generated from the user's research data within the product. Community reviews corroborate that these AI-derived insights are often more useful than manual search (undermind-comm-2, undermind-comm-6, undermind-comm-15, undermind-comm-16), though some found gaps in coverage (undermind-comm-1, undermind-comm-4). Missing for 10: independent verification of insight/synthesis accuracy at scale and clearer support for arbitrary user-uploaded (non-literature) datasets.

   • [claimed-docs] “Runs a deep literature review from an open-ended research goal and produces a ranked list of papers with a written synthesis. Plans its own …”
   • [claimed-docs] “Reads full-text PDFs in parallel and answers specific questions across many papers at once, including from figures, tables, and equations.”
   • [claimed-docs] “Creates and edits Markdown notes, syntheses, and reports in the workspace. Citations link back to the source papers, and files stay availabl…”
   • [claimed-docs] “Get notified whenever relevant papers are published.”
   • [claimed-docs] “Trace any statement by following in-line citations back to the source paper”
   • [community] “I actually was able to find at least 4 new informative papers... in less than six minutes, your search engine was able to give me more relev…”
   • [community] “These are the best results that I've gotten from an AI research assistant. I really don't mind the long latency... The 'Discovery Progress a…”

   proof ↗⚑ flag

2. ai-native userSet up automations that run autonomously in the background

   weight 2 · round to Undermind
   FutureHouse Platform—none0/10

   The evidence describes an API client for submitting tasks/queries and autonomous agents that complete a single task end-to-end, but there is no mention of scheduling, triggers, or background automations that run without explicit invocation. missing for 10: scheduling/cron mechanism, event-driven triggers, persistent background job management, and any docs describing recurring or unattended automation setup.

   • [claimed-docs] “task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…”
   • [claimed-docs] “task_response = client.run_tasks_until_done(task_data)”
   • [claimed-docs] “Kosmos autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report.”

   proof ↗⚑ flag

   Undermind~partialclaimed3/10

   The only evidence of background automation is a single line about being notified when relevant papers are published, with no detail on how such alerts are configured, scheduled, or run autonomously as multi-step agent workflows. missing for 10: documentation of automation/scheduling setup, evidence of autonomous multi-step background agent tasks, and any hands-on confirmation of the notification feature working.

   • [claimed-docs] “Get notified whenever relevant papers are published.”

   proof ↗⚑ flag

3. ai-native userDelegate tasks to a built-in AI assistant inside the product

   weight 3 · round to FutureHouse Platform
   FutureHouse Platform✓fullclaimed8/10

   The platform ships built-in autonomous agents (Kosmos, Precedent, Molecules, literature review agent) that users can delegate research tasks to via API/client, with docs describing task submission and autonomous execution producing cited reports. Missing for 10: independent/hands-on corroboration beyond vendor docs, and clearer in-product (non-API) delegation UI evidence.

   • [claimed-docs] “task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…”
   • [claimed-docs] “Kosmos autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report.”
   • [claimed-docs] “You can ask it a complex scientific question and get a high-accuracy, cited response, or task it with a deep literature review synthesizing …”
   • [claimed-docs] “Precedent determines whether a research idea has been tried before. It searches across fields to assess novelty and identify gaps”
   • [claimed-docs] “Molecules is a chemistry-focused agent for molecular design and analysis.”
   • [claimed-docs] “task_response = client.run_tasks_until_done(task_data)”
   • [claimed-docs] “Given a research objective and one or more datasets, Kosmos autonomously reads literature, writes and executes analysis code, generates hypo…”

   proof ↗⚑ flag

   Undermind—none0/10

   Undermind is positioned as an MCP server/tool that other AI clients (Cursor, Claude, ChatGPT) connect to, not as a product with its own built-in AI assistant that users delegate tasks to within Undermind itself; evidence describes it being added to external agent tools rather than an in-product assistant. missing for 10: any evidence of a native, built-in AI assistant/chat agent inside Undermind's own UI that a user can delegate tasks to.

   • [claimed-docs] “This adds Undermind directly to Cursor”
   • [claimed-docs] “Undermind works with any MCP-compatible client. The protocol's recommended way for a new client to identify itself is a Client ID Metadata D…”
   • [claimed-docs] “Undermind is available as a published ChatGPT app.”

   proof ↗⚑ flag

4. ai-native userOperate the product with natural-language commands

   weight 2 · round to Undermind
   FutureHouse Platform~partialclaimed7/10

   The platform's core interaction model is natural-language queries (e.g., the query string "Which neglected diseases had a treatment developed by artificial intelligence?") passed to autonomous agents like Kosmos, Precedent, and Molecules, and docs explicitly say users can 'ask it a complex scientific question' or 'task it with a deep literature review.' However, the demonstrated flow is via a Python client (task_data dict + run_tasks_until_done), not a conversational chat UI, so it's NL-as-input-parameter rather than full conversational natural-language operation of the whole platform (e.g., no evidence of multi-turn dialogue or a chat interface). Missing for 10: evidence of a conversational/chat UI, multi-turn NL interaction, or NL control over platform features beyond the initial query.

   • [claimed-docs] “task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…”
   • [claimed-docs] “Kosmos autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report.”
   • [claimed-docs] “You can ask it a complex scientific question and get a high-accuracy, cited response, or task it with a deep literature review synthesizing …”
   • [claimed-docs] “Given a research objective and one or more datasets, Kosmos autonomously reads literature, writes and executes analysis code, generates hypo…”
   • [claimed-docs] “task it with a deep literature review synthesizing conflicting evidence across hundreds of papers”

   proof ↗⚑ flag

   Undermind✓fullprobed8/10

   Undermind ships an official MCP server (Cursor, VS Code, Claude, ChatGPT app) that lets users issue open-ended natural-language research goals which the tool autonomously plans, searches, and synthesizes into reports, fitting the ai-native/agentic story well. Missing for 10: independent hands-on confirmation of the MCP natural-language workflow specifically (community evidence covers the web search UI, not the MCP NL commands) and any public usage examples/logs.

   • [claimed-docs] “Undermind works with any MCP-compatible client. The protocol's recommended way for a new client to identify itself is a Client ID Metadata D…”
   • [claimed-docs] “Runs a deep literature review from an open-ended research goal and produces a ranked list of papers with a written synthesis. Plans its own …”
   • [claimed-docs] “This adds Undermind to VS Code as an MCP server.”
   • [claimed-docs] “claude mcp add --transport http undermind https://mcp.undermind.ai/mcp”
   • [claimed-docs] “Undermind is available as a published ChatGPT app.”
   • [probe] “official MCP server documented at https://undermind.ai/mcp”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   FutureHouse Platform—none0/10

   Evidence shows only a quickstart guide with basic client code snippets, not an interactive API reference with runnable examples; probes for OpenAPI/swagger specs and doc endpoints all returned 404s, indicating no interactive reference exists.

   • [probe] “PROBE openapi: all candidate paths 404 (https://futurehouse.gitbook.io/openapi.json, https://futurehouse.gitbook.io/swagger.json, https://fu…”
   • [probe] “PROBE docs-md: HTTP 200 at https://futurehouse.gitbook.io/futurehouse-cookbook.md # Page Not Found The URL `futurehouse-cookbook` does not …”
   • [claimed-docs] “uv pip install edison-client”
   • [claimed-docs] “task_response = client.run_tasks_until_done(task_data)”

   proof ↗⚑ flag

   Undermind—none0/10

   There is no evidence of an interactive API reference or runnable examples; probes explicitly show no OpenAPI/Swagger spec and no docs.md/llms.txt exist. The 'Programmatic queries via API' mention is a bare feature claim with no interactive reference or runnable examples provided.

   • [probe] “PROBE openapi: all candidate paths 404 (https://undermind.ai/openapi.json, https://undermind.ai/swagger.json, https://undermind.ai/api/opena…”
   • [probe] “PROBE docs-md: HTTP 404 at https://undermind.ai/mcp.md”
   • [probe] “PROBE llms.txt: HTTP 404 at https://undermind.ai/llms.txt”
   • [claimed-docs] “Programmatic queries via API”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   FutureHouse Platform—none0/10

   Probes for OpenAPI/swagger spec paths all returned 404, and no docs reference a machine-readable API spec; only a Python client library and manual REST auth flow are documented. Missing for 10: any OpenAPI/Swagger endpoint or downloadable spec file, any documentation mentioning API schema export.

   • [probe] “PROBE openapi: all candidate paths 404 (https://futurehouse.gitbook.io/openapi.json, https://futurehouse.gitbook.io/swagger.json, https://fu…”

   proof ↗⚑ flag

   Undermind—none0/10

   Undermind mentions 'Programmatic queries via API' for enterprise but there is no evidence of a downloadable OpenAPI/Swagger spec; direct probes for openapi.json, swagger.json, and llms.txt all returned 404. Missing for 10: any published machine-readable API spec, documented API schema, or discoverable spec endpoint.

   • [claimed-docs] “Programmatic queries via API”
   • [probe] “PROBE openapi: all candidate paths 404 (https://undermind.ai/openapi.json, https://undermind.ai/swagger.json, https://undermind.ai/api/opena…”
   • [probe] “PROBE llms.txt: HTTP 404 at https://undermind.ai/llms.txt”
   • [probe] “PROBE docs-md: HTTP 404 at https://undermind.ai/mcp.md”

   proof ↗⚑ flag

3. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   FutureHouse Platform—none0/10

   No evidence of API versioning scheme, changelog, or deprecation policy anywhere in the docs; OpenAPI spec probes all 404 and no versioning documentation is present. missing for 10: versioned API endpoints, changelog/release notes, documented deprecation policy or timelines.

   • [probe] “PROBE openapi: all candidate paths 404 (https://futurehouse.gitbook.io/openapi.json, https://futurehouse.gitbook.io/swagger.json, https://fu…”

   proof ↗⚑ flag

   Undermind—none0/10

   There is a mention of a 'Programmatic queries via API' for enterprise, but no evidence of versioning or a documented deprecation policy; probes for OpenAPI/docs all returned 404s. Missing for 10: any API versioning scheme, changelog, or deprecation policy documentation.

   • [claimed-docs] “Programmatic queries via API”
   • [probe] “PROBE openapi: all candidate paths 404 (https://undermind.ai/openapi.json, https://undermind.ai/swagger.json, https://undermind.ai/api/opena…”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round to Undermind
   FutureHouse Platform—none0/10

   The docs show only single-task submission via task_data/run_tasks_until_done, with no mention of batch endpoints, bulk task lists, or parallel multi-item processing APIs; missing for 10: batch/bulk submission API, documented loop or list-based task queuing, evidence of concurrent multi-item execution.

   • [claimed-docs] “task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…”
   • [claimed-docs] “task_response = client.run_tasks_until_done(task_data)”

   proof ↗⚑ flag

   Undermind~partialclaimed6/10

   Undermind's MCP tools explicitly support bulk-style operations: reading full-text PDFs in parallel and answering questions across many papers at once, and running a deep literature review that autonomously searches and synthesizes across large numbers of papers. Enterprise API access also enables programmatic bulk queries. However, there is no evidence of bulk editing/tagging/exporting or batch management operations across items (e.g., bulk-star, bulk-move to folders) — missing for 10: documented batch update/edit/export APIs, evidence of bulk actions beyond reading/analysis, independent verification of parallel-processing claims.

   • [claimed-docs] “Reads full-text PDFs in parallel and answers specific questions across many papers at once, including from figures, tables, and equations.”
   • [claimed-docs] “Runs a deep literature review from an open-ended research goal and produces a ranked list of papers with a written synthesis. Plans its own …”
   • [claimed-docs] “Programmatic queries via API”
   • [claimed-docs] “Curate papers into a folder for long-term use.”
   • [claimed-docs] “Star important papers across the workspace.”

   proof ↗⚑ flag

2. ai-native userSchedule recurring jobs or workflows

   weight 2 · round drawn
   FutureHouse Platform—none0/10

   Evidence covers running one-off tasks via the client (run_tasks_until_done) but there is no mention of scheduling, cron-like recurrence, or persistent workflow automation anywhere in the docs or probes.

   • [claimed-docs] “task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…”
   • [claimed-docs] “task_response = client.run_tasks_until_done(task_data)”

   proof ↗⚑ flag

   Undermind—none0/10

   There's a notification feature for new papers (undermind-docs-9) but no evidence of scheduling recurring jobs/workflows, cron-like automation, or configurable recurring tasks; the product focuses on on-demand deep research via MCP tools rather than persistent scheduled automation.

   • [claimed-docs] “Get notified whenever relevant papers are published.”

   proof ↗⚑ flag

Sharing

1. analystShare a research session or report with collaborators who can view or build on it

   weight 2 · round drawn
   FutureHouse Platform—none0/10

   No evidence in the pack mentions sharing sessions, reports, or collaborative access controls; documentation focuses on API usage, agents, and task execution, not collaboration features. Missing for 10: any mention of sharing/exporting reports, collaborator invites, or team/workspace features.

   ⚑ flag

   Undermind—none0/10

   Evidence covers workspace files, folders, and note creation but never mentions sharing sessions/reports with collaborators, multi-user access, or permission controls. missing for 10: any mention of sharing/collaboration features, invite/permission mechanisms, or multi-user workspace access.

   ⚑ flag

Alerts

1. researcherSet up standing searches or alerts that surface new relevant sources as they appear

   weight 1 · round to Undermind
   FutureHouse Platform—none0/10

   The evidence describes on-demand literature search, review, and analysis agents (Kosmos, Precedent, etc.) invoked via API calls, but nothing describes standing searches, saved alerts, or recurring notifications when new relevant sources appear.

   ⚑ flag

   Undermind~partialclaimed6/10

   Undermind explicitly offers a 'Get notified whenever relevant papers are published' alert feature, which directly matches the standing-search/alert story, plus curated folders and starred papers for ongoing tracking. However, there is no independent/hands-on evidence of how the alert system works in practice (frequency, delivery channel, reliability), and community discussion focuses on one-off search quality rather than alerting. Missing for 10: independent corroboration of alert functionality, details on alert configuration/frequency, and hands-on user reports of ongoing alerts working as described.

   • [claimed-docs] “Get notified whenever relevant papers are published.”
   • [claimed-docs] “Curate papers into a folder for long-term use.”
   • [claimed-docs] “Star important papers across the workspace.”

   proof ↗⚑ flag

Corpus

1. researcherUpload my own PDFs or corpus and have the agent research over them

   weight 2 · round drawn
   FutureHouse Platform—none0/10

   The docs describe FutureHouse's agents (Kosmos, literature review, Precedent) searching a fixed 175M+ paper corpus and analyzing 'datasets' for experiments, but nowhere describe a mechanism for a researcher to upload their own PDFs or personal corpus for the agent to search/reason over. Without evidence of a file-upload or custom-corpus ingestion feature, this capability is not shown.

   • [claimed-docs] “Given a research objective and one or more datasets, Kosmos autonomously reads literature, writes and executes analysis code, generates hypo…”
   • [claimed-docs] “It accesses 175M+ papers, trials, and patents with native understanding of citation graphs, journal quality, and clinical trial data.”
   • [claimed-docs] “specializes in processing complex experimental data, including flow cytometry, RNA-seq, and other biological datasets”
   • [claimed-docs] “task it with a deep literature review synthesizing conflicting evidence across hundreds of papers”

   proof ↗⚑ flag

   Undermind—none0/10

   Evidence shows Undermind reads full-text PDFs and lets users curate/star papers discovered via its own search engine, but nothing indicates a feature to upload arbitrary personal PDFs or a private corpus for the agent to research over — all workflows described start from Undermind's own literature search rather than user-supplied documents.

   • [claimed-docs] “Reads full-text PDFs in parallel and answers specific questions across many papers at once, including from figures, tables, and equations.”
   • [claimed-docs] “Curate papers into a folder for long-term use.”
   • [claimed-docs] “Star important papers across the workspace.”

   proof ↗⚑ flag

Reviews

1. researcherRun a systematic screening and extraction workflow across many papers with consistent criteria

   weight 2 · round to Undermind
   FutureHouse Platform~partialprobed5/10

   The API client supports programmatic task submission (e.g., JobNames.LITERATURE) and batch execution via run_tasks_until_done, and the docs describe a literature agent that can synthesize evidence across hundreds of papers with citations, which could be scripted for repeated, consistent queries. However, there is no explicit documented workflow for structured screening/extraction with defined inclusion/exclusion criteria across a large corpus (e.g., systematic review style tables), and the cookbook page itself returns 404 on the probe, reducing confidence in the full pipeline. missing for 10: explicit systematic-review/extraction workflow docs, example of consistent-criteria batch screening, working cookbook page confirming the API details.

   • [claimed-docs] “task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…”
   • [claimed-docs] “task_response = client.run_tasks_until_done(task_data)”
   • [claimed-docs] “task it with a deep literature review synthesizing conflicting evidence across hundreds of papers”
   • [claimed-docs] “You can ask it a complex scientific question and get a high-accuracy, cited response, or task it with a deep literature review synthesizing …”
   • [probe] “PROBE docs-md: HTTP 200 at https://futurehouse.gitbook.io/futurehouse-cookbook.md # Page Not Found The URL `futurehouse-cookbook` does not …”

   proof ↗⚑ flag

   Undermind~partialcommunity6/10

   Undermind's MCP tools support open-ended literature review with ranked results, full-text PDF question-answering across many papers, and note/synthesis creation, which covers much of a screening-and-extraction workflow (undermind-docs-4, undermind-docs-5, undermind-docs-6, undermind-docs-7). However, there is no evidence of formal criteria-based screening (e.g., inclusion/exclusion rules) or structured, consistent extraction fields across a batch of papers, and community feedback notes gaps in coverage (missed gray literature/theses) that could undermine consistency (undermind-comm-1, undermind-comm-4). Missing for 10: explicit support for defined screening criteria, structured/consistent extraction schema, and independent validation of consistency at scale.

   • [claimed-docs] “Runs a deep literature review from an open-ended research goal and produces a ranked list of papers with a written synthesis. Plans its own …”
   • [claimed-docs] “Reads full-text PDFs in parallel and answers specific questions across many papers at once, including from figures, tables, and equations.”
   • [claimed-docs] “Creates and edits Markdown notes, syntheses, and reports in the workspace. Citations link back to the source papers, and files stay availabl…”
   • [claimed-docs] “Curate papers into a folder for long-term use.”
   • [community] “OK, I'm both impressed and disappointed. It seems to rely too much on citation count, and I don't see much dissertations/theses in the resul…”
   • [community] “Overall, I am impressed, and I can easily imagine my lab paying for this... but it seems to have missed a lot of the gray literature, partic…”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round drawn
   FutureHouse Platform~partialprobed5/10

   The edison-client library and quickstart docs show that core agent tasks (e.g., JobNames.LITERATURE) can be invoked via API with authentication and task submission (docs-1–4,12), and the cookbook describes multiple agents (Kosmos, Precedent, Molecules) that presumably map to API job types, suggesting broad functional overlap with the UI. However, there is no explicit documentation stating full feature parity between UI and API, no OpenAPI/swagger spec discoverable (probe-3 shows 404s across all candidate paths), and no confirmation that all UI-exposed capabilities (e.g., account management, dataset uploads, report viewing) are also API-accessible. missing for 10: explicit parity statement, discoverable OpenAPI schema, evidence covering non-task UI features (billing, dataset management, report exports) via API.

   • [claimed-docs] “task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…”
   • [claimed-docs] “task_response = client.run_tasks_until_done(task_data)”
   • [claimed-docs] “Kosmos autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report.”
   • [claimed-docs] “Precedent determines whether a research idea has been tried before. It searches across fields to assess novelty and identify gaps”
   • [claimed-docs] “Molecules is a chemistry-focused agent for molecular design and analysis.”
   • [probe] “PROBE openapi: all candidate paths 404 (https://futurehouse.gitbook.io/openapi.json, https://futurehouse.gitbook.io/swagger.json, https://fu…”

   proof ↗⚑ flag

   Undermind~partialprobed5/10

   Undermind exposes core research capabilities (deep literature search, PDF Q&A, note creation, curation, starring) via an official MCP server, and separately claims 'Programmatic queries via API' for enterprise customers, showing some AI-native parity. However, there is no public API/OpenAPI documentation (all probes 404), no evidence that UI-only features like notifications/alerts or workspace management are exposed programmatically, and the API claim is a single unelaborated enterprise line rather than a documented full-parity API. Missing for 10: public API docs/OpenAPI spec, confirmation that all UI features (alerts, workspace/library management) are API-accessible, and independent verification of API completeness.

   • [claimed-docs] “Runs a deep literature review from an open-ended research goal and produces a ranked list of papers with a written synthesis. Plans its own …”
   • [claimed-docs] “Reads full-text PDFs in parallel and answers specific questions across many papers at once, including from figures, tables, and equations.”
   • [claimed-docs] “Creates and edits Markdown notes, syntheses, and reports in the workspace. Citations link back to the source papers, and files stay availabl…”
   • [claimed-docs] “Curate papers into a folder for long-term use.”
   • [claimed-docs] “Star important papers across the workspace.”
   • [claimed-docs] “Programmatic queries via API”
   • [claimed-docs] “Get notified whenever relevant papers are published.”
   • [probe] “PROBE llms.txt: HTTP 404 at https://undermind.ai/llms.txt”
   • [probe] “PROBE docs-md: HTTP 404 at https://undermind.ai/mcp.md”
   • [probe] “PROBE openapi: all candidate paths 404 (https://undermind.ai/openapi.json, https://undermind.ai/swagger.json, https://undermind.ai/api/opena…”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round drawn
   FutureHouse Platform—none0/10

   No evidence of any data export capability, open-format export, or account portability/deletion feature; documentation covers agent/task usage and API access but nothing about exporting user data or leaving with it.

   ⚑ flag

   Undermind—none0/10

   No evidence of any data export feature or open-format export capability for user data/papers/workspaces; probes for docs/API endpoints also 404. The evidence covers MCP integration and search quality, not data portability or export/leave capability.

   ⚑ flag

Pricing

1. researcherTry the product meaningfully on a free tier or trial

   weight 1 · round to FutureHouse Platform
   FutureHouse Platform✓fullclaimed7/10

   Docs explicitly state Edison maintains a generous free tier for academics, with paid plans only needed for higher rate limits/additional features, and the quickstart shows a full API key + client workflow to run real tasks (literature review job). Missing for 10: independent/hands-on confirmation of free tier limits and no clarity on exact quota/duration restrictions.

   • [claimed-docs] “Edison maintains a generous free tier for academics. Researchers who need higher rate limits or additional features can subscribe to paid pl…”
   • [claimed-docs] “Researchers who need higher rate limits or additional features can subscribe to paid plans.”
   • [claimed-docs] “Authentication is done by providing an API key, which can be obtained directly from your profile page in the Edison platform”
   • [claimed-docs] “Under "API Tokens", click "Create New Token."”
   • [claimed-docs] “task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…”
   • [claimed-docs] “task_response = client.run_tasks_until_done(task_data)”

   proof ↗⚑ flag

   Undermind!disputedcontradicted4/10

   Docs imply a tiered system (e.g. '10x higher usage limits' for paid vs default) suggesting a free/limited tier exists, but community evidence shows a hard institutional/company email requirement blocking sign-up, with an independent researcher explicitly reporting they 'can't get in' and another calling the requirement 'obnoxious' and a 'roadblock' — concretely contradicting the ability for many researchers to try it meaningfully for free. Missing for 10: explicit vendor documentation of a free tier or trial with stated limits, and confirmation the email gate has been removed or has an exception path for independent researchers.

   • [claimed-docs] “Deepest analysis of full texts 10x higher usage limits Unlimited workspaces, files, and paper libraries”
   • [community] “Independent researcher without academic address; can't get in. Best of luck.”
   • [community] “'Please use a valid institutional or company email address.' This is obnoxious. Please remove this unnecessary roadblock.”

   proof ↗⚑ flag

2. researcherUnderstand plan pricing and usage limits before committing

   weight 2 · round to FutureHouse Platform
   FutureHouse Platform~partialclaimed4/10

   Docs confirm a free tier for academics and paid plans for higher rate limits/features, but no concrete pricing figures, tier breakdowns, or specific usage limit numbers are provided anywhere in the pack. missing for 10: actual price points, detailed rate-limit thresholds, feature comparison table across tiers, billing/usage dashboard evidence.

   • [claimed-docs] “Researchers who need higher rate limits or additional features can subscribe to paid plans.”
   • [claimed-docs] “Edison maintains a generous free tier for academics. Researchers who need higher rate limits or additional features can subscribe to paid pl…”

   proof ↗⚑ flag

   Undermind—none0/10

   Evidence includes only a fragmentary marketing snippet ('10x higher usage limits, unlimited workspaces...') with no actual price points, plan names, or explicit usage caps, and no dedicated pricing page is cited. A researcher cannot compare plans or understand limits before committing from this evidence alone.

   • [claimed-docs] “Deepest analysis of full texts 10x higher usage limits Unlimited workspaces, files, and paper libraries”

   proof ↗⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round drawn
   FutureHouse Platform—none0/10

   No evidence in the pack addresses data residency, regional storage options, or geographic control over where data is processed/stored; documentation only covers API usage, agents, and pricing tiers.

   ⚑ flag

   Undermind—none0/10

   No evidence in the pack mentions data residency, regional storage options, or any data-location controls for Undermind. missing for 10: any mention of region selection, data residency policy, or storage location controls.

   ⚑ flag

2. ai-native userPrevent my data from being used to train AI models

   weight 3 · round drawn
   FutureHouse Platform—none0/10

   No evidence in the pack addresses data usage for AI training opt-out, data privacy controls, or any training-data policy; the docs focus entirely on product features and API usage.

   ⚑ flag

   Undermind—none0/10

   No evidence in the pack addresses data privacy, opt-out of AI training, or data usage policies for Undermind; nothing here confirms or denies such a control exists.

   ⚑ flag

3. ai-native userControl data retention and deletion

   weight 2 · round drawn
   FutureHouse Platform—none0/10

   No evidence in the pack addresses data retention policies, deletion controls, or privacy/data lifecycle management; documentation focuses on client setup, agents, and use cases only.

   ⚑ flag

   Undermind—none0/10

   No evidence in the pack addresses data retention policies, deletion controls, or privacy/data-handling settings for Undermind; all evidence concerns search quality, MCP integration, and product features. This axis applies to any AI-native product handling user data/queries, but there is no documented capability for controlling or deleting stored data.

   ⚑ flag

4. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   FutureHouse Platform—none0/10

   No evidence in the pack mentions telemetry, usage tracking, analytics collection, or an opt-out mechanism anywhere in the FutureHouse docs or probes; the documentation focuses on client setup and agent features only.

   ⚑ flag

   Undermind—none0/10

   No evidence in the pack mentions telemetry, usage tracking, opt-out settings, or privacy controls for Undermind; all evidence covers product features, MCP integration, and general reception.

   ⚑ flag

Reports

1. researcherExport results to common formats, including documents, spreadsheets, and reference-manager files

   weight 1 · round drawn
   FutureHouse Platform—none0/10

   The evidence describes cited reports and analyses produced by Kosmos/agents but nowhere mentions exporting to document, spreadsheet, or reference-manager formats (e.g., PDF/DOCX, CSV/XLSX, BibTeX/RIS); the docs-cookbook page itself is even reported as not found in the probe.

   • [claimed-docs] “Kosmos autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report.”
   • [claimed-docs] “Every conclusion is fully auditable. You can trace any finding back to the specific code or literature passage that produced it.”
   • [claimed-docs] “It turns raw data into detailed analyses, statistical results, and publication-ready figures.”
   • [probe] “PROBE docs-md: HTTP 200 at https://futurehouse.gitbook.io/futurehouse-cookbook.md # Page Not Found The URL `futurehouse-cookbook` does not …”

   proof ↗⚑ flag

   Undermind—none0/10

   Evidence shows Undermind creates/edits Markdown notes and reports with citations, and can curate/star papers, but there is no mention of exporting to common document formats (e.g., Word/PDF), spreadsheets (e.g., CSV/Excel), or reference-manager formats (e.g., BibTeX, RIS, EndNote).

   ⚑ flag

2. analystGet a structured report with sections, tables, and a summary that I can share with stakeholders

   weight 3 · round to FutureHouse Platform
   FutureHouse Platform~partialclaimed6/10

   Docs describe Kosmos/agents producing a 'comprehensive cited report' with statistical results, publication-ready figures, and traceable citations, which implies structured, shareable output, but there's no explicit mention of report sections/tables format or export/sharing mechanisms for stakeholders. missing for 10: explicit description of report structure (sections/tables), export or sharing functionality, and independent confirmation of report format quality.

   • [claimed-docs] “Kosmos autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report.”
   • [claimed-docs] “Every conclusion is fully auditable. You can trace any finding back to the specific code or literature passage that produced it.”
   • [claimed-docs] “It turns raw data into detailed analyses, statistical results, and publication-ready figures.”
   • [claimed-docs] “Given a research objective and one or more datasets, Kosmos autonomously reads literature, writes and executes analysis code, generates hypo…”
   • [claimed-docs] “You can trace any finding back to the specific code or literature passage that produced it.”

   proof ↗⚑ flag

   Undermind~partialclaimed5/10

   Undermind produces a ranked list of papers with a written synthesis and can create/edit Markdown notes, syntheses, and reports with citations linking back to sources, which supports shareable structured output. However, there is no evidence of built-in tables, formal 'sections' structuring, or a dedicated stakeholder-facing report/export format beyond Markdown notes. Missing for 10: explicit table generation, multi-section report templates, and export/sharing formats (PDF/Word) for stakeholders beyond in-workspace Markdown.

   • [claimed-docs] “Runs a deep literature review from an open-ended research goal and produces a ranked list of papers with a written synthesis. Plans its own …”
   • [claimed-docs] “Creates and edits Markdown notes, syntheses, and reports in the workspace. Citations link back to the source papers, and files stay availabl…”
   • [claimed-docs] “Reads full-text PDFs in parallel and answers specific questions across many papers at once, including from figures, tables, and equations.”

   proof ↗⚑ flag

Agent runs

1. researcherPose a research question and get an autonomous multi-step investigation, not just a single-pass summary

   weight 3 · round drawn
   FutureHouse Platform✓fullclaimed8/10

   Docs describe Kosmos and other agents autonomously reading literature, writing/executing analysis code, generating hypotheses, and producing cited multi-step reports (not single-pass summaries), with auditable traceability of findings and multi-agent capabilities (Precedent, Molecules, literature review across hundreds of papers). This directly matches the story of posing a question and getting an autonomous multi-step investigation. Missing for 10: independent/hands-on verification of the multi-step autonomy claims (probe shows docs page 404'd, reducing corroboration strength) and no third-party case study confirming real-world multi-step behavior.

   • [claimed-docs] “Kosmos autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report.”
   • [claimed-docs] “Given a research objective and one or more datasets, Kosmos autonomously reads literature, writes and executes analysis code, generates hypo…”
   • [claimed-docs] “Every conclusion is fully auditable. You can trace any finding back to the specific code or literature passage that produced it.”
   • [claimed-docs] “You can ask it a complex scientific question and get a high-accuracy, cited response, or task it with a deep literature review synthesizing …”
   • [claimed-docs] “task it with a deep literature review synthesizing conflicting evidence across hundreds of papers”
   • [claimed-docs] “Precedent determines whether a research idea has been tried before. It searches across fields to assess novelty and identify gaps”

   proof ↗⚑ flag

   Undermind✓fullcommunity8/10

   Docs explicitly describe an autonomous multi-step deep literature review agent that plans its own searches, follows citations and key authors, reads full-text PDFs in parallel, and stops only when new searches stop finding relevant papers — not a single-pass summary. Community reviews corroborate multi-minute, iterative search behavior yielding comprehensive results beyond a simple query-response. Missing for 10: independent technical breakdown of the multi-step planning/agentic loop and more recent hands-on validation of the 'stops when exhausted' claim.

   • [claimed-docs] “Runs a deep literature review from an open-ended research goal and produces a ranked list of papers with a written synthesis. Plans its own …”
   • [claimed-docs] “Plans its own searches, follows citations and key authors, and stops only when new searches stop finding relevant papers.”
   • [claimed-docs] “Reads full-text PDFs in parallel and answers specific questions across many papers at once, including from figures, tables, and equations.”
   • [community] “Been using Undermind for several months now and it's honestly been a lifesaver in getting a comprehensive understanding of a research topic.”
   • [community] “These are the best results that I've gotten from an AI research assistant. I really don't mind the long latency... The 'Discovery Progress a…”
   • [community] “I actually was able to find at least 4 new informative papers... in less than six minutes, your search engine was able to give me more relev…”

   proof ↗⚑ flag

2. analystStart a long research job that keeps working unattended and notifies me when the result is ready

   weight 2 · round drawn
   FutureHouse Platform~partialclaimed5/10

   The client SDK exposes a task submission and `run_tasks_until_done` pattern (docs-4, docs-12) plus descriptions of Kosmos autonomously running long, multi-step research (literature review, code execution, hypothesis generation) unattended (docs-5, docs-13, docs-19), supporting the 'long unattended job' half of the story. However there is no evidence of any notification mechanism (email, webhook, push) when a job completes — the only documented pattern is a client-side polling call that blocks until done, not an async notify-on-completion flow. Missing for 10: explicit notification/webhook/email-on-completion feature, evidence of true async fire-and-forget job submission separate from polling.

   • [claimed-docs] “task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…”
   • [claimed-docs] “task_response = client.run_tasks_until_done(task_data)”
   • [claimed-docs] “Kosmos autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report.”
   • [claimed-docs] “Given a research objective and one or more datasets, Kosmos autonomously reads literature, writes and executes analysis code, generates hypo…”
   • [claimed-docs] “task it with a deep literature review synthesizing conflicting evidence across hundreds of papers”

   proof ↗⚑ flag

   Undermind~partialcommunity5/10

   Undermind's deep literature review runs autonomously, planning its own searches and stopping only when exhausted (undermind-docs-4/16), and it has a notification feature for relevant papers (undermind-docs-9), suggesting async, unattended operation. However, the notification feature is documented as an ongoing 'new paper published' alert rather than a 'job complete, come see results' notification, and community reports describe run times of minutes (3-6 min) rather than long unattended background jobs (undermind-comm-14/15/16). Missing for 10: explicit documentation that a single research job can run for extended/unattended periods (hours+) and trigger a completion notification, plus independent confirmation of this exact workflow.

   • [claimed-docs] “Runs a deep literature review from an open-ended research goal and produces a ranked list of papers with a written synthesis. Plans its own …”
   • [claimed-docs] “Plans its own searches, follows citations and key authors, and stops only when new searches stop finding relevant papers.”
   • [claimed-docs] “Get notified whenever relevant papers are published.”
   • [community] “Hmm, does this just use traditional term frequency search under the hood with query expansion? Using GPT4 as a cross encoder seems really wa…”
   • [community] “I actually was able to find at least 4 new informative papers... in less than six minutes, your search engine was able to give me more relev…”
   • [community] “These are the best results that I've gotten from an AI research assistant. I really don't mind the long latency... The 'Discovery Progress a…”

   proof ↗⚑ flag

3. researcherSteer the depth, effort, and scope of a research run before or while it executes

   weight 1 · round drawn
   FutureHouse Platform—none0/10

   Evidence shows job submission with a query and job name, plus a mid-run polling helper (run_tasks_until_done), but no documented parameters or controls for adjusting depth, effort level, or scope of a run before or during execution.

   • [claimed-docs] “task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…”
   • [claimed-docs] “task_response = client.run_tasks_until_done(task_data)”

   proof ↗⚑ flag

   Undermind—none0/10

   Undermind's research runs are described as fully autonomous — it 'plans its own searches... and stops only when new searches stop finding relevant papers' (undermind-docs-4, undermind-docs-16), with no documented controls for setting depth, effort, or scope before or during a run. A user comment notes confusion about the 'Discovery Progress and Exhaustiveness' indicator (undermind-comm-16), suggesting only passive visibility, not steering. No evidence of parameters, sliders, or mid-run adjustments exists in the pack.

   • [claimed-docs] “Runs a deep literature review from an open-ended research goal and produces a ranked list of papers with a written synthesis. Plans its own …”
   • [claimed-docs] “Plans its own searches, follows citations and key authors, and stops only when new searches stop finding relevant papers.”
   • [community] “These are the best results that I've gotten from an AI research assistant. I really don't mind the long latency... The 'Discovery Progress a…”

   proof ↗⚑ flag

Citations

1. researcherSee citations for every substantive claim so I can verify it against the underlying source

   weight 3 · round drawn
   FutureHouse Platform✓fullclaimed7/10

   Docs state that outputs are cited and auditable, with claims traceable back to specific literature passages or code, and that literature review/answer agents produce cited responses. This is first-party documentation only, with no independent/hands-on corroboration of citation accuracy or completeness. missing for 10: independent verification of citation accuracy, example of an actual cited report output, detail on citation format/traceability mechanism.

   • [claimed-docs] “Every conclusion is fully auditable. You can trace any finding back to the specific code or literature passage that produced it.”
   • [claimed-docs] “You can trace any finding back to the specific code or literature passage that produced it.”
   • [claimed-docs] “You can ask it a complex scientific question and get a high-accuracy, cited response, or task it with a deep literature review synthesizing …”
   • [claimed-docs] “Kosmos autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report.”
   • [claimed-docs] “Given a research objective and one or more datasets, Kosmos autonomously reads literature, writes and executes analysis code, generates hypo…”

   proof ↗⚑ flag

   Undermind✓fullcommunity7/10

   Undermind's docs explicitly claim in-line citations traceable to source papers ([undermind-docs-12], [undermind-docs-6]) and community reviews corroborate it reliably surfaces cited references (e.g. [undermind-comm-8] notes it 'solves' the reference-provision problem unlike a chatbot). However, no independent hands-on verification of citation accuracy/completeness at the claim level is present, and some reviewers note gaps in coverage (missing gray literature/theses) which could affect verifiability of some claims. Missing for 10: independent audit of citation accuracy per-claim, and confirmation citations withstand scrutiny across all source types.

   • [claimed-docs] “Trace any statement by following in-line citations back to the source paper”
   • [claimed-docs] “Creates and edits Markdown notes, syntheses, and reports in the workspace. Citations link back to the source papers, and files stay availabl…”
   • [community] “Compared roughly similar research questions using Claude 3.5 Sonnet and Undermind. Claude is reluctant to provide references, but Undermind …”
   • [community] “OK, I'm both impressed and disappointed. It seems to rely too much on citation count, and I don't see much dissertations/theses in the resul…”
   • [community] “Overall, I am impressed, and I can easily imagine my lab paying for this... but it seems to have missed a lot of the gray literature, partic…”

   proof ↗⚑ flag

Corpus

1. researcherSearch scholarly literature and primary sources, not just the open web

   weight 2 · round drawn
   FutureHouse Platform✓fullclaimed8/10

   Docs describe agents (LITERATURE task, Kosmos, Precedent) that explicitly search scholarly literature, trials, and patents, accessing '175M+ papers, trials, and patents' with citation graph and journal quality awareness, and produce cited reports auditable back to specific literature passages. This directly matches the researcher story of searching scholarly/primary sources beyond the open web. Missing for 10: independent/hands-on corroboration of search coverage and source quality beyond vendor docs, and clarity on which specific databases/indices are used.

   • [claimed-docs] “It accesses 175M+ papers, trials, and patents with native understanding of citation graphs, journal quality, and clinical trial data.”
   • [claimed-docs] “Precedent determines whether a research idea has been tried before. It searches across fields to assess novelty and identify gaps”
   • [claimed-docs] “Precedent determines whether a research idea has been tried before. It searches across fields to assess novelty and identify gaps, helping y…”
   • [claimed-docs] “Kosmos autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report.”
   • [claimed-docs] “Every conclusion is fully auditable. You can trace any finding back to the specific code or literature passage that produced it.”
   • [claimed-docs] “task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…”

   proof ↗⚑ flag

   Undermind✓fullcommunity8/10

   Undermind is purpose-built for scholarly literature search: it runs deep literature reviews over papers, reads full-text PDFs including figures/tables/equations, and traces claims back to source papers via citations, with independent community reports confirming it surfaces relevant academic papers beyond Google Scholar. Some community feedback notes gaps in coverage (dissertations, gray literature), which tempers but doesn't negate the core capability. Missing for 10: no independent benchmark on primary-source/preprint coverage breadth, and some users report missed gray literature/theses.

   • [claimed-docs] “Runs a deep literature review from an open-ended research goal and produces a ranked list of papers with a written synthesis. Plans its own …”
   • [claimed-docs] “Reads full-text PDFs in parallel and answers specific questions across many papers at once, including from figures, tables, and equations.”
   • [claimed-docs] “Trace any statement by following in-line citations back to the source paper”
   • [claimed-docs] “Undermind's v1 search engine delivered 10x better results than Google Scholar”
   • [community] “This is a nice search engine. I found it to be more effective than crawling with Google Scholar. Good work guys!”
   • [community] “As a CS academic, the top 10 results contained two items I really ought to have found myself... overall I'm very impressed with this.”
   • [community] “I actually was able to find at least 4 new informative papers... in less than six minutes, your search engine was able to give me more relev…”
   • [community] “OK, I'm both impressed and disappointed. It seems to rely too much on citation count, and I don't see much dissertations/theses in the resul…”
   • [community] “Overall, I am impressed, and I can easily imagine my lab paying for this... but it seems to have missed a lot of the gray literature, partic…”

   proof ↗⚑ flag

Synthesis

1. analystSee where sources agree and disagree instead of a single unqualified answer

   weight 2 · round to FutureHouse Platform
   FutureHouse Platform~partialclaimed5/10

   Docs mention tasking the platform with 'a deep literature review synthesizing conflicting evidence across hundreds of papers' and full auditability back to source passages, suggesting some capability to surface disagreement across sources, but there is no dedicated UI/feature shown for explicitly contrasting agreeing vs disagreeing sources, no example output demonstrating this, and no independent/hands-on verification. missing for 10: concrete example output showing conflicting evidence highlighted, dedicated UI feature for source agreement/disagreement, independent corroboration beyond vendor docs.

   • [claimed-docs] “You can ask it a complex scientific question and get a high-accuracy, cited response, or task it with a deep literature review synthesizing …”
   • [claimed-docs] “task it with a deep literature review synthesizing conflicting evidence across hundreds of papers”
   • [claimed-docs] “Every conclusion is fully auditable. You can trace any finding back to the specific code or literature passage that produced it.”
   • [claimed-docs] “You can trace any finding back to the specific code or literature passage that produced it.”

   proof ↗⚑ flag

   Undermind—none0/10

   The evidence pack describes literature search, synthesis, citation tracing, and PDF Q&A features, but nowhere does it mention surfacing conflicting findings, agreement/disagreement across sources, or qualifying claims by consensus vs. dispute. Citation tracing (docs-12) only supports tracing a single claim to its source, not comparing multiple sources' stances. Missing for 10: any feature or documentation showing detection/display of cross-source agreement or contradiction, any UI element flagging conflicting conclusions, community evidence of this behavior.

   • [claimed-docs] “Trace any statement by following in-line citations back to the source paper”
   • [claimed-docs] “Runs a deep literature review from an open-ended research goal and produces a ranked list of papers with a written synthesis. Plans its own …”

   proof ↗⚑ flag