Skip to content

AI Research Agents Arena

FutureHouse Platform vs Gemini Notebook (NotebookLM)

FutureHouse Platform wins · 137 (13 drawn)

Agenticness — how well agents can access and operate the productAgenticness

How well agents can access and operate the product

Agent access

  1. ai-native userRun the product headlessly / in CI for automation

    weight 2 · round to FutureHouse Platform
    FutureHouse Platformfullclaimed7/10

    The Edison client is a pip-installable Python SDK (`edison-client`) authenticated via API key, with a scriptable interface (`client.run_tasks_until_done(task_data)`) that requires no UI, making it straightforward to invoke headlessly or from CI pipelines. Missing for 10: explicit CI/automation examples (e.g., GitHub Actions workflow), documentation of exit codes/async polling patterns for pipelines, and independent confirmation of real-world CI usage.

    • [claimed-docs] uv pip install edison-client
    • [claimed-docs] Authentication is done by providing an API key, which can be obtained directly from your profile page in the Edison platform
    • [claimed-docs] task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…
    • [claimed-docs] task_response = client.run_tasks_until_done(task_data)
    Gemini Notebook (NotebookLM)none0/10

    No evidence of an API, CLI, or headless/automation mode for NotebookLM/Gemini Notebook; all documented usage is through the interactive web/app UI, and community comments even complain about lacking programmatic source ingestion.

    • [claimed-docs] Easily upload PDFs, websites, YouTube videos, audio files, Google Docs, or Google Slides or discover new sources.
    • [community] My problem with NotebookLM was that you had to give it URLs or docs, you couldn't just have it search for docs and then incorporate them int…
  2. ai-native userUse an official CLI

    weight 2 · round drawn
    FutureHouse Platformnone0/10

    Evidence shows only a Python client library (edison-client, installed via pip, used programmatically with client.run_tasks_until_done) rather than a command-line interface; no CLI tool, command syntax, or terminal usage is documented anywhere in the pack.

    Gemini Notebook (NotebookLM)none0/10

    No evidence of an official CLI for Gemini Notebook; all documentation describes a web/app UI with no command-line interface mentioned.

    • ai-native userDrive the product through a documented public API

      weight 3 · round to FutureHouse Platform
      FutureHouse Platformfullprobed7/10

      Docs describe an installable Python client (edison-client) with API-key/token auth and a documented workflow (client.run_tasks_until_done) to submit tasks like literature review, matching a genuine public API for driving the platform programmatically. Missing for 10: no OpenAPI/swagger spec found (404s), and no independent/hands-on corroboration beyond first-party docs.

      • [claimed-docs] uv pip install edison-client
      • [claimed-docs] Authentication is done by providing an API key, which can be obtained directly from your profile page in the Edison platform
      • [claimed-docs] Under "API Tokens", click "Create New Token."
      • [claimed-docs] task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…
      • [claimed-docs] task_response = client.run_tasks_until_done(task_data)
      • [probe] PROBE openapi: all candidate paths 404 (https://futurehouse.gitbook.io/openapi.json, https://futurehouse.gitbook.io/swagger.json, https://fu…
      Gemini Notebook (NotebookLM)none0/10

      No evidence pack item mentions a documented public API, SDK, or programmatic access method for Gemini Notebook; all evidence describes UI-driven features (sources, chat, audio/video overviews, sharing) and community commentary about workflow limitations, not API access.

      • ai-native userBuild against official SDKs

        weight 2 · round to FutureHouse Platform
        FutureHouse Platformpartialprobed6/10

        There is a documented official Python client (edison-client) with install, auth via API key, and task submission/run examples, showing a real SDK path for programmatic/agentic use. However, no OpenAPI spec was found (all probed paths 404), and the docs page itself returned 404 on a direct fetch, suggesting incomplete or unstable documentation coverage. Missing for 10: verified OpenAPI/REST spec, broader SDK language coverage, independent developer corroboration of SDK reliability.

        • [claimed-docs] uv pip install edison-client
        • [claimed-docs] Authentication is done by providing an API key, which can be obtained directly from your profile page in the Edison platform
        • [claimed-docs] Under "API Tokens", click "Create New Token."
        • [claimed-docs] task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…
        • [claimed-docs] task_response = client.run_tasks_until_done(task_data)
        • [probe] PROBE openapi: all candidate paths 404 (https://futurehouse.gitbook.io/openapi.json, https://futurehouse.gitbook.io/swagger.json, https://fu…
        • [probe] PROBE docs-md: HTTP 200 at https://futurehouse.gitbook.io/futurehouse-cookbook.md # Page Not Found The URL `futurehouse-cookbook` does not …
        Gemini Notebook (NotebookLM)none0/10

        No evidence of an official SDK, API, or developer library for building against Gemini Notebook; all evidence describes end-user features (uploading sources, chat, audio/video overviews, notes) rather than programmatic/developer access.

        • ai-native userSubscribe to events via webhooks

          weight 2 · round drawn
          FutureHouse Platformnone0/10

          No mention of webhooks, event subscriptions, or callback mechanisms anywhere in the docs; the client is a polling/run-tasks style API and OpenAPI probe returned 404s, giving no evidence of webhook support.

          • [claimed-docs] task_response = client.run_tasks_until_done(task_data)
          • [probe] PROBE openapi: all candidate paths 404 (https://futurehouse.gitbook.io/openapi.json, https://futurehouse.gitbook.io/swagger.json, https://fu…
          Gemini Notebook (NotebookLM)none0/10

          No evidence of any webhook subscription, API event, or outbound notification capability in Gemini Notebook's docs or community feedback; the product is a research/notebook tool with no mentioned event-driven integration surface.

          Agentic features

          1. ai-native userGet AI-generated insights and suggestions from my data inside the product

            weight 2 · round to Gemini Notebook (NotebookLM)
            FutureHouse Platformfullprobed7/10

            The platform's core agents (Kosmos, Precedent, Molecules) are documented as autonomously generating insights, hypotheses, and cited reports directly from user-supplied data and literature, matching the story closely (e.g., Kosmos 'autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report'). Missing for 10: independent/hands-on corroboration beyond vendor docs, and some doc links returned 404 in probes, weakening confidence in documentation completeness.

            • [claimed-docs] Kosmos autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report.
            • [claimed-docs] Given a research objective and one or more datasets, Kosmos autonomously reads literature, writes and executes analysis code, generates hypo…
            • [claimed-docs] You can ask it a complex scientific question and get a high-accuracy, cited response, or task it with a deep literature review synthesizing …
            • [claimed-docs] It turns raw data into detailed analyses, statistical results, and publication-ready figures.
            • [claimed-docs] Precedent determines whether a research idea has been tried before. It searches across fields to assess novelty and identify gaps
            • [claimed-docs] Every conclusion is fully auditable. You can trace any finding back to the specific code or literature passage that produced it.
            • [probe] PROBE docs-md: HTTP 200 at https://futurehouse.gitbook.io/futurehouse-cookbook.md # Page Not Found The URL `futurehouse-cookbook` does not …
            Gemini Notebook (NotebookLM)fullcommunity8/10

            NotebookLM generates summaries, insights, mind maps, briefing documents, AI-suggested report types, and answers with grounded citations directly from uploaded sources, and community feedback confirms real-world use for research insight extraction (e.g., audio overviews, code walkthroughs). Missing for 10: independent benchmarking of insight quality/accuracy and evidence of more proactive/agentic suggestion behavior beyond user-initiated Q&A.

            • [claimed-docs] Chat with your notebook to get grounded information based on your sources with clear in-line citations for accuracy, transparency, and trust…
            • [claimed-docs] Find an auto-generated summary of the entire source in the Source Guide.
            • [claimed-docs] Create your own, or choose from FAQ, study guide, briefing document, or an AI suggested report type.
            • [claimed-docs] Get insights in the moment by asking questions about your sources.
            • [claimed-docs] Mind Maps in Gemini Notebook visually summarize your uploaded sources, showing main topics and related ideas as a branching diagram.
            • [community] NotebookLM is genuinely useful for structured research workflows. My concern is not the name. It's whether the "notebook" metaphor survives …
            • [community] I used it to listen to a high level walkthrough of interesting and entertaining code while driving to work... Hearing it talk about Ken Thom…
          2. ai-native userSet up automations that run autonomously in the background

            weight 2 · round drawn
            FutureHouse Platformnone0/10

            The evidence describes an API client for submitting tasks/queries and autonomous agents that complete a single task end-to-end, but there is no mention of scheduling, triggers, or background automations that run without explicit invocation. missing for 10: scheduling/cron mechanism, event-driven triggers, persistent background job management, and any docs describing recurring or unattended automation setup.

            • [claimed-docs] task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…
            • [claimed-docs] task_response = client.run_tasks_until_done(task_data)
            • [claimed-docs] Kosmos autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report.
            Gemini Notebook (NotebookLM)none0/10

            No evidence of scheduling, triggers, or autonomous background automations in NotebookLM's documented features; all capabilities (chat, source summarization, audio/video overviews, notes) are user-initiated within a session.

            • ai-native userDelegate tasks to a built-in AI assistant inside the product

              weight 3 · round to FutureHouse Platform
              FutureHouse Platformfullclaimed8/10

              The platform ships built-in autonomous agents (Kosmos, Precedent, Molecules, literature review agent) that users can delegate research tasks to via API/client, with docs describing task submission and autonomous execution producing cited reports. Missing for 10: independent/hands-on corroboration beyond vendor docs, and clearer in-product (non-API) delegation UI evidence.

              • [claimed-docs] task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…
              • [claimed-docs] Kosmos autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report.
              • [claimed-docs] You can ask it a complex scientific question and get a high-accuracy, cited response, or task it with a deep literature review synthesizing …
              • [claimed-docs] Precedent determines whether a research idea has been tried before. It searches across fields to assess novelty and identify gaps
              • [claimed-docs] Molecules is a chemistry-focused agent for molecular design and analysis.
              • [claimed-docs] task_response = client.run_tasks_until_done(task_data)
              • [claimed-docs] Given a research objective and one or more datasets, Kosmos autonomously reads literature, writes and executes analysis code, generates hypo…
              Gemini Notebook (NotebookLM)partialcommunity4/10

              NotebookLM lets users chat with sources and generate transformed outputs (summaries, audio/video overviews, study guides, slide decks), which is a form of task delegation, but there is no evidence of an agentic assistant that can take autonomous multi-step actions, use tools, browse/search independently, or execute tasks beyond source-grounded Q&A and content transformation; community feedback even notes it can't search for docs on its own (notebooklm-comm-4). missing for 10: autonomous multi-step task execution, tool/agent orchestration, independent web search/retrieval, evidence of true delegation beyond content generation.

              • [claimed-docs] Chat with your notebook to get grounded information based on your sources with clear in-line citations for accuracy, transparency, and trust…
              • [claimed-docs] Transform your sources into approachable formats such as study guides, briefings, audio overviews, mind maps, and more.
              • [claimed-docs] Audio Overviews are deep-dive discussions between AI hosts that provide in-depth summaries of the key topics in your uploaded sources.
              • [claimed-docs] Transform your sources into a polished presentation by generating a Slide Deck, perfect for sharing or reviewing key concepts.
              • [community] My problem with NotebookLM was that you had to give it URLs or docs, you couldn't just have it search for docs and then incorporate them int…
            • ai-native userOperate the product with natural-language commands

              weight 2 · round to FutureHouse Platform
              FutureHouse Platformpartialclaimed7/10

              The platform's core interaction model is natural-language queries (e.g., the query string "Which neglected diseases had a treatment developed by artificial intelligence?") passed to autonomous agents like Kosmos, Precedent, and Molecules, and docs explicitly say users can 'ask it a complex scientific question' or 'task it with a deep literature review.' However, the demonstrated flow is via a Python client (task_data dict + run_tasks_until_done), not a conversational chat UI, so it's NL-as-input-parameter rather than full conversational natural-language operation of the whole platform (e.g., no evidence of multi-turn dialogue or a chat interface). Missing for 10: evidence of a conversational/chat UI, multi-turn NL interaction, or NL control over platform features beyond the initial query.

              • [claimed-docs] task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…
              • [claimed-docs] Kosmos autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report.
              • [claimed-docs] You can ask it a complex scientific question and get a high-accuracy, cited response, or task it with a deep literature review synthesizing …
              • [claimed-docs] Given a research objective and one or more datasets, Kosmos autonomously reads literature, writes and executes analysis code, generates hypo…
              • [claimed-docs] task it with a deep literature review synthesizing conflicting evidence across hundreds of papers
              Gemini Notebook (NotebookLM)partialcommunity6/10

              NotebookLM lets users interact via natural-language chat to query sources, request custom conversational styles/roles, and ask questions (notebooklm-docs-2, notebooklm-docs-7, notebooklm-docs-28), which is a genuine natural-language interface. However, many core actions (uploading sources, converting notes, generating mind maps/slide decks/audio overviews) are triggered via UI menus/buttons rather than free-form commands, and community feedback notes limits like being unable to have it search for and incorporate new sources on its own (notebooklm-comm-4). Missing for 10: evidence of natural-language commands driving broader agentic actions (e.g., source discovery, format generation, sharing) rather than just chat Q&A and style prompts, and independent confirmation of command robustness.

              • [claimed-docs] Chat with your notebook to get grounded information based on your sources with clear in-line citations for accuracy, transparency, and trust…
              • [claimed-docs] Custom: Best for choosing a specific style ("Respond like a PhD student") or suggesting a role ("Pretend to be a role-playing game host.").
              • [claimed-docs] Get insights in the moment by asking questions about your sources.
              • [community] My problem with NotebookLM was that you had to give it URLs or docs, you couldn't just have it search for docs and then incorporate them int…

            Api quality

            1. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

              weight 2 · round drawn
              FutureHouse Platformnone0/10

              Probes for OpenAPI/swagger spec paths all returned 404, and no docs reference a machine-readable API spec; only a Python client library and manual REST auth flow are documented. Missing for 10: any OpenAPI/Swagger endpoint or downloadable spec file, any documentation mentioning API schema export.

              • [probe] PROBE openapi: all candidate paths 404 (https://futurehouse.gitbook.io/openapi.json, https://futurehouse.gitbook.io/swagger.json, https://fu…
              Gemini Notebook (NotebookLM)none0/10

              No evidence anywhere in the pack of an OpenAPI spec, API documentation, or any machine-readable API for Gemini Notebook; all evidence covers UI features (sources, notes, audio/video overviews, sharing). missing for 10: any mention of a public API, OpenAPI/Swagger spec, or SDK.

              • ai-native userRely on versioned APIs with a documented deprecation policy

                weight 2 · round drawn
                FutureHouse Platformnone0/10

                No evidence of API versioning scheme, changelog, or deprecation policy anywhere in the docs; OpenAPI spec probes all 404 and no versioning documentation is present. missing for 10: versioned API endpoints, changelog/release notes, documented deprecation policy or timelines.

                • [probe] PROBE openapi: all candidate paths 404 (https://futurehouse.gitbook.io/openapi.json, https://futurehouse.gitbook.io/swagger.json, https://fu…
                Gemini Notebook (NotebookLM)none0/10

                No evidence of a versioned public API for NotebookLM/Gemini Notebook, let alone a documented deprecation policy; all evidence covers end-user features (uploads, summaries, audio/video overviews) not developer API access. Missing for 10: any mention of an API, versioning scheme, or deprecation/support-lifecycle policy.

                Automation depth — how much of the product can run unattendedAutomation depth

                How much of the product can run unattended

                1. ai-native userPerform bulk operations across many items at once

                  weight 2 · round drawn
                  FutureHouse Platformnone0/10

                  The docs show only single-task submission via task_data/run_tasks_until_done, with no mention of batch endpoints, bulk task lists, or parallel multi-item processing APIs; missing for 10: batch/bulk submission API, documented loop or list-based task queuing, evidence of concurrent multi-item execution.

                  • [claimed-docs] task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…
                  • [claimed-docs] task_response = client.run_tasks_until_done(task_data)
                  Gemini Notebook (NotebookLM)none0/10

                  The evidence only shows per-source actions (uploading individual sources, converting a single note to a source, generating one summary or audio overview at a time) with no mention of bulk/batch operations across many sources, notebooks, or items simultaneously. There is no documented bulk-select, bulk-delete, bulk-export, or batch-processing capability anywhere in the docs or community feedback.

                  • [claimed-docs] Easily upload PDFs, websites, YouTube videos, audio files, Google Docs, or Google Slides or discover new sources.
                  • [claimed-docs] To convert a single note to a source: Open the note. Select Convert to source.
                  • [claimed-docs] Each source can contain up to 500,000 words or up to 200MB for uploaded files.
                2. ai-native userSchedule recurring jobs or workflows

                  weight 2 · round drawn
                  FutureHouse Platformnone0/10

                  Evidence covers running one-off tasks via the client (run_tasks_until_done) but there is no mention of scheduling, cron-like recurrence, or persistent workflow automation anywhere in the docs or probes.

                  • [claimed-docs] task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…
                  • [claimed-docs] task_response = client.run_tasks_until_done(task_data)
                  Gemini Notebook (NotebookLM)none0/10

                  No evidence anywhere in the pack of scheduling, recurring jobs, triggers, or automated workflow execution; NotebookLM is a research/notebook tool driven by manual source uploads and on-demand queries.

                  Collaboration sharing — stories about collaboration sharing in this arenaCollaboration sharing

                  Stories about collaboration sharing in this arena

                  Sharing

                  1. analystShare a research session or report with collaborators who can view or build on it

                    weight 2 · round to Gemini Notebook (NotebookLM)
                    FutureHouse Platformnone0/10

                    No evidence in the pack mentions sharing sessions, reports, or collaborative access controls; documentation focuses on API usage, agents, and task execution, not collaboration features. Missing for 10: any mention of sharing/exporting reports, collaborator invites, or team/workspace features.

                      Gemini Notebook (NotebookLM)partialclaimed7/10

                      Docs confirm notebooks can be made public via shareable link, that Workspace admins now treat Gemini Notebook as a core service (implying org-level sharing), and that notes/slide decks/reports can be exported to Google Docs/Sheets or PDF so collaborators can view and continue working on them. However, evidence only describes public link sharing and analytics, not granular collaborator invite/edit permissions typical of collaborative docs, and there's no independent/hands-on account of real-world collaboration workflows. Missing for 10: explicit collaborator role/permission controls (view vs edit), independent corroboration of multi-user collaboration in practice.

                      • [claimed-docs] You can make your notebook public and share a link with anyone with a Google account.
                      • [claimed-docs] Owners of public notebooks with paid subscriptions can view usage analytics in the analytics tab.
                      • [claimed-docs] You can export your notes to Google Docs or Google Sheets.
                      • [claimed-docs] For Workspace and Workspace for Education users, Gemini Notebook is now a core service (like Gmail, Google Calendar, and Docs).
                      • [claimed-docs] Transform your sources into a polished presentation by generating a Slide Deck, perfect for sharing or reviewing key concepts. Present your …

                    Literature workflow — stories about literature workflow in this arenaLiterature workflow

                    Stories about literature workflow in this arena

                    Alerts

                    1. researcherSet up standing searches or alerts that surface new relevant sources as they appear

                      weight 1 · round drawn
                      FutureHouse Platformnone0/10

                      The evidence describes on-demand literature search, review, and analysis agents (Kosmos, Precedent, etc.) invoked via API calls, but nothing describes standing searches, saved alerts, or recurring notifications when new relevant sources appear.

                        Gemini Notebook (NotebookLM)none0/10

                        No evidence of any standing-search/alert capability that surfaces new relevant sources automatically; sources must be manually uploaded or discovered by the user. Community feedback explicitly confirms this gap: 'you had to give it URLs or docs, you couldn't just have it search for docs and then incorporate them into your research' ([notebooklm-comm-4]).

                        • [community] My problem with NotebookLM was that you had to give it URLs or docs, you couldn't just have it search for docs and then incorporate them int…
                        • [claimed-docs] Easily upload PDFs, websites, YouTube videos, audio files, Google Docs, or Google Slides or discover new sources.

                      Corpus

                      1. researcherUpload my own PDFs or corpus and have the agent research over them

                        weight 2 · round to Gemini Notebook (NotebookLM)
                        FutureHouse Platformnone0/10

                        The docs describe FutureHouse's agents (Kosmos, literature review, Precedent) searching a fixed 175M+ paper corpus and analyzing 'datasets' for experiments, but nowhere describe a mechanism for a researcher to upload their own PDFs or personal corpus for the agent to search/reason over. Without evidence of a file-upload or custom-corpus ingestion feature, this capability is not shown.

                        • [claimed-docs] Given a research objective and one or more datasets, Kosmos autonomously reads literature, writes and executes analysis code, generates hypo…
                        • [claimed-docs] It accesses 175M+ papers, trials, and patents with native understanding of citation graphs, journal quality, and clinical trial data.
                        • [claimed-docs] specializes in processing complex experimental data, including flow cytometry, RNA-seq, and other biological datasets
                        • [claimed-docs] task it with a deep literature review synthesizing conflicting evidence across hundreds of papers
                        Gemini Notebook (NotebookLM)fullcommunity8/10

                        Docs confirm PDF/document upload (up to 500k words/200MB per source), grounded Q&A with inline citations navigable to source location, source guide summaries, and note-taking over uploaded corpora—directly matching the story. Community evidence corroborates real-world use for structured research and studying, though one comment notes it can't proactively search for and pull in new docs beyond what's uploaded. Missing for 10: independent benchmarking of research quality/accuracy across large multi-PDF corpora, and confirmation of robust handling of very large or messy academic corpora at scale.

                        • [claimed-docs] Easily upload PDFs, websites, YouTube videos, audio files, Google Docs, or Google Slides or discover new sources.
                        • [claimed-docs] Chat with your notebook to get grounded information based on your sources with clear in-line citations for accuracy, transparency, and trust…
                        • [claimed-docs] Each source can contain up to 500,000 words or up to 200MB for uploaded files.
                        • [claimed-docs] Find an auto-generated summary of the entire source in the Source Guide.
                        • [claimed-docs] You can hover over any citation to get the full quoted text right away.
                        • [claimed-docs] If you select a citation, Gemini Notebook automatically navigates to the location of the quote, so you can easily view it in context.
                        • [claimed-docs] You can use notes to capture and organize information, identify key insights and interpretations from your sources, and add your own thought…
                        • [community] NotebookLM is genuinely useful for structured research workflows. My concern is not the name. It's whether the "notebook" metaphor survives …
                        • [community] My problem with NotebookLM was that you had to give it URLs or docs, you couldn't just have it search for docs and then incorporate them int…

                      Reviews

                      1. researcherRun a systematic screening and extraction workflow across many papers with consistent criteria

                        weight 2 · round to FutureHouse Platform
                        FutureHouse Platformpartialprobed5/10

                        The API client supports programmatic task submission (e.g., JobNames.LITERATURE) and batch execution via run_tasks_until_done, and the docs describe a literature agent that can synthesize evidence across hundreds of papers with citations, which could be scripted for repeated, consistent queries. However, there is no explicit documented workflow for structured screening/extraction with defined inclusion/exclusion criteria across a large corpus (e.g., systematic review style tables), and the cookbook page itself returns 404 on the probe, reducing confidence in the full pipeline. missing for 10: explicit systematic-review/extraction workflow docs, example of consistent-criteria batch screening, working cookbook page confirming the API details.

                        • [claimed-docs] task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…
                        • [claimed-docs] task_response = client.run_tasks_until_done(task_data)
                        • [claimed-docs] task it with a deep literature review synthesizing conflicting evidence across hundreds of papers
                        • [claimed-docs] You can ask it a complex scientific question and get a high-accuracy, cited response, or task it with a deep literature review synthesizing …
                        • [probe] PROBE docs-md: HTTP 200 at https://futurehouse.gitbook.io/futurehouse-cookbook.md # Page Not Found The URL `futurehouse-cookbook` does not …
                        Gemini Notebook (NotebookLM)partialcommunity3/10

                        NotebookLM supports uploading many sources into one notebook and chatting with grounded, cited answers, which could be used ad hoc to ask the same question across sources, but there is no evidence of a systematic screening/extraction feature (e.g., structured criteria templates, per-paper data tables, PRISMA-style tracking, or bulk batch processing) needed for a repeatable literature review workflow. Community feedback also notes the product cannot auto-search/pull in new sources itself, requiring manual curation, which undercuts a large-scale systematic workflow. Missing for 10: structured extraction templates/tables applied consistently across many papers, evidence of handling large paper sets with defined screening criteria, and confirmation this scales beyond source-count limits.

                        • [claimed-docs] Chat with your notebook to get grounded information based on your sources with clear in-line citations for accuracy, transparency, and trust…
                        • [claimed-docs] You can hover over any citation to get the full quoted text right away.
                        • [claimed-docs] Get insights in the moment by asking questions about your sources.
                        • [community] My problem with NotebookLM was that you had to give it URLs or docs, you couldn't just have it search for docs and then incorporate them int…

                      Openness — open source, data portability, and self-hosting storiesOpenness

                      Open source, data portability, and self-hosting stories

                      1. ai-native userDo everything through the API that I can do in the UI

                        weight 2 · round to FutureHouse Platform
                        FutureHouse Platformpartialprobed5/10

                        The edison-client library and quickstart docs show that core agent tasks (e.g., JobNames.LITERATURE) can be invoked via API with authentication and task submission (docs-1–4,12), and the cookbook describes multiple agents (Kosmos, Precedent, Molecules) that presumably map to API job types, suggesting broad functional overlap with the UI. However, there is no explicit documentation stating full feature parity between UI and API, no OpenAPI/swagger spec discoverable (probe-3 shows 404s across all candidate paths), and no confirmation that all UI-exposed capabilities (e.g., account management, dataset uploads, report viewing) are also API-accessible. missing for 10: explicit parity statement, discoverable OpenAPI schema, evidence covering non-task UI features (billing, dataset management, report exports) via API.

                        • [claimed-docs] task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…
                        • [claimed-docs] task_response = client.run_tasks_until_done(task_data)
                        • [claimed-docs] Kosmos autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report.
                        • [claimed-docs] Precedent determines whether a research idea has been tried before. It searches across fields to assess novelty and identify gaps
                        • [claimed-docs] Molecules is a chemistry-focused agent for molecular design and analysis.
                        • [probe] PROBE openapi: all candidate paths 404 (https://futurehouse.gitbook.io/openapi.json, https://futurehouse.gitbook.io/swagger.json, https://fu…
                        Gemini Notebook (NotebookLM)none0/10

                        No evidence pack items mention any public API, developer SDK, or programmatic access to Gemini Notebook features; all evidence describes UI-only workflows (upload, chat, generate audio/video/slides, notes, sharing). Community comments only discuss UI limitations, not API availability. missing for 10: any mention of a public API, SDK, or programmatic endpoints; documentation of API parity with UI features; developer access/authentication details.

                        • ai-native userExport all of my data in open formats and leave

                          weight 3 · round to Gemini Notebook (NotebookLM)
                          FutureHouse Platformnone0/10

                          No evidence of any data export capability, open-format export, or account portability/deletion feature; documentation covers agent/task usage and API access but nothing about exporting user data or leaving with it.

                            Gemini Notebook (NotebookLM)partialclaimed3/10

                            Docs show only piecemeal export options—notes to Google Docs/Sheets, audio overview downloads, and slide decks as PDF—but no evidence of a full data export (sources, chat history, citations, mind maps) or a genuine 'leave the platform' workflow in open, portable formats. Missing for 10: bulk/full-notebook export, open-format guarantees (vs Google Docs/Sheets lock-in), and any documentation or community confirmation of a complete account data export/exit path.

                            • [claimed-docs] You can export your notes to Google Docs or Google Sheets.
                            • [claimed-docs] Listen to Audio Overviews on the go and download them for offline playback.
                            • [claimed-docs] Transform your sources into a polished presentation by generating a Slide Deck, perfect for sharing or reviewing key concepts. Present your …

                          Pricing limits — free-tier ceilings, usage caps, and rate limits before you have to payPricing limits

                          Free-tier ceilings, usage caps, and rate limits before you have to pay

                          Pricing

                          1. researcherTry the product meaningfully on a free tier or trial

                            weight 1 · round to FutureHouse Platform
                            FutureHouse Platformfullclaimed7/10

                            Docs explicitly state Edison maintains a generous free tier for academics, with paid plans only needed for higher rate limits/additional features, and the quickstart shows a full API key + client workflow to run real tasks (literature review job). Missing for 10: independent/hands-on confirmation of free tier limits and no clarity on exact quota/duration restrictions.

                            • [claimed-docs] Edison maintains a generous free tier for academics. Researchers who need higher rate limits or additional features can subscribe to paid pl…
                            • [claimed-docs] Researchers who need higher rate limits or additional features can subscribe to paid plans.
                            • [claimed-docs] Authentication is done by providing an API key, which can be obtained directly from your profile page in the Edison platform
                            • [claimed-docs] Under "API Tokens", click "Create New Token."
                            • [claimed-docs] task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…
                            • [claimed-docs] task_response = client.run_tasks_until_done(task_data)
                            Gemini Notebook (NotebookLM)partialcommunity4/10

                            Evidence confirms a paid subscription tier exists (analytics for paid users) implying a separate free tier, and community comments describe hands-on use (audio overviews, code walkthroughs) without mention of payment, suggesting meaningful free access. However, no doc explicitly states free-tier quotas, source limits, or trial duration for researchers. Missing for 10: explicit free-tier usage caps/limits documentation, direct confirmation of no-cost access, and independent confirmation of research-scale usability without upgrading.

                            • [claimed-docs] Owners of public notebooks with paid subscriptions can view usage analytics in the analytics tab.
                            • [community] I used it to listen to a high level walkthrough of interesting and entertaining code while driving to work... Hearing it talk about Ken Thom…
                            • [community] Much better name for this well-designed product. I do wish they made it easier to pull in emails. I had to use gyb and mhonarc to retrieve e…
                          2. researcherUnderstand plan pricing and usage limits before committing

                            weight 2 · round to FutureHouse Platform
                            FutureHouse Platformpartialclaimed4/10

                            Docs confirm a free tier for academics and paid plans for higher rate limits/features, but no concrete pricing figures, tier breakdowns, or specific usage limit numbers are provided anywhere in the pack. missing for 10: actual price points, detailed rate-limit thresholds, feature comparison table across tiers, billing/usage dashboard evidence.

                            • [claimed-docs] Researchers who need higher rate limits or additional features can subscribe to paid plans.
                            • [claimed-docs] Edison maintains a generous free tier for academics. Researchers who need higher rate limits or additional features can subscribe to paid pl…
                            Gemini Notebook (NotebookLM)partialclaimed3/10

                            Docs mention some usage limits (500,000 words/200MB per source) and note that usage analytics require a paid subscription and Workspace inclusion, but there is no comprehensive pricing page, tier comparison, or explicit usage caps (e.g., number of notebooks, audio overviews per day) that a researcher would need to fully evaluate cost before committing. Missing for 10: full pricing tier breakdown, free vs paid feature/usage limits, and independent confirmation of these limits.

                            • [claimed-docs] Each source can contain up to 500,000 words or up to 200MB for uploaded files.
                            • [claimed-docs] Owners of public notebooks with paid subscriptions can view usage analytics in the analytics tab.
                            • [claimed-docs] For Workspace and Workspace for Education users, Gemini Notebook is now a core service (like Gmail, Google Calendar, and Docs).

                          Privacy posture — data-handling and privacy storiesPrivacy posture

                          Data-handling and privacy stories

                          1. ai-native userChoose where my data is stored (region/residency)

                            weight 2 · round drawn
                            FutureHouse Platformnone0/10

                            No evidence in the pack addresses data residency, regional storage options, or geographic control over where data is processed/stored; documentation only covers API usage, agents, and pricing tiers.

                              Gemini Notebook (NotebookLM)none0/10

                              No evidence in the pack mentions data residency, region selection, or storage location controls for Gemini Notebook; the docs cover features like sources, audio/video overviews, and sharing but say nothing about where data is stored or user control over it.

                              • ai-native userPrevent my data from being used to train AI models

                                weight 3 · round drawn
                                FutureHouse Platformnone0/10

                                No evidence in the pack addresses data usage for AI training opt-out, data privacy controls, or any training-data policy; the docs focus entirely on product features and API usage.

                                  Gemini Notebook (NotebookLM)none0/10

                                  No evidence in the pack addresses data-training opt-out, privacy policy, or data usage controls for Gemini Notebook; all citations concern features (sources, summaries, audio/video overviews) rather than training-data privacy posture.

                                  • ai-native userControl data retention and deletion

                                    weight 2 · round drawn
                                    FutureHouse Platformnone0/10

                                    No evidence in the pack addresses data retention policies, deletion controls, or privacy/data lifecycle management; documentation focuses on client setup, agents, and use cases only.

                                      Gemini Notebook (NotebookLM)none0/10

                                      No evidence in the pack addresses data retention controls, deletion policies, or user ability to manage/export/delete stored data and sources beyond basic note export; the axis clearly applies to an AI-native privacy-conscious user but no documentation or community mention covers retention/deletion controls.

                                      • ai-native userOpt out of telemetry and usage tracking

                                        weight 2 · round drawn
                                        FutureHouse Platformnone0/10

                                        No evidence in the pack mentions telemetry, usage tracking, analytics collection, or an opt-out mechanism anywhere in the FutureHouse docs or probes; the documentation focuses on client setup and agent features only.

                                          Gemini Notebook (NotebookLM)none0/10

                                          No evidence pack item mentions telemetry, usage-tracking opt-out, privacy controls, or data-collection settings; only feature docs and general community commentary are present.

                                          Report output — stories about report output in this arenaReport output

                                          Stories about report output in this arena

                                          Reports

                                          1. researcherExport results to common formats, including documents, spreadsheets, and reference-manager files

                                            weight 1 · round to Gemini Notebook (NotebookLM)
                                            FutureHouse Platformnone0/10

                                            The evidence describes cited reports and analyses produced by Kosmos/agents but nowhere mentions exporting to document, spreadsheet, or reference-manager formats (e.g., PDF/DOCX, CSV/XLSX, BibTeX/RIS); the docs-cookbook page itself is even reported as not found in the probe.

                                            • [claimed-docs] Kosmos autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report.
                                            • [claimed-docs] Every conclusion is fully auditable. You can trace any finding back to the specific code or literature passage that produced it.
                                            • [claimed-docs] It turns raw data into detailed analyses, statistical results, and publication-ready figures.
                                            • [probe] PROBE docs-md: HTTP 200 at https://futurehouse.gitbook.io/futurehouse-cookbook.md # Page Not Found The URL `futurehouse-cookbook` does not …
                                            Gemini Notebook (NotebookLM)partialclaimed5/10

                                            NotebookLM supports exporting notes to Google Docs or Google Sheets and can generate a Slide Deck exportable as PDF, covering document/spreadsheet-style outputs, but there is no evidence of export to reference-manager formats (e.g., BibTeX, RIS, EndNote) which the story explicitly requires. missing for 10: reference-manager file export, independent corroboration of export fidelity.

                                            • [claimed-docs] You can export your notes to Google Docs or Google Sheets.
                                            • [claimed-docs] Transform your sources into a polished presentation by generating a Slide Deck, perfect for sharing or reviewing key concepts. Present your …
                                          2. analystGet a structured report with sections, tables, and a summary that I can share with stakeholders

                                            weight 3 · round drawn
                                            FutureHouse Platformpartialclaimed6/10

                                            Docs describe Kosmos/agents producing a 'comprehensive cited report' with statistical results, publication-ready figures, and traceable citations, which implies structured, shareable output, but there's no explicit mention of report sections/tables format or export/sharing mechanisms for stakeholders. missing for 10: explicit description of report structure (sections/tables), export or sharing functionality, and independent confirmation of report format quality.

                                            • [claimed-docs] Kosmos autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report.
                                            • [claimed-docs] Every conclusion is fully auditable. You can trace any finding back to the specific code or literature passage that produced it.
                                            • [claimed-docs] It turns raw data into detailed analyses, statistical results, and publication-ready figures.
                                            • [claimed-docs] Given a research objective and one or more datasets, Kosmos autonomously reads literature, writes and executes analysis code, generates hypo…
                                            • [claimed-docs] You can trace any finding back to the specific code or literature passage that produced it.
                                            Gemini Notebook (NotebookLM)partialclaimed6/10

                                            NotebookLM supports Reports with FAQ/study guide/briefing document/custom formats and auto-generated source summaries, plus export of notes to Google Docs/Sheets and Slide Deck generation for sharing, which together cover structured, shareable outputs. However, evidence doesn't confirm native table generation within reports or a single unified 'report' combining sections+tables+summary in one export artifact. missing for 10: explicit table support in generated reports, a single cohesive exportable report format (vs separate docs/sheets/slides), independent/hands-on validation of report quality for stakeholder sharing.

                                            • [claimed-docs] Create your own, or choose from FAQ, study guide, briefing document, or an AI suggested report type.
                                            • [claimed-docs] Find an auto-generated summary of the entire source in the Source Guide.
                                            • [claimed-docs] You can export your notes to Google Docs or Google Sheets.
                                            • [claimed-docs] Transform your sources into a polished presentation by generating a Slide Deck, perfect for sharing or reviewing key concepts.
                                            • [claimed-docs] Transform your sources into a polished presentation by generating a Slide Deck, perfect for sharing or reviewing key concepts. Present your …

                                          Research depth — stories about research depth in this arenaResearch depth

                                          Stories about research depth in this arena

                                          Agent runs

                                          1. researcherPose a research question and get an autonomous multi-step investigation, not just a single-pass summary

                                            weight 3 · round to FutureHouse Platform
                                            FutureHouse Platformfullclaimed8/10

                                            Docs describe Kosmos and other agents autonomously reading literature, writing/executing analysis code, generating hypotheses, and producing cited multi-step reports (not single-pass summaries), with auditable traceability of findings and multi-agent capabilities (Precedent, Molecules, literature review across hundreds of papers). This directly matches the story of posing a question and getting an autonomous multi-step investigation. Missing for 10: independent/hands-on verification of the multi-step autonomy claims (probe shows docs page 404'd, reducing corroboration strength) and no third-party case study confirming real-world multi-step behavior.

                                            • [claimed-docs] Kosmos autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report.
                                            • [claimed-docs] Given a research objective and one or more datasets, Kosmos autonomously reads literature, writes and executes analysis code, generates hypo…
                                            • [claimed-docs] Every conclusion is fully auditable. You can trace any finding back to the specific code or literature passage that produced it.
                                            • [claimed-docs] You can ask it a complex scientific question and get a high-accuracy, cited response, or task it with a deep literature review synthesizing …
                                            • [claimed-docs] task it with a deep literature review synthesizing conflicting evidence across hundreds of papers
                                            • [claimed-docs] Precedent determines whether a research idea has been tried before. It searches across fields to assess novelty and identify gaps
                                            Gemini Notebook (NotebookLM)none0/10

                                            All evidence describes NotebookLM as a source-grounded chat/summarization tool: users upload sources, ask questions, and get single-pass answers, summaries, or generated formats (audio overviews, mind maps, slide decks) — there is no evidence of an autonomous multi-step investigation loop that iteratively searches, plans, or expands beyond the provided sources. A community report explicitly notes it cannot autonomously search for and incorporate new documents into research, reinforcing that it is single-pass/source-bound rather than agentic.

                                            • [claimed-docs] Chat with your notebook to get grounded information based on your sources with clear in-line citations for accuracy, transparency, and trust…
                                            • [claimed-docs] Find an auto-generated summary of the entire source in the Source Guide.
                                            • [claimed-docs] Create your own, or choose from FAQ, study guide, briefing document, or an AI suggested report type.
                                            • [community] My problem with NotebookLM was that you had to give it URLs or docs, you couldn't just have it search for docs and then incorporate them int…
                                          2. analystStart a long research job that keeps working unattended and notifies me when the result is ready

                                            weight 2 · round to FutureHouse Platform
                                            FutureHouse Platformpartialclaimed5/10

                                            The client SDK exposes a task submission and `run_tasks_until_done` pattern (docs-4, docs-12) plus descriptions of Kosmos autonomously running long, multi-step research (literature review, code execution, hypothesis generation) unattended (docs-5, docs-13, docs-19), supporting the 'long unattended job' half of the story. However there is no evidence of any notification mechanism (email, webhook, push) when a job completes — the only documented pattern is a client-side polling call that blocks until done, not an async notify-on-completion flow. Missing for 10: explicit notification/webhook/email-on-completion feature, evidence of true async fire-and-forget job submission separate from polling.

                                            • [claimed-docs] task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…
                                            • [claimed-docs] task_response = client.run_tasks_until_done(task_data)
                                            • [claimed-docs] Kosmos autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report.
                                            • [claimed-docs] Given a research objective and one or more datasets, Kosmos autonomously reads literature, writes and executes analysis code, generates hypo…
                                            • [claimed-docs] task it with a deep literature review synthesizing conflicting evidence across hundreds of papers
                                            Gemini Notebook (NotebookLM)none0/10

                                            No evidence in the pack describes an asynchronous/background 'long research job' mode with completion notifications; all cited capabilities (chat, summaries, audio/video overviews, flashcards, etc.) are generated within an active session, not unattended long-running jobs with alerts. Missing for 10: any documented async job queue, background processing status, or notification/alert mechanism when a task completes.

                                            • researcherSteer the depth, effort, and scope of a research run before or while it executes

                                              weight 1 · round to Gemini Notebook (NotebookLM)
                                              FutureHouse Platformnone0/10

                                              Evidence shows job submission with a query and job name, plus a mid-run polling helper (run_tasks_until_done), but no documented parameters or controls for adjusting depth, effort level, or scope of a run before or during execution.

                                              • [claimed-docs] task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…
                                              • [claimed-docs] task_response = client.run_tasks_until_done(task_data)
                                              Gemini Notebook (NotebookLM)partialclaimed4/10

                                              NotebookLM lets users pick conversational style/persona (Default, Custom "PhD student" etc.) and choose report/output formats (FAQ, briefing, study guide), which offers some steering of scope and tone, but there is no evidence of controlling depth/effort (e.g., reasoning length, source breadth, iteration count) or adjusting a run mid-execution — outputs like Audio/Video Overviews are generated as fixed artifacts rather than steerable in-progress runs. missing for 10: mid-run steering controls, explicit depth/effort sliders or parameters, ability to expand/narrow scope during execution, independent evidence of these controls working as intended.

                                              • [claimed-docs] Custom: Best for choosing a specific style ("Respond like a PhD student") or suggesting a role ("Pretend to be a role-playing game host.").
                                              • [claimed-docs] Select a conversational style. Default: Best for general purpose research and brainstorming tasks.
                                              • [claimed-docs] Create your own, or choose from FAQ, study guide, briefing document, or an AI suggested report type.
                                              • [claimed-docs] Choose a format. Deep Dive (default) for an in-depth conversation. Two hosts unpack and connect topics from your sources in a lively convers…

                                            Source quality — stories about source quality in this arenaSource quality

                                            Stories about source quality in this arena

                                            Citations

                                            1. researcherSee citations for every substantive claim so I can verify it against the underlying source

                                              weight 3 · round to Gemini Notebook (NotebookLM)
                                              FutureHouse Platformfullclaimed7/10

                                              Docs state that outputs are cited and auditable, with claims traceable back to specific literature passages or code, and that literature review/answer agents produce cited responses. This is first-party documentation only, with no independent/hands-on corroboration of citation accuracy or completeness. missing for 10: independent verification of citation accuracy, example of an actual cited report output, detail on citation format/traceability mechanism.

                                              • [claimed-docs] Every conclusion is fully auditable. You can trace any finding back to the specific code or literature passage that produced it.
                                              • [claimed-docs] You can trace any finding back to the specific code or literature passage that produced it.
                                              • [claimed-docs] You can ask it a complex scientific question and get a high-accuracy, cited response, or task it with a deep literature review synthesizing …
                                              • [claimed-docs] Kosmos autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report.
                                              • [claimed-docs] Given a research objective and one or more datasets, Kosmos autonomously reads literature, writes and executes analysis code, generates hypo…
                                              Gemini Notebook (NotebookLM)fullclaimed9/10

                                              Docs confirm grounded chat responses come with clear in-line citations, hovering shows the exact quoted text, and clicking a citation navigates directly to the source location for verification — directly matching the story's requirement to verify claims against underlying sources. Community evidence corroborates general usefulness but doesn't specifically contest citation accuracy. Missing for 10: independent hands-on verification of citation accuracy/completeness across diverse source types.

                                              • [claimed-docs] Chat with your notebook to get grounded information based on your sources with clear in-line citations for accuracy, transparency, and trust…
                                              • [claimed-docs] You can hover over any citation to get the full quoted text right away.
                                              • [claimed-docs] If you select a citation, Gemini Notebook automatically navigates to the location of the quote, so you can easily view it in context.

                                            Corpus

                                            1. researcherSearch scholarly literature and primary sources, not just the open web

                                              weight 2 · round to FutureHouse Platform
                                              FutureHouse Platformfullclaimed8/10

                                              Docs describe agents (LITERATURE task, Kosmos, Precedent) that explicitly search scholarly literature, trials, and patents, accessing '175M+ papers, trials, and patents' with citation graph and journal quality awareness, and produce cited reports auditable back to specific literature passages. This directly matches the researcher story of searching scholarly/primary sources beyond the open web. Missing for 10: independent/hands-on corroboration of search coverage and source quality beyond vendor docs, and clarity on which specific databases/indices are used.

                                              • [claimed-docs] It accesses 175M+ papers, trials, and patents with native understanding of citation graphs, journal quality, and clinical trial data.
                                              • [claimed-docs] Precedent determines whether a research idea has been tried before. It searches across fields to assess novelty and identify gaps
                                              • [claimed-docs] Precedent determines whether a research idea has been tried before. It searches across fields to assess novelty and identify gaps, helping y…
                                              • [claimed-docs] Kosmos autonomously reads literature, writes and executes analysis code, generates hypotheses, and produces a comprehensive cited report.
                                              • [claimed-docs] Every conclusion is fully auditable. You can trace any finding back to the specific code or literature passage that produced it.
                                              • [claimed-docs] task_data = { "name": JobNames.LITERATURE, "query": "Which neglected diseases had a treatment developed by artificial intelligence?"…
                                              Gemini Notebook (NotebookLM)disputedcontradicted3/10

                                              Docs vaguely mention users can 'discover new sources' (notebooklm-docs-1), but there is no evidence of integration with scholarly databases (e.g., PubMed, JSTOR, Google Scholar) for searching primary literature, and a hands-on community report explicitly states you cannot have NotebookLM search for and incorporate sources itself—you must supply URLs or docs yourself (notebooklm-comm-4). This directly contradicts the implied 'discover' capability, making it a disputed claim rather than a confirmed feature. missing for 10: evidence of any scholarly/primary-source search integration, first-party documentation describing how 'discover sources' actually searches literature, and independent confirmation that discovery works as claimed.

                                              • [claimed-docs] Easily upload PDFs, websites, YouTube videos, audio files, Google Docs, or Google Slides or discover new sources.
                                              • [community] My problem with NotebookLM was that you had to give it URLs or docs, you couldn't just have it search for docs and then incorporate them int…

                                            Synthesis

                                            1. analystSee where sources agree and disagree instead of a single unqualified answer

                                              weight 2 · round to FutureHouse Platform
                                              FutureHouse Platformpartialclaimed5/10

                                              Docs mention tasking the platform with 'a deep literature review synthesizing conflicting evidence across hundreds of papers' and full auditability back to source passages, suggesting some capability to surface disagreement across sources, but there is no dedicated UI/feature shown for explicitly contrasting agreeing vs disagreeing sources, no example output demonstrating this, and no independent/hands-on verification. missing for 10: concrete example output showing conflicting evidence highlighted, dedicated UI feature for source agreement/disagreement, independent corroboration beyond vendor docs.

                                              • [claimed-docs] You can ask it a complex scientific question and get a high-accuracy, cited response, or task it with a deep literature review synthesizing …
                                              • [claimed-docs] task it with a deep literature review synthesizing conflicting evidence across hundreds of papers
                                              • [claimed-docs] Every conclusion is fully auditable. You can trace any finding back to the specific code or literature passage that produced it.
                                              • [claimed-docs] You can trace any finding back to the specific code or literature passage that produced it.
                                              Gemini Notebook (NotebookLM)none0/10

                                              The evidence describes grounded chat with citations, summaries, mind maps, and study aids, but nothing about explicitly surfacing agreement or contradiction across multiple sources. This is a fair capability to expect from a multi-source research tool, but no docs or community evidence mention comparative or conflict-detection analysis between sources.

                                              Not comparable on these axes

                                              1. ai-native userPoint an agent at llms.txt or agent-oriented docs

                                                weight 2 · not comparable
                                                FutureHouse Platformfullprobed7/10

                                                A working llms.txt file is directly confirmed via probe at futurehouse.gitbook.io/llms.txt returning HTTP 200 with structured agent-oriented documentation links, satisfying the core ask of pointing an agent at llms.txt. missing for 10: markdown-per-page fallback (docs-md probe returned 404), no OpenAPI/machine-readable schema confirmation, and no independent third-party corroboration of agents actually consuming this file successfully.

                                                • [probe] PROBE llms.txt: HTTP 200 at https://futurehouse.gitbook.io/llms.txt # Edison Scientific Documentation ## Home - [Overview](https://docs.ed…
                                                • [probe] PROBE docs-md: HTTP 200 at https://futurehouse.gitbook.io/futurehouse-cookbook.md # Page Not Found The URL `futurehouse-cookbook` does not …
                                                • [probe] PROBE openapi: all candidate paths 404 (https://futurehouse.gitbook.io/openapi.json, https://futurehouse.gitbook.io/swagger.json, https://fu…
                                                Gemini Notebook (NotebookLM)n/a

                                                Gemini Notebook is an end-user research/synthesis tool, not an agent framework or platform that consumes agent-oriented documentation like llms.txt; nothing in the evidence suggests it acts as an agent client consuming machine-readable docs. This axis is a category error for this product type.

                                                • ai-native userPlug MCP servers into this product so it can use their tools

                                                  weight 3 · not comparable
                                                  FutureHouse Platformn/a

                                                  FutureHouse Platform is a science-agent product (Kosmos, Precedent, Molecules, etc.) accessed via API/client, not an MCP-client tool for plugging in external MCP servers; no evidence discusses MCP integration at all, and the product's role is as an agent/service rather than an MCP-consuming host.

                                                    Gemini Notebook (NotebookLM)none0/10

                                                    No evidence that Gemini Notebook supports MCP server integration or any external tool-plugging mechanism; the product only supports fixed source uploads and generated outputs (audio/video/study guides). missing for 10: any mention of MCP support, tool/plugin ecosystem, or extensibility API.

                                                    • [claimed-docs] Easily upload PDFs, websites, YouTube videos, audio files, Google Docs, or Google Slides or discover new sources.
                                                    • [claimed-docs] Each source can contain up to 500,000 words or up to 200MB for uploaded files.
                                                  • ai-native userConnect an agent via an official MCP server

                                                    weight 3 · not comparable
                                                    FutureHouse Platformnone0/10

                                                    The evidence pack only shows a Python client (edison-client) for calling FutureHouse agents via API key, plus probes confirming no OpenAPI/MCP-related endpoints were found; there is no mention of an official MCP server for connecting agents.

                                                    • [claimed-docs] uv pip install edison-client
                                                    • [probe] PROBE openapi: all candidate paths 404 (https://futurehouse.gitbook.io/openapi.json, https://futurehouse.gitbook.io/swagger.json, https://fu…
                                                    Gemini Notebook (NotebookLM)n/a

                                                    Gemini Notebook is a research/document-synthesis product, not an agent framework or platform exposing an official MCP server for external agents; no evidence pack item mentions MCP or agent connectivity, and the product's role is a consumer-facing knowledge tool rather than an integration surface for agents.

                                                    • ai-native userIssue scoped/least-privilege API credentials for an agent

                                                      weight 2 · not comparable
                                                      FutureHouse Platformnone0/10

                                                      Evidence shows only a single, account-wide API token creation flow with no mention of scopes, permissions, or least-privilege controls for agents; no evidence of scoped or restricted credential issuance.

                                                      • [claimed-docs] Authentication is done by providing an API key, which can be obtained directly from your profile page in the Edison platform
                                                      • [claimed-docs] Under "API Tokens", click "Create New Token."
                                                      • [claimed-docs] Under "API Tokens," click "Create New Token."
                                                      Gemini Notebook (NotebookLM)n/a

                                                      Gemini Notebook (NotebookLM) is a research/knowledge-synthesis tool for end users, not a platform that issues API credentials for agents; no evidence pack content relates to credential scoping or agent access control, and this axis is a category error for this product type.

                                                      • ai-native userExplore an interactive API reference with runnable examples

                                                        weight 2 · not comparable
                                                        FutureHouse Platformnone0/10

                                                        Evidence shows only a quickstart guide with basic client code snippets, not an interactive API reference with runnable examples; probes for OpenAPI/swagger specs and doc endpoints all returned 404s, indicating no interactive reference exists.

                                                        • [probe] PROBE openapi: all candidate paths 404 (https://futurehouse.gitbook.io/openapi.json, https://futurehouse.gitbook.io/swagger.json, https://fu…
                                                        • [probe] PROBE docs-md: HTTP 200 at https://futurehouse.gitbook.io/futurehouse-cookbook.md # Page Not Found The URL `futurehouse-cookbook` does not …
                                                        • [claimed-docs] uv pip install edison-client
                                                        • [claimed-docs] task_response = client.run_tasks_until_done(task_data)
                                                        Gemini Notebook (NotebookLM)n/a

                                                        NotebookLM is a research/notebook assistant for document synthesis, not a developer tool or platform with an API to document; the story about an interactive API reference with runnable examples is a category error for this product type.

                                                        • ai-native userTest against a sandbox environment without touching production data

                                                          weight 1 · not comparable
                                                          FutureHouse Platformn/a

                                                          FutureHouse is a research-agent API/platform for scientific literature and analysis tasks, not a data-processing or transactional system where sandbox-vs-production data separation is a relevant concept; the evidence pack contains no mention of environments, sandboxes, or production data at all.

                                                            Gemini Notebook (NotebookLM)n/a

                                                            NotebookLM is a research/document-synthesis product with no execution environment or production data pipeline to sandbox against; the concept of sandbox vs production testing is a category error for this product type.

                                                            • ai-native userDefine rules that trigger actions automatically on events

                                                              weight 3 · not comparable
                                                              FutureHouse Platformn/a

                                                              FutureHouse is a scientific research agent platform (literature review, hypothesis generation, data analysis) invoked via API/client calls, not an event-driven automation/rules engine; no concept of user-defined triggers or event-based rules appears anywhere in the evidence, and this is not the kind of capability the product's category would offer.

                                                                Gemini Notebook (NotebookLM)none0/10

                                                                No evidence of any rule-based or event-triggered automation (e.g., 'when X happens, do Y') in NotebookLM; the product is a research/synthesis tool that responds to manual user prompts and generates static outputs like summaries, audio/video overviews, and study guides, not automated event-driven actions.

                                                                • ai-native userVersion, review, and roll back my automations

                                                                  weight 1 · not comparable
                                                                  FutureHouse Platformn/a

                                                                  FutureHouse is a research/agent platform for scientific tasks (literature review, data analysis), not an automation-builder with versioned workflows; there is no concept of automations to version, review, or roll back in the evidence.

                                                                    Gemini Notebook (NotebookLM)n/a

                                                                    NotebookLM is a research/notes synthesis tool with no automation/workflow-building capability, so 'version, review, and roll back automations' is a category error — there is no automation feature to version or roll back.

                                                                    • ai-native userRead the product's source under an open license

                                                                      weight 2 · not comparable
                                                                      FutureHouse Platformnone0/10

                                                                      No evidence pack items reference an open-source license or public repository for the FutureHouse Platform itself; only client library install instructions and product docs are provided, with no license information. missing for 10: evidence of open-source license, public source repo link, license file or statement.

                                                                        Gemini Notebook (NotebookLM)n/a

                                                                        NotebookLM is a closed, proprietary Google SaaS product with no evidence of an open-license source release; open-source licensing is not a plausible axis for this kind of hosted commercial AI product beyond the general absence-implies-none rule, but here Google's entire distribution model (Workspace core service, paid tiers) signals a closed product category, so this is a category mismatch rather than a missing feature.

                                                                        • [claimed-docs] For Workspace and Workspace for Education users, Gemini Notebook is now a core service (like Gmail, Google Calendar, and Docs).
                                                                      • ai-native userSelf-host the core product

                                                                        weight 3 · not comparable
                                                                        FutureHouse Platformn/a

                                                                        FutureHouse Platform is a hosted API/SaaS agent service (Edison) accessed via client library and API keys, not a distributable self-hostable stack; no evidence of any self-hosting option exists or is claimed.

                                                                          Gemini Notebook (NotebookLM)n/a

                                                                          Gemini Notebook is a fully-hosted Google SaaS product with no self-hosted or open-source deployment option; self-hosting is not a plausible axis for this category of closed cloud service.