Elicit vs Consensus

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round drawn
   Elicit✓fullprobed8/10

   Elicit hosts a live llms.txt at support.elicit.com/llms.txt (HTTP 200, confirmed via probe) listing agent-oriented docs, and it also exposes an official MCP server for agent access, showing genuine agent-oriented documentation infrastructure. missing for 10: no independent/community corroboration of an agent actually consuming llms.txt successfully, and no OpenAPI/agent-doc spec beyond the llms.txt itself.

   • [probe] “PROBE llms.txt: HTTP 200 at https://support.elicit.com/llms.txt # Elicit Help Center > Help center for Elicit ## Getting Started - [Getti…”
   • [claimed-docs] “claude mcp add --transport http elicit https://elicit.com/api/mcp”
   • [claimed-docs] “All API functionality is also available via MCP (Model Context Protocol) server, enabling use from Claude Desktop, Claude Code, and other MC…”

   proof ↗⚑ flag

   Consensus✓fullprobed8/10

   A live probe confirms Consensus serves an llms.txt file at its root (HTTP 200) with a structured summary of the product, directly enabling agents to be pointed at agent-oriented docs. Missing for 10: broader agent-oriented doc formats (e.g. .md endpoints) return 404, and no independent third-party confirmation of llms.txt usage exists.

   • [probe] “PROBE llms.txt: HTTP 200 at https://consensus.app/llms.txt # Consensus > Consensus is an AI-powered scientific search engine that finds, ra…”
   • [probe] “PROBE docs-md: HTTP 404 at https://consensus.app/home/resources/how-consensus-works/.md”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round to Elicit
   Elicit~partialclaimed6/10

   Elicit documents an API (and MCP server) explicitly for running its search/report/systematic-review capabilities 'from your own code, scripts, and workflows,' which supports headless/automated use outside the UI. However, there's no explicit CI/pipeline example, and API access appears gated as a paid plan feature rather than a fully documented automation-first workflow. Missing for 10: explicit CI/automation examples or tutorials, rate-limit/auth details for unattended use, and independent confirmation that the API works reliably in automated pipelines.

   • [claimed-docs] “The Elicit API lets you access Elicit's research capabilities programmatically: search 138 million+ academic papers and generate automated r…”
   • [claimed-docs] “Systematic Review: run a full review end to end, with control over search strategy, screening criteria, extraction parameters, and reporting…”
   • [claimed-docs] “API access”
   • [claimed-docs] “All API functionality is also available via MCP (Model Context Protocol) server, enabling use from Claude Desktop, Claude Code, and other MC…”

   proof ↗⚑ flag

   Consensus~partialclaimed4/10

   Consensus offers an API for integrating its search into custom workflows and running automated searches (consensus-docs-1, consensus-docs-15), which implies some programmatic/headless usability. However, there is no explicit documentation of CI integration, headless execution modes, CLI tooling, or automation pipeline examples. Missing for 10: CI/CD integration examples, headless mode documentation, CLI or SDK for automation, and independent evidence of running in automated pipelines.

   • [claimed-docs] “Connect the Consensus API within your project to seamlessly integrate up-to-date peer-reviewed citations into your own custom workflow.”
   • [claimed-docs] “Save your team hours of manual discovery research and run automated searches with our API to quickly and easily find the most relevant and r…”

   proof ↗⚑ flag

3. ai-native userConnect an agent via an official MCP server

   weight 3 · round to Elicit
   Elicit✓fullclaimed8/10

   Elicit provides an official documented MCP server endpoint (claude mcp add --transport http elicit https://elicit.com/api/mcp) exposing full API functionality for use from Claude Desktop, Claude Code, and other MCP-compatible clients. missing for 10: no independent/hands-on corroboration of the MCP server working, and no detail on auth/tool-list scope beyond the docs.

   • [claimed-docs] “claude mcp add --transport http elicit https://elicit.com/api/mcp”
   • [claimed-docs] “All API functionality is also available via MCP (Model Context Protocol) server, enabling use from Claude Desktop, Claude Code, and other MC…”

   proof ↗⚑ flag

   Consensus—none0/10

   Consensus is not an agent product itself, so the MCP-server axis applies as an ecosystem/API capability, but evidence only shows a REST API and llms.txt file — no mention of an official MCP server for connecting agents. missing for 10: any documented MCP server endpoint, MCP spec compliance, or third-party confirmation of MCP support.

   • [claimed-docs] “Connect the Consensus API within your project to seamlessly integrate up-to-date peer-reviewed citations into your own custom workflow.”
   • [claimed-docs] “Save your team hours of manual discovery research and run automated searches with our API to quickly and easily find the most relevant and r…”
   • [probe] “PROBE llms.txt: HTTP 200 at https://consensus.app/llms.txt # Consensus > Consensus is an AI-powered scientific search engine that finds, ra…”

   proof ↗⚑ flag

4. ai-native userUse an official CLI

   weight 2 · round drawn
   Elicit—none0/10

   Evidence shows Elicit offers an API and an MCP server for programmatic/agentic access, but there is no mention anywhere of an official command-line interface (CLI) tool. Since API-based products could plausibly ship a CLI, absence of evidence means this axis is unmet rather than inapplicable.

   • [claimed-docs] “The Elicit API lets you access Elicit's research capabilities programmatically: search 138 million+ academic papers and generate automated r…”
   • [claimed-docs] “All API functionality is also available via MCP (Model Context Protocol) server, enabling use from Claude Desktop, Claude Code, and other MC…”
   • [claimed-docs] “claude mcp add --transport http elicit https://elicit.com/api/mcp”

   proof ↗⚑ flag

   Consensus—none0/10

   The evidence pack documents a REST API and an MCP server (consensus-docs-16) but no official command-line interface is mentioned anywhere in the docs or probes. Missing for 10: any mention of a CLI tool, CLI installation instructions, or CLI command reference.

   • [claimed-docs] “a REST API and an MCP server expose the same retrieval and synthesis surface that powers the web app”

   proof ↗⚑ flag

5. ai-native userDrive the product through a documented public API

   weight 3 · round to Elicit
   Elicit✓fullprobed8/10

   Elicit documents a public API with keys/auth management and specific programmatic endpoints (search 138M+ papers, automated report generation, full systematic review workflow control), plus an MCP server exposing the same functionality for agentic clients. Missing for 10: an actual OpenAPI/swagger spec was not found (404s on candidate paths) and no independent/hands-on developer report validates real-world API usage.

   • [claimed-docs] “API access”
   • [claimed-docs] “The Elicit API lets you access Elicit's research capabilities programmatically: search 138 million+ academic papers and generate automated r…”
   • [claimed-docs] “All API functionality is also available via MCP (Model Context Protocol) server, enabling use from Claude Desktop, Claude Code, and other MC…”
   • [claimed-docs] “Systematic Review: run a full review end to end, with control over search strategy, screening criteria, extraction parameters, and reporting…”
   • [claimed-docs] “claude mcp add --transport http elicit https://elicit.com/api/mcp”
   • [probe] “PROBE openapi: all candidate paths 404 (https://support.elicit.com/openapi.json, https://support.elicit.com/swagger.json, https://support.el…”

   proof ↗⚑ flag

   Consensus~partialprobed5/10

   Consensus advertises a documented API for integrating citations and running automated searches into custom workflows, and its site provides an llms.txt for AI-agent discovery, showing basic public-API and agent-friendliness. However, the evidence pack only shows marketing/landing pages, not actual API reference documentation, authentication, endpoints, or example requests/responses, and there's no independent or hands-on corroboration that the API works as described. Missing for 10: full API reference/spec details, code/SDK examples, and independent verification of API usage.

   • [claimed-docs] “Connect the Consensus API within your project to seamlessly integrate up-to-date peer-reviewed citations into your own custom workflow.”
   • [claimed-docs] “Save your team hours of manual discovery research and run automated searches with our API to quickly and easily find the most relevant and r…”
   • [probe] “PROBE llms.txt: HTTP 200 at https://consensus.app/llms.txt # Consensus > Consensus is an AI-powered scientific search engine that finds, ra…”

   proof ↗⚑ flag

6. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round drawn
   Elicit—none0/10

   Elicit does offer API keys and an MCP server for programmatic/agent access (elicit-docs-34, elicit-docs-35, elicit-docs-28), so the axis of credential management applies, but there is no evidence of scoped or least-privilege permissions, roles, or restricted-scope API keys — the docs only describe managing API keys generically, not limiting their scope.

   • [claimed-docs] “The Elicit API lets you access Elicit's research capabilities programmatically: search 138 million+ academic papers and generate automated r…”
   • [claimed-docs] “All API functionality is also available via MCP (Model Context Protocol) server, enabling use from Claude Desktop, Claude Code, and other MC…”
   • [claimed-docs] “claude mcp add --transport http elicit https://elicit.com/api/mcp”

   proof ↗⚑ flag

   Consensus—none0/10

   Evidence shows an API and MCP server exist, but there is no mention of scoped or least-privilege API keys, permission scopes, or credential management for agents — just generic API access. missing for 10: scoped/least-privilege credential issuance, API key permission controls, agent-specific auth documentation.

   • [claimed-docs] “Connect the Consensus API within your project to seamlessly integrate up-to-date peer-reviewed citations into your own custom workflow.”
   • [claimed-docs] “Save your team hours of manual discovery research and run automated searches with our API to quickly and easily find the most relevant and r…”
   • [claimed-docs] “a REST API and an MCP server expose the same retrieval and synthesis surface that powers the web app”

   proof ↗⚑ flag

7. ai-native userBuild against official SDKs

   weight 2 · round to Elicit
   Elicit~partialclaimed6/10

   Elicit documents an official API (with managed API keys) for programmatic access to search and automated research reports, plus full API functionality exposed via an MCP server for Claude Desktop/Code integration, which supports agentic, code-driven workflows. However, evidence shows only a REST-style API and API-key docs, not a dedicated official SDK/client library in specific languages, nor code samples or independent developer corroboration. Missing for 10: named client SDKs (e.g., Python/JS packages), quickstart code examples, and independent/hands-on developer confirmation of API reliability.

   • [claimed-docs] “The Elicit API lets you access Elicit's research capabilities programmatically: search 138 million+ academic papers and generate automated r…”
   • [claimed-docs] “All API functionality is also available via MCP (Model Context Protocol) server, enabling use from Claude Desktop, Claude Code, and other MC…”
   • [claimed-docs] “Systematic Review: run a full review end to end, with control over search strategy, screening criteria, extraction parameters, and reporting…”
   • [claimed-docs] “API access”
   • [claimed-docs] “claude mcp add --transport http elicit https://elicit.com/api/mcp”

   proof ↗⚑ flag

   Consensus—none0/10

   Consensus documents an API for integration (consensus-docs-1, consensus-docs-15) but no evidence pack item mentions official SDKs (Python, JS, etc.) or client libraries for AI-native development — only the raw API and llms.txt discovery file are shown.

   • [claimed-docs] “Connect the Consensus API within your project to seamlessly integrate up-to-date peer-reviewed citations into your own custom workflow.”
   • [claimed-docs] “Save your team hours of manual discovery research and run automated searches with our API to quickly and easily find the most relevant and r…”
   • [probe] “PROBE llms.txt: HTTP 200 at https://consensus.app/llms.txt # Consensus > Consensus is an AI-powered scientific search engine that finds, ra…”

   proof ↗⚑ flag

8. ai-native userSubscribe to events via webhooks

   weight 2 · round drawn
   Elicit—none0/10

   Elicit offers email alerts, an API, and an MCP server, but no evidence anywhere in the pack mentions webhook subscriptions or event-driven callbacks for programmatic integration.

   • [claimed-docs] “Turn on **Instant email alerts** to be immediately alerted whenever a new relevant research paper is found.”
   • [claimed-docs] “The Elicit API lets you access Elicit's research capabilities programmatically: search 138 million+ academic papers and generate automated r…”
   • [claimed-docs] “All API functionality is also available via MCP (Model Context Protocol) server, enabling use from Claude Desktop, Claude Code, and other MC…”

   proof ↗⚑ flag

   Consensus—none0/10

   No evidence anywhere in the pack mentions webhooks or event subscriptions; Consensus's API/MCP surface is described only as REST retrieval/synthesis, not event-driven push notifications.

   ⚑ flag

Agentic features

1. ai-native userGet AI-generated insights and suggestions from my data inside the product

   weight 2 · round to Consensus
   Elicit!disputedcontradicted5/10

   Elicit's Research Agent, columns, chat-with-papers, and Systematic Review reports are documented to generate AI insights and suggestions from ingested papers (elicit-docs-11, elicit-docs-40, elicit-docs-41), and one HN user found topic analysis genuinely useful (elicit-comm-1). However, independent hands-on reports concretely contradict reliability: users found mostly incorrect summaries, missed key papers, and fabricated/hallucinated facts even when directly quoting sources (elicit-comm-3, elicit-comm-4, elicit-comm-7), and Elicit's own docs admit ~10% inaccuracy requiring manual verification (elicit-comm-2). missing for 10: independent corroboration that generated insights are consistently accurate rather than frequently hallucinated, and resolution of the documented factual-error reports.

   • [claimed-docs] “To add a column, simply tell the research agent what column(s) you'd like to add. For example, you can say: "Add a column for study type."”
   • [claimed-docs] “Chat enables you to: Compare and contrast papers - Summarize multiple papers along specific dimensions (like their methodologies) - Cluster …”
   • [claimed-docs] “the Research Agent can pull from a wide range of sources (e.g. publications, public filings, press releases), produce flexible outputs, and …”
   • [community] “I asked about media bias detection and used the topic analysis feature. A minute or so later, I had a list of concepts with citations and li…”
   • [community] “Elicit states: 'assume that around 90% of the information you see in Elicit is accurate... it's very important for you to check the work in …”
   • [community] “I gave it a topic I researched in depth recently. It gave me mostly incorrect summaries (one said hypothesis X is confirmed; nope it hasn't)…”
   • [community] “It seems like it should hallucinate less, as it directly quotes, but nope, it still hallucinates just as much and then gives a quote that di…”
   • [community] “I tried abstract summarization with a whitepaper and it came up with completely made-up facts, describing an algorithm acronym incorrectly. …”

   proof ↗⚑ flag

   Consensus✓fullclaimed8/10

   Consensus generates AI-driven synthesis, summaries, the Consensus Meter, PICO extraction, and literature review synthesis directly from the papers in its corpus/library, with citations tracing insights back to sources. This is core native functionality (not a bolt-on), covering search, synthesis, and structured insight generation. Missing for 10: independent/hands-on third-party verification of insight quality beyond vendor docs.

   • [claimed-docs] “It searches over 200 million academic papers and uses language models to help you find, understand, and synthesize the literature faster.”
   • [claimed-docs] “Every response includes citations, so you can trace each insight back to the original source.”
   • [claimed-docs] “The Consensus Meter is a visual aggregator that, for a yes/no/possibly question, classifies each relevant paper as supporting, refuting, or …”
   • [claimed-docs] “The Consensus Meter is a visual aggregator that, for a yes/no/possibly question, classifies each relevant paper as supporting, refuting, or …”
   • [claimed-docs] “Extracted population, intervention, comparator, and outcome (PICO) where applicable”
   • [claimed-docs] “Consensus is an AI-powered research engine built to speed up literature reviews. Search, screen, extract, and synthesize evidence faster—whi…”
   • [claimed-docs] “The Consensus Library brings your entire research library into one searchable, AI-powered workspace.”

   proof ↗⚑ flag

2. ai-native userSet up automations that run autonomously in the background

   weight 2 · round to Elicit
   Elicit~partialclaimed5/10

   Elicit's Alerts feature lets users set up a background process that automatically monitors for new relevant papers and notifies them (e.g., via instant email alerts), which is a real autonomous background automation, and the API/MCP server also enables scripted automated report generation from external workflows. However, this is narrow (limited to paper-discovery alerts) rather than a general-purpose scheduling/automation system for arbitrary agentic tasks, and there's no evidence of recurring scheduled jobs, triggers, or workflow orchestration beyond alerts. Missing for 10: evidence of a general automation/scheduling engine, ability to chain multi-step autonomous tasks, and independent confirmation that alerts reliably run unattended over time.

   • [claimed-docs] “Turn on **Instant email alerts** to be immediately alerted whenever a new relevant research paper is found.”
   • [claimed-docs] “Alerts allow you to stay up to date on the latest research about topics that are relevant to you, so you can add them to your Library for fu…”
   • [claimed-docs] “Turn on Instant email alerts to be immediately alerted whenever a new relevant research paper is found.”
   • [claimed-docs] “The Elicit API lets you access Elicit's research capabilities programmatically: search 138 million+ academic papers and generate automated r…”
   • [claimed-docs] “All API functionality is also available via MCP (Model Context Protocol) server, enabling use from Claude Desktop, Claude Code, and other MC…”

   proof ↗⚑ flag

   Consensus—none0/10

   Consensus is a research search/synthesis engine with an API and MCP server for on-demand retrieval, but there is no evidence of scheduled or event-triggered automations that run autonomously in the background without user invocation. Missing for 10: any scheduling/trigger mechanism, background job execution, or autonomous recurring workflow capability.

   ⚑ flag

3. ai-native userDelegate tasks to a built-in AI assistant inside the product

   weight 3 · round to Elicit
   Elicit✓fullcommunity8/10

   Elicit ships a built-in Research Agent that users can delegate tasks to directly (search, screening, extraction, column creation, chat with papers, skills), with effort-level control and iterative outputs, all inside the product interface (elicit-docs-2,7,8,11,12,21,40,42). Community evidence corroborates real task delegation working in practice (elicit-comm-1) though also raises accuracy concerns that temper trust in outputs (elicit-comm-2,3,4). Missing for 10: independent quality benchmarking beyond anecdotal HN threads and more recent hands-on validation of the newer effort-level/skills features.

   • [claimed-docs] “Every Research Agent session runs at an effort level, which you set from the slider in the prompt box before you send your question.”
   • [claimed-docs] “Projects in Elicit allow you to link together multiple sessions for fluid reasoning across different aspects of your work.”
   • [claimed-docs] “A skill is a set of instructions you can hand to Elicit's agent so it approaches a task a particular way.”
   • [claimed-docs] “To add a column, simply tell the research agent what column(s) you'd like to add. For example, you can say: "Add a column for study type."”
   • [claimed-docs] “the Research Agent can pull from a wide range of sources (e.g. publications, public filings, press releases), produce flexible outputs, and …”
   • [claimed-docs] “you can filter and export your sources, generate figures, and draft slides, all without leaving the conversation”
   • [claimed-docs] “Chat enables you to: Compare and contrast papers - Summarize multiple papers along specific dimensions (like their methodologies) - Cluster …”
   • [claimed-docs] “A skill is a set of instructions you hand to Elicit's agent so it approaches a task a particular way.”
   • [community] “I asked about media bias detection and used the topic analysis feature. A minute or so later, I had a list of concepts with citations and li…”
   • [community] “I gave it a topic I researched in depth recently. It gave me mostly incorrect summaries (one said hypothesis X is confirmed; nope it hasn't)…”

   proof ↗⚑ flag

   Consensus✓fullprobed7/10

   Consensus ships a built-in "Research Agent" that chains citation crawling, DOI lookup, author search and similar-paper discovery on top of its search engine, and its core AI assistant performs search, screen, extract, and synthesize workflows with cited answers — this is essentially delegating research tasks to an in-product AI assistant. missing for 10: independent/hands-on validation of the agent's autonomy and reliability, and more detail on the scope/limits of delegable tasks beyond literature discovery.

   • [claimed-docs] “Citation crawling, DOI lookup, author search, similar papers, and more - chained together on top of the worlds best academic search engine.”
   • [claimed-docs] “Search, screen, extract, and synthesize evidence faster—while keeping full transparency and scholarly rigor.”
   • [claimed-docs] “Consensus is an AI-powered research engine built to speed up literature reviews. Search, screen, extract, and synthesize evidence faster—whi…”
   • [claimed-docs] “It searches over 200 million academic papers and uses language models to help you find, understand, and synthesize the literature faster.”
   • [claimed-docs] “Every response includes citations, so you can trace each insight back to the original source.”
   • [probe] “PROBE llms.txt: HTTP 200 at https://consensus.app/llms.txt # Consensus > Consensus is an AI-powered scientific search engine that finds, ra…”

   proof ↗⚑ flag

4. ai-native userOperate the product with natural-language commands

   weight 2 · round drawn
   Elicit✓fullcommunity7/10

   Elicit's Research Agent and semantic search are explicitly natural-language driven (e.g., asking questions in plain language, adding columns via natural-language commands like 'Add a column for study type'), and skills let users reference natural-language instructions instead of re-typing prompts. However, community reports raise accuracy/hallucination concerns that temper confidence in reliability of NL command execution. Missing for 10: independent hands-on verification of complex multi-step NL command chains, and no evidence of NL support outside the research/agent workflows (e.g., no broader command-line or API NL interface).

   • [claimed-docs] “To add a column, simply tell the research agent what column(s) you'd like to add. For example, you can say: "Add a column for study type."”
   • [claimed-docs] “With Elicit's semantic search engine, you can ask a question in natural language, and Elicit will find relevant papers, without you having t…”
   • [claimed-docs] “Elicit's semantic search means you don't have to know all the right keywords to get relevant results.”
   • [claimed-docs] “Instead of writing out the same detailed prompt every time you run a market sizing or a landscape review, you reference the skill and the ag…”
   • [claimed-docs] “the Research Agent can pull from a wide range of sources (e.g. publications, public filings, press releases), produce flexible outputs, and …”
   • [community] “I gave it a topic I researched in depth recently. It gave me mostly incorrect summaries (one said hypothesis X is confirmed; nope it hasn't)…”
   • [community] “It seems like it should hallucinate less, as it directly quotes, but nope, it still hallucinates just as much and then gives a quote that di…”

   proof ↗⚑ flag

   Consensus✓fullclaimed7/10

   Consensus's core interaction model is natural-language research queries (search, synthesize, Consensus Meter for yes/no questions) rather than rigid query syntax, and it exposes this same NL-driven retrieval/synthesis surface via an MCP server and REST API for programmatic/agentic use. Missing for 10: independent hands-on evidence of natural-language command execution quality, and no detailed example transcripts showing complex multi-step NL commands being interpreted.

   • [claimed-docs] “It searches over 200 million academic papers and uses language models to help you find, understand, and synthesize the literature faster.”
   • [claimed-docs] “a REST API and an MCP server expose the same retrieval and synthesis surface that powers the web app”
   • [claimed-docs] “The Consensus Meter is a visual aggregator that, for a yes/no/possibly question, classifies each relevant paper as supporting, refuting, or …”
   • [claimed-docs] “The Consensus Meter is a visual aggregator that, for a yes/no/possibly question, classifies each relevant paper as supporting, refuting, or …”
   • [claimed-docs] “Think of Consensus as an AI-native alternative to Google Scholar with a more-refined corpus.”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   Elicit—none0/10

   Elicit documents an API and MCP server (elicit-docs-6, elicit-docs-34, elicit-docs-35, elicit-docs-36) but there is no evidence of an interactive API reference/playground with runnable examples; probes for OpenAPI/swagger specs all returned 404 (elicit-probe-3), suggesting no such interactive reference exists.

   • [claimed-docs] “API access”
   • [claimed-docs] “The Elicit API lets you access Elicit's research capabilities programmatically: search 138 million+ academic papers and generate automated r…”
   • [claimed-docs] “All API functionality is also available via MCP (Model Context Protocol) server, enabling use from Claude Desktop, Claude Code, and other MC…”
   • [claimed-docs] “Systematic Review: run a full review end to end, with control over search strategy, screening criteria, extraction parameters, and reporting…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://support.elicit.com/openapi.json, https://support.elicit.com/swagger.json, https://support.el…”

   proof ↗⚑ flag

   Consensus—none0/10

   Evidence confirms Consensus offers a REST API and MCP server (consensus-docs-1, consensus-docs-15, consensus-docs-16), but there is no mention of an interactive API reference, sandbox, or runnable code examples anywhere in the pack.

   • [claimed-docs] “Connect the Consensus API within your project to seamlessly integrate up-to-date peer-reviewed citations into your own custom workflow.”
   • [claimed-docs] “Save your team hours of manual discovery research and run automated searches with our API to quickly and easily find the most relevant and r…”
   • [claimed-docs] “a REST API and an MCP server expose the same retrieval and synthesis surface that powers the web app”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   Elicit—none0/10

   Elicit documents a REST API (elicit-docs-34,36) and an MCP server (elicit-docs-28,35), but no evidence of a downloadable OpenAPI/Swagger spec exists; direct probes for openapi.json/swagger.json all returned 404 (elicit-probe-3). Missing for 10: any published OpenAPI/Swagger file, machine-readable schema, or API reference page listing such a spec.

   • [claimed-docs] “The Elicit API lets you access Elicit's research capabilities programmatically: search 138 million+ academic papers and generate automated r…”
   • [claimed-docs] “All API functionality is also available via MCP (Model Context Protocol) server, enabling use from Claude Desktop, Claude Code, and other MC…”
   • [claimed-docs] “Systematic Review: run a full review end to end, with control over search strategy, screening criteria, extraction parameters, and reporting…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://support.elicit.com/openapi.json, https://support.elicit.com/swagger.json, https://support.el…”

   proof ↗⚑ flag

   Consensus—none0/10

   Consensus documents a REST API and MCP server (consensus-docs-15, consensus-docs-16) but no evidence pack item mentions an OpenAPI spec, Swagger file, or any downloadable machine-readable API schema; the llms.txt probe returns a plain-text description, not an API spec. missing for 10: OpenAPI/Swagger file, machine-readable schema download link, independent confirmation of spec availability.

   • [claimed-docs] “Save your team hours of manual discovery research and run automated searches with our API to quickly and easily find the most relevant and r…”
   • [claimed-docs] “a REST API and an MCP server expose the same retrieval and synthesis surface that powers the web app”
   • [probe] “PROBE llms.txt: HTTP 200 at https://consensus.app/llms.txt # Consensus > Consensus is an AI-powered scientific search engine that finds, ra…”

   proof ↗⚑ flag

3. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   Elicit—none0/10

   Evidence confirms Elicit has an API and MCP server (elicit-docs-34, elicit-docs-35, elicit-docs-36) but nothing documents API versioning or a deprecation policy, and probes for an OpenAPI/spec file returned 404s (elicit-probe-3).

   • [claimed-docs] “The Elicit API lets you access Elicit's research capabilities programmatically: search 138 million+ academic papers and generate automated r…”
   • [claimed-docs] “All API functionality is also available via MCP (Model Context Protocol) server, enabling use from Claude Desktop, Claude Code, and other MC…”
   • [claimed-docs] “Systematic Review: run a full review end to end, with control over search strategy, screening criteria, extraction parameters, and reporting…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://support.elicit.com/openapi.json, https://support.elicit.com/swagger.json, https://support.el…”

   proof ↗⚑ flag

   Consensus—none0/10

   There's an API and MCP server mentioned, but no evidence of API versioning scheme or a documented deprecation policy anywhere in the pack. missing for 10: versioning scheme documentation, deprecation policy, changelog/migration guides.

   • [claimed-docs] “a REST API and an MCP server expose the same retrieval and synthesis surface that powers the web app”
   • [claimed-docs] “Save your team hours of manual discovery research and run automated searches with our API to quickly and easily find the most relevant and r…”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round to Elicit
   Elicit✓fullclaimed8/10

   Elicit's core Systematic Reviews and Tables/Columns workflows explicitly apply extraction and screening operations across many papers at once (docs-11, docs-18, docs-22, docs-45), with bulk import (RIS/BIB, Zotero) and bulk export (CSV/Excel/RIS/BIB) of entire tables (docs-25, docs-38, docs-47), plus API/MCP access to run full systematic reviews programmatically at scale (docs-34, docs-36). Missing for 10: independent/hands-on evidence specifically validating bulk-scale accuracy or performance (community evidence addresses general accuracy, not bulk-operation mechanics).

   • [claimed-docs] “To add a column, simply tell the research agent what column(s) you'd like to add. For example, you can say: "Add a column for study type."”
   • [claimed-docs] “Create a column for each data point you'd like to extract from your papers. Columns can pull data from the papers' body text or from tables …”
   • [claimed-docs] “Elicit supports the major steps of a systematic review: 1. Set up your review on the Setup page 2. Gather all papers for the systematic revi…”
   • [claimed-docs] “Tables can be exported in CSV and Excel format. Certain tables of sources can be exported as RIS or BIB files.”
   • [claimed-docs] “Systematic Review: run a full review end to end, with control over search strategy, screening criteria, extraction parameters, and reporting…”
   • [claimed-docs] “You can import RIS and BIB files into Elicit. This makes it much easier to import titles/abstracts from other tools like EndNote, Mendeley, …”
   • [claimed-docs] “Research Reports can be exported as a PDF or Word file... Pro, Scale, and Enterprise subscribers can export tables from Systematic Reviews, …”
   • [claimed-docs] “The Elicit API lets you access Elicit's research capabilities programmatically: search 138 million+ academic papers and generate automated r…”

   proof ↗⚑ flag

   Consensus~partialclaimed6/10

   Docs show bulk-style capabilities: one-click import of thousands of papers into a library, an API/MCP server for automated bulk searches, and Deep Searches across many studies — supporting bulk operations for an AI-native/automation persona. missing for 10: independent/hands-on verification of bulk API throughput or rate limits, explicit batch-processing endpoints (e.g., bulk extract/export across many items in one call), and any third-party confirmation of scale performance.

   • [claimed-docs] “Import thousands of papers in one click - then search, find gaps, and put your collection to work.”
   • [claimed-docs] “Turn your library into a research engine. Import thousands of papers in one click - then search, find gaps, and put your collection to work.”
   • [claimed-docs] “Save your team hours of manual discovery research and run automated searches with our API to quickly and easily find the most relevant and r…”
   • [claimed-docs] “a REST API and an MCP server expose the same retrieval and synthesis surface that powers the web app”
   • [claimed-docs] “Deep Searches (more comprehensive Lit Reviews across many studies)”
   • [claimed-docs] “Import from Zotero ... or import from BibTex, PDF, or RIS”

   proof ↗⚑ flag

2. ai-native userSchedule recurring jobs or workflows

   weight 2 · round to Elicit
   Elicit~partialclaimed4/10

   Elicit's Alerts feature lets users get recurring updates when new relevant papers matching a saved search appear (via instant email alerts), which is a limited form of a recurring job, but there is no evidence of general scheduling of arbitrary Research Agent workflows, systematic reviews, or API-driven jobs on a recurring cadence. Missing for 10: ability to schedule/repeat full Research Agent or Systematic Review workflows, cron-like or interval-based automation beyond paper alerts, and confirmation this works via API/MCP for programmatic recurring runs.

   • [claimed-docs] “Turn on **Instant email alerts** to be immediately alerted whenever a new relevant research paper is found.”
   • [claimed-docs] “Alerts allow you to stay up to date on the latest research about topics that are relevant to you, so you can add them to your Library for fu…”
   • [claimed-docs] “Turn on Instant email alerts to be immediately alerted whenever a new relevant research paper is found.”
   • [claimed-docs] “The Elicit API lets you access Elicit's research capabilities programmatically: search 138 million+ academic papers and generate automated r…”
   • [claimed-docs] “All API functionality is also available via MCP (Model Context Protocol) server, enabling use from Claude Desktop, Claude Code, and other MC…”

   proof ↗⚑ flag

   Consensus—none0/10

   No evidence in the pack mentions scheduling, recurring jobs, alerts, or automated re-running of searches/workflows over time; the API and MCP server are described as on-demand retrieval/synthesis interfaces, not schedulable automation. missing for 10: any scheduling/cron feature, recurring alert or saved-search re-run capability, or workflow automation trigger.

   • [claimed-docs] “Save your team hours of manual discovery research and run automated searches with our API to quickly and easily find the most relevant and r…”
   • [claimed-docs] “a REST API and an MCP server expose the same retrieval and synthesis surface that powers the web app”

   proof ↗⚑ flag

Sharing

1. analystShare a research session or report with collaborators who can view or build on it

   weight 2 · round to Elicit
   Elicit✓fullclaimed8/10

   Elicit has a documented feature to invite team members to collaborate live in Research Agent sessions, with edit-access collaborators able to ask the agent questions, produce new artifacts, and edit others' work, plus reports/tables can be exported as PDF/Word/CSV for sharing. missing for 10: no independent/hands-on corroboration of the live collaboration feature working smoothly, and no detail on view-only/read-access sharing permissions.

   • [claimed-docs] “Collaborators with edit access can ask the agent questions in your session, produce new artifacts (documents, tables, etc.), and edit other …”
   • [claimed-docs] “Invite your team members to collaborate live with you in Research Agent sessions.”
   • [claimed-docs] “Invite your team members to collaborate live with you in Research Agent sessions. Collaborators with edit access can ask the agent questions…”
   • [claimed-docs] “Research Reports can be exported as a PDF or Word file... Pro, Scale, and Enterprise subscribers can export tables from Systematic Reviews, …”

   proof ↗⚑ flag

   Consensus—none0/10

   No evidence pack items mention sharing sessions, reports, collaborators, team accounts, or collaborative viewing/editing features—only individual research, library import, and API/agent capabilities are documented. missing for 10: any mention of sharing links, collaborator invites, team workspaces, or comment/build-on functionality.

   ⚑ flag

Alerts

1. researcherSet up standing searches or alerts that surface new relevant sources as they appear

   weight 1 · round to Elicit
   Elicit✓fullclaimed7/10

   Elicit's Alerts feature explicitly lets researchers set up standing topic alerts with instant email notifications when new relevant papers are found, adding them to a Library for future use — directly matching the standing-search/alert story. Missing for 10: independent/hands-on verification of alert accuracy or timeliness, and detail on how alert relevance/topics are configured beyond docs claims.

   • [claimed-docs] “Turn on **Instant email alerts** to be immediately alerted whenever a new relevant research paper is found.”
   • [claimed-docs] “Alerts allow you to stay up to date on the latest research about topics that are relevant to you, so you can add them to your Library for fu…”
   • [claimed-docs] “Turn on Instant email alerts to be immediately alerted whenever a new relevant research paper is found.”

   proof ↗⚑ flag

   Consensus—none0/10

   No evidence of standing searches, saved-search alerts, or notification features when new relevant papers appear; the evidence pack covers search, library import, citation graph, API/MCP retrieval, and literature review synthesis but nothing about recurring/alert-based monitoring of new sources.

   ⚑ flag

Corpus

1. researcherUpload my own PDFs or corpus and have the agent research over them

   weight 2 · round to Elicit
   Elicit✓fullcommunity7/10

   Elicit supports importing/uploading a user's own corpus (RIS/BIB import, Zotero integration, Collections) and running research operations (chat, columns/data extraction, systematic reviews) over those uploaded papers, with the browser extension auto-fetching full text for extraction. Community feedback raises accuracy/hallucination concerns about summarization quality, which tempers reliability but does not contradict the upload/research capability itself. Missing for 10: independent hands-on verification of accuracy when researching over a user-uploaded corpus, and clearer documentation of raw multi-PDF drag-and-drop upload versus reference-manager import formats.

   • [claimed-docs] “Elicit's Zotero integration helps you bring your paper collections into Elicit, where you can extract data and analyze your papers!”
   • [claimed-docs] “You can import RIS and BIB files into Elicit. This makes it much easier to import titles/abstracts from other tools like EndNote, Mendeley, …”
   • [claimed-docs] “Create a column for each data point you'd like to extract from your papers. Columns can pull data from the papers' body text or from tables …”
   • [claimed-docs] “You can now organize and manage your papers in the Elicit Library using Collections. Collections help you group related research, arrange it…”
   • [claimed-docs] “Elicit will automatically fetch papers during the data extraction phase of your Systematic Reviews”
   • [claimed-docs] “you can spend less time downloading full-text PDFs from publisher websites to extract in Elicit – we'll get the papers for you”
   • [claimed-docs] “Chat enables you to: Compare and contrast papers - Summarize multiple papers along specific dimensions (like their methodologies) - Cluster …”
   • [community] “I gave it a topic I researched in depth recently. It gave me mostly incorrect summaries (one said hypothesis X is confirmed; nope it hasn't)…”
   • [community] “It seems like it should hallucinate less, as it directly quotes, but nope, it still hallucinates just as much and then gives a quote that di…”

   proof ↗⚑ flag

   Consensus~partialclaimed6/10

   Consensus's Library feature explicitly supports importing PDFs, BibTeX, RIS, and Zotero corpora and turns them into a 'searchable, AI-powered workspace' for finding gaps and using the collection, which matches the story's upload+research intent. However, evidence doesn't detail how deeply the AI synthesis/agent features (Meter, PICO extraction, literature review synthesis) operate specifically over a user's uploaded corpus versus the general 200M-paper index. Missing for 10: explicit documentation of agent-style synthesis/Q&A running directly over an uploaded private corpus, and independent/hands-on confirmation of this workflow.

   • [claimed-docs] “Import thousands of papers in one click - then search, find gaps, and put your collection to work.”
   • [claimed-docs] “Turn your library into a research engine. Import thousands of papers in one click - then search, find gaps, and put your collection to work.”
   • [claimed-docs] “Import from Zotero ... or import from BibTex, PDF, or RIS”
   • [claimed-docs] “The Consensus Library brings your entire research library into one searchable, AI-powered workspace.”
   • [claimed-docs] “Reference managers are great at saving papers — not so great at helping you use them. The Consensus Library brings your entire research libr…”

   proof ↗⚑ flag

Reviews

1. researcherRun a systematic screening and extraction workflow across many papers with consistent criteria

   weight 2 · round to Elicit
   Elicit✓fullcommunity8/10

   Elicit documents a dedicated Systematic Reviews workflow covering setup, search/gather, title-abstract screening, automated full-text screening, and data extraction via consistent custom columns applied across all papers, with export of screening/extraction tables — directly matching the story. Community evidence raises general accuracy/hallucination concerns about Elicit's paper analysis (not specifically the systematic review pipeline), which tempers confidence in perfect consistency at scale. Missing for 10: independent hands-on validation of the systematic review workflow's accuracy/consistency specifically (vs. general chat/summarization complaints), and no third-party benchmarking of screening reliability across large paper sets.

   • [claimed-docs] “The Systematic Reviews workflow provides step-by-step guidance through search, screening, and data extraction, culminating in a research rep…”
   • [claimed-docs] “Elicit supports the major steps of a systematic review: 1. Set up your review on the Setup page 2. Gather all papers for the systematic revi…”
   • [claimed-docs] “Full-Text Screening is a distinct, automated step in the Systematic Review workflow that catches these mismatches before you commit to data …”
   • [claimed-docs] “Create a column for each data point you'd like to extract from your papers. Columns can pull data from the papers' body text or from tables …”
   • [claimed-docs] “To add a column, simply tell the research agent what column(s) you'd like to add. For example, you can say: "Add a column for study type."”
   • [claimed-docs] “Pro, Scale, and Enterprise subscribers can export tables from Systematic Reviews, including the screening recommendation tables and the data…”
   • [claimed-docs] “Research Reports can be exported as a PDF or Word file... Pro, Scale, and Enterprise subscribers can export tables from Systematic Reviews, …”
   • [claimed-docs] “Elicit can help you save up to 80% of time usually spent on a systematic review without sacrificing accuracy.”
   • [community] “I gave it a topic I researched in depth recently. It gave me mostly incorrect summaries (one said hypothesis X is confirmed; nope it hasn't)…”
   • [community] “It seems like it should hallucinate less, as it directly quotes, but nope, it still hallucinates just as much and then gives a quote that di…”

   proof ↗⚑ flag

   Consensus~partialclaimed6/10

   Consensus offers literature-review features (search, screen, extract, synthesize per docs-8/12), library import at scale, PICO extraction, and filters by study type/year/discipline that support systematic screening with consistent criteria. However, there is no evidence of documented inter-rater reliability, exportable screening decision logs, or PRISMA-style workflow tracking that a systematic review would require. missing for 10: evidence of structured screening criteria configuration/audit trail, PRISMA-compliant workflow support, independent validation of extraction consistency across large paper sets.

   • [claimed-docs] “Search, screen, extract, and synthesize evidence faster—while keeping full transparency and scholarly rigor.”
   • [claimed-docs] “Consensus is an AI-powered research engine built to speed up literature reviews. Search, screen, extract, and synthesize evidence faster—whi…”
   • [claimed-docs] “Turn your library into a research engine. Import thousands of papers in one click - then search, find gaps, and put your collection to work.”
   • [claimed-docs] “Filters allow narrowing by study type (RCT, meta-analysis, systematic review, observational), publication year, journal, open-access status,…”
   • [claimed-docs] “Extracted population, intervention, comparator, and outcome (PICO) where applicable”
   • [claimed-docs] “Deep Searches (more comprehensive Lit Reviews across many studies)”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round to Consensus
   Elicit~partialclaimed4/10

   Elicit's API and MCP server expose search and end-to-end Systematic Review functionality (search, screening, extraction, reporting), giving programmatic access to core research capabilities, but there's no evidence that UI-only features like the interactive Research Agent chat, Skills, real-time collaboration, columns customization, alerts, or the browser extension are exposed via the API/MCP surface. Missing for 10: explicit documentation of full feature parity, API/MCP access to Research Agent conversational sessions, skills, collaboration, and alerts.

   • [claimed-docs] “The Elicit API lets you access Elicit's research capabilities programmatically: search 138 million+ academic papers and generate automated r…”
   • [claimed-docs] “All API functionality is also available via MCP (Model Context Protocol) server, enabling use from Claude Desktop, Claude Code, and other MC…”
   • [claimed-docs] “Systematic Review: run a full review end to end, with control over search strategy, screening criteria, extraction parameters, and reporting…”
   • [claimed-docs] “API access”
   • [claimed-docs] “claude mcp add --transport http elicit https://elicit.com/api/mcp”

   proof ↗⚑ flag

   Consensus~partialclaimed6/10

   The API/MCP server is documented to expose 'the same retrieval and synthesis surface that powers the web app' (consensus-docs-16), and supports automated search (consensus-docs-15), suggesting broad parity for core search/synthesis. However, UI-specific workflows like Library import/reference management (Zotero/BibTeX/RIS import), Citation Graph, and Consensus Meter visualizations are not explicitly confirmed as API-accessible endpoints. Missing for 10: explicit API documentation confirming library management, citation graph, and meter features are callable via API, plus independent/hands-on verification of claimed parity.

   • [claimed-docs] “a REST API and an MCP server expose the same retrieval and synthesis surface that powers the web app”
   • [claimed-docs] “Save your team hours of manual discovery research and run automated searches with our API to quickly and easily find the most relevant and r…”
   • [claimed-docs] “The Consensus Library brings your entire research library into one searchable, AI-powered workspace.”
   • [claimed-docs] “Import from Zotero ... or import from BibTex, PDF, or RIS”
   • [claimed-docs] “The Consensus Citation Graph turns a single seed paper into a complete map of the work that built it, the work it inspired, and the studies …”
   • [claimed-docs] “The Consensus Meter is a visual aggregator that, for a yes/no/possibly question, classifies each relevant paper as supporting, refuting, or …”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round to Elicit
   Elicit~partialclaimed6/10

   Elicit supports exporting Library, tables, and reports in open formats (RIS, CSV, BIB, PDF, DOCX) and offers API/MCP access for programmatic retrieval, which supports data portability. However, export of core artifacts like screening/extraction tables is gated behind Pro/Scale/Enterprise plans, and there's no evidence of full account data export (e.g., all research agent sessions, projects, skills, chat history) in open formats, nor an explicit 'delete account and take everything' workflow. missing for 10: full-account/session export beyond tables and library, confirmation of free-tier export ability, independent verification of export completeness/fidelity.

   • [claimed-docs] “Your Elicit Library can be exported as a .ris file, which you can import into Zotero, Mendeley, EndNote, or another reference manager.”
   • [claimed-docs] “Pro, Scale, and Enterprise subscribers can export tables from Systematic Reviews, including the screening recommendation tables and the data…”
   • [claimed-docs] “Tables can be exported in CSV and Excel format. Certain tables of sources can be exported as RIS or BIB files.”
   • [claimed-docs] “Export to RIS, CSV, BIB, PDF and DOCX”
   • [claimed-docs] “Research Reports can be exported as a PDF or Word file... Pro, Scale, and Enterprise subscribers can export tables from Systematic Reviews, …”
   • [claimed-docs] “The Elicit API lets you access Elicit's research capabilities programmatically: search 138 million+ academic papers and generate automated r…”
   • [claimed-docs] “All API functionality is also available via MCP (Model Context Protocol) server, enabling use from Claude Desktop, Claude Code, and other MC…”

   proof ↗⚑ flag

   Consensus—none0/10

   Evidence shows only import capabilities (Zotero, BibTeX, PDF, RIS) into the Consensus Library, with no mention of exporting a user's library, annotations, or account data back out in open formats. Data portability/export is a fair axis for a reference-manager-style product, but no evidence supports it.

   • [claimed-docs] “Import from Zotero ... or import from BibTex, PDF, or RIS”
   • [claimed-docs] “Import from Zotero”
   • [claimed-docs] “The Consensus Library brings your entire research library into one searchable, AI-powered workspace.”
   • [claimed-docs] “Turn your library into a research engine. Import thousands of papers in one click - then search, find gaps, and put your collection to work.”

   proof ↗⚑ flag

Pricing

1. researcherTry the product meaningfully on a free tier or trial

   weight 1 · round drawn
   Elicit—none0/10

   The evidence pack includes pricing-page references (e.g., elicit-docs-4, elicit-docs-6, elicit-docs-31) but none describe a free tier's scope, limits, or a trial period — no content confirms what a researcher could do without paying. Axis clearly applies to a SaaS research tool, but no evidence substantiates a meaningful free/trial experience.

   ⚑ flag

   Consensus—none0/10

   The evidence pack references a pricing page (consensus-docs-17) but only quotes a single line about 'Deep Searches' feature tiering; there is no description of a free tier, trial period, usage caps, or sign-up-free access that a researcher could evaluate. No first-party or independent evidence confirms Consensus offers a meaningful free/trial experience.

   • [claimed-docs] “Deep Searches (more comprehensive Lit Reviews across many studies)”

   proof ↗⚑ flag

2. researcherUnderstand plan pricing and usage limits before committing

   weight 2 · round to Elicit
   Elicit~partialclaimed3/10

   Evidence confirms a public pricing page exists (elicit.com/pricing) and reveals plan tier names (Pro, Scale, Enterprise) tied to feature gating like table exports, but no evidence pack content shows actual price points, free-tier limits, or usage caps that a researcher would need to compare plans before committing. Missing for 10: actual price figures per tier, usage/query limits, free-plan restrictions, and any independent confirmation of pricing transparency.

   • [claimed-docs] “Import from Zotero”
   • [claimed-docs] “API access”
   • [claimed-docs] “Export to RIS, CSV, BIB, PDF and DOCX”
   • [claimed-docs] “Pro, Scale, and Enterprise subscribers can export tables from Systematic Reviews, including the screening recommendation tables and the data…”
   • [claimed-docs] “Research Reports can be exported as a PDF or Word file... Pro, Scale, and Enterprise subscribers can export tables from Systematic Reviews, …”

   proof ↗⚑ flag

   Consensus—none0/10

   No evidence pack items mention pricing plans, tiers, free/paid limits, or usage quotas — the pack is entirely about product features (citation graph, library, API capabilities). Absence of any pricing/limits documentation for an applicable axis yields none.

   ⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round drawn
   Elicit—none0/10

   No evidence in the pack mentions data residency, regional storage, or the ability to choose where data is stored; Elicit's docs cover exports, imports, API, and workflows but not data residency options. missing for 10: any mention of region/data-residency controls, enterprise data-locality options, or storage location settings.

   ⚑ flag

   Consensus—none0/10

   No evidence in the pack mentions data residency, regional storage options, or any data-location controls for Consensus; all evidence concerns search, citation, and library features. Missing for 10: any mention of region selection, data residency policy, or storage location controls.

   ⚑ flag

2. ai-native userPrevent my data from being used to train AI models

   weight 3 · round drawn
   Elicit—none0/10

   No evidence pack items address opt-out from AI training, data usage policies for model training, or any privacy/data-control settings related to training data; all evidence covers product features (search, review workflows, exports, API/MCP) with no mention of training-data privacy controls. missing for 10: any documentation of a training opt-out setting, data usage/privacy policy statement, or enterprise data handling terms addressing model training.

   ⚑ flag

   Consensus—none0/10

   No evidence pack item addresses data-training opt-out, privacy controls, or AI training data policies for Consensus; all citations concern search, library, and API features. missing for 10: any privacy policy statement, opt-out mechanism, or data usage/training disclosure.

   ⚑ flag

3. ai-native userControl data retention and deletion

   weight 2 · round drawn
   Elicit—none0/10

   No evidence pack items address data retention policies, deletion controls, or account/data deletion mechanisms; docs cover export/import formats but not retention or deletion of user data.

   ⚑ flag

   Consensus—none0/10

   No evidence in the pack addresses data retention policies, deletion controls, or privacy settings for user data/library content; all citations focus on search, citation, library, and API features. Missing for 10: any documentation on data retention windows, user-initiated deletion, export/erasure workflows, or privacy policy specifics.

   ⚑ flag

4. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   Elicit—none0/10

   No evidence in the pack mentions telemetry, usage tracking, or an opt-out mechanism; only unrelated docs about search, exports, and API/MCP features appear. This is a reasonable privacy-posture question for a SaaS AI product, but nothing in the evidence pack supports Elicit offering telemetry opt-out.

   ⚑ flag

   Consensus—none0/10

   No evidence in the pack mentions telemetry, usage tracking, or any opt-out/privacy settings for Consensus; all citations are about search, citation, and library features unrelated to telemetry controls.

   ⚑ flag

Reports

1. researcherExport results to common formats, including documents, spreadsheets, and reference-manager files

   weight 1 · round to Elicit
   Elicit✓fullclaimed9/10

   Elicit documents export of tables/reports to CSV, Excel, PDF, DOCX, RIS, and BIB, covering documents, spreadsheets, and reference-manager formats, and also supports Zotero/EndNote/Mendeley import/export via RIS. Missing for 10: independent hands-on verification of export fidelity beyond vendor docs.

   • [claimed-docs] “Tables can be exported in CSV and Excel format. Certain tables of sources can be exported as RIS or BIB files.”
   • [claimed-docs] “Export to RIS, CSV, BIB, PDF and DOCX”
   • [claimed-docs] “Research Reports can be exported as a PDF or Word file... Pro, Scale, and Enterprise subscribers can export tables from Systematic Reviews, …”
   • [claimed-docs] “Your Elicit Library can be exported as a .ris file, which you can import into Zotero, Mendeley, EndNote, or another reference manager.”
   • [claimed-docs] “Pro, Scale, and Enterprise subscribers can export tables from Systematic Reviews, including the screening recommendation tables and the data…”
   • [claimed-docs] “You can import RIS and BIB files into Elicit. This makes it much easier to import titles/abstracts from other tools like EndNote, Mendeley, …”

   proof ↗⚑ flag

   Consensus—none0/10

   Evidence only documents importing papers into Consensus (from Zotero, BibTeX, PDF, RIS) but contains no mention of exporting results to documents, spreadsheets, or reference-manager formats. Missing for 10: any export-to-Word/PDF, export-to-CSV/spreadsheet, or export-to-Zotero/EndNote/BibTeX functionality.

   • [claimed-docs] “Import from Zotero ... or import from BibTex, PDF, or RIS”
   • [claimed-docs] “Reference managers are great at saving papers — not so great at helping you use them. The Consensus Library brings your entire research libr…”
   • [claimed-docs] “Import from Zotero”

   proof ↗⚑ flag

2. analystGet a structured report with sections, tables, and a summary that I can share with stakeholders

   weight 3 · round to Elicit
   Elicit✓fullclaimed8/10

   Elicit's Systematic Reviews workflow produces a research report summarizing papers, includes data extraction and screening tables, and both reports and tables can be exported as PDF/Word/CSV/Excel for sharing with stakeholders (elicit-docs-1, elicit-docs-22, elicit-docs-47, elicit-docs-19, elicit-docs-25). The Research Agent can also produce documents, tables, and figures within a session (elicit-docs-9, elicit-docs-21, elicit-docs-33). Missing for 10: independent/hands-on corroboration that the exported report format is polished enough for external stakeholder sharing, and no evidence of customizable report sectioning beyond the standard systematic-review structure.

   • [claimed-docs] “The Systematic Reviews workflow provides step-by-step guidance through search, screening, and data extraction, culminating in a research rep…”
   • [claimed-docs] “Elicit supports the major steps of a systematic review: 1. Set up your review on the Setup page 2. Gather all papers for the systematic revi…”
   • [claimed-docs] “Research Reports can be exported as a PDF or Word file... Pro, Scale, and Enterprise subscribers can export tables from Systematic Reviews, …”
   • [claimed-docs] “Pro, Scale, and Enterprise subscribers can export tables from Systematic Reviews, including the screening recommendation tables and the data…”
   • [claimed-docs] “Tables can be exported in CSV and Excel format. Certain tables of sources can be exported as RIS or BIB files.”
   • [claimed-docs] “Collaborators with edit access can ask the agent questions in your session, produce new artifacts (documents, tables, etc.), and edit other …”
   • [claimed-docs] “you can filter and export your sources, generate figures, and draft slides, all without leaving the conversation”

   proof ↗⚑ flag

   Consensus~partialclaimed5/10

   Consensus offers 'Literature Review' and 'Deep Search' features that synthesize evidence across papers, extract structured fields like PICO, and provide citations—suggesting output with some structure and sourcing suitable for sharing. However, there's no explicit evidence of a polished 'report' format with distinct sections, tables, and an executive summary designed for stakeholder sharing (e.g., export to PDF/Word, formatted report templates). Missing for 10: explicit documentation of report formatting/export (sections, tables, summary), evidence of stakeholder-sharing features like PDF export or presentation-ready output, and independent confirmation of report quality.

   • [claimed-docs] “Search, screen, extract, and synthesize evidence faster—while keeping full transparency and scholarly rigor.”
   • [claimed-docs] “Consensus is an AI-powered research engine built to speed up literature reviews. Search, screen, extract, and synthesize evidence faster—whi…”
   • [claimed-docs] “Deep Searches (more comprehensive Lit Reviews across many studies)”
   • [claimed-docs] “Extracted population, intervention, comparator, and outcome (PICO) where applicable”
   • [claimed-docs] “The Consensus Meter is a visual aggregator that, for a yes/no/possibly question, classifies each relevant paper as supporting, refuting, or …”

   proof ↗⚑ flag

Agent runs

1. researcherPose a research question and get an autonomous multi-step investigation, not just a single-pass summary

   weight 3 · round to Elicit
   Elicit~partialcommunity6/10

   Elicit's Research Agent and Systematic Review workflows are explicitly documented as multi-step (effort-level slider from Fastest to Smartest, pulling from multiple source types, iterating until output is complete, and separate search/screen/extract/report phases) rather than single-pass answers (elicit-docs-2, elicit-docs-12/13, elicit-docs-21/33, elicit-docs-22/45). However, hands-on community reports describe results arriving quickly and resembling a single pass over topic clusters rather than deep autonomous investigation, and multiple independent accounts report missed papers, hallucinated conclusions, and shallow reasoning that undercut confidence in true multi-step depth (elicit-comm-1, elicit-comm-3, elicit-comm-4, elicit-comm-5). Missing for 10: independent verification that the agent performs genuinely autonomous multi-step reasoning (not just sequential fixed workflow steps) and evidence rebutting the accuracy/depth complaints.

   • [claimed-docs] “Every Research Agent session runs at an effort level, which you set from the slider in the prompt box before you send your question.”
   • [claimed-docs] “the Research Agent can pull from a wide range of sources (e.g. publications, public filings, press releases), produce flexible outputs, and …”
   • [claimed-docs] “click it to open the slider and move between Fastest, Fast, Balanced, Smart, and Smartest”
   • [claimed-docs] “you can filter and export your sources, generate figures, and draft slides, all without leaving the conversation”
   • [claimed-docs] “Elicit supports the major steps of a systematic review: 1. Set up your review on the Setup page 2. Gather all papers for the systematic revi…”
   • [claimed-docs] “Full-Text Screening is a distinct, automated step in the Systematic Review workflow that catches these mismatches before you commit to data …”
   • [community] “I asked about media bias detection and used the topic analysis feature. A minute or so later, I had a list of concepts with citations and li…”
   • [community] “I gave it a topic I researched in depth recently. It gave me mostly incorrect summaries (one said hypothesis X is confirmed; nope it hasn't)…”
   • [community] “It seems like it should hallucinate less, as it directly quotes, but nope, it still hallucinates just as much and then gives a quote that di…”
   • [community] “Testing Elicit gave me quite a bit worse results than using PaperQA by futurehouse. While paperqa could understand a bit of the nuance of a …”

   proof ↗⚑ flag

   Consensus~partialclaimed5/10

   Consensus advertises a 'Research Agent' that chains citation crawling, DOI lookup, author search, and similar-papers search on top of its search engine, plus a literature-review feature that searches, screens, extracts, and synthesizes evidence — both suggesting multi-step, not single-pass, investigation. However, evidence is limited to marketing feature pages with no walkthrough, example transcript, or independent corroboration of true autonomous multi-step reasoning over a posed question. Missing for 10: a documented end-to-end example of the agent autonomously chaining steps for a specific question, independent/hands-on verification, and detail on how far it goes without user intervention.

   • [claimed-docs] “Citation crawling, DOI lookup, author search, similar papers, and more - chained together on top of the worlds best academic search engine.”
   • [claimed-docs] “Search, screen, extract, and synthesize evidence faster—while keeping full transparency and scholarly rigor.”
   • [claimed-docs] “Consensus is an AI-powered research engine built to speed up literature reviews. Search, screen, extract, and synthesize evidence faster—whi…”

   proof ↗⚑ flag

2. analystStart a long research job that keeps working unattended and notifies me when the result is ready

   weight 2 · round to Elicit
   Elicit~partialclaimed4/10

   Elicit documents an alerts feature that emails you when new relevant papers are found (elicit-docs-3, elicit-docs-15, elicit-docs-26), and long-running workflows like Systematic Reviews and Research Agent effort levels ('Smartest' mode) imply tasks that can take longer to complete (elicit-docs-2, elicit-docs-13, elicit-docs-22). However, alerts are for ongoing topic monitoring, not notification of a specific job's completion, and there's no evidence of a 'start and walk away, get notified when this specific job is done' async job model. Missing for 10: explicit documentation of background/async execution of a research job plus a completion notification (vs. topic-monitoring alerts), and any independent confirmation this works as described.

   • [claimed-docs] “Turn on **Instant email alerts** to be immediately alerted whenever a new relevant research paper is found.”
   • [claimed-docs] “Alerts allow you to stay up to date on the latest research about topics that are relevant to you, so you can add them to your Library for fu…”
   • [claimed-docs] “Turn on Instant email alerts to be immediately alerted whenever a new relevant research paper is found.”
   • [claimed-docs] “Every Research Agent session runs at an effort level, which you set from the slider in the prompt box before you send your question.”
   • [claimed-docs] “click it to open the slider and move between Fastest, Fast, Balanced, Smart, and Smartest”
   • [claimed-docs] “Elicit supports the major steps of a systematic review: 1. Set up your review on the Setup page 2. Gather all papers for the systematic revi…”

   proof ↗⚑ flag

   Consensus—none0/10

   Evidence shows Deep Searches/Lit Reviews and a research agent chaining searches, but there is no mention of async job submission, background/unattended execution, or notification when a long-running job completes.

   ⚑ flag

3. researcherSteer the depth, effort, and scope of a research run before or while it executes

   weight 1 · round to Elicit
   Elicit~partialclaimed7/10

   Elicit's Research Agent lets users set an effort level (Fastest→Smartest) via a slider before sending a query, and users can shape scope through columns, skills, and iterative follow-up prompts within a session; the API also exposes control over search strategy, screening criteria, and extraction parameters for Systematic Reviews. However, evidence only shows steering before/between turns, not genuine mid-execution adjustment of an in-flight run. Missing for 10: documentation of pausing/adjusting effort or scope while a run is actively executing, and independent verification that scope/effort controls meaningfully change output depth.

   • [claimed-docs] “Every Research Agent session runs at an effort level, which you set from the slider in the prompt box before you send your question.”
   • [claimed-docs] “click it to open the slider and move between Fastest, Fast, Balanced, Smart, and Smartest”
   • [claimed-docs] “To add a column, simply tell the research agent what column(s) you'd like to add. For example, you can say: "Add a column for study type."”
   • [claimed-docs] “Create a column for each data point you'd like to extract from your papers. Columns can pull data from the papers' body text or from tables …”
   • [claimed-docs] “you can filter and export your sources, generate figures, and draft slides, all without leaving the conversation”
   • [claimed-docs] “Systematic Review: run a full review end to end, with control over search strategy, screening criteria, extraction parameters, and reporting…”

   proof ↗⚑ flag

   Consensus—none0/10

   The evidence pack describes search, citation graph, library, and research-agent features but nowhere mentions controls for adjusting depth, effort, or scope of a research run before or during execution — no parameters, modes, or configuration options are documented.

   ⚑ flag

Citations

1. researcherSee citations for every substantive claim so I can verify it against the underlying source

   weight 3 · round to Consensus
   Elicit!disputedcontradicted5/10

   Elicit's search/columns features are built around pulling data from papers with links back to sources (elicit-docs-18, elicit-docs-20), and a hands-on user confirms getting 'a list of concepts with citations and links to papers' (elicit-comm-1). However, independent hands-on reports directly contradict the claim that citations reliably let you verify claims: users found Elicit 'hallucinates just as much and then gives a quote that directly contradicts its statement' (elicit-comm-4), produced 'mostly incorrect summaries' and missed key papers (elicit-comm-3), and Elicit itself warns only ~90% accuracy with a need to 'check the work in Elicit closely' (elicit-comm-2). Missing for 10: evidence that citation/quote extraction is reliably accurate, independent verification benchmarks, and resolution of the hallucination-despite-quoting complaints.

   • [claimed-docs] “Create a column for each data point you'd like to extract from your papers. Columns can pull data from the papers' body text or from tables …”
   • [claimed-docs] “With Elicit's semantic search engine, you can ask a question in natural language, and Elicit will find relevant papers, without you having t…”
   • [community] “I asked about media bias detection and used the topic analysis feature. A minute or so later, I had a list of concepts with citations and li…”
   • [community] “Elicit states: 'assume that around 90% of the information you see in Elicit is accurate... it's very important for you to check the work in …”
   • [community] “I gave it a topic I researched in depth recently. It gave me mostly incorrect summaries (one said hypothesis X is confirmed; nope it hasn't)…”
   • [community] “It seems like it should hallucinate less, as it directly quotes, but nope, it still hallucinates just as much and then gives a quote that di…”

   proof ↗⚑ flag

   Consensus✓fullclaimed8/10

   Consensus documents that every AI-generated response includes citations tracing back to the original source paper, and features like the Consensus Meter classify individual papers (supporting/refuting) with traceable provenance, directly matching the researcher's need to verify claims against sources. Missing for 10: independent/hands-on verification of citation accuracy and completeness beyond vendor docs.

   • [claimed-docs] “Every response includes citations, so you can trace each insight back to the original source.”
   • [claimed-docs] “The Consensus Meter is a visual aggregator that, for a yes/no/possibly question, classifies each relevant paper as supporting, refuting, or …”
   • [claimed-docs] “The Consensus Meter is a visual aggregator that, for a yes/no/possibly question, classifies each relevant paper as supporting, refuting, or …”
   • [claimed-docs] “It searches over 200 million academic papers and uses language models to help you find, understand, and synthesize the literature faster.”

   proof ↗⚑ flag

Corpus

1. researcherSearch scholarly literature and primary sources, not just the open web

   weight 2 · round drawn
   Elicit✓fullclaimed9/10

   Elicit's docs clearly show it searches scholarly literature (138M+ academic papers via semantic and keyword search), clinical trials, and journal-restricted queries, plus API/MCP access to the same corpus and systematic-review workflows built around paper screening/extraction rather than general web search. This directly matches the story of searching scholarly/primary sources rather than the open web. Missing for 10: independent corroboration specifically about breadth/quality of the scholarly corpus (community evidence addresses answer accuracy/hallucination, not source scope, so it doesn't contradict this particular axis).

   • [claimed-docs] “With Elicit's semantic search engine, you can ask a question in natural language, and Elicit will find relevant papers, without you having t…”
   • [claimed-docs] “The Elicit API lets you access Elicit's research capabilities programmatically: search 138 million+ academic papers and generate automated r…”
   • [claimed-docs] “Elicit allows you to search both research papers and clinical trials.”
   • [claimed-docs] “Elicit offers both semantic search and keyword search to find the most relevant papers for your systematic review.”
   • [claimed-docs] “Add +journal:"INSERT JOURNAL NAME HERE" to your query”
   • [claimed-docs] “Advanced search filters are a powerful hidden feature in the Find Papers workflow. You can use them to search within a particular journal, r…”
   • [claimed-docs] “Systematic Review: run a full review end to end, with control over search strategy, screening criteria, extraction parameters, and reporting…”

   proof ↗⚑ flag

   Consensus✓fullprobed9/10

   Consensus is explicitly built as a scholarly-search engine over 200M+ academic papers, including full-text and paywalled content, positioned as an AI-native alternative to Google Scholar, with citation tracing back to original sources. Missing for 10: independent third-party verification of corpus quality/coverage beyond vendor claims.

   • [claimed-docs] “Consensus analyzes the full text, including paywalled papers from major publishers, so you can find the most relevant papers.”
   • [claimed-docs] “Think of Consensus as an AI-native alternative to Google Scholar with a more-refined corpus.”
   • [claimed-docs] “It searches over 200 million academic papers and uses language models to help you find, understand, and synthesize the literature faster.”
   • [claimed-docs] “Every response includes citations, so you can trace each insight back to the original source.”
   • [probe] “PROBE llms.txt: HTTP 200 at https://consensus.app/llms.txt # Consensus > Consensus is an AI-powered scientific search engine that finds, ra…”

   proof ↗⚑ flag

Synthesis

1. analystSee where sources agree and disagree instead of a single unqualified answer

   weight 2 · round to Consensus
   Elicit~partialcommunity5/10

   Elicit's Chat/Compare feature explicitly lets analysts 'compare and contrast papers' and 'summarize multiple papers along specific dimensions,' and its column/table extraction lets you see each paper's data side-by-side, which supports spotting agreement/disagreement across sources. However, there is no dedicated feature that explicitly flags or highlights when sources conflict versus concur (no consensus/disagreement indicator), and community reports note the model can hallucinate quotes that contradict its own summaries, undermining confidence in cross-source synthesis. Missing for 10: an explicit contradiction/agreement-detection UI, and independent verification that comparisons are reliably accurate rather than hallucination-prone.

   • [claimed-docs] “Chat enables you to: Compare and contrast papers - Summarize multiple papers along specific dimensions (like their methodologies) - Cluster …”
   • [claimed-docs] “Create a column for each data point you'd like to extract from your papers. Columns can pull data from the papers' body text or from tables …”
   • [claimed-docs] “To add a column, simply tell the research agent what column(s) you'd like to add. For example, you can say: "Add a column for study type."”
   • [claimed-docs] “Elicit supports the major steps of a systematic review: 1. Set up your review on the Setup page 2. Gather all papers for the systematic revi…”
   • [community] “It seems like it should hallucinate less, as it directly quotes, but nope, it still hallucinates just as much and then gives a quote that di…”
   • [community] “I gave it a topic I researched in depth recently. It gave me mostly incorrect summaries (one said hypothesis X is confirmed; nope it hasn't)…”

   proof ↗⚑ flag

   Consensus✓fullclaimed8/10

   The Consensus Meter explicitly classifies each relevant paper as supporting, refuting, or mixed/inconclusive on a given question and displays the distribution, directly surfacing agreement/disagreement across sources rather than a single answer, and every response includes citations back to originals. Missing for 10: independent/hands-on corroboration of the Meter's accuracy and no worked example showing disagreement handling in practice.

   • [claimed-docs] “The Consensus Meter is a visual aggregator that, for a yes/no/possibly question, classifies each relevant paper as supporting, refuting, or …”
   • [claimed-docs] “The Consensus Meter is a visual aggregator that, for a yes/no/possibly question, classifies each relevant paper as supporting, refuting, or …”
   • [claimed-docs] “Every response includes citations, so you can trace each insight back to the original source.”

   proof ↗⚑ flag