Search Infrastructure — procurement report
ProductArena · rankings as of 2026-09-16 · evidence as of 2026-09-16 · 5 products · 53 judged requirements · 265 judged cells
Methodology: Every product is judged against a shared taxonomy of user stories using cited evidence — hands-on probes > repository code > independent community sources > vendor claims — never opinion. Full writeup: https://ultrametric.ai/productarena/methodology
Leaderboard
| # | Product | PA Score | Coverage score | Applicable cells | Confidence |
|---|---|---|---|---|---|
| 1 | Algolia | 30.5 | 40.2 | 53/53 | B |
| 2 | Meilisearch | 29.2 | 38.6 | 51/53 | C |
| 3 | Typesense | 27.4 | 36.6 | 50/53 | B |
| 4 | Orama | 26.8 | 27.5 | 51/53 | B |
| 5 | Elasticsearch | 17.4 | 21.0 | 50/53 | C |
PA Score = agent-readiness blend (see methodology). Coverage score = weighted share of judged requirements met. Confidence = how much of the score rests on tested vs claimed evidence (A–D).
Uncertainty note
The current #1/#2 gap in this arena is not close enough to qualify for the multi-judge uncertainty pass (or the pass has not covered it yet) — no extra caveat applies beyond the per-product confidence grades above.
Buyer checklist (RFP)
The arena's 53 judged user stories as requirements, grouped by theme. Priorities mirror the story weights our scoring uses (3 = must-have, 2 = should-have, 1 = nice-to-have). Interactive version with per-requirement verdicts for the top products: /arena/search-infra/checklist
Agent search — stories about agent search in this arenaAgent search
Stories about agent search in this arena
- ai-native userMy coding agent can create an index, add documents, and run queries end to end — through the API, CLI, or MCP without touching a dashboardmust-have
- ai-native userAgents can use my search indexes as a tool — an MCP server or tool-calling surface exposes query, analytics, and index operationsshould-have
Agenticness — how well agents can access and operate the productAgenticness
How well agents can access and operate the product
- ai-native userPlug MCP servers into this product so it can use their toolsmust-have
- ai-native userConnect an agent via an official MCP servermust-have
- ai-native userDrive the product through a documented public APImust-have
- ai-native userDelegate tasks to a built-in AI assistant inside the productmust-have
- ai-native userPoint an agent at llms.txt or agent-oriented docsshould-have
- ai-native userRun the product headlessly / in CI for automationshould-have
- ai-native userUse an official CLIshould-have
- ai-native userIssue scoped/least-privilege API credentials for an agentshould-have
- ai-native userBuild against official SDKsshould-have
- ai-native userSubscribe to events via webhooksshould-have
- ai-native userGet AI-generated insights and suggestions from my data inside the productshould-have
- ai-native userSet up automations that run autonomously in the backgroundshould-have
- ai-native userOperate the product with natural-language commandsshould-have
- ai-native userExplore an interactive API reference with runnable examplesshould-have
- ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)should-have
- ai-native userRely on versioned APIs with a documented deprecation policyshould-have
- ai-native userTest against a sandbox environment without touching production datanice-to-have
Ai search — stories about ai search in this arenaAi search
Stories about ai search in this arena
- developerRun hybrid search — semantic vector similarity fused with keyword matching — in a single querymust-have
- developerUse built-in or managed embedders so documents and queries are vectorized without running my own embedding pipelineshould-have
- developerPower RAG and conversational answers on top of my indexes with documented retrieval or answer APIsnice-to-have
Automation depth — how much of the product can run unattendedAutomation depth
How much of the product can run unattended
- ai-native userDefine rules that trigger actions automatically on eventsmust-have
- ai-native userPerform bulk operations across many items at onceshould-have
- ai-native userSchedule recurring jobs or workflowsshould-have
- ai-native userVersion, review, and roll back my automationsnice-to-have
Developer experience — stories about developer experience in this arenaDeveloper experience
Stories about developer experience in this arena
- developerCreate an index, add documents, and run my first search within minutes of starting the quickstartmust-have
- developerOfficial SDKs cover my language and framework, kept current across JavaScript, Python, PHP, Ruby, Go, and moreshould-have
- developerOfficial UI component libraries let me assemble a full search interface — box, results, facets, pagination — without building it from scratchshould-have
Indexing pipelines — stories about indexing pipelines in this arenaIndexing pipelines
Stories about indexing pipelines in this arena
- platform-engineerBulk-import millions of documents quickly, with async task tracking to know when indexing completesshould-have
- developerDocument adds, updates, and deletes become searchable in near real time without a full reindexshould-have
- founderIngest content with an official crawler or connectors instead of writing my own indexing pipelinenice-to-have
Openness — open source, data portability, and self-hosting storiesOpenness
Open source, data portability, and self-hosting stories
- ai-native userExport all of my data in open formats and leavemust-have
- ai-native userSelf-host the core productmust-have
- ai-native userDo everything through the API that I can do in the UIshould-have
- ai-native userRead the product's source under an open licenseshould-have
Operations scale — stories about operations scale in this arenaOperations scale
Stories about operations scale in this arena
- platform-engineerSelf-host the full engine — same features as the hosted product — on my own infrastructuremust-have
- founderBuilt-in analytics show top queries, no-result queries, and click-through so I know what users search for and missshould-have
- platform-engineerDocumented scaling paths — clustering, replication, high availability — carry me from prototype to hundreds of millions of documentsshould-have
Pricing plans — plan structure and value — what each tier costs and what it unlocksPricing plans
Plan structure and value — what each tier costs and what it unlocks
- founderPublished per-unit pricing (searches, records, or nodes) lets me predict what search will cost before committingmust-have
- founderCosts stay predictable as records and query volume grow — no surprise per-request cliffsshould-have
Privacy posture — data-handling and privacy storiesPrivacy posture
Data-handling and privacy stories
- ai-native userPrevent my data from being used to train AI modelsmust-have
- ai-native userChoose where my data is stored (region/residency)should-have
- ai-native userControl data retention and deletionshould-have
- ai-native userOpt out of telemetry and usage trackingshould-have
Relevance tuning — stories about relevance tuning in this arenaRelevance tuning
Stories about relevance tuning in this arena
- developerDefine synonyms and curate results — pin, boost, or hide specific hits for specific queriesshould-have
- developerShape relevance with custom ranking rules and business signals (popularity, recency, margin) beyond textual matchingshould-have
- platform-engineerInspect ranking scores or explanations to understand exactly why a result ranked where it didnice-to-have
Search experience — stories about search experience in this arenaSearch experience
Stories about search experience in this arena
- developerDeliver as-you-type instant search with millisecond responses so results update on every keystrokemust-have
- developerSearches tolerate typos and misspellings out of the box, with tunable rules for when and how fuzzy matching appliesmust-have
- developerServe query suggestions and autocomplete backed by real search traffic or a suggestions indexshould-have
- developerBuild faceted navigation — filters with live counts across categories, ranges, and attributes — from a single queryshould-have
Security multitenancy — stories about security multitenancy in this arenaSecurity multitenancy
Stories about security multitenancy in this arena
- developerScoped or tenant tokens restrict each end user's searches to their own documents without separate indexes per usershould-have
Appendix: recorded probes
Hands-on probe recordings — transcripts/videos a human can replay, the strongest evidence tier. Watch them at https://ultrametric.ai/productarena/proofs
- Algolia
npx -y @algolia/cli --versionterminal · recorded 2026-09-06 · exit 0 - Algolia
curl -si -X POST https://mcp.algolia.com/mcp -H 'Content-Type: application/json' -d '<jsonrpc initialize>'terminal · recorded 2026-09-06 · exit 0 - Elasticsearch
docker run docker.elastic.co/elasticsearch/elasticsearch:9.5.3 (single-node, security off) # then index a doc + query q=name:infrastructure over HTTPterminal · recorded 2026-09-06 · exit 0 - Elasticsearch
printf '<jsonrpc initialize>' | ES_URL=http://127.0.0.1:9299 npx -y @elastic/mcp-server-elasticsearch # stdio handshake, no clusterterminal · recorded 2026-09-06 · exit 0 - Meilisearch
meilisearch --db-path /tmp/pa-meili-probe --http-addr 127.0.0.1:7777 # no master key, then index 2 docs + typo search q=serverles over HTTPterminal · recorded 2026-09-06 · exit 0 - Meilisearch
printf '<jsonrpc initialize>' | uvx meilisearch-mcp # stdio handshake, no Meilisearch instanceterminal · recorded 2026-09-06 · exit 0 - Meilisearch
meilisearch --version # installed via `brew install meilisearch`terminal · recorded 2026-09-06 · exit 0 - Orama
npm i @orama/orama && node -e 'create → insert ×2 → search({ term: "infrastucture", tolerance: 2 })' # in-process, no serverterminal · recorded 2026-09-06 · exit 0 - Typesense
typesense-server --data-dir /tmp/pa-ts-probe --api-key=localdev --api-port 8188 # local self-set key, then create collection + index doc + typo search q="serch infra"terminal · recorded 2026-09-06 · exit 0 - Typesense
typesense-server --version # installed via `brew install typesense/tap/typesense-server@30.2`terminal · recorded 2026-09-06 · exit 0
Cite as: ProductArena by Ultrametric Inc, Search Infrastructure arena, rankings as of 2026-09-16 — https://ultrametric.ai/productarena/arena/search-infra
License: © 2026 Ultrametric Inc. Brief quotation of individual verdicts, scores, or evidence excerpts is permitted with attribution to "ProductArena by Ultrametric Inc (ultrametric.ai/productarena)", as is use of the data to evaluate, contest, or contribute corrections. Bulk copying, redistribution, or use to build competing datasets requires prior written permission (see DATA-LICENSE in the repository).
No liability: rankings, verdicts, and scores are research outputs derived from the cited evidence at a point in time, provided "as is", without warranties. Ultrametric Inc accepts no responsibility for procurement, purchasing, or other decisions made in reliance on them — verify against the cited evidence before acting (https://ultrametric.ai/productarena/terms).