Vector Databases & Memory Stores Arena
Vector Databases & Memory Stores — procurement report
ProductArena · rankings as of 2026-09-15 · evidence as of 2026-09-15 · 7 products · 53 judged requirements · 371 judged cells
Methodology: Every product is judged against a shared taxonomy of user stories using cited evidence — hands-on probes > repository code > independent community sources > vendor claims — never opinion. Full writeup: https://ultrametric.ai/productarena/methodology
Leaderboard
| # | Product | PA Score | Coverage score | Applicable cells | Confidence |
|---|---|---|---|---|---|
| 1 | Chroma | 41.1 | 33.2 | 49/53 | B |
| 2 | Milvus | 35.5 | 34.8 | 50/53 | B |
| 3 | Qdrant | 32.2 | 37.9 | 50/53 | B |
| 4 | HelixDB | 31.7 | 25.4 | 50/53 | B |
| 5 | Weaviate | 29.4 | 33.1 | 51/53 | B |
| 6 | LanceDB | 23.4 | 29.5 | 49/53 | C |
| 7 | Pinecone | 20.7 | 29.1 | 52/53 | C |
PA Score = agent-readiness blend (see methodology). Coverage score = weighted share of judged requirements met. Confidence = how much of the score rests on tested vs claimed evidence (A–D).
Uncertainty note
The current #1/#2 gap in this arena is not close enough to qualify for the multi-judge uncertainty pass (or the pass has not covered it yet) — no extra caveat applies beyond the per-product confidence grades above.
Buyer checklist (RFP)
The arena's 53 judged user stories as requirements, grouped by theme. Priorities mirror the story weights our scoring uses (3 = must-have, 2 = should-have, 1 = nice-to-have). Interactive version with per-requirement verdicts for the top products: /arena/vector-databases/checklist
Agenticness — how well agents can access and operate the productAgenticness
How well agents can access and operate the product
- ai-native userPlug MCP servers into this product so it can use their toolsmust-have
- ai-native userConnect an agent via an official MCP servermust-have
- ai-native userDrive the product through a documented public APImust-have
- ai-native userDelegate tasks to a built-in AI assistant inside the productmust-have
- ai-native userPoint an agent at llms.txt or agent-oriented docsshould-have
- ai-native userRun the product headlessly / in CI for automationshould-have
- ai-native userUse an official CLIshould-have
- ai-native userIssue scoped/least-privilege API credentials for an agentshould-have
- ai-native userBuild against official SDKsshould-have
- ai-native userSubscribe to events via webhooksshould-have
- ai-native userGet AI-generated insights and suggestions from my data inside the productshould-have
- ai-native userSet up automations that run autonomously in the backgroundshould-have
- ai-native userOperate the product with natural-language commandsshould-have
- ai-native userExplore an interactive API reference with runnable examplesshould-have
- ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)should-have
- ai-native userRely on versioned APIs with a documented deprecation policyshould-have
- ai-native userTest against a sandbox environment without touching production datanice-to-have
Automation depth — how much of the product can run unattendedAutomation depth
How much of the product can run unattended
- ai-native userDefine rules that trigger actions automatically on eventsmust-have
- ai-native userPerform bulk operations across many items at onceshould-have
- ai-native userSchedule recurring jobs or workflowsshould-have
- ai-native userVersion, review, and roll back my automationsnice-to-have
Data lifecycle — stories about data lifecycle in this arenaData lifecycle
Stories about data lifecycle in this arena
- platform-engineerBack up collections with snapshots and restore themshould-have
- developerUpsert and delete records continuously and have changes reflected in search results quickly, with documented freshness/consistency behaviorshould-have
- developerBulk-import and bulk-export vectors plus metadata in documented formatsshould-have
Deployment modes — stories about deployment modes in this arenaDeployment modes
Stories about deployment modes in this arena
- developerRun the database embedded in-process or as a lightweight local instance for development and small workloadsshould-have
- developerUse a fully managed cloud version of the database with programmatic provisioningshould-have
- platform-engineerDeploy to production on Kubernetes with an official Helm chart or operatornice-to-have
Embeddings pipeline — stories about embeddings pipeline in this arenaEmbeddings pipeline
Stories about embeddings pipeline in this arena
- ml-engineerHave the database generate embeddings at ingest and query time using built-in or configured model providers, instead of running a separate embedding pipelinemust-have
Filtering metadata — stories about filtering metadata in this arenaFiltering metadata
Stories about filtering metadata in this arena
- developerFilter vector search by structured metadata conditions without wrecking recall or latencymust-have
- developerExpress rich filter conditions (ranges, geo, nested boolean logic, array membership) in queriesshould-have
Multi tenancy scale — stories about multi tenancy scale in this arenaMulti tenancy scale
Stories about multi tenancy scale in this arena
- platform-engineerIsolate many tenants cheaply using namespaces, partitions, or per-tenant collections with documented limitsmust-have
- platform-engineerScale beyond one node with sharding or distributed deploymentshould-have
- platform-engineerReplicate data across nodes or zones for high availability with a documented consistency modelshould-have
- platform-engineerEnforce granular access control (API keys, roles, per-collection permissions) on database operationsshould-have
Openness — open source, data portability, and self-hosting storiesOpenness
Open source, data portability, and self-hosting stories
- ai-native userExport all of my data in open formats and leavemust-have
- ai-native userSelf-host the core productmust-have
- ai-native userDo everything through the API that I can do in the UIshould-have
- ai-native userRead the product's source under an open licenseshould-have
Performance latency — stories about performance latency in this arenaPerformance latency
Stories about performance latency in this arena
- platform-engineerSee published benchmarks or measured latency/recall numbers backing the database's performance claimsshould-have
- ml-engineerTune index parameters (HNSW graph settings, index types) to trade recall against latency and memoryshould-have
- ml-engineerEnable vector quantization or compression to cut memory and storage cost with a documented accuracy trade-offshould-have
Pricing plans — plan structure and value — what each tier costs and what it unlocksPricing plans
Plan structure and value — what each tier costs and what it unlocks
- developerPay serverless usage-based pricing with transparent per-unit costs instead of provisioning fixed clustersshould-have
- developerPrototype on a meaningful free tier before paying anythingnice-to-have
Privacy posture — data-handling and privacy storiesPrivacy posture
Data-handling and privacy stories
- ai-native userPrevent my data from being used to train AI modelsmust-have
- ai-native userChoose where my data is stored (region/residency)should-have
- ai-native userControl data retention and deletionshould-have
- ai-native userOpt out of telemetry and usage trackingshould-have
Sdk integrations — stories about sdk integrations in this arenaSdk integrations
Stories about sdk integrations in this arena
- ml-engineerPlug the database into RAG and agent frameworks (LangChain, LlamaIndex, etc.) through maintained first-class integrationsshould-have
- developerBuild against official SDKs in the major languages (Python, TypeScript, Go, Java)should-have
Search quality hybrid — stories about search quality hybrid in this arenaSearch quality hybrid
Stories about search quality hybrid in this arena
- developerRun approximate nearest-neighbor similarity search over embeddings with configurable distance metricsmust-have
- developerCombine dense vector search with keyword or sparse (BM25-style) signals in one hybrid query with fusion rankingmust-have
- developerRun keyword/full-text search over documents inside the database without bolting on a separate search engineshould-have
- ml-engineerRerank search results with built-in or first-party-integrated reranking modelsshould-have
Pricing signals
Extracted verbatim from each vendor's own pricing page — never converted, averaged, or derived. Products whose page prints no unit price are recorded as unclear, honestly.
| Product | Headline price | Unit | As of |
|---|---|---|---|
| Chroma | $0.0075 | TB queried | 2026-09-07 |
| Milvus | $5 | 1M vectors/month | 2026-09-07 |
| Qdrant | pricing unclear — The page describes tiers (Free, Standard, Premium, Hybrid, Private Cloud) and mentions usage-based billing by vCPU/RAM/disk/tokens, but prints no numeric dollar figures or per-unit rates that can be extracted without conversion. | 2026-09-07 | |
| Weaviate | $0.00465 | 1M vector dimensions/month | 2026-09-07 |
| Pinecone | $0.33 | GB-month | 2026-09-07 |
Appendix: recorded probes
Hands-on probe recordings — transcripts/videos a human can replay, the strongest evidence tier. Watch them at https://ultrametric.ai/productarena/proofs
- Chroma
chroma --helpterminal · recorded 2026-09-04 · exit 0 - Chroma
chroma run --path /tmp/pa-chroma-probe --port 8765 & curl -X POST localhost:8765/api/v2/tenants/default_tenant/databases/default_database/collections -d '{"name":"pa_probe"}' && curl localhost:8765/api/v2/tenants/default_tenant/databases/default_database/collectionsterminal · recorded 2026-09-04 · exit 0 - Chroma
echo '<jsonrpc initialize>' | uvx chroma-mcp --client-type ephemeralterminal · recorded 2026-09-04 · exit 0 - HelixDB
curl -s https://docs.helix-db.com/llms.txt | head -6terminal · recorded 2026-09-10 · exit 0 - HelixDB
curl -si -X POST https://mcp.helix-db.com/mcp -H 'Content-Type: application/json' -d '<jsonrpc initialize>'terminal · recorded 2026-09-10 · exit 0 - LanceDB
curl -s https://docs.lancedb.com/llms.txt | head -4terminal · recorded 2026-09-15 · exit 0 - LanceDB
mktemp -d && uv venv && uv pip install lancedb && python -c "import lancedb; print('PA_PROBE_OK lancedb', version('lancedb'))"terminal · recorded 2026-09-15 · exit 0 - Milvus
python3 -c 'from pymilvus import MilvusClient; c = MilvusClient("/tmp/pa-milvus-probe.db"); c.create_collection("pa_probe", dimension=8); print("PA_PROBE_OK", c.list_collections())'terminal · recorded 2026-09-04 · exit 0 - Pinecone
curl -si -X POST https://docs.pinecone.io/mcp -H 'Content-Type: application/json' -d '<jsonrpc initialize>'terminal · recorded 2026-09-04 · exit 0 - Qdrant
docker run -d --name pa-qdrant-probe -p 16333:6333 qdrant/qdrant && curl -X PUT localhost:16333/collections/pa_probe -d '{"vectors":{"size":8,"distance":"Cosine"}}' && curl localhost:16333/collectionsterminal · recorded 2026-09-04 · exit 0 - Qdrant
echo '<jsonrpc initialize>' | QDRANT_LOCAL_PATH=/tmp/pa-qdrant-mcp COLLECTION_NAME=pa-probe uvx mcp-server-qdrantterminal · recorded 2026-09-04 · exit 0 - Weaviate
docker run -d --name pa-weaviate-probe -p 18080:8080 -e AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED=true cr.weaviate.io/semitechnologies/weaviate:latest && curl -X POST localhost:18080/v1/schema -d '{"class":"PaProbe","vectorizer":"none"}' && curl localhost:18080/v1/schematerminal · recorded 2026-09-04 · exit 0
Cite as: ProductArena by Ultrametric Inc, Vector Databases & Memory Stores arena, rankings as of 2026-09-15 — https://ultrametric.ai/productarena/arena/vector-databases
License: © 2026 Ultrametric Inc. Brief quotation of individual verdicts, scores, or evidence excerpts is permitted with attribution to "ProductArena by Ultrametric Inc (ultrametric.ai/productarena)", as is use of the data to evaluate, contest, or contribute corrections. Bulk copying, redistribution, or use to build competing datasets requires prior written permission (see DATA-LICENSE in the repository).
No liability: rankings, verdicts, and scores are research outputs derived from the cited evidence at a point in time, provided "as is", without warranties. Ultrametric Inc accepts no responsibility for procurement, purchasing, or other decisions made in reliance on them — verify against the cited evidence before acting (https://ultrametric.ai/productarena/terms).