AI Memory Layers — procurement report
ProductArena · rankings as of 2026-09-16 · evidence as of 2026-09-16 · 6 products · 57 judged requirements · 342 judged cells
Methodology: Every product is judged against a shared taxonomy of user stories using cited evidence — hands-on probes > repository code > independent community sources > vendor claims — never opinion. Full writeup: https://ultrametric.ai/productarena/methodology
Leaderboard
| # | Product | PA Score | Coverage score | Applicable cells | Confidence |
|---|---|---|---|---|---|
| 1 | Mem0 | 46.8 | 43.3 | 52/57 | C |
| 2 | Letta | 43.3 | 34.1 | 56/57 | C |
| 3 | Airweave | 39.8 | 26.0 | 53/57 | B |
| 4 | Zep | 29.3 | 41.8 | 54/57 | B |
| 5 | Cognee | 29.2 | 35.3 | 56/57 | B |
| 6 | Supermemory | 28.9 | 33.9 | 54/57 | B |
PA Score = agent-readiness blend (see methodology). Coverage score = weighted share of judged requirements met. Confidence = how much of the score rests on tested vs claimed evidence (A–D).
Uncertainty note
The current #1/#2 gap in this arena is not close enough to qualify for the multi-judge uncertainty pass (or the pass has not covered it yet) — no extra caveat applies beyond the per-product confidence grades above.
Buyer checklist (RFP)
The arena's 57 judged user stories as requirements, grouped by theme. Priorities mirror the story weights our scoring uses (3 = must-have, 2 = should-have, 1 = nice-to-have). Interactive version with per-requirement verdicts for the top products: /arena/ai-memory/checklist
Agenticness — how well agents can access and operate the productAgenticness
How well agents can access and operate the product
- ai-native userPlug MCP servers into this product so it can use their toolsmust-have
- ai-native userConnect an agent via an official MCP servermust-have
- ai-native userDrive the product through a documented public APImust-have
- ai-native userDelegate tasks to a built-in AI assistant inside the productmust-have
- ai-native userPoint an agent at llms.txt or agent-oriented docsshould-have
- ai-native userRun the product headlessly / in CI for automationshould-have
- ai-native userUse an official CLIshould-have
- ai-native userIssue scoped/least-privilege API credentials for an agentshould-have
- ai-native userBuild against official SDKsshould-have
- ai-native userSubscribe to events via webhooksshould-have
- ai-native userGet AI-generated insights and suggestions from my data inside the productshould-have
- ai-native userSet up automations that run autonomously in the backgroundshould-have
- ai-native userOperate the product with natural-language commandsshould-have
- ai-native userExplore an interactive API reference with runnable examplesshould-have
- ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)should-have
- ai-native userRely on versioned APIs with a documented deprecation policyshould-have
- ai-native userTest against a sandbox environment without touching production datanice-to-have
Automation depth — how much of the product can run unattendedAutomation depth
How much of the product can run unattended
- ai-native userDefine rules that trigger actions automatically on eventsmust-have
- ai-native userPerform bulk operations across many items at onceshould-have
- ai-native userSchedule recurring jobs or workflowsshould-have
- ai-native userVersion, review, and roll back my automationsnice-to-have
Data lifecycle — stories about data lifecycle in this arenaData lifecycle
Stories about data lifecycle in this arena
- platform-engineerDelete a user's memories on demand — single memory, per-entity, or full erasure — to satisfy privacy requirementsshould-have
- developerMake memories expire or decay — via TTL, expiration dates, or recency weighting — so stale facts stop surfacingshould-have
- platform-engineerExport memories in a machine-readable format so the memory store is portable and not a lock-in trapshould-have
Deployment self host — stories about deployment self host in this arenaDeployment self host
Stories about deployment self host in this arena
- platform-engineerSelf-host the memory layer from open-source code (e.g. via Docker) on infrastructure I controlmust-have
- developerRun the memory layer fully locally — embedded in-process or against local models — without any cloud dependencynice-to-have
Graph entity memory — stories about graph entity memory in this arenaGraph entity memory
Stories about graph entity memory in this arena
- ml-engineerStore memories as a knowledge graph of entities and relationships so multi-hop and entity-centric questions are answerablemust-have
- ml-engineerTrack when facts became valid or invalid (temporal reasoning) so the memory distinguishes current from outdated informationshould-have
- ml-engineerCustomize the memory schema — entity types, edge types, or ontology — to match my domainnice-to-have
Memory recall quality — stories about memory recall quality in this arenaMemory recall quality
Stories about memory recall quality in this arena
- ai-native userMy agent can manage its own memory mid-conversation — adding, searching, updating, and deleting memories through tools or API calls it invokes itselfmust-have
- developerAdd memories from conversations and retrieve them later with semantic search, so context persists across sessionsmust-have
- ai-native userThe memory layer decides for itself what is worth remembering — extracting salient facts from raw conversation and consolidating them in the backgroundshould-have
- ml-engineerSee published memory-quality benchmark results (e.g. LongMemEval, LoCoMo) backing the product's recall-accuracy claimsshould-have
- developerRely on the memory layer to update, supersede, or merge memories when new information contradicts what was storedshould-have
- developerSteer retrieval with metadata filters, keyword/hybrid search modes, or reranking instead of accepting a single fixed similarity searchshould-have
Openness — open source, data portability, and self-hosting storiesOpenness
Open source, data portability, and self-hosting stories
- ai-native userExport all of my data in open formats and leavemust-have
- ai-native userSelf-host the core productmust-have
- ai-native userDo everything through the API that I can do in the UIshould-have
- ai-native userRead the product's source under an open licenseshould-have
Pricing plans — plan structure and value — what each tier costs and what it unlocksPricing plans
Plan structure and value — what each tier costs and what it unlocks
- platform-engineerSee published pricing with a free tier and per-unit rates so I can project memory costs before committingshould-have
Privacy posture — data-handling and privacy storiesPrivacy posture
Data-handling and privacy stories
- ai-native userPrevent my data from being used to train AI modelsmust-have
- ai-native userChoose where my data is stored (region/residency)should-have
- ai-native userControl data retention and deletionshould-have
- ai-native userOpt out of telemetry and usage trackingshould-have
Retrieval performance — stories about retrieval performance in this arenaRetrieval performance
Stories about retrieval performance in this arena
- platform-engineerSee documented retrieval-latency targets or measured numbers (e.g. p50/p95) backing the product's speed claimsshould-have
- platform-engineerIngest at scale with async or batch processing and check the status of background memory operationsshould-have
Sdk integrations — stories about sdk integrations in this arenaSdk integrations
Stories about sdk integrations in this arena
- ai-native userConnect off-the-shelf assistants (Claude, ChatGPT, Cursor) to the same memory so every tool I use shares what it knows about meshould-have
- developerDrop the memory layer into agent frameworks like LangChain, LangGraph, CrewAI, or the Vercel AI SDK via documented first-party integrationsshould-have
- developerBuild against official SDKs in at least Python and TypeScript with equivalent memory APIsshould-have
- developerWire memory into real-time voice pipelines (e.g. LiveKit, Pipecat, ElevenLabs) with documented integrations fast enough for live conversationnice-to-have
Session context — stories about session context in this arenaSession context
Stories about session context in this arena
- developerGet summaries of past sessions or threads so an agent can pick up where the last conversation left offmust-have
- developerRetrieve a token-budgeted, prompt-ready context block assembled from relevant memories in one callshould-have
- developerIngest documents, JSON, and business data into memory — not just chat transcriptsshould-have
- developerStore images, PDFs, or other files as memory inputs and recall information from them laternice-to-have
Tenancy permissions — stories about tenancy permissions in this arenaTenancy permissions
Stories about tenancy permissions in this arena
- developerScope memories per user, agent, or application so one tenant's memories never leak into another's retrievalmust-have
- platform-engineerGovern who and what can read or write memory with roles, policies, or access-control lists, and audit that accessshould-have
- developerShare selected memory across multiple agents or users (team or group memory) while keeping private memory privatenice-to-have
Appendix: recorded probes
Hands-on probe recordings — transcripts/videos a human can replay, the strongest evidence tier. Watch them at https://ultrametric.ai/productarena/proofs
- Airweave
curl -s https://docs.airweave.ai/quickstart.md | head -6terminal · recorded 2026-09-10 · exit 0 - Airweave
curl -s https://docs.airweave.ai/llms.txt | head -6terminal · recorded 2026-09-10 · exit 0 - Cognee
uvx --from cognee cognee-cli demo && uvx --from cognee cognee-cli forget --dataset demoterminal · recorded 2026-09-05 · exit 0 - Cognee
echo '<jsonrpc initialize>' | uvx cognee-mcpterminal · recorded 2026-09-05 · exit 0 - Letta
npx -y @letta-ai/letta-code --versionterminal · recorded 2026-09-05 · exit 0 - Letta
npx -y @letta-ai/letta-code server --backend local --listen ws://127.0.0.1:4500 # keyless boot, then killterminal · recorded 2026-09-05 · exit 0 - Mem0
uvx --from mem0-cli mem0 --helpterminal · recorded 2026-09-05 · exit 0 - Mem0
curl -si -X POST https://mcp.mem0.ai/mcp/ -H 'Content-Type: application/json' -d '<jsonrpc initialize>'terminal · recorded 2026-09-05 · exit 0 - Mem0
uv run --with mem0ai python3 -c 'import mem0; from mem0 import Memory; print("PA_PROBE_OK mem0ai", mem0.__version__)'terminal · recorded 2026-09-05 · exit 0 - Supermemory
curl -si -X POST https://mcp.supermemory.ai/mcp -H 'Content-Type: application/json' -d '<jsonrpc initialize>'terminal · recorded 2026-09-05 · exit 0 - Supermemory
curl -s https://supermemory.ai/openapi.json | head -c 200 && curl -si -X POST https://api.supermemory.ai/v3/search -d '{"q":"probe"}'terminal · recorded 2026-09-05 · exit 0 - Zep
curl -si -X POST https://help.getzep.com/_mcp/server -H 'Content-Type: application/json' -d '<jsonrpc initialize>'terminal · recorded 2026-09-05 · exit 0 - Zep
curl -si -X POST https://api.getzep.com/mcp -H 'Content-Type: application/json' -d '<jsonrpc initialize>'terminal · recorded 2026-09-05 · exit 0 - Zep
uv run --with graphiti-core python3 -c 'from graphiti_core import Graphiti; print("PA_PROBE_OK graphiti-core imported")'terminal · recorded 2026-09-05 · exit 0
Cite as: ProductArena by Ultrametric Inc, AI Memory Layers arena, rankings as of 2026-09-16 — https://ultrametric.ai/productarena/arena/ai-memory
License: © 2026 Ultrametric Inc. Brief quotation of individual verdicts, scores, or evidence excerpts is permitted with attribution to "ProductArena by Ultrametric Inc (ultrametric.ai/productarena)", as is use of the data to evaluate, contest, or contribute corrections. Bulk copying, redistribution, or use to build competing datasets requires prior written permission (see DATA-LICENSE in the repository).
No liability: rankings, verdicts, and scores are research outputs derived from the cited evidence at a point in time, provided "as is", without warranties. Ultrametric Inc accepts no responsibility for procurement, purchasing, or other decisions made in reliance on them — verify against the cited evidence before acting (https://ultrametric.ai/productarena/terms).