Letta vs Supermemory

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round drawn
   Letta✓fullprobed8/10

   A direct probe confirms Letta publishes an llms.txt file at docs.letta.com/llms.txt returning HTTP 200 with agent-oriented summary content, and Letta's extensive docs site is clearly structured for programmatic/agent consumption (agent-sdk, CLI, MCP docs). Missing for 10: no evidence of llms.txt content depth/comprehensiveness or independent confirmation that agents successfully consume it in practice.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.letta.com/llms.txt # Letta > Letta agents learn from experience and improve with use. The Letta ag…”
   • [claimed-docs] “create an agent once, then resume it from anywhere”

   proof ↗⚑ flag

   Supermemory✓fullprobed8/10

   A probe confirms a working llms.txt endpoint (HTTP 200) at docs.supermemory.ai, and first-party docs describe agent-oriented resources: a remote MCP server for agents to search Supermemory documentation while implementing integrations, an npx supermemory setup CLI that detects the project and launches the integration flow, and an installable skill so agents use real endpoints instead of hallucinating APIs. Missing for 10: independent/hands-on report of an agent successfully using llms.txt or the docs-MCP end-to-end, and no discoverable OpenAPI spec (probe found 404s on all candidate paths).

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.supermemory.ai/llms.txt # supermemory | Memory API for the AI era - [Introduction](https://superme…”
   • [claimed-docs] “Remote MCP that lets the agent search Supermemory documentation while it implements an integration.”
   • [claimed-docs] “npx supermemory setup # detect project, launch/print integration flow”
   • [claimed-docs] “Install the official skill so the agent uses the real endpoints, auth, and `containerTag` rules instead of hallucinating APIs”
   • [claimed-docs] “Set up coding agents to integrate Supermemory — CLI, skill, and docs MCP.”
   • [probe] “official MCP server documented at https://supermemory.ai/docs/supermemory-mcp/mcp”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round to Letta
   Letta✓fullclaimed8/10

   Letta explicitly documents a headless CLI mode built for non-interactive use in scripts, CI/CD pipelines, and UNIX tool composition, plus an --ephemeral one-shot flag for stateless CI runs, backed by a self-hostable App Server and SDK for programmatic/automated invocation. Missing for 10: independent hands-on CI usage report or example CI config demonstrating it in practice.

   • [claimed-docs] “Headless mode allows you to run Letta Code non-interactively, making it easy to integrate into scripts, CI/CD pipelines, or compose with oth…”
   • [claimed-docs] “Headless mode allows you to run Letta Code non-interactively, making it easy to integrate into scripts, CI/CD pipelines, or compose with oth…”
   • [claimed-docs] “Use --ephemeral to run a one-shot task without creating, resuming, or persisting an agent, memory blocks, or MemFS”
   • [claimed-docs] “Headless mode allows you to run Letta Code non-interactively, making it easy to integrate into scripts, CI/CD pipelines”
   • [claimed-docs] “The Letta App Server lets you deploy the Letta agent harness as a service.”
   • [claimed-docs] “Use `backend: "cloud"` to host agents in Letta Cloud and run tools in managed cloud environments.”

   proof ↗⚑ flag

   Supermemory✓fullclaimed7/10

   Supermemory is API-first: official TypeScript/Python SDKs and a documented REST API let content be added and searched entirely programmatically (docs-24, docs-26, docs-27), and the self-hosted binary explicitly boots with no Docker/config, suitable for automated/CI environments (docs-7, docs-31, docs-44). A CLI (npx supermemory setup) also supports scripted agent setup (docs-6). missing for 10: explicit CI/automation examples or docs, independent confirmation of unattended API-key usage, and clarity on whether initial key minting can be fully non-interactive

   • [claimed-docs] “Official clients wrap this API: TypeScript: npm install supermemory Python: pip install supermemory”
   • [claimed-docs] “await client.add({ content: "The user loves Paris.", containerTag: "user_123" });”
   • [claimed-docs] “const results = await client.search({ q: "where does Alex work?", containerTag: "user_123", include: { relatedMemories: true }, });”
   • [claimed-docs] “Supermemory runs on your own hardware. It's the same memory engine behind the hosted platform... as a single self-contained binary.”
   • [claimed-docs] “No Docker. No database to provision. No config files. It boots in seconds with everything built in”
   • [claimed-docs] “State-of-the-art memory, running on your machine. One binary, zero config.”
   • [claimed-docs] “npx supermemory setup # detect project, launch/print integration flow”
   • [claimed-docs] “app.supermemory.ai is the consumer product on the same engine — not where you mint API keys.”

   proof ↗⚑ flag

3. ai-native userUse an official CLI

   weight 2 · round to Letta
   Letta✓fullprobed8/10

   Letta ships an official CLI ("Letta Code") with rich documentation covering interactive slash commands, headless/non-interactive scripting mode for CI/CD, local/self-hosted mode, and cloud sync — confirmed by first-party docs and a probe hitting the CLI docs page directly. Community evidence (HN threads with Letta employees) corroborates real usage of CLI features like --yolo mode and /init, indicating it's a genuine, actively used tool. Missing for 10: independent hands-on reviews specifically benchmarking the CLI's reliability/UX beyond employee-provided community threads, and no direct evidence of install/version metrics or broad third-party adoption data.

   • [claimed-docs] “Headless mode allows you to run Letta Code non-interactively, making it easy to integrate into scripts, CI/CD pipelines, or compose with oth…”
   • [claimed-docs] “Headless mode allows you to run Letta Code non-interactively, making it easy to integrate into scripts, CI/CD pipelines, or compose with oth…”
   • [claimed-docs] “This default local mode stores agent state on your machine and does not require a Letta login.”
   • [claimed-docs] “Control Letta Code's behavior during an interactive session with slash commands.”
   • [claimed-docs] “Run /login to back up your agents to the cloud and make them available through chat.letta.com, the desktop app, remote machines, and messagi…”
   • [probe] “official CLI documented at https://docs.letta.com/platform/cli”
   • [community] “Letta Code described by Letta's Charles as 'memory-first': agents build learned context over time via tools like /init and /remember; a /cle…”
   • [community] “User asks if Letta Code supports a 'yolo'/dangerous-skip-permissions mode like Claude Code; Letta's Charles confirms: 'Letta Code also has a…”
   • [community] “Letta employee describes Letta Code: 'a fundamentally different approach to coding agents with a focus on continual learning and statefulnes…”

   proof ↗⚑ flag

   Supermemory~partialclaimed5/10

   Docs mention an official CLI command (`npx supermemory setup`) that detects a project and launches the integration flow, and reference 'CLI, skill, and docs MCP' as part of agent setup, showing a CLI exists for AI-native/agentic workflows. However, evidence is thin — no dedicated CLI reference page, no list of full command set, and no independent/hands-on confirmation of its use. Missing for 10: a full CLI command reference, examples beyond the single setup command, independent user confirmation of the CLI in practice.

   • [claimed-docs] “npx supermemory setup # detect project, launch/print integration flow”
   • [claimed-docs] “Set up coding agents to integrate Supermemory — CLI, skill, and docs MCP.”
   • [claimed-docs] “Install the official skill so the agent uses the real endpoints, auth, and `containerTag` rules instead of hallucinating APIs”

   proof ↗⚑ flag

4. ai-native userDrive the product through a documented public API

   weight 3 · round drawn
   Letta✓fullprobed8/10

   Letta ships a fully documented Agent SDK (TypeScript, with browser/mobile variants) covering agent creation, sessions, streaming, permissions, MCP tool proxying, and an OpenAI-compatible App Server API, all backed by extensive first-party docs and a public GitHub SDK repo. This clearly satisfies driving the product programmatically as an AI-native user via a documented public API. missing for 10: a discoverable formal OpenAPI/swagger spec (probe found only 404s), and independent/hands-on developer corroboration of the SDK's public API stability.

   • [claimed-docs] “Create a stateful agent with `client.createAgent()`. It returns the agent's ID as a string, which you use to open sessions and manage the ag…”
   • [claimed-docs] “The stream yields typed events for the whole turn — reasoning, assistant text, tool calls, tool results”
   • [claimed-docs] “a **`canUseTool` callback** that approves, denies, or edits individual tool calls — interactively, if your application involves a human.”
   • [claimed-docs] “`resumeSession(id)` accepts either kind of ID: pass an `agent-xxx` ID to resume the agent's default conversation, or a `conv-xxx` ID to resu…”
   • [claimed-docs] “The stream yields typed events for the whole turn — reasoning, assistant text, tool calls, tool results — and terminates after the turn's `r…”
   • [claimed-docs] “The SDK supports local stdio processes, Streamable HTTP, and legacy SSE”
   • [claimed-docs] “MCP tools are discovered from servers supplied through `mcpServers` and proxied by the SDK host to the MCP server.”
   • [claimed-docs] “Send a message with `session.send()` and read the response with `session.stream()`. The stream yields typed events for the whole turn”
   • [claimed-docs] “use the App Server’s OpenAI-compatible API with clients such as Open WebUI, or the ACP adapter with ACP clients such as Zed”
   • [github] “the Letta Agent SDK for building agents into TypeScript applications”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.letta.com/openapi.json, https://docs.letta.com/swagger.json, https://docs.letta.com/api…”

   proof ↗⚑ flag

   Supermemory✓fullprobed8/10

   Supermemory documents a full public API (add/search/memory-operations, containerTag scoping, filtering) plus official TypeScript and Python client libraries wrapping it, and an API reference overview — clearly enabling AI-native users to drive the product programmatically. Missing for 10: a discoverable OpenAPI/swagger spec (probe found only 404s) and independent hands-on confirmation of API robustness beyond docs.

   • [claimed-docs] “Official clients wrap this API: TypeScript: npm install supermemory Python: pip install supermemory”
   • [claimed-docs] “Create memories directly without going through the document ingestion workflow. Memories are embedded and immediately searchable.”
   • [claimed-docs] “await client.add({ content: "The user loves Paris.", containerTag: "user_123" });”
   • [claimed-docs] “const results = await client.search({ q: "where does Alex work?", containerTag: "user_123", include: { relatedMemories: true }, });”
   • [claimed-docs] “Use `searchMode: "hybrid"` for best results. It searches both memories and document chunks, returning the most relevant content.”
   • [claimed-docs] “Metadata Filtering ... Query memories by custom properties like category, status, or date”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.supermemory.ai/openapi.json, https://docs.supermemory.ai/swagger.json, https://docs.sup…”

   proof ↗⚑ flag

5. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round to Letta
   Letta~partialclaimed4/10

   Letta offers some related scoped-access mechanisms — Secrets that let an agent use API keys/tokens without ever seeing the raw values, GitHub integration where agents can only access repositories explicitly granted, and tool/action permission modes (unrestricted/acceptEdits/standard, canUseTool callback) — but there's no documented mechanism for issuing a distinct, scoped/least-privilege API credential specifically for an agent's own access to the Letta API or external services. missing for 10: explicit per-agent API key/credential issuance with configurable scopes, documentation of least-privilege credential lifecycle (rotation/revocation), and independent confirmation of scoped-credential behavior in practice.

   • [claimed-docs] “Secrets let your agent use sensitive values like API keys and tokens in shell commands without ever seeing the actual values.”
   • [claimed-docs] “Once connected, agents in your Letta org running on cloud computers will be able to clone, read, and push to the repositories you grant acce…”
   • [claimed-docs] “Press `Shift+Tab` in interactive mode to cycle through `unrestricted`, `acceptEdits`, and `standard`.”
   • [claimed-docs] “a **`canUseTool` callback** that approves, denies, or edits individual tool calls — interactively, if your application involves a human.”
   • [claimed-docs] “You can share agents across your team (and set permissions for access) to create custom agents for your organization”

   proof ↗⚑ flag

   Supermemory—none0/10

   Evidence shows container-tag based data isolation (namespacing memories per user/tenant) and mention that API keys are minted somewhere other than the consumer app, but nothing describes issuing scoped or least-privilege API credentials/tokens (e.g., read-only vs write, per-agent permission scopes) for agents. OAuth credential replacement (docs-47) is about branding the OAuth app name, not permission scoping.

   • [claimed-docs] “A container tag is a hard boundary — its own namespace. Memories in one tag are never returned by a search scoped to another tag.”
   • [claimed-docs] “app.supermemory.ai is the consumer product on the same engine — not where you mint API keys.”
   • [claimed-docs] “every memory tagged with `user_alex` lives in its own isolated space, completely separate from memories tagged `user_jordan`”
   • [claimed-docs] “You can replace this with your own app name by providing your own OAuth credentials via the settings endpoint.”

   proof ↗⚑ flag

6. ai-native userBuild against official SDKs

   weight 2 · round drawn
   Letta✓fullclaimed8/10

   Letta ships an official Agent SDK (letta-ai/letta on GitHub, TypeScript) with extensive first-party documentation covering agent creation, sessions, streaming, MCP tool integration, permissions/canUseTool callbacks, cloud/self-host deployment options, and demo apps for browser, Expo/React Native, and desktop clients - a substantial, well-documented SDK surface for AI-native builders. Missing for 10: independent hands-on developer reviews/benchmarks of the SDK itself, and evidence of official SDKs beyond TypeScript (e.g., Python).

   • [github] “the Letta Agent SDK for building agents into TypeScript applications”
   • [claimed-docs] “Create a stateful agent with `client.createAgent()`. It returns the agent's ID as a string, which you use to open sessions and manage the ag…”
   • [claimed-docs] “The stream yields typed events for the whole turn — reasoning, assistant text, tool calls, tool results”
   • [claimed-docs] “a **`canUseTool` callback** that approves, denies, or edits individual tool calls — interactively, if your application involves a human.”
   • [claimed-docs] “Browser, Expo, and React Native applications must import the SDK from `@letta-ai/letta-agent-sdk/client`”
   • [claimed-docs] “Use `backend: "cloud"` to host agents in Letta Cloud and run tools in managed cloud environments.”
   • [claimed-docs] “`resumeSession(id)` accepts either kind of ID: pass an `agent-xxx` ID to resume the agent's default conversation, or a `conv-xxx` ID to resu…”
   • [claimed-docs] “The SDK supports local stdio processes, Streamable HTTP, and legacy SSE”
   • [claimed-docs] “MCP tools are discovered from servers supplied through `mcpServers` and proxied by the SDK host to the MCP server.”
   • [claimed-docs] “Send a message with `session.send()` and read the response with `session.stream()`. The stream yields typed events for the whole turn”
   • [claimed-docs] “a complete Expo and React Native client for persistent Letta agents”

   proof ↗⚑ flag

   Supermemory✓fullprobed8/10

   Docs explicitly reference official TypeScript and Python SDKs ('npm install supermemory', 'pip install supermemory') that wrap the full API, with extensive documented usage examples (add, search, container tags) across the docs, indicating a mature developer-facing SDK surface. Missing for 10: independent hands-on developer corroboration of the SDKs themselves (community evidence only discusses unrelated ingestion/auth complaints) and no visible changelog/versioning evidence for SDK maturity.

   • [claimed-docs] “Official clients wrap this API: TypeScript: npm install supermemory Python: pip install supermemory”
   • [claimed-docs] “await client.add({ content: "The user loves Paris.", containerTag: "user_123" });”
   • [claimed-docs] “const results = await client.search({ q: "where does Alex work?", containerTag: "user_123", include: { relatedMemories: true }, });”
   • [claimed-docs] “await client.add({ content: "Alex mentioned he just started at Stripe", containerTag: "user_123" });”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.supermemory.ai/llms.txt # supermemory | Memory API for the AI era - [Introduction](https://superme…”

   proof ↗⚑ flag

7. ai-native userSubscribe to events via webhooks

   weight 2 · round drawn
   Letta—none0/10

   The evidence pack covers agent SDK streaming, subagents, mods, scheduled tasks, and Slack/GitHub integrations, but there is no mention of a webhook subscription mechanism for events (e.g., agent state changes, task completion) that external systems could subscribe to. Streaming responses (letta-docs-43, letta-docs-60) are pull-based within an active session, not webhook push notifications.

   ⚑ flag

   Supermemory—none0/10

   The only webhook references describe internal connector sync (e.g., Google Drive/Gmail/Notion changes triggering Supermemory's own ingestion pipeline via 'real-time webhooks'), not an outbound webhook subscription API for end users to receive event notifications. No docs describe registering a webhook endpoint or subscribing to events like memory-created, sync-complete, etc.

   • [github] “Connectors Google Drive · Gmail · Notion · OneDrive · GitHub — auto-sync with real-time webhooks.”
   • [github] “Connectors | Google Drive · Gmail · Notion · OneDrive · GitHub — auto-sync with real-time webhooks.”
   • [claimed-docs] “Connect external platforms to automatically sync documents into supermemory. Supported connectors include Google Drive, Gmail, Notion, OneDr…”

   proof ↗⚑ flag

Agentic features

1. ai-native userGet AI-generated insights and suggestions from my data inside the product

   weight 2 · round to Letta
   Letta~partialcommunity6/10

   Letta's 'dreaming' feature uses background subagents to review conversation history and consolidate learned lessons into memory without interrupting the user, and /doctor audits memory placement/duplication — both are forms of AI-generated insight derived from the agent's own interaction data. However, these are framed as internal memory-maintenance mechanisms rather than explicit user-facing 'suggestions' or dashboards surfacing insights about the user's data, and there is no independent/hands-on evidence of the quality of these generated insights. Missing for 10: user-facing insight/suggestion UI or reports, independent verification that dreaming produces useful proactive suggestions, and examples of insights derived from arbitrary connected data sources (not just conversation history).

   • [claimed-docs] “Dreaming uses background subagents to review recent conversations, consolidate useful lessons, and update memory without interrupting your a…”
   • [claimed-docs] “trigger controls when dreaming runs: after a number of steps, on context compaction, or never.”
   • [claimed-docs] “If the memory hierarchy has drifted or grown too large, run /doctor to audit placement, duplication, and system-prompt token usage.”
   • [claimed-docs] “You can also teach it explicitly with /remember: > /remember always use pnpm in this repo”
   • [community] “Letta Code described by Letta's Charles as 'memory-first': agents build learned context over time via tools like /init and /remember; a /cle…”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the [MemFS Search mod]”

   proof ↗⚑ flag

   Supermemory~partialclaimed5/10

   Supermemory automatically extracts facts, builds a living knowledge graph, and maintains auto-updating user profiles from ingested data — a form of AI-generated insight derived from raw content (supermemory-docs-14, supermemory-docs-15, supermemory-docs-33, supermemory-docs-40). However, there's no evidence of proactive 'suggestions' (e.g., recommended actions, next-step prompts) surfaced to the user — the product is positioned as memory infrastructure/API for retrieval rather than an insights/suggestions dashboard. Missing for 10: evidence of proactive suggestion generation, a user-facing insights UI, or explicit 'recommendation' features beyond fact/profile extraction.

   • [claimed-docs] “Supermemory builds a living knowledge graph of facts on top of other facts — not a static folder of embeddings”
   • [claimed-docs] “User profiles are **automatically maintained collections of facts about your users** that Supermemory builds from all their interactions.”
   • [claimed-docs] “No search needed — comprehensive user info always ready Profiles update as users interact with your system”
   • [claimed-docs] “User profiles are automatically maintained collections of facts about your users that Supermemory builds from all their interactions.”
   • [claimed-docs] “Retrieve three ways — document search (RAG), memory graph traversal, and user profile”

   proof ↗⚑ flag

2. ai-native userSet up automations that run autonomously in the background

   weight 2 · round to Letta
   Letta✓fullcommunity8/10

   Letta explicitly supports autonomous background automation: scheduled/recurring prompts via 'Scheduled tasks' (letta-docs-69), always-on cloud agents that keep running when your laptop is closed (letta-docs-55, letta-docs-68), Slack integration where mentioned agents respond using their own cloud computer (letta-docs-73, letta-docs-83), and background 'dreaming' subagents that consolidate memory without interrupting active work (letta-docs-27, letta-docs-3). Community posts corroborate the always-on, memory-first agent design (letta-comm-12, letta-comm-5). Missing for 10: independent hands-on verification that scheduled/background automations run reliably in practice, and more detail on failure handling/monitoring of unattended runs.

   • [claimed-docs] “Scheduled tasks allow you to automate prompts to your agent on a recurring schedule or at a specific time in the future.”
   • [claimed-docs] “Agents can still be messaged even if your laptop is closed, since they have their own computer to run on.”
   • [claimed-docs] “Agents can still be messaged even if your laptop is closed, since they have their own computer to run on. Combining agents with Slack and sc…”
   • [claimed-docs] “Agents connected to Slack are always-on: mention them in a channel or thread and they'll respond, using their cloud computer to do real work…”
   • [claimed-docs] “Agents connected to Slack are always-on: mention them in a channel or thread and they'll respond”
   • [claimed-docs] “Dreaming uses background subagents to review recent conversations, consolidate useful lessons, and update memory without interrupting your a…”
   • [claimed-docs] “trigger controls when dreaming runs: after a number of steps, on context compaction, or never.”
   • [community] “Letta employee describes Letta Code: 'a fundamentally different approach to coding agents with a focus on continual learning and statefulnes…”

   proof ↗⚑ flag

   Supermemory~partialclaimed4/10

   Supermemory offers background automation only in the narrow sense of connector auto-sync via real-time webhooks (Google Drive, Gmail, Notion, OneDrive, GitHub) and automatically-maintained user profiles that update as users interact — both run autonomously without manual triggering. But there's no evidence of a general-purpose automation/workflow builder, scheduled agent tasks, or triggers-and-actions system that an AI-native user could configure to run arbitrary autonomous background jobs. Missing for 10: a documented automation/workflow engine, scheduling capabilities beyond data-sync connectors, and any evidence of user-configurable autonomous background tasks.

   • [github] “Connectors Google Drive · Gmail · Notion · OneDrive · GitHub — auto-sync with real-time webhooks.”
   • [github] “Connectors | Google Drive · Gmail · Notion · OneDrive · GitHub — auto-sync with real-time webhooks.”
   • [claimed-docs] “User profiles are automatically maintained collections of facts about your users that Supermemory builds from all their interactions.”
   • [claimed-docs] “No search needed — comprehensive user info always ready Profiles update as users interact with your system”
   • [claimed-docs] “Connect external platforms to automatically sync documents into supermemory. Supported connectors include Google Drive, Gmail, Notion, OneDr…”

   proof ↗⚑ flag

3. ai-native userDelegate tasks to a built-in AI assistant inside the product

   weight 3 · round to Letta
   Letta✓fullcommunity8/10

   Letta ships a built-in agent runtime (Letta Code / Letta App Server) to which users can delegate real tasks: subagents for parallelized work, autonomous cloud agents that clone/read/push GitHub repos and open PRs, Slack-integrated always-on agents, and scheduled/recurring task automation, all corroborated by first-party docs and HN threads from Letta's own team describing the 'memory-first,' continually-learning agent design. missing for 10: independent hands-on benchmarks or third-party reviews validating real-world task delegation reliability beyond vendor docs and founder HN comments.

   • [claimed-docs] “By delegating work to focused subagents, your main agent can keep its context window clean and make use of parallelism to divide and conquer…”
   • [claimed-docs] “Subagents are specialized agents that your main agent can spawn to handle complex tasks autonomously.”
   • [claimed-docs] “Once connected, agents in your Letta org running on cloud computers will be able to clone, read, and push to the repositories you grant acce…”
   • [claimed-docs] “agents running on cloud computers can access your repositories to do work — sweeping bug reports, opening pull requests, or exploring your c…”
   • [claimed-docs] “Agents can still be messaged even if your laptop is closed, since they have their own computer to run on. Combining agents with Slack and sc…”
   • [claimed-docs] “Agents connected to Slack are always-on: mention them in a channel or thread and they'll respond, using their cloud computer to do real work…”
   • [claimed-docs] “Scheduled tasks allow you to automate prompts to your agent on a recurring schedule or at a specific time in the future.”
   • [community] “Letta employee describes Letta Code: 'a fundamentally different approach to coding agents with a focus on continual learning and statefulnes…”
   • [community] “Letta Code described by Letta's Charles as 'memory-first': agents build learned context over time via tools like /init and /remember; a /cle…”

   proof ↗⚑ flag

   Supermemory—none0/10

   Supermemory is positioned as a memory infrastructure/API layer that other AI assistants connect to (via MCP, SDKs, connectors) rather than as a product with its own built-in AI assistant for task delegation; 'Ask naturally' (supermemory-docs-35) describes external assistants querying Supermemory's tools, not a native in-app agent. No evidence describes a built-in assistant inside Supermemory itself that a user can delegate tasks to.

   ⚑ flag

4. ai-native userOperate the product with natural-language commands

   weight 2 · round to Letta
   Letta✓fullcommunity8/10

   Letta Code and CLI are extensively controlled via natural-language chat plus slash commands (/remember, /init, /fork, /btw, /doctor, /agents) that wrap agent behavior in conversational form, and agents themselves interpret natural-language instructions to manage memory, tools, and subagents. missing for 10: independent hands-on evidence (beyond vendor docs/HN Q&A) confirming reliability of natural-language command interpretation across complex workflows.

   • [claimed-docs] “Control Letta Code's behavior during an interactive session with slash commands.”
   • [claimed-docs] “Run `/init` to bootstrap or refresh memory for the current project. The agent inspects the repository, asks about your working style when ne…”
   • [claimed-docs] “You can also teach it explicitly with `/remember`: `> /remember always use pnpm in this repo`. The agent decides where the lesson belongs an…”
   • [claimed-docs] “You can also teach it explicitly with /remember: > /remember always use pnpm in this repo”
   • [claimed-docs] “If the memory hierarchy has drifted or grown too large, run /doctor to audit placement, duplication, and system-prompt token usage.”
   • [claimed-docs] “Use /fork to branch the current conversation, including its in-context history. The new conversation can take a different direction without …”
   • [claimed-docs] “Use /btw for a side question. It forks the conversation in the background and returns the answer without interrupting your main task”
   • [community] “Letta Code described by Letta's Charles as 'memory-first': agents build learned context over time via tools like /init and /remember; a /cle…”
   • [community] “User asks if Letta Code supports a 'yolo'/dangerous-skip-permissions mode like Claude Code; Letta's Charles confirms: 'Letta Code also has a…”

   proof ↗⚑ flag

   Supermemory✓fullprobed7/10

   Supermemory's MCP docs explicitly describe natural-language operation: 'Ask naturally. The assistant selects the right Supermemory tool without requiring code or tool names,' and the MCP layer lets any MCP-compatible assistant collaborate using shared context. This directly matches the AI-native natural-language command story. Missing for 10: independent/hands-on confirmation of the natural-language MCP experience and broader examples of the range of commands supported beyond search/recall.

   • [claimed-docs] “Ask naturally. The assistant selects the right Supermemory tool without requiring code or tool names.”
   • [claimed-docs] “Supermemory MCP gives every MCP-compatible assistant a shared memory layer, so technical and non-technical teams can collaborate with AI usi…”
   • [claimed-docs] “A space keeps a team's documents, memories, and profile context focused, so AI retrieves the right knowledge without mixing unrelated work.”
   • [probe] “official MCP server documented at https://supermemory.ai/docs/supermemory-mcp/mcp”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   Letta—none0/10

   The evidence pack shows extensive SDK documentation (agent-sdk pages) but no interactive API reference or runnable-example explorer; a direct probe for OpenAPI/Swagger endpoints returned 404 on all candidate paths, indicating no such interactive reference exists.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.letta.com/openapi.json, https://docs.letta.com/swagger.json, https://docs.letta.com/api…”

   proof ↗⚑ flag

   Supermemory—none0/10

   Evidence shows only a static API-reference overview page and SDK code snippets (client.add, client.search), with no interactive 'try-it' console or runnable sandbox; a probe explicitly checked for an OpenAPI/Swagger spec (which typically powers interactive references) and found all candidate URLs returning 404.

   • [claimed-docs] “Official clients wrap this API: TypeScript: npm install supermemory Python: pip install supermemory”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.supermemory.ai/openapi.json, https://docs.supermemory.ai/swagger.json, https://docs.sup…”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   Letta—none0/10

   Direct probes for OpenAPI/Swagger specs at all standard paths (docs.letta.com/openapi.json, swagger.json, api/openapi.json, .well-known/openapi.json) returned 404, and no documentation elsewhere in the evidence pack references a downloadable machine-readable API spec.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.letta.com/openapi.json, https://docs.letta.com/swagger.json, https://docs.letta.com/api…”

   proof ↗⚑ flag

   Supermemory—none0/10

   A direct probe for machine-readable API specs at standard OpenAPI/Swagger paths returned 404 on all candidates, and no evidence pack item links to a downloadable OpenAPI/JSON spec despite an 'API reference' doc existing.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.supermemory.ai/openapi.json, https://docs.supermemory.ai/swagger.json, https://docs.sup…”
   • [claimed-docs] “Official clients wrap this API: TypeScript: npm install supermemory Python: pip install supermemory”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round to Letta
   Letta~partialclaimed4/10

   Letta offers isolation-oriented options that could support sandbox-style testing: an --ephemeral CLI mode that runs one-shot tasks without persisting any agent, memory, or MemFS state (letta-docs-14), and a managed 'cloud sandbox' for running agents with no local setup (letta-docs-16, letta-docs-33, letta-docs-58). However, there is no explicit documentation of a staging/production environment separation or guarantee that testing won't touch production agent data. Missing for 10: explicit staging-vs-production environment concept, documentation of data isolation guarantees, and any evidence/testimonial confirming this workflow is used for safe testing.

   • [claimed-docs] “Use --ephemeral to run a one-shot task without creating, resuming, or persisting an agent, memory blocks, or MemFS”
   • [claimed-docs] “The desktop app and CLI can use your existing API keys and coding plans, while the web app runs agents in a cloud sandbox with no local setu…”
   • [claimed-docs] “The web app uses a managed cloud sandbox by default, so you can start without configuring a computer.”
   • [claimed-docs] “the web app runs agents in a cloud sandbox with no local setup”

   proof ↗⚑ flag

   Supermemory—none0/10

   The evidence describes container-tag based data isolation between end-users/tenants, self-hosting, and a MemoryBench benchmarking tool, but none of this documents an actual sandbox/staging environment (e.g., a test API key, non-production project, or explicit test mode) that a developer could use to try things without touching production data.

   ⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   Letta—none0/10

   No evidence anywhere in the pack of API versioning schemes or a documented deprecation policy; the openapi.json probe even 404s, indicating no discoverable API spec that would carry version/deprecation info.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.letta.com/openapi.json, https://docs.letta.com/swagger.json, https://docs.letta.com/api…”

   proof ↗⚑ flag

   Supermemory—none0/10

   No evidence of API versioning scheme (e.g., v1/v2 paths) or any documented deprecation policy; OpenAPI spec probes returned 404 and no changelog/deprecation docs appear in the pack.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.supermemory.ai/openapi.json, https://docs.supermemory.ai/swagger.json, https://docs.sup…”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round to Supermemory
   Letta~partialclaimed4/10

   Letta offers some multi-item automation primitives — subagents for parallel task delegation ('divide and conquer tasks') and a cross-agent `/search` command that queries across all agents at once — which support limited bulk-style operations, but there is no documented bulk API for batch-creating, editing, importing/exporting, or mass-updating agents, memory blocks, or files. Missing for 10: explicit bulk CRUD/batch APIs, bulk import/export tooling, and evidence of large-scale multi-item operations beyond parallel subagent delegation and cross-agent search.

   • [claimed-docs] “By delegating work to focused subagents, your main agent can keep its context window clean and make use of parallelism to divide and conquer…”
   • [claimed-docs] “Subagents are specialized agents that your main agent can spawn to handle complex tasks autonomously.”
   • [claimed-docs] “`/search [query]` | Search messages across agents”
   • [claimed-docs] “Up to 20 stateful agents”
   • [claimed-docs] “With a developer plan, you can create an unlimited number of agents but pay usage-based pricing.”

   proof ↗⚑ flag

   Supermemory~partialclaimed5/10

   Evidence shows container-tag scoped deletion of all memories in a tenant at once, connector auto-sync ingesting many documents in bulk, and metadata filtering across memories, which together imply some bulk-style operations, but there is no documented dedicated bulk API (e.g., batch add/update/delete of many arbitrary items in one call) or independent confirmation of true bulk throughput. missing for 10: explicit bulk/batch endpoint documentation, evidence of bulk operations beyond tag-scoped delete and connector sync, independent/hands-on validation of bulk performance.

   • [claimed-docs] “Scope each end-user (or tenant) to a container tag. When the user requests deletion, delete that container's content via the API / console w…”
   • [claimed-docs] “Scope each end-user (or tenant) to a container tag. 2. When the user requests deletion, delete that container's content via the API / consol…”
   • [claimed-docs] “Connect external platforms to automatically sync documents into supermemory. Supported connectors include Google Drive, Gmail, Notion, OneDr…”
   • [github] “Connectors Google Drive · Gmail · Notion · OneDrive · GitHub — auto-sync with real-time webhooks.”
   • [claimed-docs] “Metadata Filtering ... Query memories by custom properties like category, status, or date”
   • [claimed-docs] “Use customId to update existing documents or conversations. When you send content with the same customId, Supermemory intelligently processe…”

   proof ↗⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round to Letta
   Letta~partialclaimed6/10

   Letta documents several trigger-based automation mechanisms: scheduled tasks that 'automate prompts to your agent on a recurring schedule or at a specific time' (letta-docs-69), mods that add 'event hooks' taking effect immediately (letta-docs-71), dreaming triggers configurable to fire 'after a number of steps, on context compaction, or never' (letta-docs-3/12), and Slack-connected agents that automatically respond when mentioned (letta-docs-73/83). These collectively show event- and time-based automatic action triggers, though there is no single documented general-purpose 'rules engine' where a user defines arbitrary event→action conditions in one place. Missing for 10: a unified rules/automation API or UI for arbitrary custom event conditions, and independent/hands-on confirmation that these triggers reliably fire as documented.

   • [claimed-docs] “Scheduled tasks allow you to automate prompts to your agent on a recurring schedule or at a specific time in the future.”
   • [claimed-docs] “Mods let your agent modify its own runtime — adding tools, commands, event hooks, permission policies, and provider adapters that take effec…”
   • [claimed-docs] “trigger controls when dreaming runs: after a number of steps, on context compaction, or never.”
   • [claimed-docs] “Agents connected to Slack are always-on: mention them in a channel or thread and they'll respond, using their cloud computer to do real work…”
   • [claimed-docs] “Agents connected to Slack are always-on: mention them in a channel or thread and they'll respond”
   • [claimed-docs] “trigger controls when dreaming runs: after a number of steps, on context compaction, or never”

   proof ↗⚑ flag

   Supermemory—none0/10

   Supermemory is a memory/storage layer for AI apps — it supports connectors, auto-sync, and memory extraction, but there is no evidence of a rules/automation engine where users define 'if event X then action Y' triggers. Ingestion and connector syncing are automatic pipelines, not user-defined conditional automation.

   ⚑ flag

Forgetting

1. platform-engineerDelete a user's memories on demand — single memory, per-entity, or full erasure — to satisfy privacy requirements

   weight 2 · round to Supermemory
   Letta—none0/10

   The evidence pack covers extensive memory creation, editing, auditing (/doctor), and consolidation (/remember, /init, dreaming) but never documents an explicit delete/erase memory command, API, or per-entity/full-wipe capability for privacy compliance. The one related command, /clear, explicitly 'resets the message buffer but keeps learned context' (letta-comm-5), meaning conversation clearing does not erase underlying memory — the opposite of the requested erasure guarantee.

   • [community] “Letta Code described by Letta's Charles as 'memory-first': agents build learned context over time via tools like /init and /remember; a /cle…”
   • [claimed-docs] “If the memory hierarchy has drifted or grown too large, run /doctor to audit placement, duplication, and system-prompt token usage.”
   • [claimed-docs] “You can also teach it explicitly with `/remember`: `> /remember always use pnpm in this repo`. The agent decides where the lesson belongs an…”

   proof ↗⚑ flag

   Supermemory~partialclaimed6/10

   Docs explicitly describe deleting a user's entire memory scope via container tags ('When the user requests deletion, delete that container's content via the API / console workflow'), which covers per-entity/full erasure for privacy compliance. However, evidence never documents a single-memory delete endpoint/method, nor any confirmation flow, audit trail, or independent verification that deletion actually purges data (e.g., from backups/embeddings). missing for 10: single-memory-level delete API/example, confirmation of hard-delete semantics (vs soft-delete), independent/hands-on evidence of successful deletion, and audit/compliance documentation beyond container-tag erasure.

   • [claimed-docs] “Scope each end-user (or tenant) to a container tag. When the user requests deletion, delete that container's content via the API / console w…”
   • [claimed-docs] “Scope each end-user (or tenant) to a **container tag**. When the user requests deletion, delete that container's content via the API / conso…”
   • [claimed-docs] “Scope each end-user (or tenant) to a container tag.”
   • [claimed-docs] “Scope each end-user (or tenant) to a container tag. 2. When the user requests deletion, delete that container's content via the API / consol…”
   • [claimed-docs] “every memory tagged with `user_alex` lives in its own isolated space, completely separate from memories tagged `user_jordan`”

   proof ↗⚑ flag

2. developerMake memories expire or decay — via TTL, expiration dates, or recency weighting — so stale facts stop surfacing

   weight 2 · round drawn
   Letta—none0/10

   The evidence describes Letta's memory system (dreaming/consolidation, /doctor audits, MemFS search, shared memory) but nowhere documents TTL, expiration dates, or recency-weighted decay causing stale facts to stop surfacing; /doctor flags drift/duplication but is manual, not automatic decay.

   • [claimed-docs] “trigger controls when dreaming runs: after a number of steps, on context compaction, or never.”
   • [claimed-docs] “If the memory hierarchy has drifted or grown too large, run /doctor to audit placement, duplication, and system-prompt token usage.”
   • [claimed-docs] “Dreaming uses background subagents to review recent conversations, consolidate useful lessons, and update memory without interrupting your a…”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the MemFS Search mod”

   proof ↗⚑ flag

   Supermemory—none0/10

   No documentation describes TTL, expiration dates, or recency-weighted decay; the only related mention (isStatic flag) is about marking facts permanent vs. default, not about expiring stale ones. Moreover the product's own comparison doc admits 'No temporal truth ("loved Adidas" then "switched to Puma", both stay equally relevant)', indicating stale facts are not down-weighted or expired.

   • [claimed-docs] “isStatic: true for permanent identity traits (name, hometown). Defaults to false”
   • [claimed-docs] “No temporal truth ("loved Adidas" then "switched to Puma", both stay equally "relevant")”

   proof ↗⚑ flag

Portability

1. platform-engineerExport memories in a machine-readable format so the memory store is portable and not a lock-in trap

   weight 2 · round to Letta
   Letta~partialclaimed5/10

   Letta's memory is stored as plain Markdown files in a Git-backed 'memory repository' (MemFS), and self-hosting keeps all agent state on-device with no proprietary lock-in, which implies inherent portability of memory data. However, there is no explicit documented export/import command, no described export file format spec, or CLI subcommand dedicated to exporting memories for migration — the pack only shows cloud backup/sync features, not a data-portability workflow. missing for 10: an explicit export/import command or documented export format, third-party confirmation that memory can be extracted and reloaded elsewhere, guidance on migrating memory out of Letta Cloud.

   • [claimed-docs] “Pass memory for full control over the agent’s starting memory. Each entry becomes a Markdown file in the agent’s memory repository”
   • [claimed-docs] “Shared memory gives multiple agents access to the same files and working context. Each shared memory repository is a Git repository hosted i…”
   • [claimed-docs] “Letta agents can run entirely on infrastructure you control”
   • [claimed-docs] “all agent state, including messages, memory, and provider connections, stays on-device, and no Letta account is required.”
   • [claimed-docs] “Run /login to back up your agents to the cloud and make them available through chat.letta.com, the desktop app, remote machines, and messagi…”

   proof ↗⚑ flag

   Supermemory—none0/10

   The evidence pack documents ingestion, search, self-hosting, and API clients, but never mentions a bulk export endpoint, data-dump command, or documented export format for memories — nothing addresses portability of existing data out of the platform. Self-hosting (supermemory-docs-7/8/16/44) is a deployment option, not an export mechanism for data already stored in the hosted service.

   ⚑ flag

Self host

1. developerRun the memory layer fully locally — embedded in-process or against local models — without any cloud dependency

   weight 1 · round drawn
   Letta~partialclaimed6/10

   Docs clearly describe a local-only mode where the CLI/agent state, memory, and provider connections all stay on-device with no Letta account or cloud backup required, and a self-hosted App Server that lets agents run entirely on infrastructure you control. However, the evidence never confirms running against fully local models (e.g., local LLM inference/embeddings) — 'model-agnostic' language points to swapping API-based providers (Claude, GPT, Gemini) rather than in-process/local-model operation, and other docs emphasize cloud sandboxes and server-hosted agents as the default path. Missing for 10: explicit support/documentation for local model backends (e.g., Ollama/local embeddings), and confirmation that memory/embedding search (MemFS Search) works without any external API calls.

   • [claimed-docs] “This default local mode stores agent state on your machine and does not require a Letta login.”
   • [claimed-docs] “all agent state, including messages, memory, and provider connections, stays on-device, and no Letta account is required.”
   • [claimed-docs] “Letta agents can run entirely on infrastructure you control”
   • [claimed-docs] “Self-hosted App Server: Run the Letta App Server to host local agents on a central, always-on machine and expose them to client applications…”
   • [claimed-docs] “Letta agents are **model-agnostic**. Agents can use Claude, GPT, Gemini, or other supported models. Users can change an agent's underlying m…”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the [MemFS Search mod]”

   proof ↗⚑ flag

   Supermemory~partialclaimed6/10

   Docs explicitly describe a self-hosting option running as a single self-contained binary with local graph engine, local embeddings, and local LLM, claiming data never leaves the building and no Docker/config needed. However, there's no independent/hands-on corroboration of fully offline operation, no detail on which local models are supported or how embedding/LLM swapping works, and self-hosting is positioned as a paid-tier feature (SOC2/HIPAA self-hosted option) rather than a fully documented open local dev path. missing for 10: independent verification of the self-hosted binary running fully offline, concrete local-model configuration docs, and evidence this isn't gated behind enterprise pricing.

   • [claimed-docs] “Supermemory runs on your own hardware. It's the same memory engine behind the hosted platform... as a single self-contained binary.”
   • [claimed-docs] “Local graph engine, local embeddings, local LLM. Your data never leaves the building.”
   • [claimed-docs] “Supermemory runs on your own hardware. It's the same memory engine behind the hosted platform ... as a single self-contained binary.”
   • [claimed-docs] “No Docker. No database to provision. No config files. It boots in seconds with everything built in”
   • [claimed-docs] “State-of-the-art memory, running on your machine. One binary, zero config.”
   • [claimed-docs] “SOC 2 · HIPAA BAA · self-hosted option”

   proof ↗⚑ flag

2. platform-engineerSelf-host the memory layer from open-source code (e.g. via Docker) on infrastructure I control

   weight 3 · round to Letta
   Letta~partialprobed6/10

   Letta explicitly documents a self-hosting path: 'Letta agents can run entirely on infrastructure you control' via the self-hosted App Server, with a local mode that stores state on-device and requires no cloud account, and the agent harness is described as fully open source. However, the evidence pack contains no explicit Docker instructions, container image, or hands-on/community confirmation that self-hosting works in practice. Missing for 10: Docker/container deployment specifics, independent verification of the self-host setup succeeding.

   • [claimed-docs] “Letta agents can run entirely on infrastructure you control”
   • [claimed-docs] “Self-hosted App Server: Run the Letta App Server to host local agents on a central, always-on machine and expose them to client applications…”
   • [claimed-docs] “The Letta App Server lets you deploy the Letta agent harness as a service.”
   • [claimed-docs] “Run the Letta App Server to host local agents on a central, always-on machine and expose them to client applications”
   • [claimed-docs] “all agent state, including messages, memory, and provider connections, stays on-device, and no Letta account is required.”
   • [claimed-docs] “This default local mode stores agent state on your machine and does not require a Letta login.”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.letta.com/llms.txt # Letta > Letta agents learn from experience and improve with use. The Letta ag…”

   proof ↗⚑ flag

   Supermemory~partialclaimed4/10

   Docs confirm a genuine self-host mode — 'runs on your own hardware... single self-contained binary' with local embeddings/LLM — but explicitly state 'No Docker' and describe a closed single-binary distribution rather than open-source code the engineer builds/deploys via Docker. The GitHub repo evidence in the pack only documents connectors/extractors, not that the self-host binary itself is open-source or containerized. missing for 10: Docker/container deployment path, confirmation the self-hosted binary is built from open-source code (license/repo), and independent hands-on verification of self-host setup.

   • [claimed-docs] “Supermemory runs on your own hardware. It's the same memory engine behind the hosted platform... as a single self-contained binary.”
   • [claimed-docs] “Supermemory runs on your own hardware. It's the same memory engine behind the hosted platform ... as a single self-contained binary.”
   • [claimed-docs] “No Docker. No database to provision. No config files. It boots in seconds with everything built in”
   • [claimed-docs] “State-of-the-art memory, running on your machine. One binary, zero config.”
   • [claimed-docs] “SOC 2 · HIPAA BAA · self-hosted option”

   proof ↗⚑ flag

Knowledge graph

1. ml-engineerStore memories as a knowledge graph of entities and relationships so multi-hop and entity-centric questions are answerable

   weight 3 · round to Supermemory
   Letta—none0/10

   Letta's documented memory system is file/text-based (MemFS, memory blocks) with optional keyword/semantic/hybrid search (letta-docs-28, letta-docs-41), not a graph of entities and relationships; a Letta employee explicitly frames its approach as 'primarily text/files based' as an alternative to structured memory graphs (letta-comm-10). No evidence describes entity/relationship modeling or multi-hop graph traversal for memory.

   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the [MemFS Search mod]”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the MemFS Search mod”
   • [community] “Letta's Wooders responds to memory skepticism: 'the problem with ChatGPT / other RAG-based memory solutions is that it's not possible to col…”

   proof ↗⚑ flag

   Supermemory✓fullclaimed7/10

   Docs explicitly describe a 'living knowledge graph of facts on top of other facts' with graph traversal as one of three retrieval modes, and show entity-centric queries (e.g. 'where does Alex work?' with relatedMemories) demonstrating multi-hop entity resolution across facts like 'Alex mentioned he just started at Stripe'. missing for 10: independent/hands-on validation of multi-hop accuracy, and deeper documentation of explicit entity/relationship schema or graph query language beyond the relatedMemories flag.

   • [claimed-docs] “Supermemory builds a living knowledge graph of facts on top of other facts — not a static folder of embeddings”
   • [claimed-docs] “Retrieve three ways — document search (RAG), memory graph traversal, and user profile”
   • [claimed-docs] “const results = await client.search({ q: "where does Alex work?", containerTag: "user_123", include: { relatedMemories: true }, });”
   • [claimed-docs] “await client.add({ content: "Alex mentioned he just started at Stripe", containerTag: "user_123" });”
   • [claimed-docs] “No temporal truth ("loved Adidas" then "switched to Puma", both stay equally "relevant")”

   proof ↗⚑ flag

2. ml-engineerTrack when facts became valid or invalid (temporal reasoning) so the memory distinguishes current from outdated information

   weight 2 · round drawn
   Letta—none0/10

   Letta's documented memory model is file/block-based (MemFS, memory blocks, dreaming/consolidation) with no evidence of tracking fact validity intervals, timestamps of when facts became true/false, or any temporal-graph reasoning distinguishing current vs outdated facts. The axis is applicable to a memory-focused product, but nothing in the evidence pack shows Letta supports temporal fact versioning.

   ⚑ flag

   Supermemory—none0/10

   Supermemory's own comparison docs explicitly admit the product lacks temporal truth tracking, giving the example that 'loved Adidas' then 'switched to Puma' both remain equally relevant rather than one being marked outdated — the opposite of the story's requirement. No other evidence describes validity windows, fact invalidation, or temporal versioning of memories.

   • [claimed-docs] “No temporal truth ("loved Adidas" then "switched to Puma", both stay equally "relevant")”

   proof ↗⚑ flag

Schema customization

1. ml-engineerCustomize the memory schema — entity types, edge types, or ontology — to match my domain

   weight 1 · round drawn
   Letta—none0/10

   Letta's documented memory model is block/file-based (Markdown memory blocks, MemFS, persona/human blocks) rather than a graph model with configurable entity types, edge types, or ontology; no evidence pack item describes schema customization of entities/relationships. missing for 10: any documentation of graph-based memory representation, entity/edge type definitions, or ontology configuration APIs.

   • [claimed-docs] “Pass memory for full control over the agent’s starting memory. Each entry becomes a Markdown file in the agent’s memory repository”
   • [claimed-docs] “`persona` and `human` are conveniences that set the agent's persona and human memory”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the [MemFS Search mod]”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the MemFS Search mod”

   proof ↗⚑ flag

   Supermemory—none0/10

   Supermemory documents automatic knowledge-graph construction, buckets for profile categorization, and metadata filtering, but no evidence anywhere describes defining custom entity types, edge/relationship types, or an ontology to fit a domain schema — extraction is described as fully automatic ('Supermemory builds a living knowledge graph... automatically').

   • [claimed-docs] “Supermemory builds a living knowledge graph of facts on top of other facts — not a static folder of embeddings”
   • [claimed-docs] “const results = await client.search({ q: "where does Alex work?", containerTag: "user_123", include: { relatedMemories: true }, });”
   • [claimed-docs] “Buckets are custom topical categories for a profile ... group them by subject (e.g. preferences, goals, work).”
   • [claimed-docs] “Metadata Filtering ... Query memories by custom properties like category, status, or date”
   • [claimed-docs] “Supermemory automatically extracts and indexes content from various formats.”

   proof ↗⚑ flag

Agent memory

1. ai-native userMy agent can manage its own memory mid-conversation — adding, searching, updating, and deleting memories through tools or API calls it invokes itself

   weight 3 · round drawn
   Letta✓fullcommunity8/10

   Letta's memory system is explicitly designed for agent-driven self-editing: agents can be taught new memories via /remember, search via MemFS keyword/semantic search mods, consolidate/update via background 'dreaming', and audit/prune via /doctor — all invoked by the agent itself mid-conversation, and community evidence corroborates this as Letta's core differentiator ('memory-first', 'fully white box' memory editing). missing for 10: no direct API/tool-call schema example showing explicit add/search/update/delete operations invoked programmatically by the agent (docs describe user-facing slash commands and background dreaming rather than a fully enumerated CRUD tool API), and no independent hands-on test confirming the agent autonomously invokes these without user prompting.

   • [claimed-docs] “You can also teach it explicitly with /remember: > /remember always use pnpm in this repo”
   • [claimed-docs] “You can also teach it explicitly with `/remember`”
   • [claimed-docs] “You can also teach it explicitly with /remember”
   • [claimed-docs] “If the memory hierarchy has drifted or grown too large, run /doctor to audit placement, duplication, and system-prompt token usage.”
   • [claimed-docs] “Dreaming uses background subagents to review recent conversations, consolidate useful lessons, and update memory without interrupting your a…”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the [MemFS Search mod]”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the MemFS Search mod”
   • [community] “Letta Code described by Letta's Charles as 'memory-first': agents build learned context over time via tools like /init and /remember; a /cle…”
   • [community] “Letta's Wooders responds to memory skepticism: 'the problem with ChatGPT / other RAG-based memory solutions is that it's not possible to col…”
   • [community] “Concern raised: 'context poisoning is a real problem that these memory providers only make worse.' Letta's Charles countered that Letta's me…”

   proof ↗⚑ flag

   Supermemory✓fullprobed8/10

   Supermemory exposes API/SDK operations (add, search hybrid, update via customId, delete via container tag) and an official MCP server that lets an agent invoke these tools mid-conversation without pre-configured tool names ('Ask naturally... selects the right Supermemory tool'). This directly matches the story of an agent managing its own memory via tool/API calls. missing for 10: no independent hands-on evidence of an agent autonomously invoking delete via MCP, and no explicit example combining add/search/update/delete in one live agent session.

   • [claimed-docs] “Create memories directly without going through the document ingestion workflow. Memories are embedded and immediately searchable.”
   • [claimed-docs] “await client.add({ content: "The user loves Paris.", containerTag: "user_123" });”
   • [claimed-docs] “const results = await client.search({ q: "where does Alex work?", containerTag: "user_123", include: { relatedMemories: true }, });”
   • [claimed-docs] “Use searchMode: "hybrid" for best results. It searches both memories and document chunks, returning the most relevant content.”
   • [claimed-docs] “Use customId to update existing documents or conversations. When you send content with the same customId, Supermemory intelligently processe…”
   • [claimed-docs] “Supermemory MCP gives every MCP-compatible assistant a shared memory layer, so technical and non-technical teams can collaborate with AI usi…”
   • [claimed-docs] “Ask naturally. The assistant selects the right Supermemory tool without requiring code or tool names.”
   • [claimed-docs] “Scope each end-user (or tenant) to a **container tag**. When the user requests deletion, delete that container's content via the API / conso…”
   • [probe] “official MCP server documented at https://supermemory.ai/docs/supermemory-mcp/mcp”

   proof ↗⚑ flag

2. ai-native userThe memory layer decides for itself what is worth remembering — extracting salient facts from raw conversation and consolidating them in the background

   weight 2 · round to Letta
   Letta✓fullcommunity8/10

   Letta's 'dreaming' feature explicitly uses background subagents to review conversations, extract salient lessons, and consolidate/update memory without interrupting active work, and /remember and /init let the agent decide where lessons belong in MemFS autonomously. Community corroboration (Charles/Wooders on HN) confirms this memory-first, self-consolidating design and contrasts it with RAG-based approaches, though skeptics questioned scaling. Missing for 10: independent hands-on benchmarking of extraction quality/accuracy and more detail on the automatic (non-explicit) salience-detection heuristics beyond trigger conditions.

   • [claimed-docs] “Dreaming uses background subagents to review recent conversations, consolidate useful lessons, and update memory without interrupting your a…”
   • [claimed-docs] “Run `/init` to bootstrap or refresh memory for the current project. The agent inspects the repository, asks about your working style when ne…”
   • [claimed-docs] “You can also teach it explicitly with `/remember`: `> /remember always use pnpm in this repo`. The agent decides where the lesson belongs an…”
   • [claimed-docs] “trigger controls when dreaming runs: after a number of steps, on context compaction, or never.”
   • [community] “Letta Code described by Letta's Charles as 'memory-first': agents build learned context over time via tools like /init and /remember; a /cle…”
   • [community] “Letta's Wooders responds to memory skepticism: 'the problem with ChatGPT / other RAG-based memory solutions is that it's not possible to col…”
   • [community] “Concern raised: 'context poisoning is a real problem that these memory providers only make worse.' Letta's Charles countered that Letta's me…”

   proof ↗⚑ flag

   Supermemory~partialclaimed7/10

   Docs clearly describe automatic extraction of salient facts from raw content (docs-1, docs-41, docs-42) and background consolidation into an evolving knowledge graph and auto-maintained user profiles (docs-14, docs-15, docs-33, docs-40) with no manual tagging required. However, the vendor's own comparison page admits a real consolidation gap — no temporal truth resolution, so outdated and current facts both remain 'relevant' (docs-48) — and there is no independent/hands-on evidence corroborating extraction quality. missing for 10: independent corroboration of extraction accuracy, resolution of the documented temporal-truth/consolidation gap.

   • [claimed-docs] “Send any raw content to Supermemory — conversations, documents, files, URLs. We extract the memories automatically.”
   • [claimed-docs] “Supermemory builds a living knowledge graph of facts on top of other facts — not a static folder of embeddings”
   • [claimed-docs] “User profiles are **automatically maintained collections of facts about your users** that Supermemory builds from all their interactions.”
   • [claimed-docs] “No search needed — comprehensive user info always ready Profiles update as users interact with your system”
   • [claimed-docs] “User profiles are automatically maintained collections of facts about your users that Supermemory builds from all their interactions.”
   • [claimed-docs] “Supermemory automatically extracts and indexes content from various formats.”
   • [claimed-docs] “Just upload — Supermemory handles the rest ... No chunking strategies to configure. No embedding models to choose. It just works.”
   • [claimed-docs] “No temporal truth ("loved Adidas" then "switched to Puma", both stay equally "relevant")”

   proof ↗⚑ flag

Benchmarks

1. ml-engineerSee published memory-quality benchmark results (e.g. LongMemEval, LoCoMo) backing the product's recall-accuracy claims

   weight 2 · round drawn
   Letta—none0/10

   The evidence pack contains extensive documentation of Letta's memory architecture (MemFS, dreaming, shared memory, etc.) and community discussion of memory quality skepticism/responses, but nowhere cites any published benchmark results (LongMemEval, LoCoMo, or similar) backing recall-accuracy claims. Since memory-recall quality is a core claimed feature of Letta, this axis clearly applies, but no benchmark evidence exists in the pack.

   • [community] “'How does the memory scale (or not!) over time... if there is a memory block limit, is that an issue for self learning over time. Claude as …”
   • [community] “Skeptic comment: 'In my experience, memory is really not that helpful in most cases... Maintaining the memory is a considerable burden... Ch…”
   • [community] “Letta's Wooders responds to memory skepticism: 'the problem with ChatGPT / other RAG-based memory solutions is that it's not possible to col…”
   • [community] “Concern raised: 'context poisoning is a real problem that these memory providers only make worse.' Letta's Charles countered that Letta's me…”
   • [claimed-docs] “Dreaming uses background subagents to review recent conversations, consolidate useful lessons, and update memory without interrupting your a…”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the [MemFS Search mod]”

   proof ↗⚑ flag

   Supermemory—none0/10

   Supermemory documents a 'MemoryBench' tool that lets users run their own benchmarks against Mem0/Zep/Supermemory, but the evidence contains no published results on standard benchmarks like LongMemEval or LoCoMo, nor any recall-accuracy numbers backing its claims.

   • [claimed-docs] “MemoryBench ships a Claude Code skill that automates the entire process of benchmarking a custom memory implementation — yours — against Sup…”
   • [claimed-docs] “you can run it yourself, against your own memory implementation, on the datasets that match your use case”

   proof ↗⚑ flag

Core memory

1. developerAdd memories from conversations and retrieve them later with semantic search, so context persists across sessions

   weight 3 · round to Supermemory
   Letta~partialcommunity6/10

   Letta clearly supports persistent memory across sessions (MemFS, /remember, stateful agents resumable across sessions/computers) and explicitly documents semantic search as an optional add-on via the 'MemFS Search mod' for keyword/semantic/hybrid search rather than a built-in default. Community commentary confirms the memory system is real and text/file-based but transparency-focused, not natively semantic-first, and some users raise unresolved concerns about memory scaling/self-improvement. missing for 10: first-party evidence that semantic search is enabled by default (not an optional mod), independent hands-on benchmarking of recall/retrieval quality, and clearer documentation of retrieval accuracy at scale.

   • [claimed-docs] “You can return to the same agent across sessions, computers, and interfaces instead of starting from scratch each time.”
   • [claimed-docs] “You can also teach it explicitly with `/remember`: `> /remember always use pnpm in this repo`. The agent decides where the lesson belongs an…”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the [MemFS Search mod]”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the MemFS Search mod”
   • [community] “Letta Code described by Letta's Charles as 'memory-first': agents build learned context over time via tools like /init and /remember; a /cle…”
   • [community] “Letta's Wooders responds to memory skepticism: 'the problem with ChatGPT / other RAG-based memory solutions is that it's not possible to col…”
   • [community] “'How does the memory scale (or not!) over time... if there is a memory block limit, is that an issue for self learning over time. Claude as …”

   proof ↗⚑ flag

   Supermemory✓fullcommunity8/10

   Docs show a complete workflow: add raw content or direct memories (client.add), semantic/hybrid search (client.search with searchMode:hybrid), container tags to scope and isolate memory per user/session, and automatic extraction/indexing of conversations and documents — directly matching the story of persisting and recalling context across sessions. Missing for 10: independent hands-on validation of recall quality/accuracy, and one HN complaint noting ingestion size limits on large linked content suggests some real-world friction in the add-memory pipeline.

   • [claimed-docs] “Send any raw content to Supermemory — conversations, documents, files, URLs. We extract the memories automatically.”
   • [claimed-docs] “Use `searchMode: "hybrid"` for best results. It searches both memories and document chunks, returning the most relevant content.”
   • [claimed-docs] “Create memories directly without going through the document ingestion workflow. Memories are embedded and immediately searchable.”
   • [claimed-docs] “await client.add({ content: "The user loves Paris.", containerTag: "user_123" });”
   • [claimed-docs] “const results = await client.search({ q: "where does Alex work?", containerTag: "user_123", include: { relatedMemories: true }, });”
   • [claimed-docs] “every memory tagged with `user_alex` lives in its own isolated space, completely separate from memories tagged `user_jordan`”
   • [community] “It's a shame it is unable to store large content from links. I tried it with a tutorial link so that I could eventually master the Nix langu…”

   proof ↗⚑ flag

2. developerRely on the memory layer to update, supersede, or merge memories when new information contradicts what was stored

   weight 2 · round to Letta
   Letta~partialcommunity6/10

   Docs describe mechanisms that plausibly support updating/merging memory — /remember lets the agent 'decide where the lesson belongs and commit the update to MemFS' (letta-docs-66), 'dreaming' subagents 'consolidate useful lessons and update memory' (letta-docs-27), and /doctor audits for 'duplication' and drift in the memory hierarchy (letta-docs-10) — and Letta staff claim memory is transparent/controllable and addresses context-poisoning concerns (letta-comm-10, letta-comm-11). However, none of this is a concrete demonstration of contradiction detection/supersession logic, and a community skeptic raises unresolved doubts about memory scaling and self-correction (letta-comm-8, letta-comm-9) without a hands-on refutation either way. Missing for 10: explicit documentation or example of the system detecting/resolving conflicting facts, independent hands-on verification that stale memories are actually superseded rather than just appended/deduplicated.

   • [claimed-docs] “You can also teach it explicitly with `/remember`: `> /remember always use pnpm in this repo`. The agent decides where the lesson belongs an…”
   • [claimed-docs] “Dreaming uses background subagents to review recent conversations, consolidate useful lessons, and update memory without interrupting your a…”
   • [claimed-docs] “If the memory hierarchy has drifted or grown too large, run /doctor to audit placement, duplication, and system-prompt token usage.”
   • [community] “Letta's Wooders responds to memory skepticism: 'the problem with ChatGPT / other RAG-based memory solutions is that it's not possible to col…”
   • [community] “Concern raised: 'context poisoning is a real problem that these memory providers only make worse.' Letta's Charles countered that Letta's me…”
   • [community] “'How does the memory scale (or not!) over time... if there is a memory block limit, is that an issue for self learning over time. Claude as …”
   • [community] “Skeptic comment: 'In my experience, memory is really not that helpful in most cases... Maintaining the memory is a considerable burden... Ch…”

   proof ↗⚑ flag

   Supermemory~partialclaimed5/10

   Supermemory documents a customId mechanism that lets you resend content and have it 'intelligently process only what's new' (supermemory-docs-30), and memory-graph docs show new facts (e.g., new employer) being added on top of old ones (supermemory-docs-27, supermemory-docs-37), suggesting some update/graph-merge behavior. But the product's own comparison page explicitly admits it lacks 'temporal truth' — contradictory facts like 'loved Adidas' then 'switched to Puma' both remain equally relevant rather than the newer one superseding the older (supermemory-docs-48) — a direct, first-party admission that conflicting memories are not reliably resolved. missing for 10: no documented mechanism for detecting/resolving direct factual contradictions (vs. just adding new facts or updating same-ID documents), no independent/community verification of merge or supersession behavior in practice.

   • [claimed-docs] “Use customId to update existing documents or conversations. When you send content with the same customId, Supermemory intelligently processe…”
   • [claimed-docs] “No temporal truth ("loved Adidas" then "switched to Puma", both stay equally "relevant")”
   • [claimed-docs] “const results = await client.search({ q: "where does Alex work?", containerTag: "user_123", include: { relatedMemories: true }, });”
   • [claimed-docs] “await client.add({ content: "Alex mentioned he just started at Stripe", containerTag: "user_123" });”

   proof ↗⚑ flag

Retrieval controls

1. developerSteer retrieval with metadata filters, keyword/hybrid search modes, or reranking instead of accepting a single fixed similarity search

   weight 2 · round to Supermemory
   Letta~partialclaimed4/10

   Letta docs mention that installing the 'MemFS Search mod' enables keyword search and optional semantic or hybrid search, showing some retrieval-mode flexibility beyond fixed similarity search, but this requires an add-on rather than being built-in. Missing for 10: explicit metadata-filter support, reranking capability, and any evidence this mod is core/first-class rather than an optional plugin.

   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the [MemFS Search mod]”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the MemFS Search mod”

   proof ↗⚑ flag

   Supermemory✓fullclaimed8/10

   Docs explicitly document hybrid search mode (searchMode: "hybrid" combining memories and document chunks), metadata filtering by custom properties like category/status/date, and container-tag scoping as retrieval controls beyond plain similarity search. Multiple retrieval modes (document RAG, memory graph traversal, user profile) are also documented as selectable. missing for 10: no explicit mention of a dedicated reranking feature/parameter, and no independent/hands-on corroboration of these filtering/hybrid modes working as described.

   • [claimed-docs] “Use `searchMode: "hybrid"` for best results. It searches both memories and document chunks, returning the most relevant content.”
   • [claimed-docs] “Use searchMode: "hybrid" for best results. It searches both memories and document chunks, returning the most relevant content.”
   • [claimed-docs] “Metadata Filtering ... Query memories by custom properties like category, status, or date”
   • [claimed-docs] “Retrieve three ways — document search (RAG), memory graph traversal, and user profile”
   • [claimed-docs] “every memory tagged with `user_alex` lives in its own isolated space, completely separate from memories tagged `user_jordan`”
   • [claimed-docs] “const results = await client.search({ q: "where does Alex work?", containerTag: "user_123", include: { relatedMemories: true }, });”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round to Supermemory
   Letta~partialprobed5/10

   Letta's SDK/App Server exposes agent creation, memory, and OpenAI-compatible API access (letta-docs-1, letta-docs-23, letta-docs-32), suggesting an API-first architecture, but the evidence never confirms that CLI/UI-specific features like /doctor memory audits, /pin, /fork, /btw, or /agents browsing have API equivalents, and a direct probe for a public OpenAPI spec returned 404s across all candidate paths (letta-probe-2), undermining a clean 'everything via API' claim. missing for 10: explicit API endpoints/SDK methods mirroring each UI/CLI convenience command, a discoverable OpenAPI spec confirming full surface parity, and independent confirmation of parity.

   • [claimed-docs] “create an agent once, then resume it from anywhere”
   • [claimed-docs] “use the App Server’s OpenAI-compatible API with clients such as Open WebUI, or the ACP adapter with ACP clients such as Zed”
   • [claimed-docs] “The Letta App Server lets you deploy the Letta agent harness as a service.”
   • [claimed-docs] “Run the Letta App Server to host local agents on a central, always-on machine and expose them to client applications”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.letta.com/openapi.json, https://docs.letta.com/swagger.json, https://docs.letta.com/api…”
   • [claimed-docs] “Use /fork to branch the current conversation, including its in-context history.”
   • [claimed-docs] “If the memory hierarchy has drifted or grown too large, run /doctor to audit placement, duplication, and system-prompt token usage.”
   • [claimed-docs] “Use /agents in the CLI to browse and switch agents. Pin an agent with /pin or the favorite control in the desktop app”

   proof ↗⚑ flag

   Supermemory~partialprobed6/10

   Docs show a comprehensive REST API (add/search/memory operations, container tags, user profiles, connectors) with official TS/Python clients, and note that app.supermemory.ai is just the consumer UI on the same engine (implying API-first parity). However, some workflows (e.g. deletion) are described as done 'via the API / console workf[low]', suggesting console-only steps exist, and a live probe found no discoverable OpenAPI spec, undercutting full API self-service. missing for 10: explicit statement/evidence that every UI action (connector setup, buckets, profile management, deletion) is fully API-equivalent, and a public OpenAPI/schema reference confirming complete API surface.

   • [claimed-docs] “Official clients wrap this API: TypeScript: npm install supermemory Python: pip install supermemory”
   • [claimed-docs] “app.supermemory.ai is the consumer product on the same engine — not where you mint API keys.”
   • [claimed-docs] “Scope each end-user (or tenant) to a container tag. When the user requests deletion, delete that container's content via the API / console w…”
   • [claimed-docs] “Connect external platforms to automatically sync documents into supermemory. Supported connectors include Google Drive, Gmail, Notion, OneDr…”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.supermemory.ai/openapi.json, https://docs.supermemory.ai/swagger.json, https://docs.sup…”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round to Letta
   Letta~partialcommunity5/10

   Letta's architecture strongly favors openness: memory is stored as plain Markdown/MemFS files (letta-docs-11, letta-docs-66), shared memory is a Git repository (letta-docs-21, letta-docs-29), and self-hosting keeps 'all agent state... on-device' with no cloud account required (letta-docs-74, letta-docs-30), reinforced by community claims of a 'fully white box' text-based memory system (letta-comm-10, letta-comm-11). However, there is no explicit documented 'export my data and delete my account' feature or guarantee that cloud-hosted shared-memory git repos can be freely cloned/exported by the user. Missing for 10: an explicit data-export/account-deletion workflow, documentation on extracting cloud-hosted agent state, and independent verification that self-hosted mode fully replaces all cloud-only features.

   • [claimed-docs] “Pass memory for full control over the agent’s starting memory. Each entry becomes a Markdown file in the agent’s memory repository”
   • [claimed-docs] “Shared memory gives multiple agents access to the same files and working context. Each shared memory repository is a Git repository hosted i…”
   • [claimed-docs] “Shared memory gives multiple agents access to the same files and working context.”
   • [claimed-docs] “Letta agents can run entirely on infrastructure you control”
   • [claimed-docs] “all agent state, including messages, memory, and provider connections, stays on-device, and no Letta account is required.”
   • [community] “Letta's Wooders responds to memory skepticism: 'the problem with ChatGPT / other RAG-based memory solutions is that it's not possible to col…”
   • [community] “Concern raised: 'context poisoning is a real problem that these memory providers only make worse.' Letta's Charles countered that Letta's me…”

   proof ↗⚑ flag

   Supermemory—none0/10

   The docs describe ingestion, search, self-hosting, and API access, but no citation documents a bulk data-export feature or open-format export tool that would let a user extract all stored memories and leave the platform. Self-hosting (docs-7/16/31/44) shows you can run your own instance, but that is not the same as exporting existing hosted data in an open format.

   • [claimed-docs] “Supermemory runs on your own hardware. It's the same memory engine behind the hosted platform... as a single self-contained binary.”
   • [claimed-docs] “Supermemory runs on your own hardware. It's the same memory engine behind the hosted platform ... as a single self-contained binary.”
   • [claimed-docs] “Official clients wrap this API: TypeScript: npm install supermemory Python: pip install supermemory”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.supermemory.ai/openapi.json, https://docs.supermemory.ai/swagger.json, https://docs.sup…”

   proof ↗⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round to Letta
   Letta~partialprobed6/10

   Letta's own docs state the agent harness is 'fully open source' and link to a public GitHub repo (letta-ai/letta) for the Agent SDK, giving concrete evidence of source availability. However, the evidence pack never names the actual license (e.g., Apache/MIT) or confirms the full product (App Server, CLI, memory subsystems) is under that same open license rather than just the harness. Missing for 10: explicit license identification/file, confirmation that all core components (not just the agent harness) are open-sourced, and independent corroboration of license terms.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.letta.com/llms.txt # Letta > Letta agents learn from experience and improve with use. The Letta ag…”
   • [github] “the Letta Agent SDK for building agents into TypeScript applications”
   • [claimed-docs] “Letta agents can run entirely on infrastructure you control”

   proof ↗⚑ flag

   Supermemory—none0/10

   There is a public GitHub repository (supermemoryai/supermemory) referenced, and self-hosting docs describe a distributable binary, but no evidence pack item states an open-source license, and the self-hosting binary is described as a closed compiled artifact rather than published source. Missing for 10: any explicit license file/name, evidence the GitHub repo contains the actual product source (vs. docs/SDK), and confirmation the self-hosted binary corresponds to inspectable open-licensed code.

   • [github] “Connectors Google Drive · Gmail · Notion · OneDrive · GitHub — auto-sync with real-time webhooks.”
   • [claimed-docs] “Supermemory runs on your own hardware. It's the same memory engine behind the hosted platform... as a single self-contained binary.”
   • [claimed-docs] “State-of-the-art memory, running on your machine. One binary, zero config.”

   proof ↗⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round drawn
   Letta✓fullclaimed8/10

   Letta's docs explicitly describe self-hosting the core product: running the Letta App Server on infrastructure you control, local CLI mode that stores state on-device without requiring a Letta account, and running agents entirely on infra you control (letta-docs-2, letta-docs-15, letta-docs-30, letta-docs-31, letta-docs-74). This is first-party documentation of a genuine self-host mode, not just cloud-hosted convenience. missing for 10: independent/hands-on verification that self-hosted deployment works as documented, and more detail on infra/ops requirements for self-hosting at scale.

   • [claimed-docs] “Run the Letta App Server to host local agents on a central, always-on machine and expose them to client applications through the Agent SDK”
   • [claimed-docs] “Self-hosted App Server: Run the Letta App Server to host local agents on a central, always-on machine and expose them to client applications…”
   • [claimed-docs] “Letta agents can run entirely on infrastructure you control”
   • [claimed-docs] “This default local mode stores agent state on your machine and does not require a Letta login.”
   • [claimed-docs] “all agent state, including messages, memory, and provider connections, stays on-device, and no Letta account is required.”
   • [claimed-docs] “Run the Letta App Server to host local agents on a central, always-on machine and expose them to client applications”

   proof ↗⚑ flag

   Supermemory✓fullclaimed8/10

   Docs explicitly document a self-hosting mode running the same memory engine as a single self-contained binary with local graph engine, embeddings, and LLM, requiring no Docker or config, and no data leaving the building. Missing for 10: independent hands-on verification of the self-host binary working, and details on feature parity/limitations vs hosted version.

   • [claimed-docs] “Supermemory runs on your own hardware. It's the same memory engine behind the hosted platform... as a single self-contained binary.”
   • [claimed-docs] “Local graph engine, local embeddings, local LLM. Your data never leaves the building.”
   • [claimed-docs] “Supermemory runs on your own hardware. It's the same memory engine behind the hosted platform ... as a single self-contained binary.”
   • [claimed-docs] “No Docker. No database to provision. No config files. It boots in seconds with everything built in”
   • [claimed-docs] “State-of-the-art memory, running on your machine. One binary, zero config.”
   • [claimed-docs] “SOC 2 · HIPAA BAA · self-hosted option”

   proof ↗⚑ flag

Pricing

1. platform-engineerSee published pricing with a free tier and per-unit rates so I can project memory costs before committing

   weight 2 · round to Letta
   Letta~partialclaimed5/10

   Letta's docs confirm a published pricing page with a free BYOK tier (letta-docs-56), a capped free stateful-agent tier (letta-docs-36) and a usage-based developer plan (letta-docs-49), which addresses the free-tier and general cost-model part of the story. However, no explicit per-unit rate table (e.g., $/agent-hour, $/GB memory, $/token) is shown in the evidence, so a platform engineer cannot precisely project memory costs. Missing for 10: concrete per-unit pricing figures/rate card, and independent confirmation of actual billing granularity for memory usage.

   • [claimed-docs] “Up to 20 stateful agents”
   • [claimed-docs] “With a developer plan, you can create an unlimited number of agents but pay usage-based pricing.”
   • [claimed-docs] “You can get started with Letta for free by using models with BYOK, including coding plans from ChatGPT/Codex, Kimi, and Z.AI.”
   • [claimed-docs] “You can share agents across your team (and set permissions for access) to create custom agents for your organization”

   proof ↗⚑ flag

   Supermemory~partialclaimed4/10

   A dedicated pricing.md page is referenced with some billing mechanics (unique-content-only billing, prompt-cache discount, plan tiers like 'Max plan' gating connectors, SOC2/HIPAA/self-host options), but no evidence pack citation actually shows a free-tier allotment or explicit per-unit dollar rates. Missing for 10: explicit free-tier limits, published per-unit/usage pricing figures, and any independent corroboration of the pricing page's numbers.

   • [claimed-docs] “only unique content we ingest is billed; repeats cost nothing (100% prompt-cache discount, baked in).”
   • [claimed-docs] “Google Drive, Notion & OneDrive connectors”
   • [claimed-docs] “Max Plan Required: The Gmail connector is available on Max plan and above.”
   • [claimed-docs] “SOC 2 · HIPAA BAA · self-hosted option”

   proof ↗⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round drawn
   Letta~partialclaimed4/10

   Letta offers self-hosting where 'agents can run entirely on infrastructure you control' and local mode where 'all agent state ... stays on-device' (letta-docs-30, letta-docs-74), giving users a way to control where data physically lives. However, there is no documented region/residency selection feature for the managed Letta Cloud offering itself (e.g., choosing EU vs US data centers). Missing for 10: explicit cloud region/residency picker, compliance certifications (GDPR/data-locality), and any documentation of where Letta Cloud stores data by default.

   • [claimed-docs] “Letta agents can run entirely on infrastructure you control”
   • [claimed-docs] “all agent state, including messages, memory, and provider connections, stays on-device, and no Letta account is required.”
   • [claimed-docs] “Self-hosted App Server: Run the Letta App Server to host local agents on a central, always-on machine and expose them to client applications…”
   • [claimed-docs] “This default local mode stores agent state on your machine and does not require a Letta login.”

   proof ↗⚑ flag

   Supermemory~partialclaimed4/10

   Supermemory offers a self-hosted binary option ("data never leaves the building") and lists SOC2/HIPAA BAA/self-hosted as compliance options, which lets an org control the physical location of data by running it on their own infrastructure, but there is no documented feature to choose a specific region/data-residency setting within the hosted SaaS offering itself. missing for 10: explicit region/residency selection controls in the hosted product, documentation of where hosted data is stored geographically, and any multi-region deployment options.

   • [claimed-docs] “Supermemory runs on your own hardware. It's the same memory engine behind the hosted platform... as a single self-contained binary.”
   • [claimed-docs] “Supermemory runs on your own hardware. It's the same memory engine behind the hosted platform ... as a single self-contained binary.”
   • [claimed-docs] “SOC 2 · HIPAA BAA · self-hosted option”

   proof ↗⚑ flag

2. ai-native userPrevent my data from being used to train AI models

   weight 3 · round to Supermemory
   Letta—none0/10

   No evidence pack item addresses data-training opt-out, a training-data policy, or any privacy controls preventing use of user data for model training; self-hosting is mentioned but not tied to a training-data guarantee.

   ⚑ flag

   Supermemory~partialclaimed5/10

   Supermemory offers a self-hosted deployment where 'your data never leaves the building' with local embeddings/LLM (supermemory-docs-8, supermemory-docs-16), which implicitly prevents data from reaching any third-party model training pipeline, and it advertises SOC2/HIPAA compliance (supermemory-docs-52). However, there is no explicit policy statement for the hosted/cloud platform about not training on customer data or an opt-out toggle. missing for 10: explicit no-training/data-use policy for the hosted SaaS offering, independent verification of self-host data isolation claims.

   • [claimed-docs] “Local graph engine, local embeddings, local LLM. Your data never leaves the building.”
   • [claimed-docs] “Supermemory runs on your own hardware. It's the same memory engine behind the hosted platform ... as a single self-contained binary.”
   • [claimed-docs] “State-of-the-art memory, running on your machine. One binary, zero config.”
   • [claimed-docs] “SOC 2 · HIPAA BAA · self-hosted option”

   proof ↗⚑ flag

3. ai-native userControl data retention and deletion

   weight 2 · round to Supermemory
   Letta~partialclaimed5/10

   Letta gives users some retention control: local mode stores state on-device without a cloud login (letta-docs-31), self-hosting keeps 'all agent state, including messages, memory, and provider connections... on-device, and no Letta account is required' (letta-docs-74, letta-docs-30), and --ephemeral mode runs a one-shot task 'without creating, resuming, or persisting an agent, memory blocks, or MemFS' (letta-docs-14). However, there is no documented explicit 'delete agent/data' command, data-retention policy for Letta Cloud, or export/erase workflow. Missing for 10: explicit deletion/erasure command or API, documented cloud data-retention/purge policy, independent confirmation that deletion actually removes cloud-stored data.

   • [claimed-docs] “Use --ephemeral to run a one-shot task without creating, resuming, or persisting an agent, memory blocks, or MemFS”
   • [claimed-docs] “This default local mode stores agent state on your machine and does not require a Letta login.”
   • [claimed-docs] “all agent state, including messages, memory, and provider connections, stays on-device, and no Letta account is required.”
   • [claimed-docs] “Letta agents can run entirely on infrastructure you control”

   proof ↗⚑ flag

   Supermemory~partialclaimed6/10

   Docs explicitly document a deletion workflow scoped by container tags (delete a user's/tenant's content via API/console) plus self-hosting for full data control and SOC2/HIPAA compliance options, giving users a documented path to control retention and deletion. However, there's no evidence of granular per-memory deletion controls, retention policy settings (auto-expiry/TTL), or an end-user-facing UI for self-service deletion — the workflow described is developer/console-mediated only. Missing for 10: user-facing self-service deletion UI, configurable retention/expiry policies, and independent confirmation that deletion actually purges data from backups/indexes.

   • [claimed-docs] “Scope each end-user (or tenant) to a container tag. When the user requests deletion, delete that container's content via the API / console w…”
   • [claimed-docs] “Scope each end-user (or tenant) to a **container tag**. When the user requests deletion, delete that container's content via the API / conso…”
   • [claimed-docs] “Scope each end-user (or tenant) to a container tag.”
   • [claimed-docs] “Scope each end-user (or tenant) to a container tag. 2. When the user requests deletion, delete that container's content via the API / consol…”
   • [claimed-docs] “SOC 2 · HIPAA BAA · self-hosted option”
   • [claimed-docs] “Supermemory runs on your own hardware. It's the same memory engine behind the hosted platform... as a single self-contained binary.”
   • [claimed-docs] “Local graph engine, local embeddings, local LLM. Your data never leaves the building.”

   proof ↗⚑ flag

4. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   Letta—none0/10

   No evidence pack item addresses telemetry, usage analytics, or an opt-out mechanism; while local self-hosted mode is documented, nothing confirms or denies data collection/telemetry settings.

   ⚑ flag

   Supermemory—none0/10

   No evidence pack item addresses telemetry opt-out or usage-tracking controls for Supermemory; docs cover security/compliance (SOC 2, HIPAA) and self-hosting but never mention a telemetry toggle or analytics opt-out mechanism.

   ⚑ flag

Latency

1. platform-engineerSee documented retrieval-latency targets or measured numbers (e.g. p50/p95) backing the product's speed claims

   weight 2 · round drawn
   Letta—none0/10

   No evidence pack item mentions retrieval latency, p50/p95, or any measured/documented performance numbers for search or memory retrieval; documentation covers features like MemFS search mods and dreaming but never quantifies speed. missing for 10: any documented latency targets, benchmark numbers, or measured p50/p95 retrieval performance data.

   ⚑ flag

   Supermemory—none0/10

   No evidence in the pack cites any documented latency targets or measured p50/p95 numbers for retrieval; docs discuss features like hybrid search, self-hosting, and MemoryBench for benchmarking against competitors, but no concrete performance figures are given. Missing for 10: published latency SLAs/targets, measured p50/p95 benchmark numbers, or any performance dashboard/report.

   ⚑ flag

Scale

1. platform-engineerIngest at scale with async or batch processing and check the status of background memory operations

   weight 2 · round drawn
   Letta—none0/10

   Letta's docs describe per-agent memory consolidation ('dreaming' background subagents, triggers on steps/compaction) but there is no evidence of a bulk/batch ingestion pipeline for scale data loading nor any API/CLI to check the status of background memory jobs, which is what this platform-engineer story requires.

   ⚑ flag

   Supermemory—none0/10

   Docs describe ingestion mechanics (raw content ingestion, customId dedup, connectors with webhooks, automatic extraction) but there is no mention of a batch/async ingestion API, job IDs, or an endpoint/mechanism to check the status of background memory-processing operations — the evidence pack contains no support for this platform-engineer-facing capability. missing for 10: batch ingestion API, async job/task status endpoint, documentation of processing queue or webhook completion callbacks.

   • [claimed-docs] “Send any raw content to Supermemory — conversations, documents, files, URLs. We extract the memories automatically.”
   • [claimed-docs] “Use customId to update existing documents or conversations. When you send content with the same customId, Supermemory intelligently processe…”
   • [claimed-docs] “Connect external platforms to automatically sync documents into supermemory. Supported connectors include Google Drive, Gmail, Notion, OneDr…”
   • [github] “Connectors Google Drive · Gmail · Notion · OneDrive · GitHub — auto-sync with real-time webhooks.”

   proof ↗⚑ flag

Agent memory

1. ai-native userConnect off-the-shelf assistants (Claude, ChatGPT, Cursor) to the same memory so every tool I use shares what it knows about me

   weight 2 · round to Supermemory
   Letta—none0/10

   Evidence shows Letta integrating with Codex, Claude Code, Hermes Agent, OpenClaw for skills (docs-57), and offering an OpenAI-compatible API for tools like Open WebUI plus an ACP adapter for Zed (docs-23), but nothing shows Letta connecting to or sharing memory with off-the-shelf Claude, ChatGPT, or Cursor specifically. Letta's MCP support is client-side (Letta consuming MCP servers, docs-61/76), and community evidence notes only a third-party 'Letta-MCP-server' built by a user, not an official integration exposing Letta memory to these named assistants.

   • [claimed-docs] “Skills are portable across the Letta app and CLI, Codex, Claude Code, Hermes Agent, OpenClaw, and other compatible agents.”
   • [claimed-docs] “use the App Server’s OpenAI-compatible API with clients such as Open WebUI, or the ACP adapter with ACP clients such as Zed”
   • [claimed-docs] “The SDK supports local stdio processes, Streamable HTTP, and legacy SSE”
   • [claimed-docs] “MCP tools are discovered from servers supplied through `mcpServers` and proxied by the SDK host to the MCP server.”
   • [community] “Charles from Letta on MCP: 'We're working on it... making Letta agents an MCP client, so that you can easily connect to MCP servers as tools…”

   proof ↗⚑ flag

   Supermemory✓fullprobed8/10

   Supermemory ships an official MCP server explicitly described as giving 'every MCP-compatible assistant a shared memory layer,' which covers Claude, ChatGPT, Cursor and other MCP clients, plus container-tag scoping to keep the shared memory user-specific and natural-language tool selection without code. Missing for 10: explicit named walkthroughs/screenshots for each specific assistant (Claude, ChatGPT, Cursor) connecting simultaneously, and independent hands-on confirmation that cross-tool memory sharing works smoothly in practice.

   • [claimed-docs] “Supermemory MCP gives every MCP-compatible assistant a shared memory layer, so technical and non-technical teams can collaborate with AI usi…”
   • [claimed-docs] “Ask naturally. The assistant selects the right Supermemory tool without requiring code or tool names.”
   • [claimed-docs] “A space keeps a team's documents, memories, and profile context focused, so AI retrieves the right knowledge without mixing unrelated work.”
   • [claimed-docs] “every memory tagged with `user_alex` lives in its own isolated space, completely separate from memories tagged `user_jordan`”
   • [probe] “official MCP server documented at https://supermemory.ai/docs/supermemory-mcp/mcp”
   • [claimed-docs] “Scope each end-user (or tenant) to a container tag.”

   proof ↗⚑ flag

Frameworks

1. developerDrop the memory layer into agent frameworks like LangChain, LangGraph, CrewAI, or the Vercel AI SDK via documented first-party integrations

   weight 2 · round drawn
   Letta—none0/10

   The evidence pack covers Letta's own Agent SDK, CLI, App Server, MCP support, and various demo apps, but contains no mention of LangChain, LangGraph, CrewAI, or Vercel AI SDK integrations anywhere in the docs, GitHub, or community sources.

   ⚑ flag

   Supermemory—none0/10

   The evidence pack documents official TypeScript/Python SDKs, an MCP server, a CLI setup flow, and third-party connectors (Google Drive, Gmail, Notion, OneDrive, GitHub) but contains no mention of first-party integrations for LangChain, LangGraph, CrewAI, or the Vercel AI SDK specifically. Since a memory-layer product plausibly could ship such framework adapters, absence of evidence here is 'none' rather than 'na'.

   • [claimed-docs] “Official clients wrap this API: TypeScript: npm install supermemory Python: pip install supermemory”
   • [claimed-docs] “Set up coding agents to integrate Supermemory — CLI, skill, and docs MCP.”
   • [claimed-docs] “Supermemory MCP gives every MCP-compatible assistant a shared memory layer, so technical and non-technical teams can collaborate with AI usi…”

   proof ↗⚑ flag

2. developerWire memory into real-time voice pipelines (e.g. LiveKit, Pipecat, ElevenLabs) with documented integrations fast enough for live conversation

   weight 1 · round drawn
   Letta—none0/10

   The evidence pack contains no documentation or mention of LiveKit, Pipecat, ElevenLabs, or any real-time voice pipeline integration. The closest evidence (letta-comm-4) is a forum reply suggesting a DIY postgres-backed memory setup for a hypothetical voice use case, not a documented, low-latency SDK integration for live conversation.

   • [community] “User with a father suffering memory loss suggested Letta's framework looked interesting for building a voice-interactive LLM with memory to …”

   proof ↗⚑ flag

   Supermemory—none0/10

   The evidence pack shows generic SDKs, connectors (Drive, Gmail, Notion, etc.), MCP integration, and self-hosting, but nowhere mentions LiveKit, Pipecat, ElevenLabs, or any real-time voice pipeline integration, latency benchmarks, or streaming APIs suited for live conversation. No documented voice-pipeline integration exists in the pack.

   ⚑ flag

Sdks

1. developerBuild against official SDKs in at least Python and TypeScript with equivalent memory APIs

   weight 2 · round to Supermemory
   Letta~partialclaimed5/10

   The evidence extensively documents a TypeScript/JavaScript Agent SDK (client.createAgent(), sessions, streaming, MCP, browser/mobile support) but no evidence pack item confirms an official Python SDK or its API parity with the TypeScript SDK — despite Letta's backend being Python-based, docs cited here only reference the JS/TS Agent SDK and CLI. missing for 10: explicit official Python SDK documentation/package, confirmation of equivalent memory API surface (createAgent, sessions, streaming, MCP) across Python and TypeScript, and any independent corroboration of Python SDK usage.

   • [github] “the Letta Agent SDK for building agents into TypeScript applications”
   • [claimed-docs] “Create a stateful agent with `client.createAgent()`. It returns the agent's ID as a string, which you use to open sessions and manage the ag…”
   • [claimed-docs] “`resumeSession(id)` accepts either kind of ID: pass an `agent-xxx` ID to resume the agent's default conversation, or a `conv-xxx` ID to resu…”
   • [claimed-docs] “Send a message with `session.send()` and read the response with `session.stream()`. The stream yields typed events for the whole turn”
   • [claimed-docs] “Browser, Expo, and React Native applications must import the SDK from `@letta-ai/letta-agent-sdk/client`”
   • [claimed-docs] “The SDK supports local stdio processes, Streamable HTTP, and legacy SSE”
   • [claimed-docs] “MCP tools are discovered from servers supplied through `mcpServers` and proxied by the SDK host to the MCP server.”

   proof ↗⚑ flag

   Supermemory~partialclaimed6/10

   Docs explicitly confirm official TypeScript (npm install supermemory) and Python (pip install supermemory) SDKs wrapping the same API, with code examples shown mainly in TS/JS (client.add, client.search). There's no direct evidence of Python-specific code samples or confirmation the Python SDK has full feature parity (e.g., container tags, hybrid search, profiles) documented equivalently. Missing for 10: Python code examples demonstrating equivalent memory operations, explicit parity statement across SDKs, independent developer corroboration of Python SDK usage.

   • [claimed-docs] “Official clients wrap this API: TypeScript: npm install supermemory Python: pip install supermemory”
   • [claimed-docs] “await client.add({ content: "The user loves Paris.", containerTag: "user_123" });”
   • [claimed-docs] “const results = await client.search({ q: "where does Alex work?", containerTag: "user_123", include: { relatedMemories: true }, });”
   • [claimed-docs] “Use searchMode: "hybrid" for best results. It searches both memories and document chunks, returning the most relevant content.”

   proof ↗⚑ flag

Context assembly

1. developerRetrieve a token-budgeted, prompt-ready context block assembled from relevant memories in one call

   weight 2 · round to Supermemory
   Letta—none0/10

   Letta's docs describe memory blocks, MemFS, /doctor token audits, and search mods, but there is no evidence of a single API/CLI call that returns a token-budgeted, prompt-ready assembled context block for developer consumption outside the agent's own turn — memory in Letta is managed internally as part of the agent's system prompt, not exposed as a discrete retrieval endpoint.

   • [claimed-docs] “If the memory hierarchy has drifted or grown too large, run /doctor to audit placement, duplication, and system-prompt token usage.”
   • [claimed-docs] “Pass memory for full control over the agent’s starting memory. Each entry becomes a Markdown file in the agent’s memory repository”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the [MemFS Search mod]”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the MemFS Search mod”
   • [claimed-docs] “`persona` and `human` are conveniences that set the agent's persona and human memory”

   proof ↗⚑ flag

   Supermemory~partialclaimed4/10

   Supermemory offers a single-call hybrid search that returns relevant memories/document chunks and related memories, and user profiles that are 'always ready' without a search call, which covers assembling relevant context in one call — but no docs mention a token-budget parameter or explicit prompt-sized context-block output. missing for 10: explicit token-budget/max-token control on retrieval, evidence of a single call returning a pre-formatted prompt-ready block combining profile+graph+documents together.

   • [claimed-docs] “Use `searchMode: "hybrid"` for best results. It searches both memories and document chunks, returning the most relevant content.”
   • [claimed-docs] “Use searchMode: "hybrid" for best results. It searches both memories and document chunks, returning the most relevant content.”
   • [claimed-docs] “No search needed — comprehensive user info always ready”
   • [claimed-docs] “No search needed — comprehensive user info always ready Profiles update as users interact with your system”
   • [claimed-docs] “Retrieve three ways — document search (RAG), memory graph traversal, and user profile”
   • [claimed-docs] “const results = await client.search({ q: "where does Alex work?", containerTag: "user_123", include: { relatedMemories: true }, });”

   proof ↗⚑ flag

2. developerGet summaries of past sessions or threads so an agent can pick up where the last conversation left off

   weight 3 · round to Letta
   Letta✓fullcommunity8/10

   Letta's core architecture is built around persistent, stateful agents that can be resumed across sessions/computers, with explicit APIs (resumeSession, agent IDs) and memory consolidation ('dreaming') that summarizes past conversations into durable memory so agents pick up where they left off. Community/founder commentary (letta-comm-5, letta-comm-12) corroborates the 'memory-first, no more forgetting' design intent. missing for 10: independent hands-on verification of summary/context fidelity over long horizons, and quantitative benchmarks of resume/summarization quality beyond vendor docs.

   • [claimed-docs] “You can return to the same agent across sessions, computers, and interfaces instead of starting from scratch each time.”
   • [claimed-docs] “`resumeSession(id)` accepts either kind of ID: pass an `agent-xxx` ID to resume the agent's default conversation, or a `conv-xxx` ID to resu…”
   • [claimed-docs] “Dreaming uses background subagents to review recent conversations, consolidate useful lessons, and update memory without interrupting your a…”
   • [claimed-docs] “Use /fork to branch the current conversation, including its in-context history.”
   • [claimed-docs] “`/search [query]` | Search messages across agents”
   • [claimed-docs] “Run `/init` to bootstrap or refresh memory for the current project. The agent inspects the repository, asks about your working style when ne…”
   • [claimed-docs] “You can also teach it explicitly with `/remember`: `> /remember always use pnpm in this repo`. The agent decides where the lesson belongs an…”
   • [community] “Letta Code described by Letta's Charles as 'memory-first': agents build learned context over time via tools like /init and /remember; a /cle…”
   • [community] “Letta employee describes Letta Code: 'a fundamentally different approach to coding agents with a focus on continual learning and statefulnes…”

   proof ↗⚑ flag

   Supermemory~partialclaimed7/10

   Supermemory ingests raw conversations and auto-extracts memories, then exposes them via hybrid search, memory graph traversal, and always-ready user profiles that update as users interact — this gives an agent a way to recall prior conversation context and continue it (docs-1, docs-2, docs-3, docs-15, docs-23, docs-33). However there is no explicit 'session/thread summary' artifact or transcript-resume feature documented, and no hands-on/independent evidence of an agent actually resuming a specific thread using this data. Missing for 10: a dedicated session-summary or thread-resume API/feature, and independent verification that agents successfully pick up prior conversations using this context.

   • [claimed-docs] “Send any raw content to Supermemory — conversations, documents, files, URLs. We extract the memories automatically.”
   • [claimed-docs] “Use `searchMode: "hybrid"` for best results. It searches both memories and document chunks, returning the most relevant content.”
   • [claimed-docs] “No search needed — comprehensive user info always ready”
   • [claimed-docs] “User profiles are **automatically maintained collections of facts about your users** that Supermemory builds from all their interactions.”
   • [claimed-docs] “Retrieve three ways — document search (RAG), memory graph traversal, and user profile”
   • [claimed-docs] “No search needed — comprehensive user info always ready Profiles update as users interact with your system”
   • [claimed-docs] “Use customId to update existing documents or conversations. When you send content with the same customId, Supermemory intelligently processe…”

   proof ↗⚑ flag

Ingestion

1. developerIngest documents, JSON, and business data into memory — not just chat transcripts

   weight 2 · round to Letta
   Letta~partialclaimed5/10

   Letta's memory system (MemFS) is file/markdown-based and the SDK lets developers 'pass memory for full control' where 'each entry becomes a Markdown file' — implying arbitrary content beyond chat transcripts can seed agent memory, and /init lets an agent ingest a repository's files. However, there is no explicit documentation of ingesting structured JSON or generic 'business data' formats into memory, nor any first-party example of a document/JSON ingestion pipeline. Missing for 10: explicit JSON/structured-data ingestion support, document-upload workflows, and independent/hands-on evidence of non-chat data being loaded into agent memory.

   • [claimed-docs] “Pass memory for full control over the agent’s starting memory. Each entry becomes a Markdown file in the agent’s memory repository”
   • [claimed-docs] “Run `/init` to bootstrap or refresh memory for the current project. The agent inspects the repository, asks about your working style when ne…”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the [MemFS Search mod]”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the MemFS Search mod”
   • [claimed-docs] “`persona` and `human` are conveniences that set the agent's persona and human memory”

   proof ↗⚑ flag

   Supermemory!disputedcontradicted6/10

   First-party docs strongly document ingesting raw documents, files, URLs, JSON-like content, code, PDFs, images, and connectors (Google Drive, Notion, OneDrive, Gmail, GitHub) beyond chat transcripts, with customId-based document updates and content-type-specific extraction. However, a hands-on community report describes a concrete failure — a large-content link being rejected with a 'content too large' error — directly contradicting the 'just upload, it just works' ingestion claim for real-world documents. Missing for 10: resolution/acknowledgment of the size-limit failure, and independent corroboration of large-scale JSON/business-data ingestion succeeding in practice.

   • [claimed-docs] “Send any raw content to Supermemory — conversations, documents, files, URLs. We extract the memories automatically.”
   • [claimed-docs] “Supermemory automatically extracts and indexes content from various formats.”
   • [claimed-docs] “Connect external platforms to automatically sync documents into supermemory. Supported connectors include Google Drive, Gmail, Notion, OneDr…”
   • [github] “Multi-modal Extractors | PDFs, images (OCR), videos (transcription), code (AST-aware chunking). Upload and it works.”
   • [claimed-docs] “Use customId to update existing documents or conversations. When you send content with the same customId, Supermemory intelligently processe…”
   • [community] “It's a shame it is unable to store large content from links. I tried it with a tutorial link so that I could eventually master the Nix langu…”

   proof ↗⚑ flag

2. developerStore images, PDFs, or other files as memory inputs and recall information from them later

   weight 1 · round to Supermemory
   Letta—none0/10

   Letta's documented memory system (MemFS, memory blocks, /remember, /init) is explicitly text/Markdown-file based, and 'image attachments' appear only in a demo chat app's UI feature list, not as a supported memory-recall input type for PDFs or images. No evidence shows agents can ingest images or PDFs into memory and recall facts from them later.

   • [claimed-docs] “Pass memory for full control over the agent’s starting memory. Each entry becomes a Markdown file in the agent’s memory repository”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the [MemFS Search mod]”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the MemFS Search mod”
   • [claimed-docs] “Use this demo when you want to study a mobile interface with conversation history, streaming, tools, approvals, interruption, queued message…”

   proof ↗⚑ flag

   Supermemory✓fullclaimed8/10

   GitHub docs explicitly list multi-modal extractors for PDFs and images (OCR) with 'upload and it works,' and docs describe raw file ingestion plus hybrid search/recall across document chunks and memories. Missing for 10: independent hands-on verification specifically for image/PDF recall (the only community evidence found concerns large link content failing, not files/images).

   • [github] “Multi-modal Extractors | PDFs, images (OCR), videos (transcription), code (AST-aware chunking). Upload and it works.”
   • [claimed-docs] “Send any raw content to Supermemory — conversations, documents, files, URLs. We extract the memories automatically.”
   • [claimed-docs] “Supermemory automatically extracts and indexes content from various formats.”
   • [claimed-docs] “Use `searchMode: "hybrid"` for best results. It searches both memories and document chunks, returning the most relevant content.”
   • [claimed-docs] “Use searchMode: "hybrid" for best results. It searches both memories and document chunks, returning the most relevant content.”
   • [claimed-docs] “Just upload — Supermemory handles the rest ... No chunking strategies to configure. No embedding models to choose. It just works.”

   proof ↗⚑ flag

Governance

1. platform-engineerGovern who and what can read or write memory with roles, policies, or access-control lists, and audit that access

   weight 2 · round to Supermemory
   Letta~partialclaimed3/10

   Letta's pricing docs mention you can 'share agents across your team (and set permissions for access)' for organizational use, implying some access control exists, but there is no documentation of roles, granular ACLs, or an audit log/trail for memory reads/writes. Missing for 10: documented role/policy definitions for memory access, explicit read/write ACLs on memory blocks, and any audit logging or access history feature.

   • [claimed-docs] “You can share agents across your team (and set permissions for access) to create custom agents for your organization”

   proof ↗⚑ flag

   Supermemory~partialclaimed4/10

   Supermemory provides container-tag based tenant isolation (hard namespace boundaries) and SOC2/HIPAA BAA compliance claims, which partially address data segregation and enterprise trust, but there is no documented role-based access control, granular read/write policies, or audit-log capability for tracking who accessed or modified memories. Missing for 10: explicit RBAC/ACL system, audit trail/logging of access events, and evidence of enforcement beyond tag-scoped isolation.

   • [claimed-docs] “A container tag is a hard boundary — its own namespace. Memories in one tag are never returned by a search scoped to another tag.”
   • [claimed-docs] “every memory tagged with `user_alex` lives in its own isolated space, completely separate from memories tagged `user_jordan`”
   • [claimed-docs] “Scope each end-user (or tenant) to a container tag. When the user requests deletion, delete that container's content via the API / console w…”
   • [claimed-docs] “SOC 2 · HIPAA BAA · self-hosted option”

   proof ↗⚑ flag

Isolation

1. developerScope memories per user, agent, or application so one tenant's memories never leak into another's retrieval

   weight 3 · round to Supermemory
   Letta~partialclaimed5/10

   Letta's core model is per-agent stateful memory (each agent has its own memory blocks/files, docs-42, docs-48), and cross-agent sharing is opt-in via explicit 'shared-memory' repos (docs-8, docs-21, docs-29) rather than automatic leakage, plus team/org permission controls exist (docs-87: 'share agents across your team and set permissions for access'). However there is no documentation of explicit multi-tenant isolation guarantees (per-user/per-application namespacing, access-control enforcement details, or audit of leakage prevention) beyond these general mechanisms. Missing for 10: explicit tenant/user isolation model, documented ACL/RBAC enforcement for memory retrieval, and any independent verification that scoping prevents cross-tenant leakage.

   • [claimed-docs] “Create a stateful agent with `client.createAgent()`. It returns the agent's ID as a string, which you use to open sessions and manage the ag…”
   • [claimed-docs] “`persona` and `human` are conveniences that set the agent's persona and human memory”
   • [claimed-docs] “letta shared-memory create --name team-memory”
   • [claimed-docs] “Shared memory gives multiple agents access to the same files and working context. Each shared memory repository is a Git repository hosted i…”
   • [claimed-docs] “Shared memory gives multiple agents access to the same files and working context.”
   • [claimed-docs] “You can share agents across your team (and set permissions for access) to create custom agents for your organization”

   proof ↗⚑ flag

   Supermemory✓fullclaimed8/10

   Docs clearly document container tags as hard isolation boundaries per user/agent/app, with explicit statements that tag-scoped searches never return cross-tenant memories and code examples showing containerTag usage for scoping and deletion. missing for 10: independent/hands-on verification of isolation guarantees beyond vendor docs, and no detail on cross-tenant leakage testing or edge cases (e.g., shared profiles/buckets interacting with tags).

   • [claimed-docs] “A container tag is a hard boundary — its own namespace. Memories in one tag are never returned by a search scoped to another tag.”
   • [claimed-docs] “every memory tagged with `user_alex` lives in its own isolated space, completely separate from memories tagged `user_jordan`”
   • [claimed-docs] “await client.add({ content: "The user loves Paris.", containerTag: "user_123" });”
   • [claimed-docs] “const results = await client.search({ q: "where does Alex work?", containerTag: "user_123", include: { relatedMemories: true }, });”
   • [claimed-docs] “await client.add({ content: "Alex mentioned he just started at Stripe", containerTag: "user_123" });”
   • [claimed-docs] “Scope each end-user (or tenant) to a container tag. When the user requests deletion, delete that container's content via the API / console w…”
   • [claimed-docs] “Scope each end-user (or tenant) to a **container tag**. When the user requests deletion, delete that container's content via the API / conso…”
   • [claimed-docs] “Scope each end-user (or tenant) to a container tag.”
   • [claimed-docs] “Scope each end-user (or tenant) to a container tag. 2. When the user requests deletion, delete that container's content via the API / consol…”

   proof ↗⚑ flag

Sharing

1. developerShare selected memory across multiple agents or users (team or group memory) while keeping private memory private

   weight 1 · round to Letta
   Letta✓fullclaimed7/10

   Letta ships a dedicated shared-memory feature (`letta shared-memory create`, Git-backed shared repositories giving multiple agents common context) alongside per-agent private memory blocks, and pricing docs explicitly mention team-level agent sharing with configurable permissions. This directly matches sharing selected memory across agents/users while keeping other memory private. Missing for 10: independent/hands-on corroboration of the privacy boundary enforcement and finer-grained detail on how access permissions are scoped per user vs team.

   • [claimed-docs] “letta shared-memory create --name team-memory”
   • [claimed-docs] “Shared memory gives multiple agents access to the same files and working context. Each shared memory repository is a Git repository hosted i…”
   • [claimed-docs] “Shared memory gives multiple agents access to the same files and working context.”
   • [claimed-docs] “You can share agents across your team (and set permissions for access) to create custom agents for your organization”

   proof ↗⚑ flag

   Supermemory~partialclaimed7/10

   Docs describe container tags as hard-boundary namespaces for per-user/tenant isolation (private memory) and separately describe 'spaces' and the Supermemory MCP as a shared memory layer letting teams collaborate on the same authorized context, which together map to the private-vs-shared story. However there's no explicit example of selectively promoting specific memories from a private container into a shared team space, nor documented fine-grained ACL/role controls for partial sharing. missing for 10: worked example of mixed private+shared memory for the same agent/user, explicit role-based access control docs, independent/hands-on verification of isolation guarantees.

   • [claimed-docs] “A container tag is a hard boundary — its own namespace. Memories in one tag are never returned by a search scoped to another tag.”
   • [claimed-docs] “every memory tagged with `user_alex` lives in its own isolated space, completely separate from memories tagged `user_jordan`”
   • [claimed-docs] “A space keeps a team's documents, memories, and profile context focused, so AI retrieves the right knowledge without mixing unrelated work.”
   • [claimed-docs] “Supermemory MCP gives every MCP-compatible assistant a shared memory layer, so technical and non-technical teams can collaborate with AI usi…”
   • [claimed-docs] “Scope each end-user (or tenant) to a **container tag**. When the user requests deletion, delete that container's content via the API / conso…”

   proof ↗⚑ flag