Zep vs Letta

Agent access

1. ai-native userPoint an agent at llms.txt or agent-oriented docs

   weight 2 · round to Zep
   Zep✓fullprobed9/10

   Zep hosts an llms.txt file at help.getzep.com/llms.txt (confirmed HTTP 200 via direct probe) with a clear summary of the product, and also runs a docs MCP server that completed a live keyless JSON-RPC handshake, giving agents multiple direct machine-readable entry points into the docs. Missing for 10: no independent/community confirmation that agents actually use these entry points successfully in practice.

   • [probe] “PROBE llms.txt: HTTP 200 at https://help.getzep.com/llms.txt # Welcome to Zep! > Zep delivers agent memory at enterprise scale — a governed…”
   • [probe] “PROBE runtime (recorded 2026-09-05): Zep's documentation MCP server at https://help.getzep.com/_mcp/server completed a full keyless JSON-RPC…”
   • [claimed-docs] “The Memory MCP Server lets an end user connect an MCP client (Claude, ChatGPT, Claude Code, Codex, Cursor, and others) to their own agent me…”

   proof ↗⚑ flag

   Letta✓fullprobed8/10

   A direct probe confirms Letta publishes an llms.txt file at docs.letta.com/llms.txt returning HTTP 200 with agent-oriented summary content, and Letta's extensive docs site is clearly structured for programmatic/agent consumption (agent-sdk, CLI, MCP docs). Missing for 10: no evidence of llms.txt content depth/comprehensiveness or independent confirmation that agents successfully consume it in practice.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.letta.com/llms.txt # Letta > Letta agents learn from experience and improve with use. The Letta ag…”
   • [claimed-docs] “create an agent once, then resume it from anywhere”

   proof ↗⚑ flag

2. ai-native userRun the product headlessly / in CI for automation

   weight 2 · round to Letta
   Zep~partialprobed6/10

   Zep is fundamentally API/SDK-driven (thread.add_messages, graph.edge.delete calls) and ships a full-access CLI (zepctl) for managing projects, users, threads, and data from the terminal — both of which are inherently scriptable/headless and usable in CI pipelines. However, there is no explicit documentation or example of CI/CD integration, automated testing, or headless deployment guidance. Missing for 10: explicit CI/CD pipeline examples, headless automation docs, non-interactive auth/service-account setup guidance for CI.

   • [claimed-docs] “zepctl is a command-line interface for administering Zep projects. It provides full access to Zep, enabling you to manage users, threads, Co…”
   • [claimed-docs] “`zepctl` is a command-line interface for administering Zep projects. It provides full access to Zep, enabling you to manage users, threads, …”
   • [claimed-docs] “zepctl is a command-line interface for administering Zep projects.”
   • [claimed-docs] “Chat messages are added to threads using `thread.add_messages`, which both adds those messages to the thread history and ingests those messa…”
   • [claimed-docs] “Here's how to delete an edge from a graph: client.graph.edge.delete(uuid_="your_edge_uuid")”
   • [probe] “official CLI documented at https://help.getzep.com/zepctl-cli”

   proof ↗⚑ flag

   Letta✓fullclaimed8/10

   Letta explicitly documents a headless CLI mode built for non-interactive use in scripts, CI/CD pipelines, and UNIX tool composition, plus an --ephemeral one-shot flag for stateless CI runs, backed by a self-hostable App Server and SDK for programmatic/automated invocation. Missing for 10: independent hands-on CI usage report or example CI config demonstrating it in practice.

   • [claimed-docs] “Headless mode allows you to run Letta Code non-interactively, making it easy to integrate into scripts, CI/CD pipelines, or compose with oth…”
   • [claimed-docs] “Headless mode allows you to run Letta Code non-interactively, making it easy to integrate into scripts, CI/CD pipelines, or compose with oth…”
   • [claimed-docs] “Use --ephemeral to run a one-shot task without creating, resuming, or persisting an agent, memory blocks, or MemFS”
   • [claimed-docs] “Headless mode allows you to run Letta Code non-interactively, making it easy to integrate into scripts, CI/CD pipelines”
   • [claimed-docs] “The Letta App Server lets you deploy the Letta agent harness as a service.”
   • [claimed-docs] “Use `backend: "cloud"` to host agents in Letta Cloud and run tools in managed cloud environments.”

   proof ↗⚑ flag

3. ai-native userUse an official CLI

   weight 2 · round drawn
   Zep✓fullprobed8/10

   Zep documents zepctl, an official CLI providing full administrative access to Zep projects (managing users, threads, Context Graphs, and data operations from the terminal), independently confirmed via probe at the documented URL. missing for 10: no independent/community hands-on usage reports of zepctl, and no detail on installation/distribution channels.

   • [claimed-docs] “zepctl is a command-line interface for administering Zep projects. It provides full access to Zep, enabling you to manage users, threads, Co…”
   • [claimed-docs] “`zepctl` is a command-line interface for administering Zep projects. It provides full access to Zep, enabling you to manage users, threads, …”
   • [claimed-docs] “zepctl is a command-line interface for administering Zep projects.”
   • [probe] “official CLI documented at https://help.getzep.com/zepctl-cli”

   proof ↗⚑ flag

   Letta✓fullprobed8/10

   Letta ships an official CLI ("Letta Code") with rich documentation covering interactive slash commands, headless/non-interactive scripting mode for CI/CD, local/self-hosted mode, and cloud sync — confirmed by first-party docs and a probe hitting the CLI docs page directly. Community evidence (HN threads with Letta employees) corroborates real usage of CLI features like --yolo mode and /init, indicating it's a genuine, actively used tool. Missing for 10: independent hands-on reviews specifically benchmarking the CLI's reliability/UX beyond employee-provided community threads, and no direct evidence of install/version metrics or broad third-party adoption data.

   • [claimed-docs] “Headless mode allows you to run Letta Code non-interactively, making it easy to integrate into scripts, CI/CD pipelines, or compose with oth…”
   • [claimed-docs] “Headless mode allows you to run Letta Code non-interactively, making it easy to integrate into scripts, CI/CD pipelines, or compose with oth…”
   • [claimed-docs] “This default local mode stores agent state on your machine and does not require a Letta login.”
   • [claimed-docs] “Control Letta Code's behavior during an interactive session with slash commands.”
   • [claimed-docs] “Run /login to back up your agents to the cloud and make them available through chat.letta.com, the desktop app, remote machines, and messagi…”
   • [probe] “official CLI documented at https://docs.letta.com/platform/cli”
   • [community] “Letta Code described by Letta's Charles as 'memory-first': agents build learned context over time via tools like /init and /remember; a /cle…”
   • [community] “User asks if Letta Code supports a 'yolo'/dangerous-skip-permissions mode like Claude Code; Letta's Charles confirms: 'Letta Code also has a…”
   • [community] “Letta employee describes Letta Code: 'a fundamentally different approach to coding agents with a focus on continual learning and statefulnes…”

   proof ↗⚑ flag

4. ai-native userDrive the product through a documented public API

   weight 3 · round to Zep
   Zep✓fullprobed9/10

   Zep documents a full REST/SDK API (thread.add_messages, graph.edge.delete, get_user_context, etc.) with quick-start guides, plus a CLI (zepctl) providing full programmatic/administrative access, and runtime probes confirm live endpoints (MCP server, docs API) behaving as documented. This is clearly a documented, agent-drivable public API surface with concrete code examples and independent runtime verification. Missing for 10: no independent third-party API reference/benchmark beyond vendor docs and probes.

   • [claimed-docs] “Add agent memory to your app in three lines of code.”
   • [claimed-docs] “zepctl is a command-line interface for administering Zep projects. It provides full access to Zep, enabling you to manage users, threads, Co…”
   • [claimed-docs] “You can request the Context Block directly in the response to the thread.add_messages() call.”
   • [claimed-docs] “Chat messages are added to threads using `thread.add_messages`, which both adds those messages to the thread history and ingests those messa…”
   • [claimed-docs] “Here's how to delete an edge from a graph: client.graph.edge.delete(uuid_="your_edge_uuid")”
   • [probe] “official CLI documented at https://help.getzep.com/zepctl-cli”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the Memory MCP endpoint https://api.getzep.com/mcp returned HTTP 40…”
   • [claimed-docs] “`zepctl` is a command-line interface for administering Zep projects. It provides full access to Zep, enabling you to manage users, threads, …”

   proof ↗⚑ flag

   Letta✓fullprobed8/10

   Letta ships a fully documented Agent SDK (TypeScript, with browser/mobile variants) covering agent creation, sessions, streaming, permissions, MCP tool proxying, and an OpenAI-compatible App Server API, all backed by extensive first-party docs and a public GitHub SDK repo. This clearly satisfies driving the product programmatically as an AI-native user via a documented public API. missing for 10: a discoverable formal OpenAPI/swagger spec (probe found only 404s), and independent/hands-on developer corroboration of the SDK's public API stability.

   • [claimed-docs] “Create a stateful agent with `client.createAgent()`. It returns the agent's ID as a string, which you use to open sessions and manage the ag…”
   • [claimed-docs] “The stream yields typed events for the whole turn — reasoning, assistant text, tool calls, tool results”
   • [claimed-docs] “a **`canUseTool` callback** that approves, denies, or edits individual tool calls — interactively, if your application involves a human.”
   • [claimed-docs] “`resumeSession(id)` accepts either kind of ID: pass an `agent-xxx` ID to resume the agent's default conversation, or a `conv-xxx` ID to resu…”
   • [claimed-docs] “The stream yields typed events for the whole turn — reasoning, assistant text, tool calls, tool results — and terminates after the turn's `r…”
   • [claimed-docs] “The SDK supports local stdio processes, Streamable HTTP, and legacy SSE”
   • [claimed-docs] “MCP tools are discovered from servers supplied through `mcpServers` and proxied by the SDK host to the MCP server.”
   • [claimed-docs] “Send a message with `session.send()` and read the response with `session.stream()`. The stream yields typed events for the whole turn”
   • [claimed-docs] “use the App Server’s OpenAI-compatible API with clients such as Open WebUI, or the ACP adapter with ACP clients such as Zed”
   • [github] “the Letta Agent SDK for building agents into TypeScript applications”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.letta.com/openapi.json, https://docs.letta.com/swagger.json, https://docs.letta.com/api…”

   proof ↗⚑ flag

5. ai-native userIssue scoped/least-privilege API credentials for an agent

   weight 2 · round to Zep
   Zep✓fullprobed8/10

   Zep explicitly documents ABAC policies attached to API keys and UserGroups to enable least-privilege access for agents and Memory MCP users (zep-docs-25, zep-docs-38, zep-docs-31, zep-docs-12), and this is corroborated at runtime by the Memory MCP endpoint being IdP-gated with OAuth-protected-resource metadata rather than open access (zep-probe-rt-2). Missing for 10: no worked example showing granular scope definitions (e.g. per-thread or per-action policy syntax) or independent third-party audit of the ABAC enforcement.

   • [claimed-docs] “Use policies when you need least-privilege access to context for agents and Memory MCP users.”
   • [claimed-docs] “Limit which actions and context each agent and Memory MCP user can reach with ABAC policies attached to API keys and UserGroups.”
   • [claimed-docs] “Role-based access control governs dashboard users. Attribute-based access control governs API keys and UserGroups.”
   • [claimed-docs] “Attribute-based access control governs API keys and UserGroups.”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the Memory MCP endpoint https://api.getzep.com/mcp returned HTTP 40…”

   proof ↗⚑ flag

   Letta~partialclaimed4/10

   Letta offers some related scoped-access mechanisms — Secrets that let an agent use API keys/tokens without ever seeing the raw values, GitHub integration where agents can only access repositories explicitly granted, and tool/action permission modes (unrestricted/acceptEdits/standard, canUseTool callback) — but there's no documented mechanism for issuing a distinct, scoped/least-privilege API credential specifically for an agent's own access to the Letta API or external services. missing for 10: explicit per-agent API key/credential issuance with configurable scopes, documentation of least-privilege credential lifecycle (rotation/revocation), and independent confirmation of scoped-credential behavior in practice.

   • [claimed-docs] “Secrets let your agent use sensitive values like API keys and tokens in shell commands without ever seeing the actual values.”
   • [claimed-docs] “Once connected, agents in your Letta org running on cloud computers will be able to clone, read, and push to the repositories you grant acce…”
   • [claimed-docs] “Press `Shift+Tab` in interactive mode to cycle through `unrestricted`, `acceptEdits`, and `standard`.”
   • [claimed-docs] “a **`canUseTool` callback** that approves, denies, or edits individual tool calls — interactively, if your application involves a human.”
   • [claimed-docs] “You can share agents across your team (and set permissions for access) to create custom agents for your organization”

   proof ↗⚑ flag

6. ai-native userBuild against official SDKs

   weight 2 · round to Letta
   Zep~partialprobed6/10

   Docs show official SDK method calls (client.graph.edge.delete, thread.add_messages, three-line quick start) and a real pip-installable OSS engine (graphiti-core) confirmed via probe, indicating an official Python SDK/client exists and works. However, there's no explicit documentation of the full SDK ecosystem (e.g., multiple language SDKs, GitHub repo for the client library itself) or independent developer corroboration of building against these SDKs. Missing for 10: explicit multi-language SDK listing, dedicated SDK GitHub repo evidence, third-party developer confirmation of SDK integration.

   • [claimed-docs] “Add agent memory to your app in three lines of code.”
   • [claimed-docs] “Add agent memory to your app in three lines of code. This Zep quick start covers users, threads, ingesting data, and retrieving a Context Bl…”
   • [claimed-docs] “Chat messages are added to threads using `thread.add_messages`, which both adds those messages to the thread history and ingests those messa…”
   • [claimed-docs] “Here's how to delete an edge from a graph: client.graph.edge.delete(uuid_="your_edge_uuid")”
   • [claimed-docs] “You can request the Context Block directly in the response to the `thread.add_messages()` call. This optimization eliminates the need for a …”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi graphiti-core 0.30.1 (Graphiti, Zep's open-source temporal knowledge-graph engine, 30.6k GitHub st…”

   proof ↗⚑ flag

   Letta✓fullclaimed8/10

   Letta ships an official Agent SDK (letta-ai/letta on GitHub, TypeScript) with extensive first-party documentation covering agent creation, sessions, streaming, MCP tool integration, permissions/canUseTool callbacks, cloud/self-host deployment options, and demo apps for browser, Expo/React Native, and desktop clients - a substantial, well-documented SDK surface for AI-native builders. Missing for 10: independent hands-on developer reviews/benchmarks of the SDK itself, and evidence of official SDKs beyond TypeScript (e.g., Python).

   • [github] “the Letta Agent SDK for building agents into TypeScript applications”
   • [claimed-docs] “Create a stateful agent with `client.createAgent()`. It returns the agent's ID as a string, which you use to open sessions and manage the ag…”
   • [claimed-docs] “The stream yields typed events for the whole turn — reasoning, assistant text, tool calls, tool results”
   • [claimed-docs] “a **`canUseTool` callback** that approves, denies, or edits individual tool calls — interactively, if your application involves a human.”
   • [claimed-docs] “Browser, Expo, and React Native applications must import the SDK from `@letta-ai/letta-agent-sdk/client`”
   • [claimed-docs] “Use `backend: "cloud"` to host agents in Letta Cloud and run tools in managed cloud environments.”
   • [claimed-docs] “`resumeSession(id)` accepts either kind of ID: pass an `agent-xxx` ID to resume the agent's default conversation, or a `conv-xxx` ID to resu…”
   • [claimed-docs] “The SDK supports local stdio processes, Streamable HTTP, and legacy SSE”
   • [claimed-docs] “MCP tools are discovered from servers supplied through `mcpServers` and proxied by the SDK host to the MCP server.”
   • [claimed-docs] “Send a message with `session.send()` and read the response with `session.stream()`. The stream yields typed events for the whole turn”
   • [claimed-docs] “a complete Expo and React Native client for persistent Letta agents”

   proof ↗⚑ flag

7. ai-native userSubscribe to events via webhooks

   weight 2 · round drawn
   Zep—none0/10

   No evidence in the pack mentions webhooks or event subscription mechanisms; Zep's docs cover MCP servers, CLI, SDKs, and API access but nothing about outbound event notifications or webhook subscriptions.

   ⚑ flag

   Letta—none0/10

   The evidence pack covers agent SDK streaming, subagents, mods, scheduled tasks, and Slack/GitHub integrations, but there is no mention of a webhook subscription mechanism for events (e.g., agent state changes, task completion) that external systems could subscribe to. Streaming responses (letta-docs-43, letta-docs-60) are pull-based within an active session, not webhook push notifications.

   ⚑ flag

Agentic features

1. ai-native userGet AI-generated insights and suggestions from my data inside the product

   weight 2 · round to Letta
   Zep~partialclaimed5/10

   Zep does generate AI-derived artifacts from ingested data — thread summaries (natural-language summaries incrementally updated by Zep) and entity/relation extraction into the Context Graph — which qualify as AI-generated insights from user data. However, there is no evidence of proactive 'suggestions' or recommendation-style outputs, nor any user-facing dashboard/UI surfacing these insights to an end user; the artifacts are primarily consumed programmatically by the calling agent/app rather than presented as insights inside a product experience. Missing for 10: evidence of suggestion/recommendation generation, and evidence of a UI or interface that surfaces these AI-generated insights directly to a human/ai-native user.

   • [claimed-docs] “A thread summary is a natural-language summary of the messages in a single thread, generated and incrementally updated by Zep.”
   • [claimed-docs] “Zep builds a Context Graph from chat messages, business data, documents, and JSON.”
   • [claimed-docs] “Custom Entity/Edge Types | Feature allowing use of Pydantic-like classes to customize creation/retrieval of entities and relations in the Co…”
   • [claimed-docs] “Zep graph search combines semantic similarity with BM25 full-text search.”
   • [claimed-docs] “set scope="auto" and let Zep dynamically compose the most relevant context across edges, nodes, episodes, observations, and thread summaries…”

   proof ↗⚑ flag

   Letta~partialcommunity6/10

   Letta's 'dreaming' feature uses background subagents to review conversation history and consolidate learned lessons into memory without interrupting the user, and /doctor audits memory placement/duplication — both are forms of AI-generated insight derived from the agent's own interaction data. However, these are framed as internal memory-maintenance mechanisms rather than explicit user-facing 'suggestions' or dashboards surfacing insights about the user's data, and there is no independent/hands-on evidence of the quality of these generated insights. Missing for 10: user-facing insight/suggestion UI or reports, independent verification that dreaming produces useful proactive suggestions, and examples of insights derived from arbitrary connected data sources (not just conversation history).

   • [claimed-docs] “Dreaming uses background subagents to review recent conversations, consolidate useful lessons, and update memory without interrupting your a…”
   • [claimed-docs] “trigger controls when dreaming runs: after a number of steps, on context compaction, or never.”
   • [claimed-docs] “If the memory hierarchy has drifted or grown too large, run /doctor to audit placement, duplication, and system-prompt token usage.”
   • [claimed-docs] “You can also teach it explicitly with /remember: > /remember always use pnpm in this repo”
   • [community] “Letta Code described by Letta's Charles as 'memory-first': agents build learned context over time via tools like /init and /remember; a /cle…”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the [MemFS Search mod]”

   proof ↗⚑ flag

2. ai-native userSet up automations that run autonomously in the background

   weight 2 · round to Letta
   Zep—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

   Letta✓fullcommunity8/10

   Letta explicitly supports autonomous background automation: scheduled/recurring prompts via 'Scheduled tasks' (letta-docs-69), always-on cloud agents that keep running when your laptop is closed (letta-docs-55, letta-docs-68), Slack integration where mentioned agents respond using their own cloud computer (letta-docs-73, letta-docs-83), and background 'dreaming' subagents that consolidate memory without interrupting active work (letta-docs-27, letta-docs-3). Community posts corroborate the always-on, memory-first agent design (letta-comm-12, letta-comm-5). Missing for 10: independent hands-on verification that scheduled/background automations run reliably in practice, and more detail on failure handling/monitoring of unattended runs.

   • [claimed-docs] “Scheduled tasks allow you to automate prompts to your agent on a recurring schedule or at a specific time in the future.”
   • [claimed-docs] “Agents can still be messaged even if your laptop is closed, since they have their own computer to run on.”
   • [claimed-docs] “Agents can still be messaged even if your laptop is closed, since they have their own computer to run on. Combining agents with Slack and sc…”
   • [claimed-docs] “Agents connected to Slack are always-on: mention them in a channel or thread and they'll respond, using their cloud computer to do real work…”
   • [claimed-docs] “Agents connected to Slack are always-on: mention them in a channel or thread and they'll respond”
   • [claimed-docs] “Dreaming uses background subagents to review recent conversations, consolidate useful lessons, and update memory without interrupting your a…”
   • [claimed-docs] “trigger controls when dreaming runs: after a number of steps, on context compaction, or never.”
   • [community] “Letta employee describes Letta Code: 'a fundamentally different approach to coding agents with a focus on continual learning and statefulnes…”

   proof ↗⚑ flag

3. ai-native userDelegate tasks to a built-in AI assistant inside the product

   weight 3 · round to Letta
   Zep—none0/10

   The axis applies to this product kind (peer products hold positive or none verdicts on this story), so lack of evidence for an applicable capability is "none", never "na". (na/none harmonized at arena bring-up — see pipeline/scripts/na-harmonize.ts.)

   ⚑ flag

   Letta✓fullcommunity8/10

   Letta ships a built-in agent runtime (Letta Code / Letta App Server) to which users can delegate real tasks: subagents for parallelized work, autonomous cloud agents that clone/read/push GitHub repos and open PRs, Slack-integrated always-on agents, and scheduled/recurring task automation, all corroborated by first-party docs and HN threads from Letta's own team describing the 'memory-first,' continually-learning agent design. missing for 10: independent hands-on benchmarks or third-party reviews validating real-world task delegation reliability beyond vendor docs and founder HN comments.

   • [claimed-docs] “By delegating work to focused subagents, your main agent can keep its context window clean and make use of parallelism to divide and conquer…”
   • [claimed-docs] “Subagents are specialized agents that your main agent can spawn to handle complex tasks autonomously.”
   • [claimed-docs] “Once connected, agents in your Letta org running on cloud computers will be able to clone, read, and push to the repositories you grant acce…”
   • [claimed-docs] “agents running on cloud computers can access your repositories to do work — sweeping bug reports, opening pull requests, or exploring your c…”
   • [claimed-docs] “Agents can still be messaged even if your laptop is closed, since they have their own computer to run on. Combining agents with Slack and sc…”
   • [claimed-docs] “Agents connected to Slack are always-on: mention them in a channel or thread and they'll respond, using their cloud computer to do real work…”
   • [claimed-docs] “Scheduled tasks allow you to automate prompts to your agent on a recurring schedule or at a specific time in the future.”
   • [community] “Letta employee describes Letta Code: 'a fundamentally different approach to coding agents with a focus on continual learning and statefulnes…”
   • [community] “Letta Code described by Letta's Charles as 'memory-first': agents build learned context over time via tools like /init and /remember; a /cle…”

   proof ↗⚑ flag

4. ai-native userOperate the product with natural-language commands

   weight 2 · round to Letta
   Zep~partialprobed5/10

   Zep exposes an MCP server so agents/AI clients can invoke memory operations via natural-language-driven tool calls, and the Memory MCP endpoint is confirmed live/IdP-gated in a runtime probe, which supports natural-language operation through an AI assistant. However, the primary administrative surface is zepctl, a traditional structured CLI, not a natural-language command interface, and there is no documented natural-language command parser or chat-driven admin console. missing for 10: evidence of natural-language command parsing/execution outside MCP-client pass-through, first-party NL command examples, independent hands-on confirmation of NL-driven operation.

   • [claimed-docs] “The Memory MCP Server lets an end user connect an MCP client (Claude, ChatGPT, Claude Code, Codex, Cursor, and others) to their own agent me…”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the Memory MCP endpoint https://api.getzep.com/mcp returned HTTP 40…”
   • [claimed-docs] “zepctl is a command-line interface for administering Zep projects. It provides full access to Zep, enabling you to manage users, threads, Co…”
   • [claimed-docs] “`zepctl` is a command-line interface for administering Zep projects. It provides full access to Zep, enabling you to manage users, threads, …”

   proof ↗⚑ flag

   Letta✓fullcommunity8/10

   Letta Code and CLI are extensively controlled via natural-language chat plus slash commands (/remember, /init, /fork, /btw, /doctor, /agents) that wrap agent behavior in conversational form, and agents themselves interpret natural-language instructions to manage memory, tools, and subagents. missing for 10: independent hands-on evidence (beyond vendor docs/HN Q&A) confirming reliability of natural-language command interpretation across complex workflows.

   • [claimed-docs] “Control Letta Code's behavior during an interactive session with slash commands.”
   • [claimed-docs] “Run `/init` to bootstrap or refresh memory for the current project. The agent inspects the repository, asks about your working style when ne…”
   • [claimed-docs] “You can also teach it explicitly with `/remember`: `> /remember always use pnpm in this repo`. The agent decides where the lesson belongs an…”
   • [claimed-docs] “You can also teach it explicitly with /remember: > /remember always use pnpm in this repo”
   • [claimed-docs] “If the memory hierarchy has drifted or grown too large, run /doctor to audit placement, duplication, and system-prompt token usage.”
   • [claimed-docs] “Use /fork to branch the current conversation, including its in-context history. The new conversation can take a different direction without …”
   • [claimed-docs] “Use /btw for a side question. It forks the conversation in the background and returns the answer without interrupting your main task”
   • [community] “Letta Code described by Letta's Charles as 'memory-first': agents build learned context over time via tools like /init and /remember; a /cle…”
   • [community] “User asks if Letta Code supports a 'yolo'/dangerous-skip-permissions mode like Claude Code; Letta's Charles confirms: 'Letta Code also has a…”

   proof ↗⚑ flag

Api quality

1. ai-native userExplore an interactive API reference with runnable examples

   weight 2 · round drawn
   Zep—none0/10

   Zep's docs pages show static code snippets (e.g., zep-docs-21, zep-docs-29) but there is no evidence of an interactive, runnable API reference (e.g., embedded sandbox, 'try it' console, Postman/Swagger integration) anywhere in the evidence pack.

   ⚑ flag

   Letta—none0/10

   The evidence pack shows extensive SDK documentation (agent-sdk pages) but no interactive API reference or runnable-example explorer; a direct probe for OpenAPI/Swagger endpoints returned 404 on all candidate paths, indicating no such interactive reference exists.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.letta.com/openapi.json, https://docs.letta.com/swagger.json, https://docs.letta.com/api…”

   proof ↗⚑ flag

2. ai-native userDownload a machine-readable API spec (OpenAPI or equivalent)

   weight 2 · round drawn
   Zep—none0/10

   Zep is an API-first service with SDKs, a CLI (zepctl), and MCP servers, so a downloadable OpenAPI spec would be a natural artifact — but no evidence pack item mentions an OpenAPI/Swagger spec, API reference export, or machine-readable schema file being available for download.

   ⚑ flag

   Letta—none0/10

   Direct probes for OpenAPI/Swagger specs at all standard paths (docs.letta.com/openapi.json, swagger.json, api/openapi.json, .well-known/openapi.json) returned 404, and no documentation elsewhere in the evidence pack references a downloadable machine-readable API spec.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.letta.com/openapi.json, https://docs.letta.com/swagger.json, https://docs.letta.com/api…”

   proof ↗⚑ flag

3. ai-native userTest against a sandbox environment without touching production data

   weight 1 · round to Letta
   Zep—none0/10

   No evidence of a dedicated sandbox/staging environment, test API keys, or non-production project mode for Zep's memory platform; the only related items are deployment options (Cloud vs VPC) and the standalone open-source Graphiti library, neither of which document a sandbox for testing against production-like data safely.

   ⚑ flag

   Letta~partialclaimed4/10

   Letta offers isolation-oriented options that could support sandbox-style testing: an --ephemeral CLI mode that runs one-shot tasks without persisting any agent, memory, or MemFS state (letta-docs-14), and a managed 'cloud sandbox' for running agents with no local setup (letta-docs-16, letta-docs-33, letta-docs-58). However, there is no explicit documentation of a staging/production environment separation or guarantee that testing won't touch production agent data. Missing for 10: explicit staging-vs-production environment concept, documentation of data isolation guarantees, and any evidence/testimonial confirming this workflow is used for safe testing.

   • [claimed-docs] “Use --ephemeral to run a one-shot task without creating, resuming, or persisting an agent, memory blocks, or MemFS”
   • [claimed-docs] “The desktop app and CLI can use your existing API keys and coding plans, while the web app runs agents in a cloud sandbox with no local setu…”
   • [claimed-docs] “The web app uses a managed cloud sandbox by default, so you can start without configuring a computer.”
   • [claimed-docs] “the web app runs agents in a cloud sandbox with no local setup”

   proof ↗⚑ flag

4. ai-native userRely on versioned APIs with a documented deprecation policy

   weight 2 · round drawn
   Zep—none0/10

   The evidence pack covers Zep's features (memory, graph, MCP, CLI) but contains no mention of API versioning scheme or a documented deprecation policy for breaking changes. This is a fair axis for an API-first product, but no evidence supports it.

   ⚑ flag

   Letta—none0/10

   No evidence anywhere in the pack of API versioning schemes or a documented deprecation policy; the openapi.json probe even 404s, indicating no discoverable API spec that would carry version/deprecation info.

   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.letta.com/openapi.json, https://docs.letta.com/swagger.json, https://docs.letta.com/api…”

   proof ↗⚑ flag

1. ai-native userPerform bulk operations across many items at once

   weight 2 · round drawn
   Zep~partialclaimed4/10

   Zep supports some bulk-style operations — deleting a User removes all associated threads/artifacts in one API call, and the zepctl CLI gives scriptable full access to manage users, threads, and graphs from the terminal — but there is no documented batch/bulk create-ingest API for processing many items in a single call. missing for 10: explicit bulk ingest/create endpoints, batch size limits or throughput guidance, and independent evidence of large-scale bulk operations being exercised in practice.

   • [claimed-docs] “Deleting a User will delete all Threads and thread artifacts associated with that User with a single API call, making it easy to handle Righ…”
   • [claimed-docs] “This single-operation approach makes it simple to handle Right To Be Forgotten (RTBF) requests and comply with privacy regulations.”
   • [claimed-docs] “zepctl is a command-line interface for administering Zep projects. It provides full access to Zep, enabling you to manage users, threads, Co…”
   • [claimed-docs] “`zepctl` is a command-line interface for administering Zep projects. It provides full access to Zep, enabling you to manage users, threads, …”

   proof ↗⚑ flag

   Letta~partialclaimed4/10

   Letta offers some multi-item automation primitives — subagents for parallel task delegation ('divide and conquer tasks') and a cross-agent `/search` command that queries across all agents at once — which support limited bulk-style operations, but there is no documented bulk API for batch-creating, editing, importing/exporting, or mass-updating agents, memory blocks, or files. Missing for 10: explicit bulk CRUD/batch APIs, bulk import/export tooling, and evidence of large-scale multi-item operations beyond parallel subagent delegation and cross-agent search.

   • [claimed-docs] “By delegating work to focused subagents, your main agent can keep its context window clean and make use of parallelism to divide and conquer…”
   • [claimed-docs] “Subagents are specialized agents that your main agent can spawn to handle complex tasks autonomously.”
   • [claimed-docs] “`/search [query]` | Search messages across agents”
   • [claimed-docs] “Up to 20 stateful agents”
   • [claimed-docs] “With a developer plan, you can create an unlimited number of agents but pay usage-based pricing.”

   proof ↗⚑ flag

2. ai-native userDefine rules that trigger actions automatically on events

   weight 3 · round to Letta
   Zep—none0/10

   Zep is a memory/context-graph layer with search, retrieval, MCP access, and governance policies, but no evidence describes a rules engine or event-trigger mechanism that automatically fires actions on defined conditions/events. Governance features (ABAC/RBAC) restrict access rather than trigger automated actions.

   ⚑ flag

   Letta~partialclaimed6/10

   Letta documents several trigger-based automation mechanisms: scheduled tasks that 'automate prompts to your agent on a recurring schedule or at a specific time' (letta-docs-69), mods that add 'event hooks' taking effect immediately (letta-docs-71), dreaming triggers configurable to fire 'after a number of steps, on context compaction, or never' (letta-docs-3/12), and Slack-connected agents that automatically respond when mentioned (letta-docs-73/83). These collectively show event- and time-based automatic action triggers, though there is no single documented general-purpose 'rules engine' where a user defines arbitrary event→action conditions in one place. Missing for 10: a unified rules/automation API or UI for arbitrary custom event conditions, and independent/hands-on confirmation that these triggers reliably fire as documented.

   • [claimed-docs] “Scheduled tasks allow you to automate prompts to your agent on a recurring schedule or at a specific time in the future.”
   • [claimed-docs] “Mods let your agent modify its own runtime — adding tools, commands, event hooks, permission policies, and provider adapters that take effec…”
   • [claimed-docs] “trigger controls when dreaming runs: after a number of steps, on context compaction, or never.”
   • [claimed-docs] “Agents connected to Slack are always-on: mention them in a channel or thread and they'll respond, using their cloud computer to do real work…”
   • [claimed-docs] “Agents connected to Slack are always-on: mention them in a channel or thread and they'll respond”
   • [claimed-docs] “trigger controls when dreaming runs: after a number of steps, on context compaction, or never”

   proof ↗⚑ flag

Forgetting

1. platform-engineerDelete a user's memories on demand — single memory, per-entity, or full erasure — to satisfy privacy requirements

   weight 2 · round to Zep
   Zep✓fullclaimed9/10

   Zep documents multiple granularities of deletion: single edge deletion via client.graph.edge.delete (with cascading edge deletion when a node is deleted), and full user erasure via a single API call that deletes all threads and graph data, explicitly framed as satisfying Right To Be Forgotten/privacy regulation requirements. This covers single-memory, per-entity (node/edge), and full-erasure use cases directly relevant to a platform-engineer's data-lifecycle needs. Missing for 10: no independent/hands-on confirmation of deletion behavior beyond first-party docs, and no explicit documented API for bulk per-entity-type deletion across a graph (only node/edge-level and full-user-level are shown).

   • [claimed-docs] “Deleting a User will delete all Threads and thread artifacts associated with that User with a single API call, making it easy to handle Righ…”
   • [claimed-docs] “This single-operation approach makes it simple to handle Right To Be Forgotten (RTBF) requests and comply with privacy regulations.”
   • [claimed-docs] “Deleting a node will also delete all edges connected to that node.”
   • [claimed-docs] “Here's how to delete an edge from a graph: client.graph.edge.delete(uuid_="your_edge_uuid")”
   • [claimed-docs] “Here's how to delete an edge from a graph: ... client.graph.edge.delete(uuid_="your_edge_uuid")”

   proof ↗⚑ flag

   Letta—none0/10

   The evidence pack covers extensive memory creation, editing, auditing (/doctor), and consolidation (/remember, /init, dreaming) but never documents an explicit delete/erase memory command, API, or per-entity/full-wipe capability for privacy compliance. The one related command, /clear, explicitly 'resets the message buffer but keeps learned context' (letta-comm-5), meaning conversation clearing does not erase underlying memory — the opposite of the requested erasure guarantee.

   • [community] “Letta Code described by Letta's Charles as 'memory-first': agents build learned context over time via tools like /init and /remember; a /cle…”
   • [claimed-docs] “If the memory hierarchy has drifted or grown too large, run /doctor to audit placement, duplication, and system-prompt token usage.”
   • [claimed-docs] “You can also teach it explicitly with `/remember`: `> /remember always use pnpm in this repo`. The agent decides where the lesson belongs an…”

   proof ↗⚑ flag

2. developerMake memories expire or decay — via TTL, expiration dates, or recency weighting — so stale facts stop surfacing

   weight 2 · round to Zep
   Zep~partialclaimed4/10

   Zep's underlying Graphiti engine is a temporal knowledge graph that tracks how facts change over time, which implies automatic invalidation of superseded facts as new information arrives — a decay-like mechanism rather than explicit TTL. However, no evidence describes explicit TTL settings, expiration dates, or recency-weighted retrieval; only manual node/edge deletion APIs are documented for removing stale data. Missing for 10: explicit TTL/expiration-date configuration, documented recency-weighting in search/ranking, and confirmation that fact invalidation actually suppresses surfacing of stale facts in retrieval.

   • [github] “Graphiti's context graphs track how facts change over time, maintain provenance to source data, and support both prescribed and learned onto…”
   • [github] “Graphiti is a framework for building and querying temporal context graphs for AI agents. Unlike static knowledge graphs, Graphiti's context …”
   • [claimed-docs] “Deleting a node will also delete all edges connected to that node.”
   • [claimed-docs] “Here's how to delete an edge from a graph: client.graph.edge.delete(uuid_="your_edge_uuid")”
   • [claimed-docs] “Zep graph search combines semantic similarity with BM25 full-text search.”

   proof ↗⚑ flag

   Letta—none0/10

   The evidence describes Letta's memory system (dreaming/consolidation, /doctor audits, MemFS search, shared memory) but nowhere documents TTL, expiration dates, or recency-weighted decay causing stale facts to stop surfacing; /doctor flags drift/duplication but is manual, not automatic decay.

   • [claimed-docs] “trigger controls when dreaming runs: after a number of steps, on context compaction, or never.”
   • [claimed-docs] “If the memory hierarchy has drifted or grown too large, run /doctor to audit placement, duplication, and system-prompt token usage.”
   • [claimed-docs] “Dreaming uses background subagents to review recent conversations, consolidate useful lessons, and update memory without interrupting your a…”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the MemFS Search mod”

   proof ↗⚑ flag

Portability

1. platform-engineerExport memories in a machine-readable format so the memory store is portable and not a lock-in trap

   weight 2 · round to Letta
   Zep~partialprobed4/10

   Zep exposes a graph API (JSON data model, node/edge delete by UUID) and a zepctl CLI described as having 'full access... to manage users, threads, Context Graphs, and data operations from the terminal,' which implies some programmatic read/write access to graph data, and the underlying Graphiti engine is open-source and pip-installable, reducing lock-in. However, no evidence explicitly documents a bulk 'export' command, a defined export file format, or a documented data-portability workflow for moving a full Context Graph out of Zep. Missing for 10: an explicit export/dump feature or format, documentation confirming zepctl or API supports full-graph export, and independent verification of successful export/import round-trips.

   • [claimed-docs] “zepctl is a command-line interface for administering Zep projects. It provides full access to Zep, enabling you to manage users, threads, Co…”
   • [claimed-docs] “`zepctl` is a command-line interface for administering Zep projects. It provides full access to Zep, enabling you to manage users, threads, …”
   • [claimed-docs] “JSON, text, and message | Data types that you can ingest into a Context Graph.”
   • [claimed-docs] “Here's how to delete an edge from a graph: client.graph.edge.delete(uuid_="your_edge_uuid")”
   • [claimed-docs] “Here's how to delete an edge from a graph: ... client.graph.edge.delete(uuid_="your_edge_uuid")”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi graphiti-core 0.30.1 (Graphiti, Zep's open-source temporal knowledge-graph engine, 30.6k GitHub st…”
   • [claimed-docs] “Graphiti is the open-source temporal knowledge graph framework. Use it to build and query a single Context Graph per subject locally”

   proof ↗⚑ flag

   Letta~partialclaimed5/10

   Letta's memory is stored as plain Markdown files in a Git-backed 'memory repository' (MemFS), and self-hosting keeps all agent state on-device with no proprietary lock-in, which implies inherent portability of memory data. However, there is no explicit documented export/import command, no described export file format spec, or CLI subcommand dedicated to exporting memories for migration — the pack only shows cloud backup/sync features, not a data-portability workflow. missing for 10: an explicit export/import command or documented export format, third-party confirmation that memory can be extracted and reloaded elsewhere, guidance on migrating memory out of Letta Cloud.

   • [claimed-docs] “Pass memory for full control over the agent’s starting memory. Each entry becomes a Markdown file in the agent’s memory repository”
   • [claimed-docs] “Shared memory gives multiple agents access to the same files and working context. Each shared memory repository is a Git repository hosted i…”
   • [claimed-docs] “Letta agents can run entirely on infrastructure you control”
   • [claimed-docs] “all agent state, including messages, memory, and provider connections, stays on-device, and no Letta account is required.”
   • [claimed-docs] “Run /login to back up your agents to the cloud and make them available through chat.letta.com, the desktop app, remote machines, and messagi…”

   proof ↗⚑ flag

Self host

1. developerRun the memory layer fully locally — embedded in-process or against local models — without any cloud dependency

   weight 1 · round to Letta
   Zep~partialprobed5/10

   Zep's open-source engine Graphiti can be pip-installed and imported with no API key and used to build/query a Context Graph 'locally' in-process, giving a genuine local path, but the main Zep memory service (the one with threads, users, MCP server, zepctl) is documented as deployed via Cloud or 'Your VPC', not as an embeddable local-only runtime, and there is no evidence of configuring it against local LLMs/embedding models. Missing for 10: explicit support for local/self-hosted LLM or embedding models, confirmation that the full Zep memory layer (not just Graphiti) can run fully offline/in-process, and any docs on air-gapped or no-cloud-dependency operation.

   • [claimed-docs] “Graphiti is the open-source temporal knowledge graph framework. Use it to build and query a single Context Graph per subject locally”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi graphiti-core 0.30.1 (Graphiti, Zep's open-source temporal knowledge-graph engine, 30.6k GitHub st…”
   • [claimed-docs] “Deploy anywhere CloudYour VPC”
   • [github] “Graphiti is a framework for building and querying temporal context graphs for AI agents. Unlike static knowledge graphs, Graphiti's context …”

   proof ↗⚑ flag

   Letta~partialclaimed6/10

   Docs clearly describe a local-only mode where the CLI/agent state, memory, and provider connections all stay on-device with no Letta account or cloud backup required, and a self-hosted App Server that lets agents run entirely on infrastructure you control. However, the evidence never confirms running against fully local models (e.g., local LLM inference/embeddings) — 'model-agnostic' language points to swapping API-based providers (Claude, GPT, Gemini) rather than in-process/local-model operation, and other docs emphasize cloud sandboxes and server-hosted agents as the default path. Missing for 10: explicit support/documentation for local model backends (e.g., Ollama/local embeddings), and confirmation that memory/embedding search (MemFS Search) works without any external API calls.

   • [claimed-docs] “This default local mode stores agent state on your machine and does not require a Letta login.”
   • [claimed-docs] “all agent state, including messages, memory, and provider connections, stays on-device, and no Letta account is required.”
   • [claimed-docs] “Letta agents can run entirely on infrastructure you control”
   • [claimed-docs] “Self-hosted App Server: Run the Letta App Server to host local agents on a central, always-on machine and expose them to client applications…”
   • [claimed-docs] “Letta agents are **model-agnostic**. Agents can use Claude, GPT, Gemini, or other supported models. Users can change an agent's underlying m…”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the [MemFS Search mod]”

   proof ↗⚑ flag

2. platform-engineerSelf-host the memory layer from open-source code (e.g. via Docker) on infrastructure I control

   weight 3 · round to Letta
   Zep~partialprobed4/10

   Zep explicitly offers a 'Your VPC' deployment option alongside cloud (zep-docs-17), and its underlying open-source engine Graphiti is pip-installable and runs locally with no API key (zep-probe-rt-3), showing some self-hostable OSS components exist. However, there is no Docker Compose file, self-hosting guide, or explicit instructions for standing up the full commercial 'memory layer' (threads/users/MCP server) on infrastructure a platform engineer controls — the evidence conflates the OSS Graphiti library with the hosted Zep service. Missing for 10: explicit Docker/self-host deployment docs for the full Zep memory service, confirmation that the main product (not just Graphiti) is open-source, and any hands-on report of a successful self-hosted deployment.

   • [claimed-docs] “Deploy anywhere CloudYour VPC”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi graphiti-core 0.30.1 (Graphiti, Zep's open-source temporal knowledge-graph engine, 30.6k GitHub st…”
   • [github] “Graphiti's context graphs track how facts change over time, maintain provenance to source data, and support both prescribed and learned onto…”

   proof ↗⚑ flag

   Letta~partialprobed6/10

   Letta explicitly documents a self-hosting path: 'Letta agents can run entirely on infrastructure you control' via the self-hosted App Server, with a local mode that stores state on-device and requires no cloud account, and the agent harness is described as fully open source. However, the evidence pack contains no explicit Docker instructions, container image, or hands-on/community confirmation that self-hosting works in practice. Missing for 10: Docker/container deployment specifics, independent verification of the self-host setup succeeding.

   • [claimed-docs] “Letta agents can run entirely on infrastructure you control”
   • [claimed-docs] “Self-hosted App Server: Run the Letta App Server to host local agents on a central, always-on machine and expose them to client applications…”
   • [claimed-docs] “The Letta App Server lets you deploy the Letta agent harness as a service.”
   • [claimed-docs] “Run the Letta App Server to host local agents on a central, always-on machine and expose them to client applications”
   • [claimed-docs] “all agent state, including messages, memory, and provider connections, stays on-device, and no Letta account is required.”
   • [claimed-docs] “This default local mode stores agent state on your machine and does not require a Letta login.”
   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.letta.com/llms.txt # Letta > Letta agents learn from experience and improve with use. The Letta ag…”

   proof ↗⚑ flag

Knowledge graph

1. ml-engineerStore memories as a knowledge graph of entities and relationships so multi-hop and entity-centric questions are answerable

   weight 3 · round to Zep
   Zep✓fullprobed9/10

   Zep explicitly builds a Context Graph (temporal knowledge graph via Graphiti) from chat/business data, with entities and relationships, custom entity/edge types, edge/node deletion, and hybrid semantic+BM25 search across nodes/edges/episodes enabling multi-hop and entity-centric retrieval; the underlying Graphiti engine is verified installable and open-source with independent GitHub corroboration. Missing for 10: no explicit hands-on multi-hop query benchmark or third-party validation of entity-centric answer accuracy beyond vendor docs.

   • [claimed-docs] “JSON, text, and message | Data types that you can ingest into a Context Graph.”
   • [claimed-docs] “Feature allowing use of Pydantic-like classes to customize creation/retrieval of entities and relations in the Context Graph.”
   • [claimed-docs] “Zep graph search combines semantic similarity with BM25 full-text search.”
   • [claimed-docs] “set scope="auto" and let Zep dynamically compose the most relevant context across edges, nodes, episodes, observations, and thread summaries…”
   • [claimed-docs] “Deleting a node will also delete all edges connected to that node.”
   • [claimed-docs] “Zep builds a Context Graph from chat messages, business data, documents, and JSON.”
   • [github] “Graphiti's context graphs track how facts change over time, maintain provenance to source data, and support both prescribed and learned onto…”
   • [github] “Graphiti is a framework for building and querying temporal context graphs for AI agents. Unlike static knowledge graphs, Graphiti's context …”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi graphiti-core 0.30.1 (Graphiti, Zep's open-source temporal knowledge-graph engine, 30.6k GitHub st…”

   proof ↗⚑ flag

   Letta—none0/10

   Letta's documented memory system is file/text-based (MemFS, memory blocks) with optional keyword/semantic/hybrid search (letta-docs-28, letta-docs-41), not a graph of entities and relationships; a Letta employee explicitly frames its approach as 'primarily text/files based' as an alternative to structured memory graphs (letta-comm-10). No evidence describes entity/relationship modeling or multi-hop graph traversal for memory.

   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the [MemFS Search mod]”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the MemFS Search mod”
   • [community] “Letta's Wooders responds to memory skepticism: 'the problem with ChatGPT / other RAG-based memory solutions is that it's not possible to col…”

   proof ↗⚑ flag

2. ml-engineerTrack when facts became valid or invalid (temporal reasoning) so the memory distinguishes current from outdated information

   weight 2 · round to Zep
   Zep✓fullprobed8/10

   Zep's Context Graph is built on Graphiti, explicitly described as a temporal knowledge graph that 'tracks how facts change over time' and maintains provenance, directly matching the ml-engineer's need to distinguish current vs outdated facts; this is corroborated by GitHub docs, the Zep team's own explanation of moving from static fact extraction to a temporal graph to fix hallucination/staleness issues, and a runtime probe confirming the open-source Graphiti engine is real and installable. Missing for 10: explicit documentation of valid_at/invalid_at timestamp fields or an API example showing querying facts as of a point in time, and independent hands-on verification of temporal query accuracy.

   • [github] “Graphiti's context graphs track how facts change over time, maintain provenance to source data, and support both prescribed and learned onto…”
   • [github] “Graphiti is a framework for building and querying temporal context graphs for AI agents. Unlike static knowledge graphs, Graphiti's context …”
   • [github] “Graphiti is a framework for building and querying temporal context graphs for AI agents.”
   • [community] “Zep team explained prior approach: LLM extracted 'facts' from chat history via RAG pipeline, but reconciling facts from complex conversation…”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi graphiti-core 0.30.1 (Graphiti, Zep's open-source temporal knowledge-graph engine, 30.6k GitHub st…”
   • [claimed-docs] “Graphiti is the open-source temporal knowledge graph framework. Use it to build and query a single Context Graph per subject locally”

   proof ↗⚑ flag

   Letta—none0/10

   Letta's documented memory model is file/block-based (MemFS, memory blocks, dreaming/consolidation) with no evidence of tracking fact validity intervals, timestamps of when facts became true/false, or any temporal-graph reasoning distinguishing current vs outdated facts. The axis is applicable to a memory-focused product, but nothing in the evidence pack shows Letta supports temporal fact versioning.

   ⚑ flag

Schema customization

1. ml-engineerCustomize the memory schema — entity types, edge types, or ontology — to match my domain

   weight 1 · round to Zep
   Zep✓fullclaimed8/10

   Zep explicitly supports Custom Entity/Edge Types via Pydantic-like classes to customize creation/retrieval of entities and relations in the Context Graph (zep-docs-3, zep-docs-33), and the underlying Graphiti engine documents support for both prescribed and learned ontology (zep-gh-1). This directly matches the ML-engineer story of customizing entity/edge types to fit a domain. Missing for 10: no independent hands-on example showing a full custom ontology in production use, and no detail on limits/complexity of ontology customization beyond entity/edge type classes.

   • [claimed-docs] “Feature allowing use of Pydantic-like classes to customize creation/retrieval of entities and relations in the Context Graph.”
   • [claimed-docs] “Custom Entity/Edge Types | Feature allowing use of Pydantic-like classes to customize creation/retrieval of entities and relations in the Co…”
   • [github] “Graphiti's context graphs track how facts change over time, maintain provenance to source data, and support both prescribed and learned onto…”

   proof ↗⚑ flag

   Letta—none0/10

   Letta's documented memory model is block/file-based (Markdown memory blocks, MemFS, persona/human blocks) rather than a graph model with configurable entity types, edge types, or ontology; no evidence pack item describes schema customization of entities/relationships. missing for 10: any documentation of graph-based memory representation, entity/edge type definitions, or ontology configuration APIs.

   • [claimed-docs] “Pass memory for full control over the agent’s starting memory. Each entry becomes a Markdown file in the agent’s memory repository”
   • [claimed-docs] “`persona` and `human` are conveniences that set the agent's persona and human memory”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the [MemFS Search mod]”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the MemFS Search mod”

   proof ↗⚑ flag

Agent memory

1. ai-native userMy agent can manage its own memory mid-conversation — adding, searching, updating, and deleting memories through tools or API calls it invokes itself

   weight 3 · round to Letta
   Zep~partialprobed7/10

   Zep exposes API/SDK calls the agent (or an MCP-connected agent) can invoke itself to add memories (thread.add_messages), search the graph (semantic+BM25, scope=auto), and delete edges/nodes (graph.edge.delete), and the Memory MCP Server is confirmed live and functioning for end-user agent memory access (zep-docs-6, zep-probe-2, zep-probe-rt-2). However, no explicit 'update' operation on existing memories is documented (only add/delete), and the MCP server is gated behind OAuth per the runtime probe, meaning autonomous mid-conversation self-service isn't fully frictionless. Missing for 10: an explicit update-memory API/tool, and clearer evidence the agent (not just external MCP clients) autonomously triggers add/search/delete without human-configured auth setup.

   • [claimed-docs] “The Memory MCP Server lets an end user connect an MCP client (Claude, ChatGPT, Claude Code, Codex, Cursor, and others) to their own agent me…”
   • [claimed-docs] “Chat messages are added to threads using `thread.add_messages`, which both adds those messages to the thread history and ingests those messa…”
   • [claimed-docs] “Here's how to delete an edge from a graph: client.graph.edge.delete(uuid_="your_edge_uuid")”
   • [claimed-docs] “Zep graph search combines semantic similarity with BM25 full-text search.”
   • [claimed-docs] “set scope="auto" and let Zep dynamically compose the most relevant context across edges, nodes, episodes, observations, and thread summaries…”
   • [probe] “official MCP server documented at https://help.getzep.com/memory-mcp-server”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the Memory MCP endpoint https://api.getzep.com/mcp returned HTTP 40…”

   proof ↗⚑ flag

   Letta✓fullcommunity8/10

   Letta's memory system is explicitly designed for agent-driven self-editing: agents can be taught new memories via /remember, search via MemFS keyword/semantic search mods, consolidate/update via background 'dreaming', and audit/prune via /doctor — all invoked by the agent itself mid-conversation, and community evidence corroborates this as Letta's core differentiator ('memory-first', 'fully white box' memory editing). missing for 10: no direct API/tool-call schema example showing explicit add/search/update/delete operations invoked programmatically by the agent (docs describe user-facing slash commands and background dreaming rather than a fully enumerated CRUD tool API), and no independent hands-on test confirming the agent autonomously invokes these without user prompting.

   • [claimed-docs] “You can also teach it explicitly with /remember: > /remember always use pnpm in this repo”
   • [claimed-docs] “You can also teach it explicitly with `/remember`”
   • [claimed-docs] “You can also teach it explicitly with /remember”
   • [claimed-docs] “If the memory hierarchy has drifted or grown too large, run /doctor to audit placement, duplication, and system-prompt token usage.”
   • [claimed-docs] “Dreaming uses background subagents to review recent conversations, consolidate useful lessons, and update memory without interrupting your a…”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the [MemFS Search mod]”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the MemFS Search mod”
   • [community] “Letta Code described by Letta's Charles as 'memory-first': agents build learned context over time via tools like /init and /remember; a /cle…”
   • [community] “Letta's Wooders responds to memory skepticism: 'the problem with ChatGPT / other RAG-based memory solutions is that it's not possible to col…”
   • [community] “Concern raised: 'context poisoning is a real problem that these memory providers only make worse.' Letta's Charles countered that Letta's me…”

   proof ↗⚑ flag

2. ai-native userThe memory layer decides for itself what is worth remembering — extracting salient facts from raw conversation and consolidating them in the background

   weight 2 · round drawn
   Zep✓fullcommunity8/10

   Zep automatically ingests raw chat messages into a temporal knowledge graph (zep-docs-20, zep-gh-1/2/3), extracting facts/entities/relations without manual tagging, and separately generates and incrementally updates natural-language thread summaries in the background (zep-docs-8). Community commentary confirms this is a deliberate design evolution from naive LLM fact-extraction toward a more robust temporal graph approach for salience and consolidation (zep-comm-1). Missing for 10: independent benchmark or hands-on evidence of extraction accuracy/salience quality, and detail on exactly how 'importance' is scored versus simply ingesting everything.

   • [claimed-docs] “Chat messages are added to threads using `thread.add_messages`, which both adds those messages to the thread history and ingests those messa…”
   • [claimed-docs] “A thread summary is a natural-language summary of the messages in a single thread, generated and incrementally updated by Zep.”
   • [github] “Graphiti's context graphs track how facts change over time, maintain provenance to source data, and support both prescribed and learned onto…”
   • [github] “Graphiti is a framework for building and querying temporal context graphs for AI agents. Unlike static knowledge graphs, Graphiti's context …”
   • [community] “Zep team explained prior approach: LLM extracted 'facts' from chat history via RAG pipeline, but reconciling facts from complex conversation…”
   • [claimed-docs] “Zep builds a Context Graph from chat messages, business data, documents, and JSON.”

   proof ↗⚑ flag

   Letta✓fullcommunity8/10

   Letta's 'dreaming' feature explicitly uses background subagents to review conversations, extract salient lessons, and consolidate/update memory without interrupting active work, and /remember and /init let the agent decide where lessons belong in MemFS autonomously. Community corroboration (Charles/Wooders on HN) confirms this memory-first, self-consolidating design and contrasts it with RAG-based approaches, though skeptics questioned scaling. Missing for 10: independent hands-on benchmarking of extraction quality/accuracy and more detail on the automatic (non-explicit) salience-detection heuristics beyond trigger conditions.

   • [claimed-docs] “Dreaming uses background subagents to review recent conversations, consolidate useful lessons, and update memory without interrupting your a…”
   • [claimed-docs] “Run `/init` to bootstrap or refresh memory for the current project. The agent inspects the repository, asks about your working style when ne…”
   • [claimed-docs] “You can also teach it explicitly with `/remember`: `> /remember always use pnpm in this repo`. The agent decides where the lesson belongs an…”
   • [claimed-docs] “trigger controls when dreaming runs: after a number of steps, on context compaction, or never.”
   • [community] “Letta Code described by Letta's Charles as 'memory-first': agents build learned context over time via tools like /init and /remember; a /cle…”
   • [community] “Letta's Wooders responds to memory skepticism: 'the problem with ChatGPT / other RAG-based memory solutions is that it's not possible to col…”
   • [community] “Concern raised: 'context poisoning is a real problem that these memory providers only make worse.' Letta's Charles countered that Letta's me…”

   proof ↗⚑ flag

Benchmarks

1. ml-engineerSee published memory-quality benchmark results (e.g. LongMemEval, LoCoMo) backing the product's recall-accuracy claims

   weight 2 · round drawn
   Zep—none0/10

   The evidence pack contains no mention of published benchmark results such as LongMemEval or LoCoMo, nor any recall-accuracy benchmark comparison; it only covers performance latency claims (sub-200ms retrieval) and general architecture/feature docs. missing for 10: any citation to LongMemEval/LoCoMo results, any third-party or first-party benchmark study, or recall-accuracy metrics.

   • [claimed-docs] “Zep delivers sub-200ms context retrieval regardless of graph size or number of graphs.”
   • [claimed-docs] “Add agent memory to your app in three lines of code... in under 200ms.”

   proof ↗⚑ flag

   Letta—none0/10

   The evidence pack contains extensive documentation of Letta's memory architecture (MemFS, dreaming, shared memory, etc.) and community discussion of memory quality skepticism/responses, but nowhere cites any published benchmark results (LongMemEval, LoCoMo, or similar) backing recall-accuracy claims. Since memory-recall quality is a core claimed feature of Letta, this axis clearly applies, but no benchmark evidence exists in the pack.

   • [community] “'How does the memory scale (or not!) over time... if there is a memory block limit, is that an issue for self learning over time. Claude as …”
   • [community] “Skeptic comment: 'In my experience, memory is really not that helpful in most cases... Maintaining the memory is a considerable burden... Ch…”
   • [community] “Letta's Wooders responds to memory skepticism: 'the problem with ChatGPT / other RAG-based memory solutions is that it's not possible to col…”
   • [community] “Concern raised: 'context poisoning is a real problem that these memory providers only make worse.' Letta's Charles countered that Letta's me…”
   • [claimed-docs] “Dreaming uses background subagents to review recent conversations, consolidate useful lessons, and update memory without interrupting your a…”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the [MemFS Search mod]”

   proof ↗⚑ flag

Core memory

1. developerAdd memories from conversations and retrieve them later with semantic search, so context persists across sessions

   weight 3 · round to Zep
   Zep✓fullprobed9/10

   Zep's docs directly cover the story: adding messages persists them into a user-level knowledge graph (thread.add_messages), retrieval via semantic + BM25 hybrid graph search with sub-200ms context blocks, and cross-session persistence via Threads/Users tied to a Context Graph. This is corroborated by runtime probes confirming the memory endpoints are live and by community discussion of the underlying Graphiti engine's design rationale. Missing for 10: independent hands-on benchmarking of recall quality/accuracy across long sessions beyond vendor-stated latency claims.

   • [claimed-docs] “Chat messages are added to threads using `thread.add_messages`, which both adds those messages to the thread history and ingests those messa…”
   • [claimed-docs] “Zep graph search combines semantic similarity with BM25 full-text search.”
   • [claimed-docs] “Zep graph search combines semantic similarity with BM25 full-text search. Semantic search finds conceptual matches, and full-text search fin…”
   • [claimed-docs] “set scope="auto" and let Zep dynamically compose the most relevant context across edges, nodes, episodes, observations, and thread summaries…”
   • [claimed-docs] “Zep delivers sub-200ms context retrieval regardless of graph size or number of graphs.”
   • [claimed-docs] “Add agent memory to your app in three lines of code. This Zep quick start covers users, threads, ingesting data, and retrieving a Context Bl…”
   • [community] “Zep team explained prior approach: LLM extracted 'facts' from chat history via RAG pipeline, but reconciling facts from complex conversation…”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the Memory MCP endpoint https://api.getzep.com/mcp returned HTTP 40…”

   proof ↗⚑ flag

   Letta~partialcommunity6/10

   Letta clearly supports persistent memory across sessions (MemFS, /remember, stateful agents resumable across sessions/computers) and explicitly documents semantic search as an optional add-on via the 'MemFS Search mod' for keyword/semantic/hybrid search rather than a built-in default. Community commentary confirms the memory system is real and text/file-based but transparency-focused, not natively semantic-first, and some users raise unresolved concerns about memory scaling/self-improvement. missing for 10: first-party evidence that semantic search is enabled by default (not an optional mod), independent hands-on benchmarking of recall/retrieval quality, and clearer documentation of retrieval accuracy at scale.

   • [claimed-docs] “You can return to the same agent across sessions, computers, and interfaces instead of starting from scratch each time.”
   • [claimed-docs] “You can also teach it explicitly with `/remember`: `> /remember always use pnpm in this repo`. The agent decides where the lesson belongs an…”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the [MemFS Search mod]”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the MemFS Search mod”
   • [community] “Letta Code described by Letta's Charles as 'memory-first': agents build learned context over time via tools like /init and /remember; a /cle…”
   • [community] “Letta's Wooders responds to memory skepticism: 'the problem with ChatGPT / other RAG-based memory solutions is that it's not possible to col…”
   • [community] “'How does the memory scale (or not!) over time... if there is a memory block limit, is that an issue for self learning over time. Claude as …”

   proof ↗⚑ flag

2. developerRely on the memory layer to update, supersede, or merge memories when new information contradicts what was stored

   weight 2 · round to Zep
   Zep✓fullcommunity8/10

   Zep's Graphiti-based temporal knowledge graph explicitly tracks how facts change over time and maintains provenance, and Zep's own team describes moving away from naive fact-extraction to a temporal graph specifically to handle contradiction/reconciliation issues, with edge/node deletion APIs supporting explicit correction. missing for 10: no explicit documented API or hands-on example showing automatic detection/supersession of a contradicting fact (e.g., 'address changed from X to Y' auto-invalidating the old edge) or independent benchmark validating merge/supersede accuracy.

   • [github] “Graphiti's context graphs track how facts change over time, maintain provenance to source data, and support both prescribed and learned onto…”
   • [github] “Graphiti is a framework for building and querying temporal context graphs for AI agents. Unlike static knowledge graphs, Graphiti's context …”
   • [community] “Zep team explained prior approach: LLM extracted 'facts' from chat history via RAG pipeline, but reconciling facts from complex conversation…”
   • [claimed-docs] “Here's how to delete an edge from a graph: client.graph.edge.delete(uuid_="your_edge_uuid")”
   • [claimed-docs] “Deleting a node will also delete all edges connected to that node.”
   • [claimed-docs] “Zep graph search combines semantic similarity with BM25 full-text search.”

   proof ↗⚑ flag

   Letta~partialcommunity6/10

   Docs describe mechanisms that plausibly support updating/merging memory — /remember lets the agent 'decide where the lesson belongs and commit the update to MemFS' (letta-docs-66), 'dreaming' subagents 'consolidate useful lessons and update memory' (letta-docs-27), and /doctor audits for 'duplication' and drift in the memory hierarchy (letta-docs-10) — and Letta staff claim memory is transparent/controllable and addresses context-poisoning concerns (letta-comm-10, letta-comm-11). However, none of this is a concrete demonstration of contradiction detection/supersession logic, and a community skeptic raises unresolved doubts about memory scaling and self-correction (letta-comm-8, letta-comm-9) without a hands-on refutation either way. Missing for 10: explicit documentation or example of the system detecting/resolving conflicting facts, independent hands-on verification that stale memories are actually superseded rather than just appended/deduplicated.

   • [claimed-docs] “You can also teach it explicitly with `/remember`: `> /remember always use pnpm in this repo`. The agent decides where the lesson belongs an…”
   • [claimed-docs] “Dreaming uses background subagents to review recent conversations, consolidate useful lessons, and update memory without interrupting your a…”
   • [claimed-docs] “If the memory hierarchy has drifted or grown too large, run /doctor to audit placement, duplication, and system-prompt token usage.”
   • [community] “Letta's Wooders responds to memory skepticism: 'the problem with ChatGPT / other RAG-based memory solutions is that it's not possible to col…”
   • [community] “Concern raised: 'context poisoning is a real problem that these memory providers only make worse.' Letta's Charles countered that Letta's me…”
   • [community] “'How does the memory scale (or not!) over time... if there is a memory block limit, is that an issue for self learning over time. Claude as …”
   • [community] “Skeptic comment: 'In my experience, memory is really not that helpful in most cases... Maintaining the memory is a considerable burden... Ch…”

   proof ↗⚑ flag

Retrieval controls

1. developerSteer retrieval with metadata filters, keyword/hybrid search modes, or reranking instead of accepting a single fixed similarity search

   weight 2 · round to Zep
   Zep~partialclaimed6/10

   Zep's docs confirm hybrid retrieval (semantic similarity + BM25 full-text) and a configurable 'scope' that composes across edges/nodes/episodes/summaries, showing retrieval is not a single fixed similarity search. However, there is no documented support for explicit metadata-based filtering or a dedicated reranking step/model in the search API. Missing for 10: metadata filter parameters in search calls, explicit reranking option/model, independent confirmation of these controls in practice.

   • [claimed-docs] “Zep graph search combines semantic similarity with BM25 full-text search.”
   • [claimed-docs] “Zep graph search combines semantic similarity with BM25 full-text search. Semantic search finds conceptual matches, and full-text search fin…”
   • [claimed-docs] “set scope="auto" and let Zep dynamically compose the most relevant context across edges, nodes, episodes, observations, and thread summaries…”

   proof ↗⚑ flag

   Letta~partialclaimed4/10

   Letta docs mention that installing the 'MemFS Search mod' enables keyword search and optional semantic or hybrid search, showing some retrieval-mode flexibility beyond fixed similarity search, but this requires an add-on rather than being built-in. Missing for 10: explicit metadata-filter support, reranking capability, and any evidence this mod is core/first-class rather than an optional plugin.

   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the [MemFS Search mod]”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the MemFS Search mod”

   proof ↗⚑ flag

1. ai-native userDo everything through the API that I can do in the UI

   weight 2 · round to Zep
   Zep~partialclaimed7/10

   Zep is fundamentally API/SDK-driven (thread.add_messages, graph.edge.delete, etc.) and zepctl/CLI is documented as providing 'full access to Zep, enabling you to manage users, threads, Context Graphs, and data operations from the terminal,' implying strong API/CLI parity with the dashboard UI. However, there is no explicit confirmation that every dashboard-only feature (e.g., RBAC/ABAC policy configuration, analytics views) is fully API-exposed. Missing for 10: explicit parity statement or docs enumerating dashboard-only features and confirming API equivalents, independent verification of full CLI/API coverage.

   • [claimed-docs] “zepctl is a command-line interface for administering Zep projects. It provides full access to Zep, enabling you to manage users, threads, Co…”
   • [claimed-docs] “`zepctl` is a command-line interface for administering Zep projects. It provides full access to Zep, enabling you to manage users, threads, …”
   • [claimed-docs] “zepctl is a command-line interface for administering Zep projects.”
   • [claimed-docs] “Role-based access control governs dashboard users. Attribute-based access control governs API keys and UserGroups.”
   • [claimed-docs] “Limit which actions and context each agent and Memory MCP user can reach with ABAC policies attached to API keys and UserGroups.”
   • [claimed-docs] “Chat messages are added to threads using `thread.add_messages`, which both adds those messages to the thread history and ingests those messa…”
   • [claimed-docs] “Here's how to delete an edge from a graph: client.graph.edge.delete(uuid_="your_edge_uuid")”

   proof ↗⚑ flag

   Letta~partialprobed5/10

   Letta's SDK/App Server exposes agent creation, memory, and OpenAI-compatible API access (letta-docs-1, letta-docs-23, letta-docs-32), suggesting an API-first architecture, but the evidence never confirms that CLI/UI-specific features like /doctor memory audits, /pin, /fork, /btw, or /agents browsing have API equivalents, and a direct probe for a public OpenAPI spec returned 404s across all candidate paths (letta-probe-2), undermining a clean 'everything via API' claim. missing for 10: explicit API endpoints/SDK methods mirroring each UI/CLI convenience command, a discoverable OpenAPI spec confirming full surface parity, and independent confirmation of parity.

   • [claimed-docs] “create an agent once, then resume it from anywhere”
   • [claimed-docs] “use the App Server’s OpenAI-compatible API with clients such as Open WebUI, or the ACP adapter with ACP clients such as Zed”
   • [claimed-docs] “The Letta App Server lets you deploy the Letta agent harness as a service.”
   • [claimed-docs] “Run the Letta App Server to host local agents on a central, always-on machine and expose them to client applications”
   • [probe] “PROBE openapi: all candidate paths 404 (https://docs.letta.com/openapi.json, https://docs.letta.com/swagger.json, https://docs.letta.com/api…”
   • [claimed-docs] “Use /fork to branch the current conversation, including its in-context history.”
   • [claimed-docs] “If the memory hierarchy has drifted or grown too large, run /doctor to audit placement, duplication, and system-prompt token usage.”
   • [claimed-docs] “Use /agents in the CLI to browse and switch agents. Pin an agent with /pin or the favorite control in the desktop app”

   proof ↗⚑ flag

2. ai-native userExport all of my data in open formats and leave

   weight 3 · round to Letta
   Zep~partialprobed4/10

   Zep exposes API/CLI access to graph data (zepctl, client.graph.edge.delete, thread/user APIs) and its underlying Graphiti engine is open-source and locally installable, giving some data portability, but there is no documented bulk 'export all my data' feature or standard export format. missing for 10: explicit data-export/portability feature or format documentation, evidence of a full account-data export tool, independent confirmation of successful full export.

   • [claimed-docs] “zepctl is a command-line interface for administering Zep projects. It provides full access to Zep, enabling you to manage users, threads, Co…”
   • [claimed-docs] “`zepctl` is a command-line interface for administering Zep projects. It provides full access to Zep, enabling you to manage users, threads, …”
   • [claimed-docs] “zepctl is a command-line interface for administering Zep projects.”
   • [claimed-docs] “Here's how to delete an edge from a graph: client.graph.edge.delete(uuid_="your_edge_uuid")”
   • [claimed-docs] “Here's how to delete an edge from a graph: ... client.graph.edge.delete(uuid_="your_edge_uuid")”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi graphiti-core 0.30.1 (Graphiti, Zep's open-source temporal knowledge-graph engine, 30.6k GitHub st…”
   • [claimed-docs] “Deleting a User will delete all Threads and thread artifacts associated with that User with a single API call, making it easy to handle Righ…”

   proof ↗⚑ flag

   Letta~partialcommunity5/10

   Letta's architecture strongly favors openness: memory is stored as plain Markdown/MemFS files (letta-docs-11, letta-docs-66), shared memory is a Git repository (letta-docs-21, letta-docs-29), and self-hosting keeps 'all agent state... on-device' with no cloud account required (letta-docs-74, letta-docs-30), reinforced by community claims of a 'fully white box' text-based memory system (letta-comm-10, letta-comm-11). However, there is no explicit documented 'export my data and delete my account' feature or guarantee that cloud-hosted shared-memory git repos can be freely cloned/exported by the user. Missing for 10: an explicit data-export/account-deletion workflow, documentation on extracting cloud-hosted agent state, and independent verification that self-hosted mode fully replaces all cloud-only features.

   • [claimed-docs] “Pass memory for full control over the agent’s starting memory. Each entry becomes a Markdown file in the agent’s memory repository”
   • [claimed-docs] “Shared memory gives multiple agents access to the same files and working context. Each shared memory repository is a Git repository hosted i…”
   • [claimed-docs] “Shared memory gives multiple agents access to the same files and working context.”
   • [claimed-docs] “Letta agents can run entirely on infrastructure you control”
   • [claimed-docs] “all agent state, including messages, memory, and provider connections, stays on-device, and no Letta account is required.”
   • [community] “Letta's Wooders responds to memory skepticism: 'the problem with ChatGPT / other RAG-based memory solutions is that it's not possible to col…”
   • [community] “Concern raised: 'context poisoning is a real problem that these memory providers only make worse.' Letta's Charles countered that Letta's me…”

   proof ↗⚑ flag

3. ai-native userRead the product's source under an open license

   weight 2 · round to Letta
   Zep~partialprobed5/10

   Zep's underlying temporal knowledge-graph engine, Graphiti, is confirmed open-source on GitHub (30.6k stars) and independently verified as pip-installable with no API key required, satisfying an open-license read for the core Context Graph technology. However, the full Zep product itself (the hosted memory service, zepctl CLI, Memory MCP server) is offered as a commercial Cloud/VPC deployment with no evidence of its own source being under an open license. Missing for 10: confirmation that Zep's actual service/CLI code (not just the Graphiti dependency) is open-licensed, and any repo/license file for Zep itself.

   • [github] “Graphiti's context graphs track how facts change over time, maintain provenance to source data, and support both prescribed and learned onto…”
   • [github] “Graphiti is a framework for building and querying temporal context graphs for AI agents. Unlike static knowledge graphs, Graphiti's context …”
   • [github] “Graphiti is a framework for building and querying temporal context graphs for AI agents.”
   • [claimed-docs] “Graphiti is the open-source temporal knowledge graph framework. Use it to build and query a single Context Graph per subject locally”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi graphiti-core 0.30.1 (Graphiti, Zep's open-source temporal knowledge-graph engine, 30.6k GitHub st…”
   • [claimed-docs] “Deploy anywhere CloudYour VPC”

   proof ↗⚑ flag

   Letta~partialprobed6/10

   Letta's own docs state the agent harness is 'fully open source' and link to a public GitHub repo (letta-ai/letta) for the Agent SDK, giving concrete evidence of source availability. However, the evidence pack never names the actual license (e.g., Apache/MIT) or confirms the full product (App Server, CLI, memory subsystems) is under that same open license rather than just the harness. Missing for 10: explicit license identification/file, confirmation that all core components (not just the agent harness) are open-sourced, and independent corroboration of license terms.

   • [probe] “PROBE llms.txt: HTTP 200 at https://docs.letta.com/llms.txt # Letta > Letta agents learn from experience and improve with use. The Letta ag…”
   • [github] “the Letta Agent SDK for building agents into TypeScript applications”
   • [claimed-docs] “Letta agents can run entirely on infrastructure you control”

   proof ↗⚑ flag

4. ai-native userSelf-host the core product

   weight 3 · round to Letta
   Zep~partialprobed5/10

   Zep offers a 'Your VPC' deployment option alongside Cloud (zep-docs-17), suggesting self-hosting is possible, and the underlying Graphiti temporal knowledge-graph engine that powers Zep's Context Graphs is genuinely open-source and pip-installable with no API key (zep-probe-rt-3, zep-gh-1/2/3). However, the full Zep product (users/threads API, zepctl, governance, Memory MCP server) is only ever described as a commercial Cloud/VPC deployment, not as an openly licensed self-hostable package — VPC deployment likely requires an enterprise contract rather than free self-hosting. Missing for 10: explicit open-source license or self-host instructions for the full Zep service (not just the Graphiti sub-component), pricing/terms for VPC self-hosting, and any community confirmation of successfully self-hosting core Zep.

   • [claimed-docs] “Deploy anywhere CloudYour VPC”
   • [probe] “PROBE runtime (recorded 2026-09-05): pypi graphiti-core 0.30.1 (Graphiti, Zep's open-source temporal knowledge-graph engine, 30.6k GitHub st…”
   • [github] “Graphiti's context graphs track how facts change over time, maintain provenance to source data, and support both prescribed and learned onto…”
   • [github] “Graphiti is a framework for building and querying temporal context graphs for AI agents. Unlike static knowledge graphs, Graphiti's context …”
   • [claimed-docs] “`zepctl` is a command-line interface for administering Zep projects. It provides full access to Zep, enabling you to manage users, threads, …”

   proof ↗⚑ flag

   Letta✓fullclaimed8/10

   Letta's docs explicitly describe self-hosting the core product: running the Letta App Server on infrastructure you control, local CLI mode that stores state on-device without requiring a Letta account, and running agents entirely on infra you control (letta-docs-2, letta-docs-15, letta-docs-30, letta-docs-31, letta-docs-74). This is first-party documentation of a genuine self-host mode, not just cloud-hosted convenience. missing for 10: independent/hands-on verification that self-hosted deployment works as documented, and more detail on infra/ops requirements for self-hosting at scale.

   • [claimed-docs] “Run the Letta App Server to host local agents on a central, always-on machine and expose them to client applications through the Agent SDK”
   • [claimed-docs] “Self-hosted App Server: Run the Letta App Server to host local agents on a central, always-on machine and expose them to client applications…”
   • [claimed-docs] “Letta agents can run entirely on infrastructure you control”
   • [claimed-docs] “This default local mode stores agent state on your machine and does not require a Letta login.”
   • [claimed-docs] “all agent state, including messages, memory, and provider connections, stays on-device, and no Letta account is required.”
   • [claimed-docs] “Run the Letta App Server to host local agents on a central, always-on machine and expose them to client applications”

   proof ↗⚑ flag

Pricing

1. platform-engineerSee published pricing with a free tier and per-unit rates so I can project memory costs before committing

   weight 2 · round to Letta
   Zep—none0/10

   The only pricing-related evidence (zep-docs-17) shows a 'Deploy anywhere / Cloud / Your VPC' snippet from the pricing page but contains no mention of a free tier, per-unit rates, or any cost figures that would let a platform engineer project costs. Missing for 10: published free-tier details, per-unit/usage pricing rates, and any cost calculator or tier breakdown.

   • [claimed-docs] “Deploy anywhere CloudYour VPC”

   proof ↗⚑ flag

   Letta~partialclaimed5/10

   Letta's docs confirm a published pricing page with a free BYOK tier (letta-docs-56), a capped free stateful-agent tier (letta-docs-36) and a usage-based developer plan (letta-docs-49), which addresses the free-tier and general cost-model part of the story. However, no explicit per-unit rate table (e.g., $/agent-hour, $/GB memory, $/token) is shown in the evidence, so a platform engineer cannot precisely project memory costs. Missing for 10: concrete per-unit pricing figures/rate card, and independent confirmation of actual billing granularity for memory usage.

   • [claimed-docs] “Up to 20 stateful agents”
   • [claimed-docs] “With a developer plan, you can create an unlimited number of agents but pay usage-based pricing.”
   • [claimed-docs] “You can get started with Letta for free by using models with BYOK, including coding plans from ChatGPT/Codex, Kimi, and Z.AI.”
   • [claimed-docs] “You can share agents across your team (and set permissions for access) to create custom agents for your organization”

   proof ↗⚑ flag

1. ai-native userChoose where my data is stored (region/residency)

   weight 2 · round to Zep
   Zep~partialclaimed5/10

   Zep's pricing page states 'Deploy anywhere: Cloud, Your VPC' (zep-docs-17), implying customers can self-host in their own VPC and thus control where data resides, and the open-source Graphiti engine can be run entirely locally (zep-docs-15). However, there is no documentation of a region-selection feature within Zep Cloud itself or explicit data-residency guarantees for hosted customers. Missing for 10: explicit region/residency options in the managed cloud offering, documented list of supported regions, and any compliance certification tied to region choice.

   • [claimed-docs] “Deploy anywhere CloudYour VPC”
   • [claimed-docs] “Graphiti is the open-source temporal knowledge graph framework. Use it to build and query a single Context Graph per subject locally”

   proof ↗⚑ flag

   Letta~partialclaimed4/10

   Letta offers self-hosting where 'agents can run entirely on infrastructure you control' and local mode where 'all agent state ... stays on-device' (letta-docs-30, letta-docs-74), giving users a way to control where data physically lives. However, there is no documented region/residency selection feature for the managed Letta Cloud offering itself (e.g., choosing EU vs US data centers). Missing for 10: explicit cloud region/residency picker, compliance certifications (GDPR/data-locality), and any documentation of where Letta Cloud stores data by default.

   • [claimed-docs] “Letta agents can run entirely on infrastructure you control”
   • [claimed-docs] “all agent state, including messages, memory, and provider connections, stays on-device, and no Letta account is required.”
   • [claimed-docs] “Self-hosted App Server: Run the Letta App Server to host local agents on a central, always-on machine and expose them to client applications…”
   • [claimed-docs] “This default local mode stores agent state on your machine and does not require a Letta login.”

   proof ↗⚑ flag

2. ai-native userPrevent my data from being used to train AI models

   weight 3 · round drawn
   Zep—none0/10

   The evidence pack covers RTBF/deletion, access control, and MCP/CLI tooling, but contains no statement about Zep's or its LLM providers' use of customer data for model training, nor any opt-out/no-training guarantee. Since Zep processes conversational data through LLMs, this is a fair privacy-posture question, but no evidence addresses it.

   ⚑ flag

   Letta—none0/10

   No evidence pack item addresses data-training opt-out, a training-data policy, or any privacy controls preventing use of user data for model training; self-hosting is mentioned but not tied to a training-data guarantee.

   ⚑ flag

3. ai-native userControl data retention and deletion

   weight 2 · round to Zep
   Zep✓fullclaimed8/10

   Zep documents user-level deletion (single API call deletes all threads/artifacts) explicitly framed for Right To Be Forgotten/privacy compliance, plus granular deletion of individual edges/nodes and cascading edge deletion. This is corroborated by concrete API examples (client.graph.edge.delete) and admin CLI (zepctl) for data operations. Missing for 10: no documented data-retention policy/TTL configuration or independent third-party audit of deletion completeness.

   • [claimed-docs] “Deleting a User will delete all Threads and thread artifacts associated with that User with a single API call, making it easy to handle Righ…”
   • [claimed-docs] “This single-operation approach makes it simple to handle Right To Be Forgotten (RTBF) requests and comply with privacy regulations.”
   • [claimed-docs] “Deleting a node will also delete all edges connected to that node.”
   • [claimed-docs] “Here's how to delete an edge from a graph: client.graph.edge.delete(uuid_="your_edge_uuid")”
   • [claimed-docs] “Here's how to delete an edge from a graph: ... client.graph.edge.delete(uuid_="your_edge_uuid")”
   • [claimed-docs] “zepctl is a command-line interface for administering Zep projects. It provides full access to Zep, enabling you to manage users, threads, Co…”

   proof ↗⚑ flag

   Letta~partialclaimed5/10

   Letta gives users some retention control: local mode stores state on-device without a cloud login (letta-docs-31), self-hosting keeps 'all agent state, including messages, memory, and provider connections... on-device, and no Letta account is required' (letta-docs-74, letta-docs-30), and --ephemeral mode runs a one-shot task 'without creating, resuming, or persisting an agent, memory blocks, or MemFS' (letta-docs-14). However, there is no documented explicit 'delete agent/data' command, data-retention policy for Letta Cloud, or export/erase workflow. Missing for 10: explicit deletion/erasure command or API, documented cloud data-retention/purge policy, independent confirmation that deletion actually removes cloud-stored data.

   • [claimed-docs] “Use --ephemeral to run a one-shot task without creating, resuming, or persisting an agent, memory blocks, or MemFS”
   • [claimed-docs] “This default local mode stores agent state on your machine and does not require a Letta login.”
   • [claimed-docs] “all agent state, including messages, memory, and provider connections, stays on-device, and no Letta account is required.”
   • [claimed-docs] “Letta agents can run entirely on infrastructure you control”

   proof ↗⚑ flag

4. ai-native userOpt out of telemetry and usage tracking

   weight 2 · round drawn
   Zep—none0/10

   No evidence in the pack addresses telemetry/usage-tracking opt-out settings for Zep itself; the closest topics are RTBF/user-deletion and access control, which do not cover product telemetry opt-out.

   ⚑ flag

   Letta—none0/10

   No evidence pack item addresses telemetry, usage analytics, or an opt-out mechanism; while local self-hosted mode is documented, nothing confirms or denies data collection/telemetry settings.

   ⚑ flag

Latency

1. platform-engineerSee documented retrieval-latency targets or measured numbers (e.g. p50/p95) backing the product's speed claims

   weight 2 · round to Zep
   Zep~partialclaimed5/10

   Zep's docs make an explicit performance claim ('sub-200ms context retrieval regardless of graph size or number of graphs') and reference it in the quick-start guide, giving platform engineers a concrete latency target to plan against. However, there is no documented p50/p95 breakdown, benchmark methodology, or dataset/scale disclosure, and no independent hands-on measurement corroborates the figure. Missing for 10: percentile (p50/p95) breakdowns, benchmark methodology/conditions, and independent/third-party latency verification.

   • [claimed-docs] “Zep delivers sub-200ms context retrieval regardless of graph size or number of graphs.”
   • [claimed-docs] “Add agent memory to your app in three lines of code... in under 200ms.”
   • [claimed-docs] “Add agent memory to your app in three lines of code. This Zep quick start covers users, threads, ingesting data, and retrieving a Context Bl…”
   • [claimed-docs] “You can request the Context Block directly in the response to the thread.add_messages() call.”
   • [claimed-docs] “You can request the Context Block directly in the response to the `thread.add_messages()` call. This optimization eliminates the need for a …”

   proof ↗⚑ flag

   Letta—none0/10

   No evidence pack item mentions retrieval latency, p50/p95, or any measured/documented performance numbers for search or memory retrieval; documentation covers features like MemFS search mods and dreaming but never quantifies speed. missing for 10: any documented latency targets, benchmark numbers, or measured p50/p95 retrieval performance data.

   ⚑ flag

Scale

1. platform-engineerIngest at scale with async or batch processing and check the status of background memory operations

   weight 2 · round drawn
   Zep—none0/10

   The evidence pack describes single-message ingestion (thread.add_messages) and various data types, but nowhere documents a batch/async ingestion API or a way to poll status of background memory-processing jobs. Missing for 10: batch ingestion endpoint, async job submission, job status/polling API.

   ⚑ flag

   Letta—none0/10

   Letta's docs describe per-agent memory consolidation ('dreaming' background subagents, triggers on steps/compaction) but there is no evidence of a bulk/batch ingestion pipeline for scale data loading nor any API/CLI to check the status of background memory jobs, which is what this platform-engineer story requires.

   ⚑ flag

Agent memory

1. ai-native userConnect off-the-shelf assistants (Claude, ChatGPT, Cursor) to the same memory so every tool I use shares what it knows about me

   weight 2 · round to Zep
   Zep✓fullprobed9/10

   Zep documents an official Memory MCP Server explicitly enabling Claude, ChatGPT, Claude Code, Codex, Cursor, and others to connect to a shared end-user memory graph, with a runtime probe confirming the endpoint is live and IdP-gated as documented. This directly satisfies the story of connecting multiple off-the-shelf assistants to shared memory; missing for 10: independent hands-on multi-assistant test showing simultaneous cross-tool memory sharing in practice.

   • [claimed-docs] “The Memory MCP Server lets an end user connect an MCP client (Claude, ChatGPT, Claude Code, Codex, Cursor, and others) to their own agent me…”
   • [claimed-docs] “Use policies when you need least-privilege access to context for agents and Memory MCP users.”
   • [claimed-docs] “Limit which actions and context each agent and Memory MCP user can reach with ABAC policies attached to API keys and UserGroups.”
   • [probe] “official MCP server documented at https://help.getzep.com/memory-mcp-server”
   • [probe] “PROBE runtime (recorded 2026-09-05): keyless JSON-RPC initialize POST to the Memory MCP endpoint https://api.getzep.com/mcp returned HTTP 40…”

   proof ↗⚑ flag

   Letta—none0/10

   Evidence shows Letta integrating with Codex, Claude Code, Hermes Agent, OpenClaw for skills (docs-57), and offering an OpenAI-compatible API for tools like Open WebUI plus an ACP adapter for Zed (docs-23), but nothing shows Letta connecting to or sharing memory with off-the-shelf Claude, ChatGPT, or Cursor specifically. Letta's MCP support is client-side (Letta consuming MCP servers, docs-61/76), and community evidence notes only a third-party 'Letta-MCP-server' built by a user, not an official integration exposing Letta memory to these named assistants.

   • [claimed-docs] “Skills are portable across the Letta app and CLI, Codex, Claude Code, Hermes Agent, OpenClaw, and other compatible agents.”
   • [claimed-docs] “use the App Server’s OpenAI-compatible API with clients such as Open WebUI, or the ACP adapter with ACP clients such as Zed”
   • [claimed-docs] “The SDK supports local stdio processes, Streamable HTTP, and legacy SSE”
   • [claimed-docs] “MCP tools are discovered from servers supplied through `mcpServers` and proxied by the SDK host to the MCP server.”
   • [community] “Charles from Letta on MCP: 'We're working on it... making Letta agents an MCP client, so that you can easily connect to MCP servers as tools…”

   proof ↗⚑ flag

Frameworks

1. developerDrop the memory layer into agent frameworks like LangChain, LangGraph, CrewAI, or the Vercel AI SDK via documented first-party integrations

   weight 2 · round to Zep
   Zep~partialclaimed6/10

   Zep docs explicitly reference 'agent frameworks' integrations (zep-docs-14, zep-docs-24) that persist and retrieve conversation data, and community/HN threads confirm real-world adoption, but the evidence pack never names or links specific first-party integration docs for LangChain, LangGraph, CrewAI, or Vercel AI SDK—only generic 'supported frameworks' language. missing for 10: named/linked docs pages for LangChain, LangGraph, CrewAI, or Vercel AI SDK integrations, and independent confirmation these integrations work as advertised.

   • [claimed-docs] “Zep provides agent memory for agents that you build with supported frameworks.”
   • [claimed-docs] “Zep provides agent memory for agents that you build with supported frameworks. The integrations persist conversation turns and retrieve data…”
   • [claimed-docs] “Add agent memory to your app in three lines of code.”
   • [claimed-docs] “Add agent memory to your app in three lines of code. This Zep quick start covers users, threads, ingesting data, and retrieving a Context Bl…”

   proof ↗⚑ flag

   Letta—none0/10

   The evidence pack covers Letta's own Agent SDK, CLI, App Server, MCP support, and various demo apps, but contains no mention of LangChain, LangGraph, CrewAI, or Vercel AI SDK integrations anywhere in the docs, GitHub, or community sources.

   ⚑ flag

2. developerWire memory into real-time voice pipelines (e.g. LiveKit, Pipecat, ElevenLabs) with documented integrations fast enough for live conversation

   weight 1 · round drawn
   Zep—none0/10

   The evidence pack shows generic 'agent framework' memory integrations, an MCP server, and a CLI, but contains no mention of LiveKit, Pipecat, ElevenLabs, or any real-time voice pipeline integration or documentation. This is an applicable axis for a memory SDK, but no supporting evidence exists.

   ⚑ flag

   Letta—none0/10

   The evidence pack contains no documentation or mention of LiveKit, Pipecat, ElevenLabs, or any real-time voice pipeline integration. The closest evidence (letta-comm-4) is a forum reply suggesting a DIY postgres-backed memory setup for a hypothetical voice use case, not a documented, low-latency SDK integration for live conversation.

   • [community] “User with a father suffering memory loss suggested Letta's framework looked interesting for building a voice-interactive LLM with memory to …”

   proof ↗⚑ flag

Sdks

1. developerBuild against official SDKs in at least Python and TypeScript with equivalent memory APIs

   weight 2 · round to Letta
   Zep~partialclaimed3/10

   Code examples in docs (e.g. client.graph.edge.delete, thread.add_messages()) show a Python-style SDK, but the evidence pack never mentions a TypeScript/JavaScript SDK or confirms parity between two language SDKs. Missing for 10: explicit TypeScript SDK docs/install instructions, side-by-side API parity confirmation, and any independent corroboration of both SDKs.

   • [claimed-docs] “Here's how to delete an edge from a graph: client.graph.edge.delete(uuid_="your_edge_uuid")”
   • [claimed-docs] “Here's how to delete an edge from a graph: ... client.graph.edge.delete(uuid_="your_edge_uuid")”
   • [claimed-docs] “Chat messages are added to threads using `thread.add_messages`, which both adds those messages to the thread history and ingests those messa…”
   • [claimed-docs] “You can request the Context Block directly in the response to the `thread.add_messages()` call. This optimization eliminates the need for a …”
   • [claimed-docs] “You can request the Context Block directly in the response to the thread.add_messages() call. This optimization eliminates the need for a se…”

   proof ↗⚑ flag

   Letta~partialclaimed5/10

   The evidence extensively documents a TypeScript/JavaScript Agent SDK (client.createAgent(), sessions, streaming, MCP, browser/mobile support) but no evidence pack item confirms an official Python SDK or its API parity with the TypeScript SDK — despite Letta's backend being Python-based, docs cited here only reference the JS/TS Agent SDK and CLI. missing for 10: explicit official Python SDK documentation/package, confirmation of equivalent memory API surface (createAgent, sessions, streaming, MCP) across Python and TypeScript, and any independent corroboration of Python SDK usage.

   • [github] “the Letta Agent SDK for building agents into TypeScript applications”
   • [claimed-docs] “Create a stateful agent with `client.createAgent()`. It returns the agent's ID as a string, which you use to open sessions and manage the ag…”
   • [claimed-docs] “`resumeSession(id)` accepts either kind of ID: pass an `agent-xxx` ID to resume the agent's default conversation, or a `conv-xxx` ID to resu…”
   • [claimed-docs] “Send a message with `session.send()` and read the response with `session.stream()`. The stream yields typed events for the whole turn”
   • [claimed-docs] “Browser, Expo, and React Native applications must import the SDK from `@letta-ai/letta-agent-sdk/client`”
   • [claimed-docs] “The SDK supports local stdio processes, Streamable HTTP, and legacy SSE”
   • [claimed-docs] “MCP tools are discovered from servers supplied through `mcpServers` and proxied by the SDK host to the MCP server.”

   proof ↗⚑ flag

Context assembly

1. developerRetrieve a token-budgeted, prompt-ready context block assembled from relevant memories in one call

   weight 2 · round to Zep
   Zep✓fullcommunity8/10

   Zep's docs describe a single-call 'Context Block' — via `scope="auto"` graph search or returned directly from `thread.add_messages()` — that composes relevant edges, nodes, episodes, and summaries into one prompt-ready block, with sub-200ms retrieval regardless of graph size, and prior HN commentary confirms the long-term-state/token-count design intent. missing for 10: explicit documentation of a configurable token budget/limit parameter and independent hands-on benchmarking of the assembled context block's quality.

   • [claimed-docs] “set scope="auto" and let Zep dynamically compose the most relevant context across edges, nodes, episodes, observations, and thread summaries…”
   • [claimed-docs] “You can request the Context Block directly in the response to the thread.add_messages() call.”
   • [claimed-docs] “You can request the Context Block directly in the response to the `thread.add_messages()` call. This optimization eliminates the need for a …”
   • [claimed-docs] “You can request the Context Block directly in the response to the thread.add_messages() call. This optimization eliminates the need for a se…”
   • [claimed-docs] “Zep delivers sub-200ms context retrieval regardless of graph size or number of graphs.”
   • [claimed-docs] “Add agent memory to your app in three lines of code. This Zep quick start covers users, threads, ingesting data, and retrieving a Context Bl…”
   • [community] “Zep team clarified differentiation: 'Many long-term memory services focus on vector search over documents... Zep is different, we focus on t…”

   proof ↗⚑ flag

   Letta—none0/10

   Letta's docs describe memory blocks, MemFS, /doctor token audits, and search mods, but there is no evidence of a single API/CLI call that returns a token-budgeted, prompt-ready assembled context block for developer consumption outside the agent's own turn — memory in Letta is managed internally as part of the agent's system prompt, not exposed as a discrete retrieval endpoint.

   • [claimed-docs] “If the memory hierarchy has drifted or grown too large, run /doctor to audit placement, duplication, and system-prompt token usage.”
   • [claimed-docs] “Pass memory for full control over the agent’s starting memory. Each entry becomes a Markdown file in the agent’s memory repository”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the [MemFS Search mod]”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the MemFS Search mod”
   • [claimed-docs] “`persona` and `human` are conveniences that set the agent's persona and human memory”

   proof ↗⚑ flag

2. developerGet summaries of past sessions or threads so an agent can pick up where the last conversation left off

   weight 3 · round to Zep
   Zep✓fullclaimed9/10

   Zep documents thread summaries that are auto-generated and incrementally updated per thread, plus a Context Block retrievable via thread.add_messages()/get_user_context() combining summaries, edges, nodes, and episodes so an agent can resume with full prior context. This directly supports resuming past sessions/threads with sub-200ms retrieval performance documented. Missing for 10: independent hands-on verification of summary quality/accuracy over long conversations beyond vendor docs.

   • [claimed-docs] “A thread summary is a natural-language summary of the messages in a single thread, generated and incrementally updated by Zep.”
   • [claimed-docs] “set scope="auto" and let Zep dynamically compose the most relevant context across edges, nodes, episodes, observations, and thread summaries…”
   • [claimed-docs] “You can request the Context Block directly in the response to the thread.add_messages() call.”
   • [claimed-docs] “You can request the Context Block directly in the response to the `thread.add_messages()` call. This optimization eliminates the need for a …”
   • [claimed-docs] “You can request the Context Block directly in the response to the thread.add_messages() call. This optimization eliminates the need for a se…”
   • [claimed-docs] “Zep delivers sub-200ms context retrieval regardless of graph size or number of graphs.”
   • [claimed-docs] “Chat messages are added to threads using `thread.add_messages`, which both adds those messages to the thread history and ingests those messa…”

   proof ↗⚑ flag

   Letta✓fullcommunity8/10

   Letta's core architecture is built around persistent, stateful agents that can be resumed across sessions/computers, with explicit APIs (resumeSession, agent IDs) and memory consolidation ('dreaming') that summarizes past conversations into durable memory so agents pick up where they left off. Community/founder commentary (letta-comm-5, letta-comm-12) corroborates the 'memory-first, no more forgetting' design intent. missing for 10: independent hands-on verification of summary/context fidelity over long horizons, and quantitative benchmarks of resume/summarization quality beyond vendor docs.

   • [claimed-docs] “You can return to the same agent across sessions, computers, and interfaces instead of starting from scratch each time.”
   • [claimed-docs] “`resumeSession(id)` accepts either kind of ID: pass an `agent-xxx` ID to resume the agent's default conversation, or a `conv-xxx` ID to resu…”
   • [claimed-docs] “Dreaming uses background subagents to review recent conversations, consolidate useful lessons, and update memory without interrupting your a…”
   • [claimed-docs] “Use /fork to branch the current conversation, including its in-context history.”
   • [claimed-docs] “`/search [query]` | Search messages across agents”
   • [claimed-docs] “Run `/init` to bootstrap or refresh memory for the current project. The agent inspects the repository, asks about your working style when ne…”
   • [claimed-docs] “You can also teach it explicitly with `/remember`: `> /remember always use pnpm in this repo`. The agent decides where the lesson belongs an…”
   • [community] “Letta Code described by Letta's Charles as 'memory-first': agents build learned context over time via tools like /init and /remember; a /cle…”
   • [community] “Letta employee describes Letta Code: 'a fundamentally different approach to coding agents with a focus on continual learning and statefulnes…”

   proof ↗⚑ flag

Ingestion

1. developerIngest documents, JSON, and business data into memory — not just chat transcripts

   weight 2 · round to Zep
   Zep✓fullclaimed8/10

   Docs explicitly state Zep ingests JSON, text, business data, and documents into the Context Graph, not just chat messages (zep-docs-2, zep-docs-27), with API examples for graph operations and custom entity/edge types for structuring business data. Community context confirms the underlying Graphiti engine tracks facts/entities beyond raw transcripts. Missing for 10: no hands-on independent verification of ingesting large documents or complex JSON schemas at scale, and no detail on supported document formats (PDF, etc.).

   • [claimed-docs] “JSON, text, and message | Data types that you can ingest into a Context Graph.”
   • [claimed-docs] “Zep builds a Context Graph from chat messages, business data, documents, and JSON.”
   • [claimed-docs] “Custom Entity/Edge Types | Feature allowing use of Pydantic-like classes to customize creation/retrieval of entities and relations in the Co…”
   • [claimed-docs] “Chat messages are added to threads using `thread.add_messages`, which both adds those messages to the thread history and ingests those messa…”
   • [github] “Graphiti's context graphs track how facts change over time, maintain provenance to source data, and support both prescribed and learned onto…”

   proof ↗⚑ flag

   Letta~partialclaimed5/10

   Letta's memory system (MemFS) is file/markdown-based and the SDK lets developers 'pass memory for full control' where 'each entry becomes a Markdown file' — implying arbitrary content beyond chat transcripts can seed agent memory, and /init lets an agent ingest a repository's files. However, there is no explicit documentation of ingesting structured JSON or generic 'business data' formats into memory, nor any first-party example of a document/JSON ingestion pipeline. Missing for 10: explicit JSON/structured-data ingestion support, document-upload workflows, and independent/hands-on evidence of non-chat data being loaded into agent memory.

   • [claimed-docs] “Pass memory for full control over the agent’s starting memory. Each entry becomes a Markdown file in the agent’s memory repository”
   • [claimed-docs] “Run `/init` to bootstrap or refresh memory for the current project. The agent inspects the repository, asks about your working style when ne…”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the [MemFS Search mod]”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the MemFS Search mod”
   • [claimed-docs] “`persona` and `human` are conveniences that set the agent's persona and human memory”

   proof ↗⚑ flag

2. developerStore images, PDFs, or other files as memory inputs and recall information from them later

   weight 1 · round drawn
   Zep—none0/10

   Zep's documentation explicitly enumerates ingestible data types as JSON, text, and message only (zep-docs-2), with no mention of images, PDFs, or file uploads anywhere in the evidence pack. No feature, API, or integration is described for storing or recalling from binary/file-based memory inputs.

   • [claimed-docs] “JSON, text, and message | Data types that you can ingest into a Context Graph.”
   • [claimed-docs] “Zep builds a Context Graph from chat messages, business data, documents, and JSON.”

   proof ↗⚑ flag

   Letta—none0/10

   Letta's documented memory system (MemFS, memory blocks, /remember, /init) is explicitly text/Markdown-file based, and 'image attachments' appear only in a demo chat app's UI feature list, not as a supported memory-recall input type for PDFs or images. No evidence shows agents can ingest images or PDFs into memory and recall facts from them later.

   • [claimed-docs] “Pass memory for full control over the agent’s starting memory. Each entry becomes a Markdown file in the agent’s memory repository”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the [MemFS Search mod]”
   • [claimed-docs] “For keyword search and optional semantic or hybrid search, install the MemFS Search mod”
   • [claimed-docs] “Use this demo when you want to study a mobile interface with conversation history, streaming, tools, approvals, interruption, queued message…”

   proof ↗⚑ flag

Governance

1. platform-engineerGovern who and what can read or write memory with roles, policies, or access-control lists, and audit that access

   weight 2 · round to Zep
   Zep~partialclaimed6/10

   Zep's governance docs describe RBAC for dashboard users and ABAC policies scoped to API keys and UserGroups, letting admins limit what agents and Memory MCP users can read/write (zep-docs-12, zep-docs-25, zep-docs-31, zep-docs-38). However, there is no documented audit logging/trail capability to verify who accessed what memory. Missing for 10: audit log/trail evidence, granular per-resource ACLs beyond API-key/UserGroup scope, and independent verification of policy enforcement.

   • [claimed-docs] “Attribute-based access control governs API keys and UserGroups.”
   • [claimed-docs] “Use policies when you need least-privilege access to context for agents and Memory MCP users.”
   • [claimed-docs] “Role-based access control governs dashboard users. Attribute-based access control governs API keys and UserGroups.”
   • [claimed-docs] “Limit which actions and context each agent and Memory MCP user can reach with ABAC policies attached to API keys and UserGroups.”

   proof ↗⚑ flag

   Letta~partialclaimed3/10

   Letta's pricing docs mention you can 'share agents across your team (and set permissions for access)' for organizational use, implying some access control exists, but there is no documentation of roles, granular ACLs, or an audit log/trail for memory reads/writes. Missing for 10: documented role/policy definitions for memory access, explicit read/write ACLs on memory blocks, and any audit logging or access history feature.

   • [claimed-docs] “You can share agents across your team (and set permissions for access) to create custom agents for your organization”

   proof ↗⚑ flag

Isolation

1. developerScope memories per user, agent, or application so one tenant's memories never leak into another's retrieval

   weight 3 · round to Zep
   Zep✓fullclaimed7/10

   Zep documents user-scoped Context Graphs (each user gets their own knowledge graph, created explicitly per app user) and thread-level isolation, plus ABAC/RBAC policies that limit which context each agent, API key, UserGroup, or Memory MCP user can reach — directly supporting tenant isolation across users/agents. Deletion is also scoped per user (RTBF), reinforcing that one user's data is a separable unit that can be walled off or purged without affecting others. missing for 10: explicit documentation of 'application'-level tenancy/namespace isolation beyond User/UserGroup scoping, and independent/hands-on verification that cross-tenant retrieval leakage cannot occur.

   • [claimed-docs] “Whenever users are created in your application, you need to trigger the creation of a Zep user.”
   • [claimed-docs] “Chat messages are added to threads using `thread.add_messages`, which both adds those messages to the thread history and ingests those messa…”
   • [claimed-docs] “Deleting a User will delete all Threads and thread artifacts associated with that User with a single API call, making it easy to handle Righ…”
   • [claimed-docs] “This single-operation approach makes it simple to handle Right To Be Forgotten (RTBF) requests and comply with privacy regulations.”
   • [claimed-docs] “Attribute-based access control governs API keys and UserGroups.”
   • [claimed-docs] “Use policies when you need least-privilege access to context for agents and Memory MCP users.”
   • [claimed-docs] “Role-based access control governs dashboard users. Attribute-based access control governs API keys and UserGroups.”
   • [claimed-docs] “Limit which actions and context each agent and Memory MCP user can reach with ABAC policies attached to API keys and UserGroups.”

   proof ↗⚑ flag

   Letta~partialclaimed5/10

   Letta's core model is per-agent stateful memory (each agent has its own memory blocks/files, docs-42, docs-48), and cross-agent sharing is opt-in via explicit 'shared-memory' repos (docs-8, docs-21, docs-29) rather than automatic leakage, plus team/org permission controls exist (docs-87: 'share agents across your team and set permissions for access'). However there is no documentation of explicit multi-tenant isolation guarantees (per-user/per-application namespacing, access-control enforcement details, or audit of leakage prevention) beyond these general mechanisms. Missing for 10: explicit tenant/user isolation model, documented ACL/RBAC enforcement for memory retrieval, and any independent verification that scoping prevents cross-tenant leakage.

   • [claimed-docs] “Create a stateful agent with `client.createAgent()`. It returns the agent's ID as a string, which you use to open sessions and manage the ag…”
   • [claimed-docs] “`persona` and `human` are conveniences that set the agent's persona and human memory”
   • [claimed-docs] “letta shared-memory create --name team-memory”
   • [claimed-docs] “Shared memory gives multiple agents access to the same files and working context. Each shared memory repository is a Git repository hosted i…”
   • [claimed-docs] “Shared memory gives multiple agents access to the same files and working context.”
   • [claimed-docs] “You can share agents across your team (and set permissions for access) to create custom agents for your organization”

   proof ↗⚑ flag

Sharing

1. developerShare selected memory across multiple agents or users (team or group memory) while keeping private memory private

   weight 1 · round to Letta
   Zep~partialclaimed5/10

   Zep's governance model provides ABAC policies attached to API keys and UserGroups to limit which context/actions each agent or MCP user can reach, implying some ability to scope shared vs. private access, and user-level Context Graphs are the default privacy boundary (zep-docs-12, zep-docs-25, zep-docs-31, zep-docs-38). However, there is no explicit documentation of a mechanism for selectively sharing specific memory items across multiple users/agents as a 'team or group memory' feature—only access-control policies, not a shared-graph or memory-merging capability. Missing for 10: explicit team/group memory graph construct, documented API for cross-user memory sharing, and any example/tutorial showing selective sharing while isolating private data.

   • [claimed-docs] “Attribute-based access control governs API keys and UserGroups.”
   • [claimed-docs] “Use policies when you need least-privilege access to context for agents and Memory MCP users.”
   • [claimed-docs] “Role-based access control governs dashboard users. Attribute-based access control governs API keys and UserGroups.”
   • [claimed-docs] “Limit which actions and context each agent and Memory MCP user can reach with ABAC policies attached to API keys and UserGroups.”
   • [claimed-docs] “Whenever users are created in your application, you need to trigger the creation of a Zep user.”
   • [claimed-docs] “Chat messages are added to threads using `thread.add_messages`, which both adds those messages to the thread history and ingests those messa…”

   proof ↗⚑ flag

   Letta✓fullclaimed7/10

   Letta ships a dedicated shared-memory feature (`letta shared-memory create`, Git-backed shared repositories giving multiple agents common context) alongside per-agent private memory blocks, and pricing docs explicitly mention team-level agent sharing with configurable permissions. This directly matches sharing selected memory across agents/users while keeping other memory private. Missing for 10: independent/hands-on corroboration of the privacy boundary enforcement and finer-grained detail on how access permissions are scoped per user vs team.

   • [claimed-docs] “letta shared-memory create --name team-memory”
   • [claimed-docs] “Shared memory gives multiple agents access to the same files and working context. Each shared memory repository is a Git repository hosted i…”
   • [claimed-docs] “Shared memory gives multiple agents access to the same files and working context.”
   • [claimed-docs] “You can share agents across your team (and set permissions for access) to create custom agents for your organization”

   proof ↗⚑ flag