Long-Term Memory
The model has no memory of its own — every "it remembers me" is your system storing and re-injecting facts. Extraction and conflict are the hard parts, and the payoff is cost.
The model has no cross-session memory at all. Every moment where "it remembers me" happens is your system storing and fetching on its behalf. locomo A memory feature adds no capability to the model; it adds a storage pipeline to your system — so the first question is whether that pipeline is worth having, and where it should sit.
After reading, you should be able to answer:
- Why does the value of a memory system not show up in accuracy?
- What are the stages of storage and retrieval, and where is each one hard?
- When a temporary fact and a lasting one collide, what decides — and what does the whole system cost?
- Which facts belong in a settings field, and which actually earn automatic memory?
The payoff is latency and cost, not accuracy
Judging a memory system by accuracy gets the direction wrong from the start.
One set of benchmarks quantifies this neatly: the dumbest possible setup — cram the entire conversation history into the context — scores higher overall than every dedicated memory system. The full-context baseline reached 72.9, against 66.9 for Mem0 and 68.4 for the graph-structured Mem0g. mem0 On proportion of correct answers, no memory management at all came out ahead.
So where is the value? Latency and cost. In the same evaluation the full-context baseline ran at roughly 17.1 seconds p95, while Mem0 ran at about 1.4 seconds and consumed over 90% fewer tokens. mem0
That trade pays because the cost of the full-context approach keeps climbing with conversation length: the longer a user stays, the slower and more expensive every request, until the window limit is reached. A memory system flattens that curve and gives up a small step in accuracy to do it. Where responses have to come back in about a second, that step is worth giving up; in offline batch work, it may not be.
So the question to ask is not "how much accuracy does memory buy me" but "will I trade a little accuracy for an order of magnitude in latency and cost."
Three stages, and the same mechanism as RAG
Storage and retrieval run in three stages.
One: extraction. Once a conversation wraps up, or while it is still running, a model call makes the judgment: which content will matter again later. Say a user mentions that customer-facing documents are always in English, that customer data cannot leave the country, and that weekly reports are easier to read when split by project.
Two: conflict resolution. Before a new fact gets written, it is checked against what is already stored for disagreement. The user first said their manager was Chen; later they mention Chen moved on and they now report to Lin. Those two cannot both stand, so the job is to update, not to append.
Three: injection. When the next session opens, a few relevant entries are pulled from the store and joined to the system prompt or the opening context. The model reads them and the effect is "it still remembers me."
Step three is worth spelling out: it is the same mechanism as RAG — retrieve relevant content, put it into context. What actually differs is the thing being retrieved: here it is facts about the current user, not passages sliced out of documents. Research like MemGPT makes this loop explicit: the model needs an "operating system" to store and fetch memory in layers, which is also the most common blueprint for production memory features. memgpt
Extraction has no standard to work from
Here is the first hard part: "what is worth remembering" has no objective standard.
Remember too much and the store soon fills with marginal material — a joke tossed off in passing, a request that only ever applied once, all filed away as long-term preferences. From then on they get re-injected every session, and both the spend and the interference with judgment climb. Remember too little and there is no point at all; the user cannot tell any difference.
What works in practice is drawing the categories first: keep identity facts, long-term preferences, and explicit prohibitions, and nothing outside those three. A blanket "remember what matters" is close to guaranteed to drift.
There is also a hazard that is hard to see: memories come from a model, so getting one wrong is always possible. A single bad entry, once written, colors every conversation after it, and it rarely surfaces — the user has no way of knowing what the system recorded behind their back.
Conflict resolution fails on temporary versus lasting
The default rule takes the newer one by timestamp, but there are exceptions — that is the second hard part.
"This time, quote the previous version of the rule" applied only to that one request and should not displace "we go by the current policy by default," a standing setting. By the same token, "Chen has moved on and I report to Lin now" is a lasting change and must displace the old reporting line. Form tells these two new facts apart not at all; one should override and one should not, and the dividing line is duration alone.
A workable approach is to have extraction record the time property as it goes: does this hold only inside the current task, or does it stand long term? Miss that layer at extraction time and no later stage can tell — it can only guess. Confusing the temporary with the lasting is where this stage slips most often. Some research argues for changing the storage model entirely: make "storing a fact" and "acting on that fact" one thing, expressed as executable rules about the user, because splitting storage from action is exactly why contradictions are hard to resolve and rules are hard to honor. user-as-code
What injection costs, and the three bills to add up
The third stage is a question of timing and volume. Fetch too many memories and every session opens carrying a bag of facts that get resent every round, adding a fixed cost. Fetch too few or the wrong ones and the user feels "I told you this already." Injected memories also occupy the window — the window is a budget to allocate, so every slot memory takes is a slot documents lose. So injection has to answer three questions: which memories relate to this session, how many to fetch, and where to put them. The first decides accuracy; the other two decide cost.
Building the whole system adds three more bills to settle:
One: an extra extraction call per conversation. One more model request, billed to you, invisible to the user.
Two: injected content at the start of every conversation. The more memories, the larger that fixed cost.
Three: privacy and compliance. The most easily overlooked. Personal information about users will be kept by you for the long haul, which pulls in your privacy policy, retention periods, and users' right to deletion. Have legal read it before you start.
There is also one product requirement that has to be met: users must be able to view and delete the memories that belong to them. That means a page listing "here is what the AI knows about you," with per-item deletion. It satisfies compliance and it builds trust at the same time — a user who finds out the system has been quietly keeping things behind their back reacts badly.
Most products do not need full memory
Skip general memory for now; start from a small, well-defined set of fields. That is the most cost-saving judgment in the whole topic.
A complete memory system is unnecessary for most products. What you probably need is to remember which language answers come back in, which time zone timestamps display in, and which citation style to follow — and all of those can be explicit settings the user fills in. The result is predictable, editable, free of extraction error and of privacy argument, at close to zero cost.
The test is blunt: information the user would happily volunteer belongs in a setting, not in a guess by the AI.
What genuinely earns automatic memory is the material users will not volunteer but that does affect the experience — how they work, where they tend to slip, the phrasing they prefer.
One boundary still has to be drawn: what lives in memory is facts about the user, not the user's entire chat log. The log is a separate concern, with different storage, retention, and compliance rules. Do not mix them.
This store-and-fetch pipeline is still answering the same underlying question: what goes into the next request. Every block it inserts takes budget and competes for attention — What Actually Goes Into Context breaks that budget down further. And the fact that a model can extract wrongly is a reminder that anything accumulated across sessions needs a way for people to see and correct it, which is the other side of Saving Experience Is Not Learning From It.