Nearly every memory system for language agents performs some transformation at write time. Raw conversation goes in; facts, summaries, entities, or relations come out. The transformation is usually justified on storage and precision grounds, and it does deliver both.
It also does something that is discussed much less often. Any write-time transformation is a prediction about which questions will be asked later, committed to before any of them have been. It helps the queries it anticipated. It hurts the queries it did not. And because the transformation happens once, at ingest, a bad prediction is not a ranking problem you can tune away later — it is information that is no longer there.
A live disagreement#
This is not a settled question, and the 2026 literature is unusually direct about disagreeing with itself. AtomMem argues that selectively extracting high-value atomic facts from long-form interactions yields a simple, economical, and strong memory representation. Work published weeks earlier argues, in its title, for going beyond exactly that.AtomMem: Building Simple and Effective Memory System for LLM Agents via Atomic Facts (arXiv:2606.19847) versus Rethinking How to Remember: Beyond Atomic Facts in Lifelong LLM Agent Memory (arXiv:2605.19952). Reading them side by side is the fastest way to see that the field has not converged here.
Both results are credible. We think they are compatible, and that the apparent contradiction dissolves once you stop asking which representation is better and start asking which questions each one is a bet on.
The fragmentation cost#
Consider a session in which someone mentions six things about the same topic — six hobbies, six constraints on a project, six symptoms. Stored raw, those six facts sit inside one retrievable unit. One retrieval event brings all six into context, and their adjacency is preserved for free, because nobody had to decide it was worth preserving.
Atomised, they become six independently indexed items. Now a question like "what are all of her hobbies" is no longer one retrieval. It is a requirement that most of six separate items each rank inside the same top-k budget, competing against everything else in the store. Recall of any single fact improves — that is the real win of extraction, and it is why atomic systems look strong on single-hop questions. Recall of the set degrades, because it is now a conjunction of independent ranking events.
The asymmetry in Fig. 1 and Fig. 2 explains a result that otherwise looks like a paradox: the same extraction pipeline can raise single-hop accuracy and lower aggregation and multi-hop accuracy at the same time. Those are not two measurements of one quality. They are measurements of two different bets.
The opposite failure#
None of this is an argument for storing raw text and nothing else, because raw-only systems fail hard in a way that atomic systems handle well: knowledge updates.
When someone changes jobs, a transcript contains both the old employer and the new one, and both are equally retrievable. Nothing in the raw substrate marks one as superseded. Asked "where does she work," a raw-only system will happily surface a two-year-old answer with the same confidence as last week's, and the reader has no principled basis for choosing. This is precisely the gap that recent work has begun to isolate and train against.Supersede: Diagnosing and Training the Memory-Update Gap in LLM Agents (arXiv:2606.27472). The framing as a distinct diagnosable gap, rather than a retrieval quality issue, is the useful contribution.
So each pole owns a failure mode the other pole handles. Extraction fragments aggregation. Raw storage cannot express supersession. Any system that picks one and calls it an architecture has chosen which of its users to disappoint.
Additivity as the only safe posture#
The resolution we find most defensible is not a compromise between the poles but a constraint on how they combine: derived layers must never replace the substrate they were derived from. Keep raw as ground truth. Add facts, summaries, entity observations, and graph structure as indexes over it. Let retrieval use whichever serves the query.
The argument for this is asymmetric risk rather than average accuracy. A pipeline where extraction replaces raw has an unrecoverable failure mode: when the extractor mispredicts — drops a detail, resolves a pronoun wrongly, silently degrades after a model upgrade — the original is gone and no downstream improvement can recover it. A pipeline where extraction merely adds an index has a bounded failure mode: a bad derived layer costs precision and can be regenerated. One is a bug, the other is data loss.
This also has an underappreciated operational consequence. Extraction quality drifts silently as prompts and models change, and nobody notices, because evaluations exercise retrieval over a frozen corpus and never re-examine the ingest that produced it. If raw is retained, that drift is repairable by re-deriving. If it is not, the drift is permanent and invisible.
Where the intelligence lives#
Underneath the extraction debate is a more general design axis: does the system spend its effort at write time or at read time? Write time has abundant latency budget and no knowledge of the question. Read time has the question and almost no latency budget. Every architecture in this space is a position on that trade.
Write-time synthesis is the most interesting version of the bet, because it is a bet made deliberately rather than incidentally. Pre-computing a per-entity summary turns a multi-hop aggregation question into a single-hop retrieval — the expensive stitching happens where there is time to do it well. The cost is that the synthesis anticipates some questions and not others, and now staleness and regeneration policy become moving parts of their own. It is a real technique with a real bill.Vectorize's write-up on the consolidation problem in agent memory is a good practitioner account of how quickly regeneration policy becomes the hard part.
Which pole is correct depends on the read/write ratio of the workload, which is to say it depends on the product. Per-message extraction cost dominates for chatty users; per-query reranking cost dominates for retrieval-heavy agents. This is a business decision wearing an architecture decision's clothes, and it is usually made implicitly.
What stays open#
We hold the additivity constraint with reasonable confidence, because its failure modes are well attested across independent systems and the asymmetry argument does not depend on any particular benchmark. Several things around it we genuinely do not know.
- Whether derived layers should compete in one ranked list with raw, or be routed to by query type. A shared index avoids a brittle router; separate channels give control but every fusion scheme introduces failure modes of its own.
- How aggressive supersession should be. Marking validity intervals without deleting preserves recoverability and enables "as of" queries, but a store that never discards anything eventually loses precision to accumulation — and detecting genuine contradiction is hard, since "I work at A" and "I work at B" may be an update, a contradiction, or both true.
- Whether rerankers trained on web question answering transfer to personal-memory relevance at all. "Relevant to a query" and "relevant to this person's history" are different notions, and we consider the mismatch an under-examined weakness of most current stacks, ours included.
What we would ask of any memory system, including our own, is simply that the bet be legible: which questions did this pipeline decide were worth anticipating, what did it discard to do so, and can that decision be reversed after the fact. A system that cannot answer those three questions is not neutral about the future. It has just declined to tell you what it wagered.