What we learned building a temporal knowledge graph for AI memory
Why a flat list of facts fails as AI memory, how a temporal knowledge graph handles contradictions, and the four-stage retrieval cascade behind Esment.
In short
- A flat list of facts can't answer "how does this relate to that?" and can't handle contradiction — so it fails as assistant memory.
- In a graph, memories are grounded in entities (people, projects, tools) and linked by typed relations, so retrieval can expand outward.
- Every memory has a lifecycle; when a new fact contradicts an old one, the old one is archived, not deleted, so the past stays queryable.
- Retrieval is a cascade — keywords, meaning, graph expansion, then a diversity-aware rerank — because no single technique is enough.
The first version of Esment treated memory as a flat list: facts in, facts out, ranked by recency and importance. It worked, and it was wrong. A flat list can't answer the questions that matter — how does this relate to that? — and it can't handle contradiction. This is what we learned replacing it with a temporal knowledge graph, and why the retrieval on top of it is a cascade rather than a single search.
Why a flat list of facts fails§
Facts aren't independent. "Alex prefers dark mode" and "Alex works on the Atlas redesign" are both true, but they're different kinds of truth: one is a preference attached to a person, the other an activity attached to a project. In a list, they're two rows. Ask about the redesign and the list can find rows that mention it — but not the person, the tools or the decisions around it unless they happen to share the same words.
Contradiction is worse. When Alex moves from the Atlas redesign to a new project, a list now holds two facts that can't both be current, and nothing in the data says which one wins.
Memories grounded in entities§
In a graph, memories become nodes grounded in entities, and relations
become typed edges — WORKS_ON, USES, DEPENDS_ON, CREATES. Retrieval
can then do something a keyword search can't: it expands. Ask about a
project and the graph walks out to the people, tools and decisions connected
to it.
From a question to its neighbours
Atlas redesign → Alex → Figma → Ship by March
The point isn't the data structure for its own sake. It's that "what's going on with the redesign?" deserves an answer that includes who, with what and what we decided, even when the question names none of them.
The lifecycle problem§
A memory that can't forget becomes noise. Every store needs a lifecycle, and ours has five states:
A memory's life
Generated → Activated → Merged → Archived → Expired → back to the start
The subtle part is merging: when new information contradicts old information, which one survives? Our reconciler resolves conflicts in time — a newer fact supersedes an older one — but the old fact is never deleted. It's archived, and the commit history keeps exactly what was known at every point.
That turns the store into a time machine. What did we believe about this project in March? is a query, not a guess. It's also why supersession, not decay, is the default: a fact doesn't become less true because it's old, only because something newer replaced it. We worked through that distinction carefully in a memory that cannot forget.
Retrieval is a cascade, not a search§
We ended up with a four-stage pipeline because no single technique is enough:
- Full-text (BM25) — exact words and names, in milliseconds.
- Vectors — meaning, for things worded differently than you stored them.
- Graph expansion — walking the relations out from the best hits; the stage that makes the graph pay for itself.
- Diversity-aware reranking (MMR) — the final order, so the context the assistant gets isn't five near-copies of the same fact.
Each stage covers a blind spot of the one before. Keyword search misses paraphrases; vectors miss exact names and rare terms; neither knows that a person is connected to a project; and without a final rerank the top results tend to repeat each other. If embeddings are new to you, what is an embedding explains the vector step in plain terms.
The retrieval cascade
Full-text · exact words, then Vectors · similar meaning, then Graph · connected facts, then Rerank · diverse top results
Each stage is fast on its own; the cascade is what makes the whole thing feel like a memory rather than a search engine. Since this post was first written, the cascade got faster and multilingual — see retrieval v2.
The lesson§
Building a memory engine is 20% storage and 80% judgment: what to keep, what to merge, what to let die and what to surface. The graph gave us the structure to make those judgments well. The lifecycle gave us the courage to make them at all.
If you want the case for why any of this matters, start with why AI assistants need a memory they can't lose; for where the graph lives, read local-first AI memory. To see what each retrieval stage adds, read what each stage of our retrieval cascade is worth, and for failures that start at write time, why AI memory answers fail.
Questions, answered§
What is a temporal knowledge graph?
A knowledge graph stores facts as entities (people, projects, tools) connected by typed relations. A temporal one also records when each fact was true, so a newer fact can supersede an older one without erasing it, and you can ask what was believed at any point in time.
Why use a knowledge graph for AI memory instead of plain search?
Because facts aren't independent. A graph lets retrieval walk from a project to the people, tools and decisions connected to it, which plain keyword or similarity search can't do. It also gives contradictions a place to be resolved.
How does Esment handle contradictory memories?
A reconciler compares new information with what's stored. When a newer fact contradicts an older one, the newer one becomes current and the older one is archived with its history, so recall stays clean and nothing is silently lost.
Does a knowledge graph make retrieval slower?
Not in practice. Each stage of the cascade is fast on its own and runs on the device; the graph step only expands from a handful of candidates the earlier stages already found.
Pass it on →