Guides / Long-term memory for agents.

1437 words | Esment

Long-term memory for agents.

Short-term vs long-term memory in AI, the three kinds agents keep, the four ways to add it, and the five parts a memory engine needs to work.

╌╌╌╌

A language model has no memory of its own between calls. Everything an agent seems to know in a session is text in its context window: your instructions, the files it opened, the conversation so far. When the session ends, that text is gone. Long-term memory, also called persistent memory, is whatever puts the right part of it back next time.

Short-term and long-term memory in AI

The terms are borrowed from psychology, and the analogy holds well enough to be useful. Short-term memory is the context window: fast, exact, and limited in size. The model sees all of it at once, and it is emptied when the session ends or replaced by a summary when it fills. Long-term memory lives outside the model, in files or a database, and has no practical size limit, but the model sees none of it until something retrieves a piece and puts it into the window.

Short-term (context window)Long-term (external memory)
Where it livesInside the model’s current inputFiles, a database, a memory engine
SizeFixed by the model, then compactedEffectively unlimited
What the model seesAll of it, every turnOnly what is retrieved
How long it lastsOne sessionUntil it is deleted or archived
Main failureOverflow: old details are summarised awayRetrieval: the right fact is not brought back

That last row is the whole design problem. Long-term memory is easy to store and hard to use: it only helps if the right entries reach the short-term window at the right moment, and nothing else does.

The three kinds of long-term memory

Research on agents usually splits long-term memory the way cognitive science does. The split is practical, because each kind is written and read differently.

  • Semantic memory: facts. “The API uses Postgres.” “The client’s fiscal year starts in April.” Retrieved by meaning when a question touches them.
  • Episodic memory: what happened. “On Tuesday we tried the cache fix and it broke the tests.” Dated, tied to a session, useful for not repeating mistakes.
  • Procedural memory: how to do things. “Run the migrations before the seed.” Often lives in instruction files or skills, because it applies every time.

There are four common ways to do that. They are not rivals; most real setups use two or three together.

What it isGood forBreaks down when
Instruction filesCLAUDE.md, AGENTS.md, rules files read at startConventions that rarely changeKnowledge grows or changes weekly
Built-in assistant memoryThe memory inside ChatGPT or ClaudePersonal preferences in one appYou use more than one assistant or need exact recall
Chat history searchRetrieval over past conversationsFinding what was saidFacts changed since, or the agent never searches
A memory engineA separate store agents read and write over a toolFacts shared across sessions and agentsIt is not connected or the agent is not told to use it

Why files and chat history are not enough

Instruction files are the right home for things a person has reviewed: how the project is built, how tests run. They are read in full every time, so they grow expensive and noisy as they grow. Our comparison of CLAUDE.md and a memory server goes through where each belongs.

Searching old conversations finds what was said, not what is true now. If a decision was reversed last week, both versions are in the history and nothing marks which one won. And search only runs if the agent thinks to search.

What a memory engine has to do

A memory engine is a store built for this job, which agents reach through tools, usually over MCP. Whatever product you choose, five parts decide whether it works.

1. Extraction: decide what to keep

Storing whole transcripts is easy and nearly useless; the useful part of a session is a few facts and decisions. A memory engine turns conversation into small, separate entries, either when the agent calls a save tool or by reading the stream itself. Each entry should carry when it was saved and which agent saved it.

2. Search by meaning

An agent asks “what database do we use?” and the memory says “we moved to Postgres in March”. No words in common. Search has to work on meaning, which is what embeddings are for, usually combined with keyword search and a re-ranking step so exact names are not lost.

3. Versions instead of overwrites

Facts change. An engine that overwrites loses the trail, and one that only appends returns both answers. The useful behaviour is to keep the new fact as current and the old one as history, so the agent gets one answer and you can still see what changed and when.

4. Forgetting

A memory that never forgets fills with stale detail and search gets worse. Entries need states: active, fading, archived, pinned. Our research note on why a memory must be able to forget explains the reasoning.

5. A brief the agent reads first

The most common failure is not bad search but no search: the agent never thinks to look. The fix is to prepare a short brief of what matters, such as the current project, recent decisions and pinned facts, and hand it to the agent before it starts. Search is then for the details.

Rules load into every chat; the chat itself does not carry over; search and a memory server reach back only when the agent uses themCHAT 1CHAT 2CHAT 3RULES,AGENTS.MDLOADED INTO EVERY CHAT, AUTOMATICALLYTHE CHATITSELFCHATSEARCHIF THE AGENT SEARCHESMEMORYSERVERSAVERECALLWHEN THE AGENT CALLS IT · OTHER TOOLS CAN USE IT TOO

How to judge a memory for your agents

  • Can more than one agent use it? Coding agents, chat assistants and your own scripts should see the same memory.
  • Where does it run? On your machine, in your cloud, or a vendor’s. Our guide to local MCP memory servers explains what “local” really covers.
  • Does it measure recall? Ask for results on a public benchmark and the method behind them. Ours, and its limits, are in The Benchmark Is Not the Territory.
  • Can you see and fix what it holds? An agent memory you cannot read is one you cannot trust.
  • What happens when a fact changes? Test it: save, correct, then ask.

Where Esment fits

Esment is our memory engine, built along the lines above, so read this part with that in mind. Agents connect over MCP: Claude Code, Cursor, Codex and other desktop tools in one click from the Mac app, and web assistants through the hosted endpoint. Memories are individual entries with type, tags, source and date; recall combines text search, vector search, a knowledge graph and re-ranking; changes keep their history; and every session starts from a brief. In Claude Code, hooks add relevant memory to every turn automatically. It reaches 98% Recall@20 on LongMemEval-S. How Esment works has the details, and the MCP tools reference lists what an agent can call.

Give your agents a memory →

Common questions

In:

What is long-term memory in AI?

Out:

Information an AI system keeps outside the model, in files or a database, so it can be brought back in later sessions. The model itself does not learn from your conversations; long-term memory is a separate store that retrieves relevant facts into the context window when they are needed.

In:

What is the difference between short-term and long-term memory in AI?

Out:

Short-term memory is the context window: everything the model sees in the current session, limited in size and gone when the session ends. Long-term memory is stored outside the model, lasts across sessions and has no practical size limit, but only helps when the right part is retrieved into the window.

In:

Do AI agents have memory?

Out:

Not on their own. A model only sees what is in its context window for the current call. Memory between sessions comes from something outside the model: instruction files, the app’s built-in memory, chat history search or a separate memory engine.

In:

What is the difference between RAG and agent memory?

Out:

RAG retrieves from documents someone else wrote. Agent memory is written by the agents themselves as they work, and has to handle what documents rarely do: facts that change, entries that go stale, and several agents writing at once.

In:

Is a vector database enough for agent memory?

Out:

It covers search by meaning, which is one of five parts. You still need extraction, versions, forgetting and a way to put the right memories in front of the agent before it answers.

In:

Can different agents share one memory?

Out:

Yes, if they all connect to the same store. With MCP, Claude Code, Cursor, ChatGPT and others can read and write one memory. Our guide to sharing memory between Claude Code and Cursor shows the setup.

Written by the team that builds Esment, so we are not a neutral party. Tools change quickly: check each one before you rely on it.

Get the next chapter by email.

One email per new chapter. Nothing else.

0.00