An exploded DRAM memory moduleD5-1D5-2D5-3D5-5D5-6D5-7D5-8DDR5 · 32 GBHEAT SPREADERMEMORY ARRAYDRAM PACKAGEDRAM CHIPGOLD CONTACTSPCB · 8 LAYERS
FIG_001
[ DDR5 memory module ]

Have you ever wondered how ChatGPT knows what a word means? It does not keep a dictionary. Every word is turned into a long list of numbers, and words that mean similar things end up with similar numbers.

Words as points in a three dimensional space, grouped by meaningANIMALSFOODPLACES“CAT”QUERYDIM 1DIM 2DIM 3
FIG_002

Plot those numbers and meaning becomes geography. Animals gather in one corner, food in another, and a question lands right next to its answer. That is most of what “search by meaning” is.

Or maybe you have wondered how a model decides which earlier words matter. When it reads “the cat sat on the mat”, every word scores every word that came before it, and the scores decide where its attention goes.

Attention weights drawn as a grid under a glass plateTHECATSATONTHEMATKEYSQUERIES
FIG_003

The scores are plain multiplication, millions of them at once, which is why this kind of maths runs on graphics cards and not on the processor in your laptop.

Perhaps you have always wanted to know what a “weight” is, the thing models have billions of.

A neuron: two inputs, two weights, a sum and an activationX₁0.9X₂0.4w = 0.20w = 0.80Σz = 0.20σ(z)INPUT z →10OUTPUT = 0.61
FIG_004

It is one number that says how much an input should count. Move it and the output moves. Training is nothing more than nudging billions of them until the answers come out right.

But before any of that can happen, text has to be cut into pieces a model can count. Words are split into tokens, and every token becomes an id.

The word remembering split into three tokens, each with an idremem18604TOKEN 1ber1941TOKEN 2ing278TOKEN 311 LETTERS → 3 TOKENS → 3 NUMBERS
FIG_005

That is why models are oddly bad at counting the letters in a word. They never see the letters, only the pieces, the same way you never see the pixels on this page.

If you have ever wondered about any of this, or it has sparked your curiosity, this manual is for you.

It will not teach you to build an AI product. It is not a tutorial and there is nothing to install. It explains, with a lot of pictures, how the machines you talk to every day actually work, and how they remember.

We build long-term memory for AI assistants, so we spend our days inside these systems. This is what we wish someone had drawn for us when we started.

You do not need to be technical. The diagrams do the heavy lifting; you only need to be curious.

Machine Memory

An exploded GPU package: substrate, interposer, die and memory stacksGPU · 80 GB HBM3INTEGRATED HEAT SPREADERSILICON DIE132 COMPUTE UNITSHBM STACK8 DRAM DIESHBM STACKSILICONINTERPOSERSUBSTRATESOLDER BALLS
FIG_006
[ GPU package ]

New chapters, by email

The manual is written in the open, a chapter at a time. Leave your address and each new one lands in your inbox the day it is published. Nothing else, and one click to leave.

Want to start now? Read the first chapters

Current Progress:

Finished chapters:

16 / 25

Words/chapter:

1.7K

Images/chapter:

2

Current word count:

27.4K

Est word count:

42.8K

Current image count:

36

An M.2 solid state driveCONTROLLERCONTROLLERDRAM CACHENAND FLASH3D NAND232 LAYERSM.2 KEYPCIE 5.0 × 4
FIG_007
[ M.2 solid state drive ]

Common Questions

Common Questions.

In:Is it free?
Out:

Yes. Every chapter is free to read here, with no account and no paywall.

New chapters, by email

The manual is written in the open, a chapter at a time. Leave your address and each new one lands in your inbox the day it is published. Nothing else, and one click to leave.

Want to start now? Read the first chapters

Layers of a transformer block pulled apartEMBEDDINGATTENTIONADD & NORMFEED FORWARDADD & NORMOUTPUTSELF-ATTENTION8 HEADSMLP · 4× WIDTHNEXT LAYERTOKEN VECTORS
FIG_008
[ Transformer block ]