2.4
Memory And RAG In Plain Language
Why this exists
Three lessons in, you've got two problems and one lever. It guesses. It decays. Your first prompt is where you push.
None of that solves the underlying thing: it doesn't remember, and clearing a session wipes it completely. So how does anything ever persist? How does a system answer questions about a business it was never trained on?
This is also the lesson that explains our AI setter. It holds conversations about our programme, our pricing, our objections — none of which any model was ever trained on. By the end of this you'll know exactly how, and the acronym everyone throws around will stop being mysterious.
The idea
There are only two ways to give it information: put it in front of it, or let it go and find it.
That's the entire subject. Everything else is plumbing.
Way one — put it in front of it. Paste the document. Drop the file in the project folder. Write it into a file it always reads. The information sits in the context window and it reads it every time, like notes on the desk. Simple, reliable, and it doesn't scale — a thousand documents won't fit on the desk, and even if they did you'd be paying to re-read all thousand to answer a question about one.
Way two — let it go and find it. Keep the documents somewhere searchable. When a question comes in, search first, pull back the handful of relevant bits, and then answer using those. The desk stays clear. Only what's needed comes out.
That second one is what people mean by RAG. The letters stand for retrieval-augmented generation, which is three words doing the work of one:
Look it up, then answer.
That's it. That's the whole acronym. Search the documents, find the relevant pieces, put those in front of the model, let it answer from them.
The reason it matters is the thing you learned in 2.1. A model answering from memory is continuing — plausible, confident, unverifiable. A model answering from a document it just retrieved is reading. Same tool, completely different reliability, and you can point at where the answer came from.
A doctor who memorised the textbook, versus one who looks it up in front of you. The first feels more impressive. The second is more likely to be right, and you can see what they're looking at.
How you decide which way to use, in practice:
| Situation | Use |
|---|---|
| A handful of documents | Put them in the folder |
| Something it should always know | A file it always reads (Level 3) |
| Hundreds or thousands of documents | Retrieval |
| Changes constantly | Retrieval |
| Needs to cite where it came from | Retrieval |
For nearly everything you'll do for a while, the answer is "put it in the folder." Retrieval is what you reach for when the pile gets too big for the desk — and you'll build one properly in Level 8.
And our AI setter is just this, aimed at DMs. Someone asks about pricing; it searches what we've written about pricing; it answers from what it found. It isn't remembering our business. It's looking it up, every time.
Do
- Make a folder called
knowledgeinside your course folder. - Put four or five real documents in it. Actual ones — notes, a process you've written, a PDF, something from your own work. It has to be material no model could possibly know.
- Point Claude at that folder.
- Ask it a question that can only be answered from those documents. Get an answer.
- Ask it: "which file did that come from?" Check it's telling the truth by opening the file yourself.
- Now the control. Open a fresh session with no access to that folder and ask the identical question. Save what it says.
- Put the two answers side by side in
retrieval.md— the one with the documents, the one without — plus one line on how you'd have told them apart if you didn't already know.
Step 6 is the lesson. Both answers will read as confident. One is reading, one is continuing, and from the outside they look the same — which is exactly the point of lesson 2.1, now shown rather than told.
Done when
Never do
Don't put anything in a searchable knowledge base that you wouldn't want the whole system to surface. Passwords, keys, personal data about real people, anything under an NDA.
Retrieval does what you asked it to do: it finds things. Anything indexed can come back in an answer, to whoever asked. Assume everything in there is quotable, and put the sensitive material somewhere else.