The shape underneath every architecture
Every RAG system, from the simplest to the most elaborate agentic setup in Lesson 17, is built from the same three stages. Learn these once, and every later lesson is a story about which stage got more sophisticated and why.
Stage 1: Indexing (done ahead of time)
Before any question is asked, the source documents are prepared for search: split into smaller pieces called chunks, converted into numeric vectors (Lesson 3 explains how), and stored in a searchable index. This happens once, offline, whenever the underlying documents change, not per question.
Stage 2: Retrieval (done per question)
When a question arrives, it is converted into the same kind of vector as the indexed chunks, and the index is searched for the chunks whose vectors are closest to it. The result is a short list of candidate chunks, typically the top 3 to 10, judged most relevant to the question.
Stage 3: Generation (done per question)
The retrieved chunks are inserted into the prompt alongside the original question, and the language model generates an answer grounded in that material. A well-built prompt also asks the model to say when the retrieved material does not actually answer the question, rather than falling back on its own possibly-stale memory.
| Stage | When it runs | What it produces |
|---|---|---|
| Indexing | Offline, ahead of time | A searchable store of chunk vectors |
| Retrieval | Per question, in real time | A short list of candidate chunks |
| Generation | Per question, in real time | The final answer, grounded in the retrieved chunks |
Checkpoint
- Indexing: preparing documents for search ahead of time (chunk, embed, store); happens whenever source data changes, not per question.
- Retrieval: converting a question into the same vector space and finding the closest indexed chunks.
- Generation: handing the retrieved chunks to the language model to produce a grounded answer.
If anything here still feels unclear, ask before moving to Lesson 3.