Capstone: Fully Offline RAG Agent

What this combines

Every idea from this course, in one program:

• Readiness checking (Lesson 22): wait_for_ollama() runs before anything else, so a cold-started Ollama service doesn't cause a confusing crash.
• RAG, as a tool (Lesson 21): the split/embed/retrieve pipeline is built once into a NotesIndex, then exposed to the model as a search_notes tool, rather than always running unconditionally on every question. The model itself decides when a question actually needs the notes.
• Multi-tool calling loop (Lesson 17): search_notes and calculate both available at once, dispatched through the same functions dict pattern.
• Real multi-turn memory (Lesson 12): one growing messages list across two related questions, the second only answerable using the first's result.
• Streaming (Lesson 5): the model's final, natural-language answer streams live; only the (much shorter) tool-call decision is a single non-streamed chunk, since Ollama doesn't stream those token by token.

And, underneath everything, the whole course's throughline: .env never even gets imported here. No API key, no cloud call, at any point.

Why RAG as a tool, not a fixed step

Lesson 21 always ran retrieval before generation, every question got a context chunk whether it needed one or not. This capstone makes that a choice: search_notes is offered as a tool alongside calculate, and the model decides, per question, whether it needs the notes at all. That's a closer match to a real assistant, which fields plenty of questions (chit-chat, general knowledge, pure arithmetic) that have nothing to do with any specific knowledge base.

The code, piece by piece

class NotesIndex:
    def __init__(self, path: Path):
        text = path.read_text()
        splitter = RecursiveCharacterTextSplitter(chunk_size=300, chunk_overlap=30)
        self.chunks = splitter.split_text(text)
        self.embeddings = ollama.embed(model="nomic-embed-text", input=self.chunks).embeddings

    def search(self, query: str) -> str:
        ...

Splitting and embedding happen once, in __init__, not on every search. This mirrors why the pgvector course stores embeddings in a real database instead of recomputing them per query: the expensive work (embedding every chunk) should happen once, searches after that are cheap.

stream = ollama.chat(model=MODEL, messages=messages, tools=tools, stream=True)
content = ""
tool_calls = []
for chunk in stream:
    if chunk.message.tool_calls:
        tool_calls = chunk.message.tool_calls
    if chunk.message.content:
        print(chunk.message.content, end="", flush=True)
        content += chunk.message.content

Streaming and tool calling, combined: every call streams, but a tool-requesting response arrives as one early chunk carrying the full tool_calls list (Ollama doesn't stream a decision to call a tool token by token), while a direct answer arrives incrementally, printed live exactly like Lesson 5.

messages.append({"role": "assistant", "content": content, "tool_calls": tool_calls})

Reconstructing the assistant message manually here (rather than appending response.message directly, as Lesson 11 and 17 did) because streaming never hands you one complete message object, just chunks, so the loop builds the equivalent by hand from what it collected.