Wrapping It as a Service

Where we left off

This is a short recipe, not a new concept. naive_rag Lesson 24 already taught the whole idea, ingest() runs once at startup via FastAPI's lifespan, ask() runs per request. Lesson 23 built this course's own ingest()/ask() pair; this lesson just wires them into the identical FastAPI shape, worth doing because the loop underneath (function calling, multi-step tool dispatch) is meaningfully more complex than a plain retrieve-and-generate pipeline, and it's worth confirming that complexity disappears completely once it's behind ask().

The code, piece by piece

@asynccontextmanager
async def lifespan(app: FastAPI):
    chroma_client = chromadb.Client()
    app.state.agent_state = ingest(NOTES_DIR, chroma_client)
    yield

Identical structure to naive_rag's version, app.state.agent_state here instead of app.state.collection, holding this course's full State (collection and tool registry both) instead of just a collection.

@app.get("/ask", response_model=AskResponse)
def ask_endpoint(q: str) -> AskResponse:
    return AskResponse(answer=ask(q, app.state.agent_state))

One route, exactly naive_rag's shape. Everything about function calling, multi-step loops, the tool registry, is fully contained inside ask(), this handler has no idea any of it exists, which is precisely the point of drawing the ingest()/ask() boundary in Lesson 23.