Instrumenting Strategy Choice

Where we left off

Lesson 23's service returns an answer and which strategy produced it, but that information disappears the moment the response is sent, nothing records it anywhere. This lesson adds a routing log: every request's question, classified label, the classifier's stated reason, which strategy handled it, and how long the whole thing took, recorded and retrievable.

The code, piece by piece

@dataclass
class RoutingLogEntry:
    question: str
    label: str
    reason: str
    strategy: str
    latency_seconds: float


ROUTING_LOG: list[RoutingLogEntry] = []

Lesson 13's persisted-decision idea (question, decision, outcome, saved for later analysis), moved from a script's local list into the service every request now runs through. reason comes from extending the classifier prompt to return a short justification alongside its label, not a separate call.

entry = RoutingLogEntry(
    question=query, label=label, reason=reason, strategy=strategy_name, latency_seconds=round(elapsed, 2)
)
ROUTING_LOG.append(entry)
print(f"[routing log] {json.dumps(asdict(entry))}")

Every call to answer() appends one entry and prints it, so the log is visible in real time (useful for a real deployment's log aggregator) and also queryable afterward.

@app.get("/logs")
def logs_endpoint() -> list[dict]:
    return [asdict(entry) for entry in ROUTING_LOG]

A second endpoint exposing the whole log, so "why did this question get routed the way it did" is answerable after the fact, not only by reading console output at the moment it happened.