A Rule-Based Pre-Filter

Where we left off

Lesson 19 built three cheap signals and showed they're directional but noisy, good enough to spot a clearly simple question, not precise enough to tell "multi_hop" apart from "ambiguous" on their own. This lesson uses that limitation on purpose: instead of replacing Lesson 3's LLM classifier, use the cheap signals only to catch the confidently-simple cases, and fall through to the real classifier for everything else.

The code, piece by piece

PRE_FILTER_THRESHOLD = 0.20

Chosen by inspecting Lesson 19's own five example scores directly (the two clean simple_factual questions scored 0.135 and 0.120, everything else scored 0.31 or higher), not from this course's Lesson 17 evaluation set. That distinction matters and gets said explicitly here rather than assumed: tuning a threshold against a different signal than the one you later evaluate against is the safe pattern (see naive_rag Lesson 14's threshold, chosen the same way), and Lesson 17 later in this course is the deliberately-flagged case where that separation does NOT hold.

def classify(query: str) -> tuple[str, bool]:
    score = complexity_score(query)
    if score < PRE_FILTER_THRESHOLD:
        return "simple_factual", False
    return classify_with_llm(query), True

Below the threshold, skip Gemini entirely and return simple_factual directly. At or above it, fall through to the real LLM classifier from Lesson 3, rebuilt here as classify_with_llm(). The pre-filter only ever short-circuits to one label, on purpose: it isn't confident enough to distinguish multi_hop from ambiguous, only "clearly simple" from "needs the real classifier."

On a small, mixed four-question demo run, half the questions never touch the network. On a larger, more skewed question set (most real traffic to a Q&A tool tends to be simple lookups), that ratio would save considerably more.