Where we left off

Every retrieval mode this course built, dense, sparse, hybrid, gets compared here, side by side, precision@1 on one shared labeled question set. Before reading the numbers below, read naive_rag Lesson 17's "Why this doesn't generalize (yet)" section if you haven't already, it covers exactly what a score like this can and can't tell you, and this lesson doesn't repeat that ground.

The code, piece by piece

LABELED_QUESTIONS = [
    ("20240115", "home_network.md"),
    ...
    ("What change finally made things work reliably again?", "old_travel_router.md"),
]

Eleven questions: the same ten from Lessons 6-9 and 11, plus Lesson 16's known-hard case, included deliberately. Leaving it out would have made hybrid retrieval look flawless, which would misrepresent Lesson 16's own finding.

dense_ok = dense[0] == expected
sparse_ok = sparse[0] == expected
hybrid_ok = fused[0] == expected

Three separate precision@1 scores from the exact same eleven questions and the exact same underlying rankings, so the comparison is as apples-to-apples as this course's tools allow.

Checkpoint

  • Hybrid retrieval scores strictly higher than either retriever alone on this course's labeled set, and the specific question it recovers matches the specific blind spots demonstrated all the way back in Lesson 6.
  • It doesn't recover Lesson 16's failure case, confirming that fusion redistributes existing signal rather than creating new signal.
  • Eleven questions is enough to illustrate the mechanism, not enough to trust the exact numbers, the same caveat naive_rag Lesson 17 raised applies here without modification.

If anything here still feels unclear, ask before moving to Lesson 18.