Checkpoint: Auto-Detecting OCR Need

The gap this closes

Lesson 5's checkpoint parsed every file with OCR off, fast, but wrong for scanned_notice.pdf. Lesson 8 turned OCR on and got that file's text back, but at the cost of an OCR pass on every page, including the four that never needed it. Neither extreme is right for a real mixed batch. This checkpoint tries the cheap path first and only pays for OCR on the files that actually need it.

The strategy: try fast, fall back on near-zero yield

result = parser_no_ocr.parse(pdf_path)
if len(result.text.strip()) < NEAR_ZERO_TEXT_CHARS:
    result = parser_ocr.parse(pdf_path)

Parse with ocr_enabled=False first. If the text that comes back is near-empty (under a small character threshold), the document almost certainly has no usable native text layer, so re-parse it with ocr_enabled=True to recover text via OCR. This is deliberately blunt (a genuinely tiny native-text page could, in principle, trip it), but it's fast, simple, and gets every file in this batch right.

Why is_complex().needs_ocr alone is not enough here

Lesson 8 introduced is_complex() as a cheap way to see LiteParse's own OCR decision without a full parse. Run it against this whole batch and needs_ocr comes back True for every single file, including the three plain, perfectly native-text documents whose OCR-off parse already recovered complete, correct text. The reason is the sparse-text heuristic: these are short, mostly-whitespace pages (a one-page memo, a one-page form), so native text covers only 2-14% of the page area, low enough to trip a heuristic tuned to catch pages where whatever native text exists might be an artifact rather than the whole story. The heuristic isn't wrong to be cautious, but it's not a reliable "does this file need OCR" verdict for a batch like this one.

The text-yield fallback in this lesson doesn't have that problem: it checks what the fast parse actually returned, not a layout proportion, so it correctly leaves the three native-text files alone and only re-parses scanned_notice.pdf with OCR. The practical takeaway: use is_complex() as a cheap pre-parse triage signal when you want it (e.g. to sort a huge batch before committing CPU time), but for a correctness-critical fallback decision, checking the real output of a fast parse is more reliable than trusting a heuristic verdict blindly.