Checkpoint: Batch-Parsing a Folder of PDFs

Putting the beginner tier together

Four lessons in, you have everything a real first pipeline step needs: build one parser (Lesson 3's ocr_enabled=False for known native-text documents), run .parse() over every file in a folder (Lesson 2), and read back .text and .num_pages for each (Lessons 2 and 4). This checkpoint does exactly that over the entire sample_data/ folder and writes one .txt file per PDF, the shape of the very first stage in almost any document-processing pipeline: PDFs in, plain text out, everything downstream (chunking, embedding, search) reads text files or strings, not PDFs.

The code, piece by piece

pdf_paths = sorted(SAMPLE_DATA_DIR.glob("*.pdf"))

Globs only the top-level .pdf files in sample_data/, which leaves out the _src/ subfolder (the plain-text/PNG sources those PDFs were originally built from, not something to parse).

parser = liteparse.LiteParse(ocr_enabled=False, quiet=True)

One parser, reused across every file in the loop, ocr_enabled=False since these are all documents you already know the nature of.

for pdf_path in pdf_paths:
    result = parser.parse(pdf_path)
    txt_path = output_dir / f"{pdf_path.stem}.txt"
    txt_path.write_text(result.text, encoding="utf-8")

The core loop: parse, then write .text straight to a .txt file with the same base name. Output goes to a temp directory (tempfile.mkdtemp) so running this lesson repeatedly never leaves generated files lying around in the repo, in a real pipeline you'd point output_dir at wherever your next stage reads from.

Why scanned_notice.pdf comes back empty

This batch deliberately includes scanned_notice.pdf, a genuinely scanned page with no extractable text layer at all (it was built by rendering a PNG image into a PDF, there's no text object in it for LiteParse to find). With ocr_enabled=False, LiteParse never attempts to recover text from the rendered page image, so its .txt output is empty. This isn't a bug in this lesson, it's the realistic shape of a mixed batch: some PDFs need OCR and some don't, and Lesson 8 covers how to tell the difference and turn OCR on for the ones that need it.