ENGINEERING INDEX / ENTRY 19
CASE 19 / MEDIA INGESTION / SOURCE STRUCTURE
Keep the page attached to the passage.
A listening edition needs manageable spoken passages, but blind splitting can sever headings, page references, and dialogue.
Source project: Twilight ReaderFocused extraction tests passedReviewed September 25, 2026
How can a document become narration chunks without losing where each passage came from?
02 / THE SYSTEM
Where the handoffs happen.
The extractor tests preserve page ranges and a heading, can flush chunks on page changes, keep dialogue and abbreviations together, reject table-like false headings, and read PDF metadata.
- 01Extract blocks
- 02Detect headings and sentences
- 03Chunk by source page
- 04Retain metadata
What the record earns.
Six PDF extraction/chunking checks passed.
- Observed receipt
- On September 25, 2026, six Reader extractor tests passed within a seven-test extraction/router batch.
- Source and scope
- Twilight Reader extractor and tests/test_reader_extractor.py; Reader source/tests untracked in dirty checkout with base HEAD efd4149.
04 / THE LIMIT
What this does not prove.
Fixture PDFs and hand-built blocks do not prove accuracy for scanned documents, every layout, or copyrighted source material.
Where the work goes next.
Build a rights-cleared mixed-layout fixture set and score extraction against human-reviewed source passages.