Skip to content
← back to projects
2026·shipped·Forward Deployed Engineer

Document Intelligence Refinery

A five-stage escalating pipeline that turns mixed PDFs, scans, and tables into structured, spatially-indexed knowledge, using vision only when text extraction cannot answer.

10 Academy TRP · 1-week brief
PythonPDFOCRVision ModelsSemantic Chunking
Code ↗

Pipeline

1
Triage
Classify document
↓
2
Extraction
Text, then layout, then vision
↓
3
Chunking
Structure-aware
↓
4
PageIndex
Spatial coordinates
↓
5
Query interface
Retrieval + citations

Highlights

  • ▸Escalating strategy: cheap text extraction first, layout-aware next, vision-augmented only as last resort.
  • ▸PageIndex layer preserves spatial relationships so the query interface can point at exact regions.
  • ▸Triage stage classifies documents so each one hits the cheapest strategy that will actually work.

Design principle

Every document intelligence pipeline eventually meets a PDF that breaks it: a scan, a form, a table that flows across pages. The Refinery treats that reality by building an escalating pipeline where each stage only runs if earlier stages could not produce a confident answer.

Stages

  1. Triage: classify the document (born-digital, scan, form, mixed).
  2. Multi-strategy extraction: cheap text extraction first, then layout-aware parsers, then vision-augmented models only when needed.
  3. Semantic chunking: chunks are shaped by document structure, not fixed-size windows.
  4. PageIndex: chunks carry spatial coordinates so the query layer can cite regions.
  5. Query interface: retrieval + generation with citations back to original coordinates.

Result

Cost drops significantly on the common case (born-digital PDFs), while quality holds on the hard cases (scans and tables) because vision only runs when it earns its price.