The Extraction Stack

The seven layers

Production document-extraction systems separate concerns into discrete layers because each one has different failure modes, different scale characteristics, and different teams who own it. Knowing the boundaries lets you diagnose efficiently — when extraction is wrong, you can usually narrow the root cause to one layer.

1. Preprocessing

The unglamorous first layer. What it does:

• Format detection. PDF (native text vs image-based), DOCX, RTF, image (PNG/JPEG/TIFF). Picks the route through the next layer.
• Page splitting. Multi-page documents → per-page processing. Lets layers downstream parallelize.
• De-skewing & rotation correction. Scanned documents often arrive slightly rotated; OCR engines struggle past ~3° skew.
• Denoising. Remove speckle, watermark bleed-through, scan artifacts.
• Resolution normalization. Upsample low-DPI scans before OCR.

For native PDFs, preprocessing often collapses to "extract embedded text directly with pdfplumber/pdfminer; skip OCR." Knowing which route a document takes is the most important preprocessing decision.

2. OCR

Optical Character Recognition: pixels → text. The major options today:

• Cloud OCR services: AWS Textract, Google Document AI, Azure Document Intelligence (formerly Form Recognizer). Generally excellent. The Form-Recognizer flavor of Document Intelligence and Textract have layout-aware variants that produce structured output, not just flat text.
• Open-source: Tesseract (battle-tested, sometimes inferior on complex layouts), PaddleOCR (better on Asian languages, layout-aware variant).
• Specialty: ABBYY (legacy, still strong on print quality), Mathpix (math/equations), Nougat (academic papers).

Output is typically per-word with bounding boxes — every recognized word carries its (x, y, width, height) coordinates on the page. Downstream layers depend on those coordinates.

Covered in depth in chapter 02.

3. Layout analysis

Given text with spatial positions, classify the page into regions: text paragraphs, tables, headers, footers, signature blocks, page numbers. This step is what makes "this number is in the 'Payment Terms' section" distinguishable from "this number is in the 'Termination Notice' section."

Two flavors:

• Heuristic: rule-based on coordinates, font sizes, indentation. Works for simple, consistent layouts. Brittle.
• Model-based: LayoutLM family, LiLT, DocFormer. Train jointly on text + spatial features. Robust to layout variance.

Cloud services (Textract, Document AI) ship strong layout analysis bundled with OCR. If you're building from scratch, this is the most valuable layer to use a managed service for.

Covered in chapter 03.

4. Field extraction

Given regions and text, extract specific field values: party names, effective date, renewal terms, indemnification clauses, payment amounts. The architectural choice here is the biggest single decision in the stack:

• Rule-based: regex + pattern matching. Brittle, fast, debuggable.
• Task-specific encoders: fine-tuned BERT-class models per field. Mid-range complexity.
• LLM-based: Claude / GPT-4 / etc. with structured output prompts. Most flexible, most expensive, hardest to fully debug.
• Hybrid: rules for high-confidence fields, models for the rest. Production-default.

Covered in chapter 04.

5. Normalization

Extracted values are messy. Normalization brings them to canonical form:

• Dates: "March 15, 2024" / "03/15/2024" / "15-Mar-2024" → 2024-03-15.
• Currency: "$1,234.50" / "USD 1234.50" / "$1.2K" → {amount: 1234.50, currency: "USD"}.
• Entity resolution: "Acme Corp" / "Acme Corporation" / "Acme Inc." → canonical supplier_id.
• Unit conversion: "30 days" / "1 month" / "thirty days" → 30 days (or a canonical unit per field).

Normalization failures are often invisible — they don't fail loudly, they produce subtly wrong canonical values. Good systems log raw + normalized so you can audit.

6. Validation & HITL

Below a confidence threshold, route to a human reviewer queue. Reviewer either confirms, corrects, or escalates. The pieces:

• Confidence calibration determines which extractions hit HITL.
• Queue infrastructure manages reviewer assignment, deadline tracking, capacity.
• Reviewer UI shows the source document, the extracted value, the proposed correction.
• Feedback loop: reviewer corrections become training signal for the next model version.

HITL economics (chapter 07) determines where the threshold sits. Too low: queue overflows, customer-visible delays. Too high: low-confidence extractions land in customer output, accuracy slips.

7. Persistence

Confirmed values are written to a structured store with full provenance — which document, which page, which bounding box, which model version, which reviewer. The persistence layer is also the one that interfaces with the warehouse / API / downstream consumers.

Two structural patterns:

• Wide-narrow: one row per (document, field, extraction_id). Easy to extend.
• Wide-wide: one row per document with typed columns per field. Faster to query; harder to evolve.

Production-default: wide-narrow for ground truth + extraction records; wide marts on top via dbt for query convenience.

How errors compound

The reason the layered model matters: errors at any layer cascade downward. An OCR mistake at layer 2 becomes a wrong extraction at layer 4 becomes a wrong normalization at layer 5 becomes a wrong row at layer 7.

Concrete examples:

• OCR misreads "12-month" as "1-month" → field-extraction returns the wrong term → renewal-alert pipeline fires 11 months early.
• Layout analysis treats a signature block as a paragraph → field-extraction extracts "John Smith" as a party name → supplier resolution creates a phantom supplier.
• Field extraction misses the contingency clause → normalization has nothing to normalize → contract risk surfacing fails silently.

The diagnostic implication: when extraction looks wrong, walk down through the layers. Start at layer 2 (is OCR clean?). Then layer 3 (is the right region identified?). Then layer 4 (is the model picking the right span?). And so on.

Who owns each layer in practice

At a document-AI platform, the ownership map typically looks like:

For an FDE: layers 4, 5, 6, 7 are your daily territory. Layers 1, 2, 3 are usually managed services or platform team work; you consume their output and file bugs against them.