The Extraction Stack

The seven layers

Production document-extraction systems separate concerns into discrete layers because each one has different failure modes, different scale characteristics, and different teams who own it. Knowing the boundaries lets you diagnose efficiently — when extraction is wrong, you can usually narrow the root cause to one layer.

1. Preprocessing

The unglamorous first layer. What it does:

• Format detection. PDF (native text vs image-based), DOCX, RTF, image (PNG/JPEG/TIFF). Picks the route through the next layer.
• Page splitting. Multi-page documents → per-page processing. Lets layers downstream parallelize.
• De-skewing & rotation correction. Scanned documents often arrive slightly rotated; OCR engines struggle past ~3° skew.
• Denoising. Remove speckle, watermark bleed-through, scan artifacts.
• Resolution normalization. Upsample low-DPI scans before OCR.

For native PDFs, preprocessing often collapses to "extract embedded text directly with pdfplumber/pdfminer; skip OCR." Knowing which route a document takes is the most important preprocessing decision.

2. OCR

Optical Character Recognition: pixels → text. The major options today:

• Cloud OCR services: AWS Textract, Google Document AI, Azure Document Intelligence (formerly Form Recognizer). Generally excellent. The Form-Recognizer flavor of Document Intelligence and Textract have layout-aware variants that produce structured output, not just flat text.
• Open-source: Tesseract (battle-tested, sometimes inferior on complex layouts), PaddleOCR (better on Asian languages, layout-aware variant).
• Specialty: ABBYY (legacy, still strong on print quality), Mathpix (math/equations), Nougat (academic papers).

Output is typically per-word with bounding boxes — every recognized word carries its (x, y, width, height) coordinates on the page. Downstream layers depend on those coordinates.

Covered in depth in chapter 02.

3. Layout analysis

Given text with spatial positions, classify the page into regions: text paragraphs, tables, headers, footers, signature blocks, page numbers. This step is what makes "this number is in the 'Payment Terms' section" distinguishable from "this number is in the 'Termination Notice' section."

Two flavors:

• Heuristic: rule-based on coordinates, font sizes, indentation. Works for simple, consistent layouts. Brittle.
• Model-based: LayoutLM family, LiLT, DocFormer. Train jointly on text + spatial features. Robust to layout variance.

Cloud services (Textract, Document AI) ship strong layout analysis bundled with OCR. If you're building from scratch, this is the most valuable layer to use a managed service for.

Covered in chapter 03.

4. Field extraction

Given regions and text, extract specific field values: party names, effective date, renewal terms, indemnification clauses, payment amounts. The architectural choice here is the biggest single decision in the stack:

• Rule-based: regex + pattern matching. Brittle, fast, debuggable.
• Task-specific encoders: fine-tuned BERT-class models per field. Mid-range complexity.
• LLM-based: Claude / GPT-4 / etc. with structured output prompts. Most flexible, most expensive, hardest to fully debug.
• Hybrid: rules for high-confidence fields, models for the rest. Production-default.

Covered in chapter 04.

5. Normalization

Extracted values are messy. Normalization brings them to canonical form:

• Dates: "March 15, 2024" / "03/15/2024" / "15-Mar-2024" → 2024-03-15.
• Currency: "$1,234.50" / "USD 1234.50" / "$1.2K" → {amount: 1234.50, currency: "USD"}.
• Entity resolution: "Acme Corp" / "Acme Corporation" / "Acme Inc." → canonical supplier_id.
• Unit conversion: "30 days" / "1 month" / "thirty days" → 30 days (or a canonical unit per field).

Normalization failures are often invisible — they don't fail loudly, they produce subtly wrong canonical values. Good systems log raw + normalized so you can audit.

6. Validation & HITL

Below a confidence threshold, route to a human reviewer queue. Reviewer either confirms, corrects, or escalates. The pieces:

• Confidence calibration determines which extractions hit HITL.
• Queue infrastructure manages reviewer assignment, deadline tracking, capacity.
• Reviewer UI shows the source document, the extracted value, the proposed correction.
• Feedback loop: reviewer corrections become training signal for the next model version.

HITL economics (chapter 07) determines where the threshold sits. Too low: queue overflows, customer-visible delays. Too high: low-confidence extractions land in customer output, accuracy slips.

7. Persistence

Confirmed values are written to a structured store with full provenance — which document, which page, which bounding box, which model version, which reviewer. The persistence layer is also the one that interfaces with the warehouse / API / downstream consumers.

Two structural patterns:

• Wide-narrow: one row per (document, field, extraction_id). Easy to extend.
• Wide-wide: one row per document with typed columns per field. Faster to query; harder to evolve.

Production-default: wide-narrow for ground truth + extraction records; wide marts on top via dbt for query convenience.

How errors compound

The reason the layered model matters: errors at any layer cascade downward. An OCR mistake at layer 2 becomes a wrong extraction at layer 4 becomes a wrong normalization at layer 5 becomes a wrong row at layer 7.

Concrete examples:

• OCR misreads "12-month" as "1-month" → field-extraction returns the wrong term → renewal-alert pipeline fires 11 months early.
• Layout analysis treats a signature block as a paragraph → field-extraction extracts "John Smith" as a party name → supplier resolution creates a phantom supplier.
• Field extraction misses the contingency clause → normalization has nothing to normalize → contract risk surfacing fails silently.

The diagnostic implication: when extraction looks wrong, walk down through the layers. Start at layer 2 (is OCR clean?). Then layer 3 (is the right region identified?). Then layer 4 (is the model picking the right span?). And so on.

Who owns each layer in practice

At a document-AI platform, the ownership map typically looks like:

For an FDE: layers 4, 5, 6, 7 are your daily territory. Layers 1, 2, 3 are usually managed services or platform team work; you consume their output and file bugs against them.

✓ Check yourself

Answer from memory before opening each one.

1. Each of the seven layers is 95% accurate and their errors are independent. Roughly what share of documents come out fully right, and why is the real figure usually worse?

   Show answer

   About 70%, because 0.95 to the seventh power is 0.70. The real figure is usually worse because errors are not independent: a bad scan hurts OCR, layout and extraction at once, and a layer that is only 90% accurate drags the product down fast. Measure each layer on its own. See how errors compound.

2. A renewal alert fires eleven months early on one contract. Where do you start looking, and in what order do you move?

   Show answer

   Start at layer 2 and read the OCR text for the term clause, because a misread "12-month" as "1-month" explains the symptom. Then check that layout found the right region, then that extraction picked the right span, then the normaliser. Each step rules out everything upstream of it. See how errors compound.

3. Finance reports a new supplier in the master table named after a person who signed a contract. Which layers could have produced it, and which do you inspect first?

   Show answer

   Layout may have labelled the signature block as a paragraph, extraction then read the signer as a party, and entity resolution created a supplier for it. Inspect the region labels on the signature page first, since that is the earliest layer that can be wrong. See how errors compound.

4. Why does the stack send a native PDF down a different route from a scan, and what goes wrong if you OCR every file anyway?

   Show answer

   A native PDF already holds exact characters and coordinates, so parsing it is faster, free and error-free. Running OCR on it pays per page and adds recognition mistakes that then cascade down every later layer. Choosing the route is the first preprocessing decision. See preprocessing.

5. The team adds two or three fields a month and the BI group queries the results daily. Which persistence shape do you choose, and what do you give the BI group?

   Show answer

   Store one row per document, field and extraction, because a new field then needs no schema change and every value keeps its provenance. Give BI wide tables built on top with dbt, so they get typed columns without constraining how the ground truth evolves. See persistence.

Takeaway

• A document-extraction system is seven layers with different failure modes and owners, so the first step in any bug is to name the layer.
• Errors cascade: seven layers at 95% each give about 70% end to end, so each layer needs its own measurement.
• Debug from the top down: OCR text first, then regions, then the extracted span, then normalisation.
• Route each file or page by whether it has a text layer, because native PDFs skip OCR.
• Normalisation fails silently with plausible values, so keep both the raw and the normalised value.
• Persist one row per document, field and extraction with provenance, and publish wide marts on top.

Layer 2 is where most downstream trouble starts, so it gets its own chapter. → OCR Layer in Depth

Further reading

• What is Amazon Textract? — AWS documentation, 2026. What a managed OCR and layout service returns, including key-value pairs and tables.
• Document AI overview — Google Cloud documentation, 2026. Processor types and how a managed service bundles OCR, layout and extraction.
• What is Azure AI Document Intelligence? — Microsoft documentation, 2026. The prebuilt and custom models, and where layout analysis sits among them.
• pdfplumber — Open-source project, ongoing. The library for pulling text, characters and coordinates from native PDFs.