OCR Layer in Depth

Native vs scanned PDFs

The first decision in OCR is whether you need OCR at all. PDFs come in three flavors:

Native PDF (text-layer present)

Text is stored as actual character data — generated by Word, Adobe, or another authoring tool. Parse directly with pdfplumber, pdfminer.six, or PyMuPDF. No OCR needed. Output is fast, accurate, and includes coordinates for free.

Image-based PDF (scan or photo)

Text is rendered as pixels. Requires OCR. Quality depends on scan resolution, skew, and image cleanliness.

Mixed PDF

Some pages native, some scanned. Common for contracts with scanned signature pages appended to native documents. Process page-by-page: detect format per page, route appropriately.

How to detect

import pdfplumber

def page_is_native(page) -> bool:
    """Native if the page has embedded text characters."""
    # pdfplumber returns the text layer; empty/whitespace = no native text
    text = page.extract_text() or ""
    return len(text.strip()) > 20  # threshold avoids false positives on stubs

def classify_pdf(path: str) -> str:
    """Returns 'native', 'image', or 'mixed'."""
    with pdfplumber.open(path) as pdf:
        flags = [page_is_native(p) for p in pdf.pages]
    if all(flags): return "native"
    if not any(flags): return "image"
    return "mixed"

Threshold of 20 characters per page avoids classifying near-empty pages (cover sheets, blank dividers) as native; tune for your corpus.

OCR engines

The major options grouped by cost and capability:

Cloud OCR (recommended default)

• AWS Textract — strong general OCR; "Analyze Document" variant returns layout + tables + key-value pairs. Pricing scales per-page; volume discounts at scale. Often the default for AWS-shop customers.
• Google Document AI — formerly Document Understanding AI. Strong on form-heavy documents. Custom processor training available for specific document types.
• Azure AI Document Intelligence — formerly Form Recognizer. Strong on layout-aware extraction; competitive with Textract on general OCR.

Open-source

• Tesseract — the workhorse. 30+ years old, fully open. Reasonable on clean, single-column documents; struggles on complex layouts. Run with --psm mode tuned per document type.
• PaddleOCR — Baidu-developed; strong on Asian-language documents; includes layout-aware variant (PP-Structure).
• EasyOCR — newer Python-native; good for prototyping; less battle-tested at scale.

Specialty / legacy

• ABBYY FineReader — legacy strong-print quality; common in legal-tech and government document workflows.
• Adobe Document Cloud — Acrobat-flavored OCR; common when customers already license Adobe enterprise.

Modern alternatives

• Vision-capable LLMs (Claude with vision, GPT-4o, Gemini) can perform OCR + extraction in a single step on small documents. Faster prototyping; cost and latency are not competitive with dedicated OCR at corpus scale.
• Nougat, Donut — research-grade end-to-end document-understanding models. Useful in narrow domains (academic papers, receipts); not general-purpose yet.

Output shape

OCR output should always include three things per recognized text:

• Text string — what the engine recognized.
• Bounding box — (x, y, width, height) in page coordinates. Required for layout analysis downstream.
• Confidence — per-word or per-token confidence from the OCR engine.

Cloud services produce hierarchical output (page → block → line → word) with bounding boxes at each level. Use the line level for most extraction logic; the word level when you need precise spans.

{
  "BlockType": "LINE",
  "Text": "This Master Service Agreement (the \"Agreement\") is entered into",
  "Confidence": 99.4,
  "Geometry": {
    "BoundingBox": {"Width": 0.521, "Height": 0.014, "Left": 0.131, "Top": 0.231},
    "Polygon": [...]
  },
  "Page": 1,
  "Relationships": [{"Type": "CHILD", "Ids": ["word-id-1", "word-id-2", ...]}]
}

BoundingBox coordinates are normalized (0.0–1.0) on AWS; absolute pixel coordinates on others. Normalize before further use.

Failure modes

The catalog of how OCR breaks. Most appear in real customer corpora; the question is which.

Multi-column reading order

Documents laid out in columns can be read row-by-row (correct) or top-to-bottom column-by-column (wrong). Different OCR engines have different defaults; some let you configure. Tesseract --psm 1 attempts automatic page-segmentation; --psm 4 assumes single column.

Table flattening

Tables can come out as cleanly-structured cells (cloud services with layout) or as flat text where row/column relationships are lost (basic OCR). For contract tables (pricing schedules, deliverable lists), the difference is critical.

Rotation

Even 1–2° rotation degrades OCR quality. 90°/180°/270° rotation usually fails entirely without explicit rotation handling. Preprocessing should detect and correct.

Watermarks & stamps

"DRAFT" / "CONFIDENTIAL" / company seals overlap with text; OCR may include them as content or partially overlap with the real text below. Mitigate with image-preprocessing (HSV filtering on common watermark colors) or post-process to remove known watermark phrases.

Handwriting

Annotations, signatures, notes in margins. Most OCR engines have a separate "handwriting" mode (Textract, Document AI); enable explicitly when expected.

Low resolution

Below ~200 DPI, OCR quality degrades. Upsampling can help marginally; the right fix is requesting better scans.

Compressed PDFs / artifacts

Some PDFs render fine in Acrobat but compress text into image blocks. Detect by checking native-text presence per page; route to OCR.

Languages not configured

Tesseract requires language packs; cloud services support most major languages but require explicit language hints. Multi-language documents (English contract with German appendix) need per-page or per-region detection.

The OCR preflight

Before signing an SLA with a new customer, run an OCR quality preflight on a representative corpus sample. The 30-minute investment in discovery prevents weeks of remediation later.

"""
Sample 50 documents from the customer's corpus, run OCR, eyeball results.
"""
import random
from pathlib import Path

def run_preflight(corpus_dir: Path, sample_size: int = 50):
    pdfs = list(corpus_dir.glob("**/*.pdf"))
    sample = random.sample(pdfs, min(sample_size, len(pdfs)))

    classifications = {"native": [], "image": [], "mixed": []}
    for pdf_path in sample:
        kind = classify_pdf(pdf_path)
        classifications[kind].append(pdf_path)

    print(f"Native PDFs:  {len(classifications['native'])}")
    print(f"Image PDFs:   {len(classifications['image'])}")
    print(f"Mixed PDFs:   {len(classifications['mixed'])}")

    # For image PDFs, run OCR and produce a quality report
    for pdf_path in classifications["image"] + classifications["mixed"]:
        ocr_result = run_textract(pdf_path)
        avg_confidence = sum(b["Confidence"] for b in ocr_result["Blocks"]
                             if b["BlockType"] == "LINE") / max(1, sum(1 for b in ocr_result["Blocks"] if b["BlockType"] == "LINE"))
        write_report(pdf_path, ocr_result, avg_confidence)

    # Spot-check by eye: open the first 5 reports, compare to source
    print("Open data/preflight/report-*.html in the browser and compare side-by-side.")

What to look for

• What fraction of the corpus is image-based? Sets the OCR cost.
• Average OCR confidence per page? Below 90% means trouble.
• How often does multi-column reading order break?
• Tables — do they extract as structured cells or flat text?
• Languages other than English?
• Watermarks, stamps, handwritten annotations — frequency?

The report becomes evidence in the working-agreement conversation. If 30% of the corpus is below OCR confidence threshold, the SLA needs a carve-out for that subset.

Quality measurement

OCR quality has two flavors:

Engine-reported confidence

Every modern OCR engine produces per-word confidence. Aggregate (mean, p95) per page. Useful for triage and HITL routing.

Ground-truth comparison

Label a sample of pages by hand or via dual annotation. Score OCR output against the ground truth. Standard metrics:

• CER (Character Error Rate): edit distance between OCR text and ground truth, normalized by length. Lower is better.
• WER (Word Error Rate): same but word-level.
• Field-level accuracy: did OCR get the specific values you care about right? This matters more than raw CER/WER for downstream extraction.

For contract intelligence, field-level matters most: a 5% CER might be fine if the parties, dates, and key terms are all readable. A 2% CER is unacceptable if the 2% concentrates in dollar amounts.

Cost & throughput

OCR is a meaningful line item at scale.

• Cloud OCR cost: roughly $1.50–$15 per 1000 pages depending on service and "analyze" tier. Layout-aware variants are 3–10× the base OCR rate.
• Throughput: cloud services are roughly 1–5 pages/second per request; parallelize across requests for higher throughput. Batch endpoints (Textract async, Document AI batch) for cost optimization on large historical backfills.
• Caching: same document re-OCRed should hit cache. Use file-content hash as cache key.

For a customer with a 50,000-document corpus and 90% image-based PDFs at ~10 pages each = 450,000 pages. At $5/1000 pages, the OCR bill for historical backfill is $2,250. Ongoing flow at 500 new docs/month with similar shape: ~$22/month. Most customers don't blink at OCR cost; the cost discussion is usually about the LLM-extraction step downstream.