Output Schema & Provenance

Why output design is load-bearing

Extraction output is the artifact every downstream system depends on. Marts, dashboards, alerts, customer outputs — all read from it. If the output shape is wrong, every downstream consumer pays for it. Inversely, if the output shape is right, you get auditability, debuggability, and re-extraction for free.

Three properties to engineer in:

• Provenance — for every extracted value, where it came from in the source document.
• Confidence — per-field, calibrated.
• Versioning — which model / pipeline / config produced this value.

Anatomy of a good extraction record

{
  "extraction_id": "ext_8f3a2b1c",
  "document_id": "doc_a4e2c9d1",
  "customer_id": "cust_acme",
  "doc_class": "master_service_agreement",
  "extracted_at": "2026-05-27T14:32:01Z",

  "model_version": "extract-v4.2.1",
  "pipeline_version": "pipeline-v2.7",
  "customer_config_version": "acme-config-v1.4",

  "fields": {
    "parties": {
      "value": ["Acme Corp", "Customer Inc"],
      "raw_value": ["ACME Corp.", "Customer, Inc."],
      "confidence": 0.97,
      "provenance": [
        {"page": 1, "bbox": [120, 240, 480, 280], "quote": "ACME Corp."},
        {"page": 1, "bbox": [120, 295, 510, 335], "quote": "Customer, Inc."}
      ]
    },
    "effective_date": {
      "value": "2024-03-15",
      "raw_value": "March 15, 2024",
      "confidence": 0.94,
      "provenance": {"page": 1, "bbox": [120, 310, 290, 330], "quote": "effective as of March 15, 2024"}
    },
    "renewal_terms": {
      "value": {
        "auto_renew": true,
        "term_length_months": 12,
        "notice_period_days": 60
      },
      "confidence": 0.78,
      "provenance": {"page": 8, "bbox": [80, 540, 520, 615],
                     "quote": "This Agreement shall automatically renew for successive 12-month terms unless either party provides written notice of non-renewal at least 60 days prior to the end of the then-current term."},
      "review_status": "pending_human"
    }
  },

  "qa": {
    "schema_valid": true,
    "completeness": 1.0,
    "ocr_avg_confidence": 0.96
  }
}

Walking through what each piece earns its place:

• extraction_id: stable identifier for this specific extraction event. Often document_id + model_version.
• document_id, customer_id, doc_class: routing keys for downstream marts.
• extracted_at: when this record was produced. Useful for monitoring drift over time.
• model_version, pipeline_version, customer_config_version: forensic stack. When something looks wrong, you can reconstruct exactly which combination of components produced it.
• fields: the actual extracted values, each with value, raw_value, confidence, provenance.
• qa: meta about the extraction itself — schema validation result, completeness (what fraction of required fields were extracted), OCR quality signal.

Provenance

For every extracted value, store where it came from. The minimal shape:

• page — page number (1-indexed).
• bbox — bounding box (x0, y0, x1, y1) in normalized or pixel coordinates.
• quote — the exact source text the model used as evidence.

Why all three

• Auditability: when a customer disputes a value, you can point at the exact page and quote.
• Debugging: when extraction is wrong, knowing which region the model looked at tells you whether the failure is layout (wrong region) or extraction (right region, wrong span).
• UI: customer-facing reviewers want to click a field and see the source highlighted. Provenance enables that.
• Selective re-extraction: when you ship a model update, you can re-process only documents where provenance looks suspicious (no bbox, low overlap with the field region) instead of reprocessing everything.

Multi-source provenance

Some fields have multiple sources — the "parties" field in an MSA might cite the title block on page 1 and the signature block on page N. Use a list of provenance entries, not a single one.

When you can't get provenance

Pure-LLM extraction with no anchoring sometimes produces values without traceable provenance ("the model summarized something from page 3-4"). Two mitigations:

• Ask the model to cite — the citation goes in the provenance.quote field.
• Validate after extraction by string-matching the quote against the OCR text; flag if not found.

Per-field confidence

One number per field, between 0.0 and 1.0, calibrated such that fields with confidence 0.9 are correct ~90% of the time. (Calibration is its own chapter — 06.)

Where confidence comes from

• OCR confidence: per-word confidence aggregated over the field's tokens.
• Model confidence: token-classifier softmax, or LLM logprobs, or self-reported confidence from the LLM.
• Validation rules: schema match, format match, plausibility checks.

The composite confidence is typically a weighted combination, calibrated against held-out gold data.

Confidence at the wrong granularity

Per-document confidence is mostly useless — the same document can have very-confident parties and not-confident renewal-terms. Always store confidence at field granularity, not document granularity.

Model versioning

Three versioned components, all stored on every extraction:

model_version

The extraction model that produced this. For LLM-based, includes the model name + the prompt version (Claude 3.6 with extract-prompt-v4.2). For task-specific encoders, the trained-model checkpoint identifier.

pipeline_version

The overall extraction pipeline (preprocessing, OCR engine, layout analysis, extraction). A git SHA or semver string.

customer_config_version

Customer-specific configuration — doc-class taxonomy, field mappings, prompt customizations. Versioned per customer because each customer's config evolves independently.

Why all three

When something looks wrong, you want to reconstruct exactly what combination produced this row. "The supplier-agreement extractions started looking wrong around 2026-05-15" → you query for the model/pipeline/config versions in use that day; one of them is likely the culprit.

Schema design choices

Wide-narrow extraction table

One row per (document_id, model_version, field_name). The flexible default — adding a new field doesn't require schema migration.

CREATE TABLE extractions (
  extraction_id TEXT PRIMARY KEY,
  document_id TEXT NOT NULL,
  customer_id TEXT NOT NULL,
  model_version TEXT NOT NULL,
  pipeline_version TEXT NOT NULL,
  field_name TEXT NOT NULL,
  value JSONB NOT NULL,
  raw_value JSONB,
  confidence FLOAT NOT NULL,
  provenance JSONB NOT NULL,
  review_status TEXT,
  extracted_at TIMESTAMP NOT NULL,
  UNIQUE (document_id, model_version, field_name)
);
CREATE INDEX idx_extractions_doc ON extractions (document_id, model_version);
CREATE INDEX idx_extractions_field ON extractions (field_name);

Wide-wide extraction table

One row per (document_id, model_version) with typed columns per field. Faster to query for known fields; rigid schema.

CREATE TABLE extracted_msas (
  extraction_id TEXT PRIMARY KEY,
  document_id TEXT NOT NULL,
  customer_id TEXT NOT NULL,
  model_version TEXT NOT NULL,
  pipeline_version TEXT NOT NULL,
  extracted_at TIMESTAMP NOT NULL,

  parties TEXT[],
  parties_confidence FLOAT,
  parties_provenance JSONB,

  effective_date DATE,
  effective_date_confidence FLOAT,
  effective_date_provenance JSONB,

  -- ... per field
  UNIQUE (document_id, model_version)
);

Combined: wide-narrow ground truth + wide marts

The production-default: wide-narrow for the source-of-truth table (rare to evolve once schema settles); dbt models on top that produce per-doc-class wide marts for query convenience.

Where confidence/provenance lives

• Inline with the field (wide-wide) — convenient when querying a single field.
• JSONB sidecar (wide-narrow) — flexible, slightly slower to query.
• Separate provenance table — overkill for most platforms; only worth it if provenance is queried independently of values.

The audit story

The shape lets you answer key questions at any time:

• "Why does the system say the renewal term is 12 months?" → query the extraction record for renewal_terms; show the page, bbox, and quote.
• "What did the system say about this document 6 months ago?" → query by document_id; sort by extracted_at; pick the historical extraction.
• "All documents extracted with model version X had this issue — which are they?" → query by model_version.
• "What changed when we deployed v4.2.1 vs v4.1.0?" → join extractions where document_id is in both versions; diff the values.

None of these queries require special infrastructure — they're SQL. But they only work if the data shape supports them. Most "we can't audit our extractions" stories trace back to a schema decision made early that didn't include provenance or versioning.

Anti-patterns

Single-row updates that overwrite history

"Latest extraction wins; we update the row in place." Convenient; destroys auditability. Always insert new extraction rows; let marts present the "current" view by partitioning on model_version + extracted_at.

No provenance because "the LLM already saw the document"

Saving 5% disk and losing 95% of debuggability. Always store provenance — at minimum a quote.

Document-level confidence only

Easy to compute, hard to use. Store confidence per field; aggregate up if you need a document-level number.

Mixing extracted and normalized values

Store both. raw_value is what the model said; value is the normalized canonical version. Normalization is a step that can be redone; the raw extraction is the unrepeatable artifact.

Pipeline version not tied to a git SHA

"v2.7" of the pipeline could mean any of several states. Tie pipeline_version to either a semver release or a git SHA — something the team can resolve to specific code months later.