Output Schema & Provenance

Why output design is load-bearing

Extraction output is the artifact every downstream system depends on. Marts, dashboards, alerts, customer outputs — all read from it. If the output shape is wrong, every downstream consumer pays for it. Inversely, if the output shape is right, you get auditability, debuggability, and re-extraction for free.

Three properties to engineer in:

• Provenance — for every extracted value, where it came from in the source document.
• Confidence — per-field, calibrated.
• Versioning — which model / pipeline / config produced this value.

Anatomy of a good extraction record

{
  "extraction_id": "ext_8f3a2b1c",
  "document_id": "doc_a4e2c9d1",
  "customer_id": "cust_acme",
  "doc_class": "master_service_agreement",
  "extracted_at": "2026-05-27T14:32:01Z",

  "model_version": "extract-v4.2.1",
  "pipeline_version": "pipeline-v2.7",
  "customer_config_version": "acme-config-v1.4",

  "fields": {
    "parties": {
      "value": ["Acme Corp", "Customer Inc"],
      "raw_value": ["ACME Corp.", "Customer, Inc."],
      "confidence": 0.97,
      "provenance": [
        {"page": 1, "bbox": [120, 240, 480, 280], "quote": "ACME Corp."},
        {"page": 1, "bbox": [120, 295, 510, 335], "quote": "Customer, Inc."}
      ]
    },
    "effective_date": {
      "value": "2024-03-15",
      "raw_value": "March 15, 2024",
      "confidence": 0.94,
      "provenance": {"page": 1, "bbox": [120, 310, 290, 330], "quote": "effective as of March 15, 2024"}
    },
    "renewal_terms": {
      "value": {
        "auto_renew": true,
        "term_length_months": 12,
        "notice_period_days": 60
      },
      "confidence": 0.78,
      "provenance": {"page": 8, "bbox": [80, 540, 520, 615],
                     "quote": "This Agreement shall automatically renew for successive 12-month terms unless either party provides written notice of non-renewal at least 60 days prior to the end of the then-current term."},
      "review_status": "pending_human"
    }
  },

  "qa": {
    "schema_valid": true,
    "completeness": 1.0,
    "ocr_avg_confidence": 0.96
  }
}

Walking through what each piece earns its place:

• extraction_id: stable identifier for this specific extraction event. Often document_id + model_version.
• document_id, customer_id, doc_class: routing keys for downstream marts.
• extracted_at: when this record was produced. Useful for monitoring drift over time.
• model_version, pipeline_version, customer_config_version: forensic stack. When something looks wrong, you can reconstruct exactly which combination of components produced it.
• fields: the actual extracted values, each with value, raw_value, confidence, provenance.
• qa: meta about the extraction itself — schema validation result, completeness (what fraction of required fields were extracted), OCR quality signal.

Provenance

For every extracted value, store where it came from. The minimal shape:

• page — page number (1-indexed).
• bbox — bounding box (x0, y0, x1, y1) in normalized or pixel coordinates.
• quote — the exact source text the model used as evidence.

Why all three

• Auditability: when a customer disputes a value, you can point at the exact page and quote.
• Debugging: when extraction is wrong, knowing which region the model looked at tells you whether the failure is layout (wrong region) or extraction (right region, wrong span).
• UI: customer-facing reviewers want to click a field and see the source highlighted. Provenance enables that.
• Selective re-extraction: when you ship a model update, you can re-process only documents where provenance looks suspicious (no bbox, low overlap with the field region) instead of reprocessing everything.

Multi-source provenance

Some fields have multiple sources — the "parties" field in an MSA might cite the title block on page 1 and the signature block on page N. Use a list of provenance entries, not a single one.

When you can't get provenance

Pure-LLM extraction with no anchoring sometimes produces values without traceable provenance ("the model summarized something from page 3-4"). Two mitigations:

• Ask the model to cite — the citation goes in the provenance.quote field.
• Validate after extraction by string-matching the quote against the OCR text; flag if not found.

Per-field confidence

One number per field, between 0.0 and 1.0, calibrated such that fields with confidence 0.9 are correct ~90% of the time. (Calibration is its own chapter — 06.)

Where confidence comes from

• OCR confidence: per-word confidence aggregated over the field's tokens.
• Model confidence: token-classifier softmax, or LLM logprobs, or self-reported confidence from the LLM.
• Validation rules: schema match, format match, plausibility checks.

The composite confidence is typically a weighted combination, calibrated against held-out gold data.

Confidence at the wrong granularity

Per-document confidence is mostly useless — the same document can have very-confident parties and not-confident renewal-terms. Always store confidence at field granularity, not document granularity.

Model versioning

Three versioned components, all stored on every extraction:

model_version

The extraction model that produced this. For LLM-based, includes the model name + the prompt version (claude-sonnet-5-5 with extract-prompt-v4.2). For task-specific encoders, the trained-model checkpoint identifier.

pipeline_version

The overall extraction pipeline (preprocessing, OCR engine, layout analysis, extraction). A git SHA or semver string.

customer_config_version

Customer-specific configuration — doc-class taxonomy, field mappings, prompt customizations. Versioned per customer because each customer's config evolves independently.

Why all three

When something looks wrong, you want to reconstruct exactly what combination produced this row. "The supplier-agreement extractions started looking wrong around 2026-05-15" → you query for the model/pipeline/config versions in use that day; one of them is likely the culprit.

Schema design choices

Wide-narrow extraction table

One row per (document_id, model_version, field_name). The flexible default — adding a new field doesn't require schema migration.

CREATE TABLE extractions (
  extraction_id TEXT PRIMARY KEY,
  document_id TEXT NOT NULL,
  customer_id TEXT NOT NULL,
  model_version TEXT NOT NULL,
  pipeline_version TEXT NOT NULL,
  field_name TEXT NOT NULL,
  value JSONB NOT NULL,
  raw_value JSONB,
  confidence FLOAT NOT NULL,
  provenance JSONB NOT NULL,
  review_status TEXT,
  extracted_at TIMESTAMP NOT NULL,
  UNIQUE (document_id, model_version, field_name)
);
CREATE INDEX idx_extractions_doc ON extractions (document_id, model_version);
CREATE INDEX idx_extractions_field ON extractions (field_name);

Wide-wide extraction table

One row per (document_id, model_version) with typed columns per field. Faster to query for known fields; rigid schema.

CREATE TABLE extracted_msas (
  extraction_id TEXT PRIMARY KEY,
  document_id TEXT NOT NULL,
  customer_id TEXT NOT NULL,
  model_version TEXT NOT NULL,
  pipeline_version TEXT NOT NULL,
  extracted_at TIMESTAMP NOT NULL,

  parties TEXT[],
  parties_confidence FLOAT,
  parties_provenance JSONB,

  effective_date DATE,
  effective_date_confidence FLOAT,
  effective_date_provenance JSONB,

  -- ... per field
  UNIQUE (document_id, model_version)
);

Combined: wide-narrow ground truth + wide marts

The production-default: wide-narrow for the source-of-truth table (rare to evolve once schema settles); dbt models on top that produce per-doc-class wide marts for query convenience.

Where confidence/provenance lives

• Inline with the field (wide-wide) — convenient when querying a single field.
• JSONB sidecar (wide-narrow) — flexible, slightly slower to query.
• Separate provenance table — overkill for most platforms; only worth it if provenance is queried independently of values.

The audit story

The shape lets you answer key questions at any time:

• "Why does the system say the renewal term is 12 months?" → query the extraction record for renewal_terms; show the page, bbox, and quote.
• "What did the system say about this document 6 months ago?" → query by document_id; sort by extracted_at; pick the historical extraction.
• "All documents extracted with model version X had this issue — which are they?" → query by model_version.
• "What changed when we deployed v4.2.1 vs v4.1.0?" → join extractions where document_id is in both versions; diff the values.

None of these queries require special infrastructure — they're SQL. But they only work if the data shape supports them. Most "we can't audit our extractions" stories trace back to a schema decision made early that didn't include provenance or versioning.

Anti-patterns

Single-row updates that overwrite history

"Latest extraction wins; we update the row in place." Convenient; destroys auditability. Always insert new extraction rows; let marts present the "current" view by partitioning on model_version + extracted_at.

No provenance because "the LLM already saw the document"

Saving 5% disk and losing 95% of debuggability. Always store provenance — at minimum a quote.

Document-level confidence only

Easy to compute, hard to use. Store confidence per field; aggregate up if you need a document-level number.

Mixing extracted and normalized values

Store both. raw_value is what the model said; value is the normalized canonical version. Normalization is a step that can be redone; the raw extraction is the unrepeatable artifact.

Pipeline version not tied to a git SHA

"v2.7" of the pipeline could mean any of several states. Tie pipeline_version to either a semver release or a git SHA — something the team can resolve to specific code months later.

✓ Check yourself

Answer from memory before opening each one.

1. A customer disputes an effective date and the stored value is wrong. How do the page, bounding box and quote each help you decide whether layout or extraction failed?

   Show answer

   The box shows which region the extractor read. If the box sits in the wrong section, layout or region choice failed. If the box is right but the quote shows a different date, the extractor chose the wrong span. The quote also gives the customer something to check by eye. See provenance.

2. An LLM-only extractor returns values with a quote for each. What check do you run on the quote, and what happens when it fails?

   Show answer

   Search for the quote in the OCR text of the stated page, after normalising whitespace. If it is absent, the value has no support, so flag the field, lower its confidence and send it to review. Do this check before you trust the model's own confidence. See provenance.

3. A date normaliser turns out to have read 03/04/2024 as March 4 for a European customer. You store only the normalised value. What does that cost you, and what would the raw value have bought?

   Show answer

   You cannot tell which rows are wrong without re-running extraction, which costs OCR and model spend and may give different answers. With the raw value kept, you re-run only the normaliser on stored strings. Normalisation can be repeated; the raw extraction cannot. See anti-patterns.

4. A colleague proposes updating the extraction row in place so the table stays small. Which questions can you no longer answer, and what do you do instead?

   Show answer

   You lose what the system said six months ago, which documents a given model version touched, and what changed between two versions. Insert a new row per extraction and let a view or mart present the current one. Storage is cheap next to an unanswerable dispute. See the audit story.

5. Extractions of supplier agreements started to look wrong around the fifteenth. Which three versions do you compare, and why not just the model version?

   Show answer

   Compare the model and prompt version, the pipeline version, and the customer config version in use that day. Any of the three can change behaviour, and the model version alone misses a pipeline or configuration change. The pipeline version only helps if it resolves to a git SHA or release. See model versioning.

6. When does the wide-wide table beat wide-narrow as the source of truth, and what do you give up?

   Show answer

   Wide-wide wins when the field set is stable and most queries read known fields, since typed columns are faster to query. You give up cheap schema change: each new field is a migration, and provenance and confidence columns multiply. For a platform that keeps adding fields, keep wide-narrow as the record and build wide marts. See schema design choices.

Takeaway

• Output design is load-bearing: provenance, per-field calibrated confidence and versioning decide whether you can audit and re-run the system.
• Every field stores its value, its raw value, a confidence and its source as page, box and quote; a field with several sources stores a list.
• Check each quote against the OCR text, because a quote that is not in the page means the value is unsupported.
• Record the model, pipeline and customer config version on every extraction, and tie the pipeline version to a git SHA.
• Insert new rows rather than updating in place, so history and version comparisons stay answerable.
• Keep a wide-narrow table as the record and build wide marts on top for queries.

The confidence stored on each field is only useful if it is honest, which the next chapter shows how to test. → Confidence & Calibration

Further reading

• Citations — Anthropic documentation, 2026. Having Claude return the source passages behind each claim, useful for the quote field.
• PROV-Overview — W3C, 2013. The standard vocabulary for describing where data came from and what produced it.
• JSON types — PostgreSQL documentation, 2026. Storage and indexing of the JSONB columns used for values and provenance.
• About dbt models — dbt documentation, 2026. How to build the wide marts on top of the narrow record table.
• Understanding JSON Schema — JSON Schema project, 2026. Defining and validating the shape of extraction records.