Confidence & Calibration

Why calibration matters

Calibration is whether the model's stated confidence matches its observed accuracy. A perfectly-calibrated model: of all extractions scored at 0.9 confidence, 90% are actually correct.

Three operational decisions depend on calibration:

• HITL routing thresholds. If you route everything below 0.85 to reviewers, you're assuming 0.85+ predictions are accurate enough to skip review. If actual accuracy at 0.85 is only 0.7, customer-visible accuracy slips.
• SLA reporting. If the model is over-confident, the SLA dashboard reads "we're hitting 95%" while reality is 87%. The customer finds out the hard way.
• Cost-quality tradeoffs. Tuning the threshold is the lever for HITL volume. The lever doesn't work if the input (confidence) is uncalibrated.

Reliability diagrams

The standard calibration diagnostic. Bin predictions by confidence (deciles or finer), then plot the bin's mean confidence vs the bin's actual accuracy. A perfectly calibrated model lies on the y=x diagonal.

from collections import defaultdict

def reliability_curve(predictions: list[dict], n_bins: int = 10) -> list[dict]:
    """
    Each prediction: {"confidence": float, "was_correct": bool}
    Returns per-bin: mean_confidence, mean_accuracy, n, gap.
    """
    buckets = defaultdict(list)
    for p in predictions:
        b = min(int(p["confidence"] * n_bins), n_bins - 1)
        buckets[b].append(p)
    out = []
    for b in range(n_bins):
        items = buckets.get(b, [])
        if not items:
            continue
        mc = sum(i["confidence"] for i in items) / len(items)
        ma = sum(1 for i in items if i["was_correct"]) / len(items)
        out.append({"bin": b, "n": len(items),
                    "mean_confidence": mc, "mean_accuracy": ma,
                    "gap": ma - mc})
    return out

def expected_calibration_error(predictions, n_bins=10) -> float:
    """ECE — the standard scalar summary of calibration."""
    curve = reliability_curve(predictions, n_bins)
    total = sum(b["n"] for b in curve)
    return sum(b["n"] * abs(b["gap"]) for b in curve) / total

ECE (Expected Calibration Error) is the weighted average of the calibration gap across bins. ECE under 0.02 is excellent; over 0.05 means meaningful miscalibration.

Sources of confidence

Each layer in the extraction stack produces a confidence signal; they're often combined.

OCR confidence

Per-word from the OCR engine. Aggregate across the tokens of an extracted field. Reliable signal for "did we read the text correctly."

Layout/region confidence

From layout-aware models — confidence that this region is what we classified it as (paragraph vs table vs signature).

Token-classifier softmax

For token-classification extractors (BIO tagging with BERT/LayoutLM), the softmax probability of the predicted label. Usually over-confident out of the box; needs calibration.

LLM logprobs

For LLM-based extraction, the logprobs of the generated tokens. Available from most provider APIs. Indirect signal — high logprobs mean the model is fluent, not necessarily correct.

LLM self-reported confidence

Asking the LLM "rate your confidence in this extraction from 0 to 1." Notoriously miscalibrated unless you train against ground truth and Platt-scale.

Validation-rule confidence

Did the extracted value pass format checks? Range checks? Cross-field consistency checks? Each can boost or reduce composite confidence.

Calibration methods

Platt scaling

Fit a logistic regression mapping raw confidence to calibrated probability. Two parameters; works well when miscalibration is roughly sigmoidal. Cheap to fit, cheap to apply.

from sklearn.linear_model import LogisticRegression
import numpy as np

def fit_platt(raw_confidences: np.ndarray, was_correct: np.ndarray) -> LogisticRegression:
    """raw_confidences: floats in [0,1]; was_correct: 0/1."""
    X = raw_confidences.reshape(-1, 1)
    model = LogisticRegression()
    model.fit(X, was_correct)
    return model

def apply_platt(model, raw_confidence: float) -> float:
    return float(model.predict_proba([[raw_confidence]])[0, 1])

Isotonic regression

Fit a monotonic step function mapping raw confidence to calibrated probability. More flexible than Platt; handles non-sigmoidal miscalibration. Needs more held-out data (500+ examples per bin for stable fit).

from sklearn.isotonic import IsotonicRegression

def fit_isotonic(raw_confidences: np.ndarray, was_correct: np.ndarray) -> IsotonicRegression:
    model = IsotonicRegression(out_of_bounds="clip")
    model.fit(raw_confidences, was_correct)
    return model

def apply_isotonic(model, raw_confidence: float) -> float:
    return float(model.predict([raw_confidence])[0])

Temperature scaling

For models that produce logits (token-classifiers), divide logits by a learned temperature T before softmax. Single-parameter; preserves rank ordering. Common for neural-network classifier calibration.

Binning

The simplest: replace raw confidence with the empirical accuracy of its bin. Coarse but defensible. Use when sample sizes are small.

LLM confidence specifically

LLM-based extraction has unusual confidence properties worth knowing.

Logprobs are about fluency, not correctness

An LLM generating "March 15, 2024" with high logprob means the model finds that string fluent given the context. The string might still be wrong (e.g., the date is actually on a different page). Logprobs over-estimate accuracy.

Self-reported confidence is miscalibrated by default

Asking the model "how confident are you?" returns plausible-sounding numbers (usually 0.85, 0.9, 0.95). They don't correspond to reality without calibration.

Consistency-as-confidence

A more reliable signal: sample the model N times, look at agreement. Fields where 5 of 5 runs agree are typically more accurate than fields where 3 of 5 agree. Costs N× the inference; worth it for high-stakes extractions or for offline confidence training.

Citation-grounding

If the model produces a citation (quote from source), validate by string-matching against the OCR text. Failed citations are a strong "low confidence" signal.

from collections import Counter

def consistency_confidence(values: list, n_samples: int) -> tuple[any, float]:
    """Given N sampled extractions of the same field, return the
    most-common value and its agreement rate."""
    counts = Counter(values)
    most_common_value, count = counts.most_common(1)[0]
    return most_common_value, count / n_samples

Composite confidence

In production, you usually combine multiple confidence signals into one number used by HITL routing and reporting. A common shape:

def composite_confidence(
    model_conf: float,
    ocr_conf: float,
    citation_validated: bool,
    schema_valid: bool,
    weights: dict[str, float] = None,
) -> float:
    """Combine signals into a calibrated composite.
    Then apply isotonic / Platt against ground truth to keep calibration."""
    weights = weights or {"model": 0.6, "ocr": 0.2,
                          "citation": 0.1, "schema": 0.1}
    base = (
        weights["model"] * model_conf
        + weights["ocr"] * ocr_conf
        + weights["citation"] * (1.0 if citation_validated else 0.0)
        + weights["schema"] * (1.0 if schema_valid else 0.0)
    )
    return base  # then pass through isotonic calibrator

The weights and calibrator are fit against ground truth per document class. They drift; recompute periodically.

The cost of miscalibration

Miscalibration has asymmetric costs depending on direction.

Over-confidence

Model says 0.9 but is actually right only 0.7. Threshold-based HITL auto-approves predictions that should have been reviewed. Customer-visible accuracy degrades; SLA breaches.

Under-confidence

Model says 0.7 but is actually right 0.9. HITL queue fills with predictions that would have been correct. Reviewer cost rises; throughput suffers; customer-visible latency increases.

Over-confidence is usually worse — it produces silent quality issues. Under-confidence is loud (queue overflow, latency complaints) and easier to detect and fix.

Operating calibration in production

Recompute calibration regularly

Model drift, customer data drift, OCR vendor changes — calibration shifts. Recompute monthly or after every significant model deploy.

Per-document-class calibration

Calibration that's correct for MSAs may be wrong for invoices. Fit calibrators per document class; store the calibrator version alongside the model version.

Calibrate against representative ground truth

Calibration data should reflect the production distribution of inputs. Calibrating on cherry-picked easy examples produces calibrators that fail on hard ones.

Monitor calibration in production

When labeled outcomes arrive (HITL corrections, customer feedback), update a rolling reliability diagram. If ECE creeps up, recalibrate before it affects SLA.

Cold-start calibration

New customer, new document class — you don't have ground truth yet. Start with the calibrator from a similar customer / class; collect HITL outcomes for the first 200–500 documents; refit. Until then, lean conservative on the threshold (route more to HITL than strictly necessary).