Confidence & Calibration

Why calibration matters

Calibration is whether the model's stated confidence matches its observed accuracy. A perfectly-calibrated model: of all extractions scored at 0.9 confidence, 90% are actually correct.

Three operational decisions depend on calibration:

• HITL routing thresholds. If you route everything below 0.85 to reviewers, you're assuming 0.85+ predictions are accurate enough to skip review. If actual accuracy at 0.85 is only 0.7, customer-visible accuracy slips.
• SLA reporting. If the model is over-confident, the SLA dashboard reads "we're hitting 95%" while reality is 87%. The customer finds out the hard way.
• Cost-quality tradeoffs. Tuning the threshold is the lever for HITL volume. The lever doesn't work if the input (confidence) is uncalibrated.

Reliability diagrams

The standard calibration diagnostic. Bin predictions by confidence (deciles or finer), then plot the bin's mean confidence vs the bin's actual accuracy. A perfectly calibrated model lies on the y=x diagonal.

from collections import defaultdict

def reliability_curve(predictions: list[dict], n_bins: int = 10) -> list[dict]:
    """
    Each prediction: {"confidence": float, "was_correct": bool}
    Returns per-bin: mean_confidence, mean_accuracy, n, gap.
    """
    buckets = defaultdict(list)
    for p in predictions:
        b = min(int(p["confidence"] * n_bins), n_bins - 1)
        buckets[b].append(p)
    out = []
    for b in range(n_bins):
        items = buckets.get(b, [])
        if not items:
            continue
        mc = sum(i["confidence"] for i in items) / len(items)
        ma = sum(1 for i in items if i["was_correct"]) / len(items)
        out.append({"bin": b, "n": len(items),
                    "mean_confidence": mc, "mean_accuracy": ma,
                    "gap": ma - mc})
    return out

def expected_calibration_error(predictions, n_bins=10) -> float:
    """ECE — the standard scalar summary of calibration."""
    curve = reliability_curve(predictions, n_bins)
    total = sum(b["n"] for b in curve)
    return sum(b["n"] * abs(b["gap"]) for b in curve) / total

ECE (Expected Calibration Error) is the weighted average of the calibration gap across bins. ECE under 0.02 is excellent; over 0.05 means meaningful miscalibration.

Sources of confidence

Each layer in the extraction stack produces a confidence signal; they're often combined.

OCR confidence

Per-word from the OCR engine. Aggregate across the tokens of an extracted field. Reliable signal for "did we read the text correctly."

Layout/region confidence

From layout-aware models — confidence that this region is what we classified it as (paragraph vs table vs signature).

Token-classifier softmax

For token-classification extractors (BIO tagging with BERT/LayoutLM), the softmax probability of the predicted label. Usually over-confident out of the box; needs calibration.

LLM logprobs

For LLM-based extraction, the logprobs of the generated tokens. Available from most provider APIs. Indirect signal — high logprobs mean the model is fluent, not necessarily correct.

LLM self-reported confidence

Asking the LLM "rate your confidence in this extraction from 0 to 1." Notoriously miscalibrated unless you train against ground truth and Platt-scale.

Validation-rule confidence

Did the extracted value pass format checks? Range checks? Cross-field consistency checks? Each can boost or reduce composite confidence.

Calibration methods

Platt scaling

Fit a logistic regression mapping raw confidence to calibrated probability. Two parameters; works well when miscalibration is roughly sigmoidal. Cheap to fit, cheap to apply.

from sklearn.linear_model import LogisticRegression
import numpy as np

def fit_platt(raw_confidences: np.ndarray, was_correct: np.ndarray) -> LogisticRegression:
    """raw_confidences: floats in [0,1]; was_correct: 0/1."""
    X = raw_confidences.reshape(-1, 1)
    model = LogisticRegression()
    model.fit(X, was_correct)
    return model

def apply_platt(model, raw_confidence: float) -> float:
    return float(model.predict_proba([[raw_confidence]])[0, 1])

Isotonic regression

Fit a monotonic step function mapping raw confidence to calibrated probability. More flexible than Platt; handles non-sigmoidal miscalibration. Needs more held-out data (500+ examples per bin for stable fit).

from sklearn.isotonic import IsotonicRegression

def fit_isotonic(raw_confidences: np.ndarray, was_correct: np.ndarray) -> IsotonicRegression:
    model = IsotonicRegression(out_of_bounds="clip")
    model.fit(raw_confidences, was_correct)
    return model

def apply_isotonic(model, raw_confidence: float) -> float:
    return float(model.predict([raw_confidence])[0])

Temperature scaling

For models that produce logits (token-classifiers), divide logits by a learned temperature T before softmax. Single-parameter; preserves rank ordering. Common for neural-network classifier calibration.

Binning

The simplest: replace raw confidence with the empirical accuracy of its bin. Coarse but defensible. Use when sample sizes are small.

LLM confidence specifically

LLM-based extraction has unusual confidence properties worth knowing.

Logprobs are about fluency, not correctness

An LLM generating "March 15, 2024" with high logprob means the model finds that string fluent given the context. The string might still be wrong (e.g., the date is actually on a different page). Logprobs over-estimate accuracy.

Self-reported confidence is miscalibrated by default

Asking the model "how confident are you?" returns plausible-sounding numbers (usually 0.85, 0.9, 0.95). They don't correspond to reality without calibration.

Consistency-as-confidence

A more reliable signal: sample the model N times, look at agreement. Fields where 5 of 5 runs agree are typically more accurate than fields where 3 of 5 agree. Costs N× the inference; worth it for high-stakes extractions or for offline confidence training.

Citation-grounding

If the model produces a citation (quote from source), validate by string-matching against the OCR text. Failed citations are a strong "low confidence" signal.

from collections import Counter

def consistency_confidence(values: list, n_samples: int) -> tuple[any, float]:
    """Given N sampled extractions of the same field, return the
    most-common value and its agreement rate."""
    counts = Counter(values)
    most_common_value, count = counts.most_common(1)[0]
    return most_common_value, count / n_samples

Composite confidence

In production, you usually combine multiple confidence signals into one number used by HITL routing and reporting. A common shape:

def composite_confidence(
    model_conf: float,
    ocr_conf: float,
    citation_validated: bool,
    schema_valid: bool,
    weights: dict[str, float] = None,
) -> float:
    """Combine signals into a calibrated composite.
    Then apply isotonic / Platt against ground truth to keep calibration."""
    weights = weights or {"model": 0.6, "ocr": 0.2,
                          "citation": 0.1, "schema": 0.1}
    base = (
        weights["model"] * model_conf
        + weights["ocr"] * ocr_conf
        + weights["citation"] * (1.0 if citation_validated else 0.0)
        + weights["schema"] * (1.0 if schema_valid else 0.0)
    )
    return base  # then pass through isotonic calibrator

The weights and calibrator are fit against ground truth per document class. They drift; recompute periodically.

The cost of miscalibration

Miscalibration has asymmetric costs depending on direction.

Over-confidence

Model says 0.9 but is actually right only 0.7. Threshold-based HITL auto-approves predictions that should have been reviewed. Customer-visible accuracy degrades; SLA breaches.

Under-confidence

Model says 0.7 but is actually right 0.9. HITL queue fills with predictions that would have been correct. Reviewer cost rises; throughput suffers; customer-visible latency increases.

Over-confidence is usually worse — it produces silent quality issues. Under-confidence is loud (queue overflow, latency complaints) and easier to detect and fix.

Operating calibration in production

Recompute calibration regularly

Model drift, customer data drift, OCR vendor changes — calibration shifts. Recompute monthly or after every significant model deploy.

Per-document-class calibration

Calibration that's correct for MSAs may be wrong for invoices. Fit calibrators per document class; store the calibrator version alongside the model version.

Calibrate against representative ground truth

Calibration data should reflect the production distribution of inputs. Calibrating on cherry-picked easy examples produces calibrators that fail on hard ones.

Monitor calibration in production

When labeled outcomes arrive (HITL corrections, customer feedback), update a rolling reliability diagram. If ECE creeps up, recalibrate before it affects SLA.

Cold-start calibration

New customer, new document class — you don't have ground truth yet. Start with the calibrator from a similar customer / class; collect HITL outcomes for the first 200–500 documents; refit. Until then, lean conservative on the threshold (route more to HITL than strictly necessary).

✓ Check yourself

Answer from memory before opening each one.

1. The top confidence bin holds 800 predictions with mean confidence 0.95 and observed accuracy 0.82. What is its gap, and what does it mean for a rule that auto-approves at 0.9?

   Show answer

   The gap is 0.13 in the optimistic direction, so the model is over-confident there. An auto-approve rule at 0.9 would accept values that are wrong about 18% of the time, while the dashboard implies about 5%. Fix the calibrator before you rely on the threshold. See reliability diagrams.

2. Why do token logprobs from an LLM extractor over-estimate accuracy, and what signal is stronger?

   Show answer

   Logprobs measure how fluent the string is given the context, not whether it matches the document. Agreement across several sampled runs is stronger, as is whether the quoted evidence appears in the OCR text. Sampling costs several times the inference, so use it for high-stakes fields or for training the calibrator. See LLM confidence.

3. You have 300 labelled outcomes for a new document class. Platt scaling, isotonic regression or binning?

   Show answer

   Platt scaling, because it has two parameters and fits stably on small data when the distortion is roughly sigmoidal. Isotonic is more flexible but needs far more data per bin and overfits at 300. Binning is the fallback when the sample is smaller still. See calibration methods.

4. A team ships a composite score as 0.6 times model confidence plus 0.2 times OCR confidence plus 0.1 each for citation and schema checks, and sets the threshold at 0.85. What is missing?

   Show answer

   A weighted sum of scores is not a probability. Fit a calibrator on ground truth for the composite, per document class, then set the threshold on the calibrated value. Without that, 0.85 means nothing about accuracy. See composite confidence.

5. Which is worse, over-confidence or under-confidence, and how does each show up?

   Show answer

   Over-confidence is worse, because wrong values pass silently into customer output and show up only in audits or complaints. Under-confidence is loud: the review queue grows and latency rises, so you notice and fix it. See the cost of miscalibration.

6. A new customer goes live next week with a document class you have never calibrated. What do you set up for the first month?

   Show answer

   Borrow the calibrator from the most similar customer or class, set a conservative threshold so more fields go to review, and collect review outcomes for the first 200 to 500 documents. Then refit on that data and relax the threshold. See operating calibration in production.

Takeaway

• A calibrated score of 0.9 means 90% of such values are right; routing thresholds, SLA figures and queue cost all depend on that.
• Reliability diagrams and expected calibration error measure it, and an error above 0.05 is meaningful miscalibration.
• Raw classifier softmax, LLM logprobs and self-reported confidence are miscalibrated by default; agreement across samples and verified citations are stronger signals.
• Platt scaling suits small data, isotonic regression suits large data, and temperature scaling suits models that output logits.
• Over-confidence is the worse failure because it is silent, while under-confidence floods the queue.
• Fit calibrators per document class on representative ground truth, and refit regularly.

Calibrated confidence sets the threshold, and the next chapter prices what it costs to review below it. → HITL Economics

Further reading

• On Calibration of Modern Neural Networks — Guo et al., 2017. Shows that modern networks are over-confident and introduces temperature scaling.
• Probability calibration — scikit-learn documentation, 2026. Reliability curves, Platt scaling and isotonic regression in practice.
• Language Models (Mostly) Know What They Know — Kadavath et al., 2022. When a language model's own probability estimates are calibrated and when they are not.
• Can LLMs Express Their Uncertainty? — Xiong et al., 2023. Compares verbalised confidence and sampling consistency as uncertainty signals.