Evaluation & Quality

Why finance evals are different

Chatbot evals tend to measure helpfulness, harmlessness, tone, factuality on a small benchmark. Finance evals measure something much narrower and harder:

• Numerical correctness to the penny. "Close" is not OK. $4,231.07 ≠ $4,231.06.
• Citation correctness. Every claim must trace back to a real transaction or balance.
• No fabrication. A made-up vendor name or invented transaction is a SOX issue.
• Stability across cycles. The agent should make the same call on the same data two months later.
• Auditability of the eval itself. If you can't show external auditors how you measure agent quality, the agent is not production-grade.

Golden datasets

The foundation of any eval program: a curated, versioned set of inputs with known-correct outputs. For finance:

How to build one

1. Sample real historical cases. Take last 6 months of completed reconciliations, post-Controller-approval. These are your ground truth.
2. De-identify where required. Strip PII / vendor confidentiality where policy demands; preserve transaction structure.
3. Capture intermediate state too: not just final outputs, but tool-call sequences from prior runs.
4. Curate edge cases. The 5% of cases that broke historically are 50% of the value of the dataset.
5. Version it. Golden v1, v2, v3 — each tied to a model version and a prompt version.

Categories your golden set must include

• Happy path: clean recons with all items matching.
• Timing differences: outstanding checks, in-transit transfers.
• Foreign currency complications: FX revaluation, settlement vs. trade dates.
• Edge cases: fee accruals, intercompany breaks, on-chain confirmations vs. accounting events.
• Known-wrong cases: inputs designed to fail reconciliation. The right answer is "halt and escalate."
• Adversarial cases: malformed inputs, duplicated transactions, missing fields. The agent must not invent values.

Metrics that matter

Pick metrics that map to business outcomes. A working list for a recon agent:

The metric that separates senior from junior is false-confidence rate. Anyone can measure accuracy. Measuring "how often is the agent both wrong and confident" is what makes the system trustworthy.

LLM-as-judge — and when not to

For some outputs (variance commentary, audit narratives) the right answer is text, not a number, and exact-match doesn't apply. LLM-as-judge is the technique: a second LLM call grades the first's output against a rubric.

JUDGE_RUBRIC = """Grade the following variance commentary on three axes:
1. Citation accuracy: every numeric claim is supported by the input transactions. 0-5.
2. Driver identification: the commentary correctly identifies the top 1-3 drivers. 0-5.
3. No fabrication: no invented vendor names, dates, or amounts. PASS/FAIL.

Return JSON: {citation_accuracy, driver_identification, no_fabrication, rationale}.
"""

Where LLM-judge is useful

• Narrative quality (variance, flux, audit responses).
• Structural correctness ("did the agent follow the required output format?").
• Pre-flighting outputs in production (judge runs as a critic before the output ships).

Where LLM-judge is dangerous

• Numerical correctness. Don't use an LLM to grade a number. Compute it.
• Citation existence. Don't ask an LLM if a transaction id is valid. Look it up.
• SOX-relevant decisions. "Did the agent's proposed JE follow the controls policy?" — code the rules; don't trust an LLM to evaluate compliance.

The rule: deterministic checks for numbers and rules; LLM-judge for text quality.

Regression suites

Every prompt change, model upgrade, or tool change runs the full golden set and compares to the previous baseline. Failures block the change.

def run_regression(prompt_version, model_version, golden_set):
    results = []
    for case in golden_set:
        out = run_agent(case.inputs, prompt_version, model_version)
        results.append({
            "case_id": case.id,
            "variance_accurate": out.variance == case.expected_variance,
            "match_precision": precision(out.matches, case.true_matches),
            "match_recall": recall(out.matches, case.true_matches),
            "citation_accuracy": citation_check(out, case.universe),
            "tokens": out.tokens, "latency_ms": out.latency_ms,
        })
    return aggregate(results)

def gate(new_results, baseline):
    # Block if any metric drops more than tolerance
    return all(
        new_results[m] >= baseline[m] - tolerance[m]
        for m in CRITICAL_METRICS
    )

The CI pipeline that runs this is part of your SOX change-management story: "Every change to the production agent is gated by a regression run that exercises 500 golden cases, with results archived 7 years."

Drift detection

Inputs change over time. New transaction types appear. Counterparty mix shifts. Crypto introduces new chains. Your agent's performance can quietly degrade even if the prompt is unchanged.

• Distribution drift: track input statistics (volume, account mix, transaction sizes) and alert when current distribution diverges from the training/eval distribution.
• Outcome drift: track the rate of human overrides on agent proposals. If overrides are climbing, the agent is getting it wrong more often.
• Tool-call drift: the average sequence of tool calls per case. If the agent suddenly calls 12 tools where it used to call 5, something has shifted.

Surface these in a finance dashboard. The Process Owner should see them weekly.

Production evals — every run is a sample

Offline regression is necessary but not sufficient. In production:

• Every production run logs structured tool calls, outputs, confidence, and ultimate human decision (approved / edited / rejected).
• A weekly job samples N production runs and re-grades them with the same metrics as the golden set.
• Human overrides become candidate additions to the golden set (with PII review).

This closes the loop: the agent's failures in production feed back into the eval set that catches future regressions.

Evals as SOX change gates

The deepest reason to invest in evals: they are the substrate of your change-management story. The control narrative reads:

External auditors will walk through this control quarterly. If your eval suite is just "we tried it and it seemed fine," you fail. If you have versioned golden sets, archived regression runs, and a gate that blocks deploys, you pass.

A first eval suite in a week

If you're starting from zero, here's the cheapest viable eval program for your first agent:

1. Day 1-2: collect 50 historical reconciliations the Controller approved. Get inputs + expected outputs.
2. Day 3: write a runner that executes the agent on each case and computes variance accuracy, match precision, citation accuracy.
3. Day 4: baseline the current prompt/model. Document the numbers as v1.
4. Day 5: wire the runner into CI. Any PR that changes the prompt must run it; comments post the diff.
5. Day 6: add 10 adversarial cases (malformed inputs, fake transactions, edge currencies). Verify agent halts on each.
6. Day 7: share the dashboard with the Process Owner.

Not glamorous, but it is the artifact that turns your agent from "demo" into "production." It is also exactly the artifact the JD's section 4 (governance) requires.

The eval answer in interviews

When asked "how do you eval a finance agent," a strong four-beat answer:

1. Golden set: curated historical cases, including adversarial. Versioned.
2. Metrics: numeric correctness deterministically; narrative quality via LLM-judge; and a false-confidence metric.
3. Regression as change gate: every prompt/model change runs the suite; thresholds block deploys; runs archived for SOX.
4. Production loop: human overrides feed the next golden set; drift dashboards surface degradation early.

If you say "false confidence" out loud, an experienced interviewer will lean in. Most candidates skip it.

✓ Check yourself

Answer from memory before opening each one.

1. Agent A is right on 96% of reconciliations and nearly all of its misses carry low confidence. Agent B is right on 98%, but its misses carry confidence above 0.9. Which do you deploy, and why?

   Show answer

   Agent A. Low-confidence misses are routed to a human, so the error is caught. Confident misses skip review and reach the ledger. Accuracy ignores that routing, while the false-confidence rate, the share of cases that are both wrong and confident, captures it. See metrics that matter.

2. You can tune the matcher to be stricter or looser. Which of match precision and match recall do you protect, and what does it cost you?

   Show answer

   Protect precision. A false match hides a break, which is the dangerous direction, while a missed match only creates work for a human. Stricter matching lowers recall and adds review load, which is a safe cost you can plan staffing for. See metrics that matter.

3. A teammate proposes using a model judge to confirm that cited transaction ids are valid and that the stated totals add up. What do you use instead, and what do you keep the judge for?

   Show answer

   Look up each id against the transaction universe and compute the totals in code. Both are exact and repeatable, whereas a judge can approve a plausible wrong number. Keep the judge for text qualities that need reading, such as whether a commentary identifies the real drivers. See LLM-as-judge.

4. Nothing about the prompt or model has changed, yet the rate at which humans edit or reject the agent's proposals rises from 8% to 20% over three months. What is happening, and what else do you check?

   Show answer

   That is outcome drift, and the likely cause is a change in the inputs, such as new transaction types or a new chain. Check input distribution drift and the average tool-call sequence per case. Then add the overridden cases to the golden set. See drift detection.

5. A prompt edit improves commentary quality but cuts variance accuracy by half a point. Do you ship it?

   Show answer

   Not if variance accuracy is a critical metric with a tolerance fixed in advance and the drop exceeds it. The gate decides, not the reviewer's taste on the day. If the drop is inside tolerance and the Process Owner accepts the trade, ship it with the run archived. See regression suites and change gates.

6. Why does the golden set include inputs built to fail, and what counts as a correct answer on them?

   Show answer

   A set made only of clean cases rewards forging ahead. For known-wrong and adversarial inputs, such as duplicated transactions or missing fields, the correct output is to halt and escalate without inventing values. They test the bail behaviour that protects the books. See golden datasets.

Takeaway

• Finance evals test numeric correctness to the cent, citation correctness, no fabrication, stability across cycles, and whether the eval itself can be shown to an auditor.
• A golden set is built from approved historical cases, versioned with the model and prompt, and it includes known-wrong and adversarial cases where the right answer is to halt.
• False-confidence rate, cases that are both wrong and confident, is the metric that shows whether the agent can be trusted.
• Use deterministic checks for numbers, citations and rule compliance, and use a model judge only for text quality.
• Every prompt, model or tool change runs the full regression suite and is blocked if a critical metric falls past its preset tolerance.
• In production, sample runs, track human overrides and drift, and feed failures back into the golden set.
• Archived regression runs are the evidence for the change-management control that auditors walk through.

Evals tell you how often the agent is wrong. The next chapter covers what you do when it, or the systems around it, fail. → Error Handling & Failure Modes

Further reading

• Define success criteria and build evaluations — Anthropic docs, accessed 2026. Eval design, grading methods and the case for automated grading at volume.
• Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena — Zheng et al., 2023. The evidence on where model judges agree with humans and where they carry bias.
• SR 11-7: Guidance on Model Risk Management — Federal Reserve, 2011. Validation and ongoing monitoring expectations that regulated firms apply to models.
• AI Risk Management Framework — NIST. Govern, map, measure and manage, as a vocabulary for AI risk programs.
• AS 2201: An Audit of Internal Control Over Financial Reporting — PCAOB. How auditors test whether a control operated, which your eval evidence must support.