Evaluation & Quality

The big six payments metrics

Memorize these. Be able to define each, name the failure mode, and write a SQL skeleton for it.

Authorization rate, rigorously

AAR sounds simple. It isn't. There are many ways to count it, and which you pick changes the story.

-- First-attempt AAR by issuer BIN, last 7 days
WITH first_attempts AS (
  SELECT DISTINCT ON (payment_intent_id)
    payment_intent_id,
    bin,
    issuer_country,
    status,
    created_at
  FROM auth_attempts
  WHERE created_at >= NOW() - INTERVAL '7 days'
  ORDER BY payment_intent_id, created_at ASC
)
SELECT
  bin,
  issuer_country,
  COUNT(*) AS attempts,
  COUNT(*) FILTER (WHERE status='approved') AS approved,
  ROUND(100.0 * COUNT(*) FILTER (WHERE status='approved') / COUNT(*), 2) AS aar_pct
FROM first_attempts
GROUP BY bin, issuer_country
HAVING COUNT(*) > 100
ORDER BY attempts DESC;

Conversion vs AAR — the trap interviewers love

AAR measures issuer behaviour. Conversion measures the whole funnel. They diverge whenever UX dropoff or pre-auth abandonment is material.

• If your AAR is 92% and conversion is 70%, you have a UX problem, not an issuer problem.
• If your AAR is 65% and conversion is 60%, you have an issuer problem; the funnel is mostly waiting on auth.
• Lifting AAR can sometimes lower conversion — e.g. forcing 3DS step-up improves AAR via liability shift, but the step-up itself causes dropoff.

Cost per successful payment — the right denominator

Naïve: total spend ÷ all txns. Wrong. You're charged for retries, declines, chargebacks. The right framing:

-- All-in cost per successful payment, last 30 days, by rail
SELECT
  rail,
  geo,
  SUM(approved_count) AS successes,
  SUM(processor_fees + scheme_fees + interchange + xborder_fees + fx_spread
      + retry_fees + chargeback_fees + fraud_loss) AS total_cost_cents,
  ROUND(
    SUM(processor_fees + scheme_fees + interchange + xborder_fees + fx_spread
        + retry_fees + chargeback_fees + fraud_loss)::numeric
    / NULLIF(SUM(approved_count), 0)::numeric,
    2
  ) AS cost_per_success
FROM payment_economics_daily
WHERE date >= CURRENT_DATE - 30
GROUP BY rail, geo
ORDER BY rail, geo;

Many companies under-report this because retry/fraud costs sit in different ledgers. Owning the all-in number is a senior signal.

Fraud and chargeback rates

Two different things, often confused.

• Fraud rate — confirmed fraudulent transactions / volume. Internal classification.
• Chargeback rate — scheme-recorded disputes. External. Card-scheme programs count this.
• 3DS-attempted rate — what % of eligible txns are routed through 3DS.
• 3DS-frictionless rate — of 3DS-attempted, what % were issuer-decided frictionless.

Visa's VDMP / Mastercard's ECP set thresholds (e.g. 0.9% dispute rate, 100 disputes/month). Crossing thresholds triggers fees, then de-platforming. PMs must instrument early-warning at e.g. 0.5%.

Time to availability — the user-perceived speed

Settlement speed is bank-side. Availability is what your customer sees. The user-facing clock:

• UPI / PIX: seconds.
• Faster Payments: usually seconds, sometimes minutes for first-time payee.
• SEPA Instant: ~10s where supported.
• Card: auth instant, but you may impose a hold-then-release policy for fraud reasons (sometimes hours-days).
• ACH: hours-days; hold policies often days.

The PM question: do you release funds on auth, or do you wait for settlement? Tradeoff: trust vs fraud loss. Best practice: risk-tier the hold — long-history customer + low-amount = instant release; new customer + high-amount = delayed.

Defining a rail-health SLO

SLOs aren't only for engineering. Define an SLO per rail that bundles uptime, AAR, latency, error budget. Example for "BR Card":

• Availability: 99.95% successful API request rate from the company to PSP, measured at minute-granularity.
• AAR floor: first-attempt AAR ≥ 78% (BR market baseline) over rolling 7 days.
• Latency: p99 auth response < 1500 ms.
• Settlement: T+1 funds in nostro for 99.5% of txns.

Below the floor, the rail is "in incident" and ops opens a partner ticket. Define the off-ramp explicitly: at AAR floor breach for 3 consecutive days, traffic shifts X% to the secondary route automatically.

Cohort analysis by issuer / BIN

Aggregate AAR hides everything. The real diagnoses live in cohort cuts.

-- AAR change week-over-week by issuer × MCC × country, last 4 weeks
WITH weekly AS (
  SELECT
    DATE_TRUNC('week', created_at) AS wk,
    issuer_name,
    mcc,
    issuer_country,
    COUNT(*) AS attempts,
    COUNT(*) FILTER (WHERE status='approved') AS approved
  FROM auth_attempts
  WHERE created_at >= NOW() - INTERVAL '4 weeks'
  GROUP BY 1, 2, 3, 4
  HAVING COUNT(*) > 200
)
SELECT
  wk,
  issuer_name,
  mcc,
  issuer_country,
  ROUND(100.0 * approved / attempts, 2) AS aar_pct,
  ROUND(100.0 * approved / attempts
        - LAG(100.0 * approved / attempts)
          OVER (PARTITION BY issuer_name, mcc, issuer_country ORDER BY wk),
        2) AS wow_change_pp
FROM weekly
ORDER BY issuer_name, issuer_country, wk;

This kind of query is what you'd open in front of an interviewer who asks "AAR is down — how do you investigate?"

Instrumentation requirements

None of the above metrics are recoverable if you didn't log them correctly at the time. Non-negotiable fields per attempt:

• payment_intent_id (single identity across retries)
• attempt_seq (1, 2, 3...)
• psp_name, acquirer, route_rule_id
• bin, issuer_country, card_type, scheme, mcc
• amount_cents, currency
• status, decline_reason, decline_family
• auth_latency_ms, three_ds_outcome, network_token_used
• created_at, responded_at

If any of these are missing, you can't do post-hoc cohort cuts. The earliest PM lever in a new rail is the event schema.