The Scorecard

Scope first — before the scorecard

The scorecard only makes sense once you know what you're scoring against. Pin these answers before writing weights. Disagreement between Product, Compliance, and Finance on any of these is the most common reason scorecards collapse mid-evaluation.

Kill switches — anything that auto-disqualifies

A kill switch is a hard floor. Vendors that fail any of these are eliminated before scoring. Be ruthless here; it saves weeks downstream. Customize for your situation — every kill switch should have an owner who'll defend it.

Weighting the seven categories

Weights are not universal. They reflect your business stage and regulatory posture. A pre-launch fintech weights compliance and capability higher; a scale-stage business with an existing book weights cost and operations higher. Here are three reference weightings as starting points.

The 38-criterion scorecard

Each criterion is scored 0–5. Within a category, all criteria are weighted equally unless flagged. Mark a criterion (★) to give it 2× weight inside its category.

Scoring rubric (0–5)

Capability (7 criteria)

Performance (5 criteria)

Cost (5 criteria)

Compliance (6 criteria)

Integration (6 criteria)

Operations (5 criteria)

Commercial (4 criteria)

Worked example — four vendors, pre-launch fintech weights

This is illustrative. Real scores depend on your testing, your jurisdictions, your population. Use this to see how the math behaves, not to learn the answers — and never use a generic scorecard score as a substitute for a sandbox bake-off.

Vendor A wins this scorecard narrowly, but the spread (3.78–4.04) is tight enough that any single re-scored row could flip it. That's normal and it's a feature — it means you've reached the point where qualitative judgment, reference calls, and the bake-off matter more than the scorecard.

What the scorecard tells you when it's close

• Which dimensions are differentiated. Cost (4.2 vs 2.6) and Integration (4.6 vs 3.2) are the dimensions with real spread. Those should drive the negotiation.
• Where every vendor is mediocre. If everyone scores ~3.5 on Performance, none of them have measured what you need. That's a signal to demand the bake-off be on your data, not theirs.
• Where the bar is high enough. If everyone is ≥ 4.0 on Compliance, you can stop investing scoring cycles there and reallocate.

Show the math

The scorecard is a weighted mean. Each category score is the arithmetic mean of its 0–5 criterion scores (with starred criteria counting twice). The final score is the weighted mean of category scores. Both are stable to ±0.05 against small re-scorings — which is intentionally larger than typical inter-rater variance.

Per-category score

def category_score(criteria):
    """
    criteria: list of (score_0_to_5, weight_1_or_2)
    """
    total_weight = sum(w for _, w in criteria)
    total_score  = sum(s * w for s, w in criteria)
    return total_score / total_weight

# Capability example for Vendor A
capability_A = category_score([
    (4, 2),  # C1 doc coverage (starred = weight 2)
    (4, 1),  # C2 biometric
    (5, 2),  # C3 liveness (starred)
    (4, 1),  # C4 sanctions
    (3, 1),  # C5 KYB
    (4, 1),  # C6 re-verification
    (4, 1),  # C7 workflow
])
# = (8 + 4 + 10 + 4 + 3 + 4 + 4) / 9
# = 37 / 9
# = 4.11

Weighted total

CATEGORY_WEIGHTS = {
    "capability":  0.22,
    "performance": 0.18,
    "cost":        0.10,
    "compliance":  0.20,
    "integration": 0.15,
    "operations":  0.08,
    "commercial":  0.07,
}

def total_score(vendor):
    return sum(
        vendor[cat] * w
        for cat, w in CATEGORY_WEIGHTS.items()
    )

vendor_A = {
    "capability":  4.1, "performance": 4.0, "cost":        3.5,
    "compliance":  3.8, "integration": 4.6, "operations":  3.9,
    "commercial":  4.0,
}
# total_score(vendor_A) == 4.04

Spreadsheet equivalent

For non-coders, the same math in a Google Sheet:

A          B           C          D          E          F
1 Category   Weight      Vendor A   Vendor B   Vendor C   Vendor D
2 Capability 0.22        4.1        4.5        3.6        4.4
3 Performance 0.18       4.0        3.4        3.8        3.5
…
9 TOTAL                  =SUMPRODUCT($B$2:$B$8,C2:C8)  (drag right)

Export & templates

Copy the JSON skeleton into your tool of choice (Notion DB, Coda, Airtable, sheet). It mirrors the structure above so vendor responses, criterion scores, and weights all live in one schema.

{
  "scorecard_version": "1.0",
  "weights": {
    "capability": 0.22, "performance": 0.18, "cost": 0.10,
    "compliance": 0.20, "integration": 0.15, "operations": 0.08,
    "commercial": 0.07
  },
  "kill_switches": [
    {"id": "K1", "label": "SOC 2 Type II current", "owner": "security"},
    {"id": "K2", "label": "GDPR DPA signed",       "owner": "legal"}
  ],
  "criteria": [
    {"id": "C1", "category": "capability", "label": "Doc coverage by country", "weight_multiplier": 2},
    {"id": "C2", "category": "capability", "label": "Biometric matching",      "weight_multiplier": 1},
    {"id": "C3", "category": "capability", "label": "Liveness (passive + active)", "weight_multiplier": 2}
  ],
  "vendors": [
    {
      "name": "Vendor A",
      "kill_switches_passed": ["K1", "K2"],
      "scores": {"C1": 4, "C2": 4, "C3": 5}
    }
  ]
}