Data Quality & Trust

Trust is the product

Restating chapter 01 because it governs this whole topic: a dashboard nobody believes is worse than none — it generates re-checking, shadow spreadsheets, and lost credibility. Your early roadmap should optimize for trust in a few metrics, not coverage. Every quality mechanism below exists to protect that trust.

The dimensions of data quality

Have a vocabulary for what "quality" means — interviewers want structured thinking, not "I'd add some tests."

Where to test — at the boundaries

Test where data enters and where it's consumed, not everywhere (that's noise).

• Source / ingest: schema and freshness — catch upstream changes immediately.
• Staging: uniqueness of the dedup key, not-null on keys, valid enums.
• Marts: grain uniqueness, referential integrity to dimensions, business invariants (e.g. revenue ≥ 0), reconciliation against source-of-truth.

The shift-left principle: catch problems as early in the pipeline as possible — a bad row caught at staging is cheaper than a wrong number caught in a board deck.

Test types (and the dbt vocabulary)

dbt is the lingua franca; know the four built-in generic tests and what a custom test is.

• Generic: unique, not_null, accepted_values, relationships (foreign-key). Declared in YAML, run on every build.
• Singular: a SQL query that should return zero rows; any rows = failure. This is how you encode business rules.
• Packages: dbt_utils / dbt_expectations add row-count ranges, recency, distribution, mutually-exclusive-ranges, etc.
• Severity: error blocks the build; warn surfaces without stopping. Use warn for noisy/soft checks so alerts stay meaningful.

models:
  - name: fct_usage_daily
    columns:
      - name: usage_id
        tests: [unique, not_null]
      - name: customer_id
        tests:
          - not_null
          - relationships:
              to: ref('dim_customer')
              field: customer_id
      - name: status
        tests:
          - accepted_values: { values: ['active','stopped','failed'] }

-- Fails the build if any row violates the rule
SELECT * FROM {{ ref('fct_usage_daily') }}
WHERE gpu_seconds < 0;

Freshness & volume — the two that catch silent breaks

Most "silent" data incidents are a feed that stopped or halved. These two checks catch the majority of real-world breakage for almost no effort:

-- Freshness: alert if newest data is too old
SELECT 'usage_events stale' AS issue
WHERE (SELECT MAX(loaded_at) FROM usage_events) < NOW() - INTERVAL '2 hours';

-- Volume: alert if today is wildly off the trailing average
WITH daily AS (
  SELECT DATE(started_at) d, COUNT(*) n FROM usage_events GROUP BY 1
)
SELECT * FROM daily
WHERE d = CURRENT_DATE
  AND n < 0.5 * (SELECT AVG(n) FROM daily WHERE d >= CURRENT_DATE - 14);

dbt has native source freshness checks; mention them. As a first hire, freshness + volume + key-uniqueness on your handful of trusted metrics is 80% of the value for 20% of the effort.

Data contracts

An agreement with upstream producers about schema, types, semantics, and SLAs — so a backend deploy doesn't silently break your pipeline. The concept matters more than any tool.

• What's in a contract: column names & types, nullability, allowed values, freshness guarantee, and a change/deprecation process.
• Enforcement: schema tests at ingest that fail loudly; dbt's contract config that enforces declared column types on a model's output.
• First-hire reality: you often can't impose formal contracts on a small eng team early. The pragmatic version is a schema test that pings you the moment a column changes, plus a lightweight agreement: "tell me before you rename a field." Show you understand both the ideal and the pragmatic.

Anomaly detection

Fixed-threshold tests miss gradual drift and don't adapt to growth. Anomaly detection flags statistically unusual values:

• Simple & effective: compare today's metric to a trailing window using z-score or percent-change bands. Catches "revenue dropped 40% overnight" without hard-coding a number.
• Seasonality: weekday/weekend and month-boundary effects — compare like-for-like (this Monday vs prior Mondays) or you'll cry wolf.
• Tooling: Elementary (dbt-native, cheap to start), Monte Carlo / Bigeye (commercial observability). First-hire lean: start with Elementary or a few custom z-score tests; don't buy a platform on day one.

Observability & lineage

Observability = knowing the health of your data over time across five pillars: freshness, volume, schema, distribution, lineage. Lineage is the map of what depends on what.

• Why lineage matters for a solo hire: when something breaks, lineage tells you the blast radius — which dashboards and metrics are affected — so you can communicate impact accurately. dbt's DAG gives you this for free.
• Tools: dbt docs/exposures (free, comes with your project), OpenLineage, DataHub. Don't over-invest early; dbt's built-in graph covers a one-person stack.

SLAs & severity

Not all data is equally critical. Set expectations explicitly so you're not paged at 3am for a non-critical model.

• Tier your data: Tier 1 = revenue/billing/exec metrics (tight freshness, hard alerts); Tier 2 = team dashboards (looser); Tier 3 = exploratory (best-effort).
• An SLA is a promise: "the daily revenue model is fresh by 8am on business days." Publish it so stakeholders know what to expect and you know what to defend.
• Severity routing: Tier-1 failures page you; Tier-3 warnings go to a channel you check. Keeps alerting trustworthy — alert fatigue is itself a quality failure.

Running a data incident

Incidents are inevitable; how you run them defines your credibility. The sequence:

1. Acknowledge fast & communicate. "We've spotted an issue with today's revenue number; investigating, update in 30 min." Silence is what erodes trust, not the bug.
2. Contain. Mark the affected dashboard stale / hold the number so no one acts on bad data.
3. Diagnose via lineage. Walk upstream — source freshness? volume drop? schema change? a logic bug in a recent merge?
4. Fix & backfill. Idempotent pipelines (chapter 04) let you reprocess the affected partitions cleanly.
5. Add a test so it can't recur silently. Every incident becomes a new tripwire. This is how a one-person team compounds reliability.
6. Brief, blameless write-up. What broke, impact, fix, prevention. Builds organizational trust in the data function.

The "the number looks wrong" scenario (script)

A near-universal interview prompt. Don't jump to "I'd check the SQL." Show a triage framework:

1. Reproduce & quantify. "Which number, which view, how wrong, since when?" Pin it down before theorizing.
2. Is the data wrong, or the expectation? Often the number is right and the definition differs ("active" means what?). Half of these are definition mismatches — check that first; it's cheap.
3. Work the pipeline top-down: source freshness/volume → staging dedup → join fan-out → metric logic → BI-layer filter. Lineage guides the walk.
4. Reconcile to a source of truth (e.g. metered usage vs the billing system — problem 12).
5. Communicate throughout, fix idempotently, add a test.