Governance, Privacy & Compliance for ML Data

Why models change the rules

Governance on an analytics platform is about tables: who can read them, how long they are kept, which columns are sensitive. Governance for ML has to cover two more things. The first is derived artifacts: a training set is a copy of governed data, a feature table is a transformation of it, a model is a lossy compression of it, and each of those inherits the obligations of its source. The second is purpose: the same rental history that is legitimately used for billing may not be legitimately used to train a model that prices a customer differently, depending on what that customer consented to and what the regulator thinks a marketplace may do.

The practical consequence is that the ML data plane needs the same four instruments the rest of the platform has — classification, access control, lineage, and retention — applied one level further, to datasets, features, and models. The bad version of this chapter is a compliance checklist. The good version is a set of structural decisions that make the compliant thing the default thing.

PII in training data

Start from classification, because nothing else works without it. Every column in silver and every feature in the registry carries a sensitivity tag — public, internal, confidential, pii, pii-special (financial account numbers, government IDs, KYC documents). Tags are set at the source contract and propagate through lineage, so a feature derived from a PII column is PII until someone explicitly declassifies it with a justification.

Then apply minimization at the feature boundary. Models rarely need the raw identifier; they need a signal derived from it. The pattern that keeps most training sets clean:

Two enforcement points make this real. The feature registry refuses to register a feature whose source columns are tagged pii-special unless a declassification review is attached. And the training-set builder refuses to run if any selected feature carries a tag above the model's approved tier. Both are simple policy checks against metadata you already have; they turn a memo into a build failure.

Consent & purpose limitation

GDPR, CCPA/CPRA, and most modern regimes tie data use to a purpose. Billing a rental is one purpose; training a dynamic-pricing model that may charge that customer more is a different one, and "improving our services" in a privacy policy does not automatically cover it. The engineering translation: consent is a feature of the entity, stored and versioned like any other, and it is a filter in every training-set builder.

-- Purpose-limited training set: only accounts whose consent state, AS OF the prediction time,
-- permitted use of their data for this model's registered purpose.
WITH consent_asof AS (
  SELECT l.account_id, l.prediction_ts, c.allows_model_training, c.jurisdiction
  FROM ml.churn_labels l
  ASOF JOIN silver.account_consent_history c
    MATCH_CONDITION (l.prediction_ts >= c.valid_from) ON l.account_id = c.account_id
)
SELECT ...
FROM consent_asof
WHERE allows_model_training = TRUE
  AND jurisdiction NOT IN (SELECT jurisdiction FROM policy.model_training_exclusions
                           WHERE model_purpose = 'churn_prediction');
-- The builder records: purpose, consent version, exclusion list version → model registry metadata.

Notice the join is point-in-time, like every other join in this guide. A customer who withdrew consent last week still legitimately appears in a training row from last year — the question is what was permitted then. And the builder records the consent-policy version it applied, so the model card can state exactly which population the model was allowed to learn from.

Retention & the deletion problem

A deletion request (GDPR Art. 17, CCPA deletion) is straightforward against tables: delete the rows, propagate through the lineage graph, expire the snapshots that still contain them. Against a model it is not: the account's data has been folded into weights and cannot be surgically removed. Machine unlearning exists as a research field and is not something you should promise a regulator. What you can do is architectural:

• Bounded retention on training data. Training sets and feature history are kept for a declared window (say 24 months for behavioural features, 90 days for raw events). Every model trained on data past its window is retrained or retired. This bounds the exposure of any individual's data in any model.
• Deletion propagates to derived sets on a clock. The deletion job removes the entity from silver, feature tables, the offline store, the online store, the served-feature log, and the vector store — then expires lakehouse snapshots older than the deletion so time travel cannot resurrect it. Iceberg and Delta both support this; you must actually schedule expire_snapshots or the data is still there.
• Retrain cadence as the deletion SLA. If the fraud model is retrained monthly on a rolling window, an account deleted today is absent from every model within a month. Write that cadence down as the SLA and monitor it.
• Exclusion lists for the interim. Between deletion and the next retrain, the entity is on a hard exclusion list at inference — no predictions are made about it and no features are read for it.

Access tiers & isolation

Access for ML follows the medallion tiers with two additions. Data scientists read gold and the feature store freely, silver with justification, bronze almost never — raw events are where the unredacted PII lives. The additions are per-feature access in the registry (a feature tagged pii is visible in the catalog to everyone but readable only by approved models and roles) and tenant isolation, which matters at GridDP because providers are businesses whose fleet telemetry is commercially sensitive.

Column masking and row-level security are catalog features in all three reference stacks (Unity Catalog, Snowflake Horizon, Polaris/Nessie + engine policies); the ML-specific work is making the feature registry and the model registry enforce the same tags rather than merely display them.

Audit for models

The auditor's questions are always the same four: what data trained this model, was that data permitted, who can read the outputs, and can you reproduce the decision it made for this person on this date. Answering them from memory is not an option; answer them from artifacts the platform produces automatically:

• Dataset card — generated by the training-set builder: source tables and snapshot IDs, feature list with sensitivity tags, consent-policy version, row count, class balance, retention window, builder commit hash.
• Model card — from the registry: dataset card reference, training code version, evaluation metrics by segment, intended purpose, approved tier, known limitations, the drift baseline.
• Lineage — the graph from source column to feature to dataset to model to serving endpoint (chapter 09), queryable both ways: "which models used kyc_outcome?" and "what fed ranking_v14?".
• Decision log — the served-feature log plus the prediction, request ID, model version and timestamp, retained for the regulatory window (five years for financial decisions is common). This is what lets you explain a specific fraud decline.

When these are produced by the pipeline rather than written by hand, an audit is a query. When they are not, an audit is a month.

Differential privacy & synthetic data, practically

Two techniques get invoked as magic words; both are useful in narrow places. Differential privacy adds calibrated noise so that any single individual's presence in a dataset has a bounded effect on the output. It is excellent for released aggregates — GridDP publishing regional utilization statistics to providers, or an internal dashboard exposed widely — and workable for training via DP-SGD when the model is small and the accuracy cost (which can be large) is acceptable. It is not something to bolt onto the ranking model to satisfy a policy; the utility loss on a high-cardinality tabular model is usually unacceptable, and a mis-set epsilon provides no real guarantee.

Synthetic data — generated to match the statistics of a real dataset — earns its keep for development environments (engineers build pipelines against realistic data without PII access), for sharing with vendors, and for augmenting rare classes. It does not earn its keep as a privacy substitute for training on the real thing: a generator overfit to a small population can reproduce real records. Rule of thumb: synthetic for non-production and for augmentation; real, minimized, purpose-checked data for the production model.

GridDP: KYC, crypto rails & payouts

GridDP's specifics sharpen every point above. KYC data — identity documents, vendor decisions — is pii-special; only the vendor outcome and score enter the platform, and the fraud model's dataset card must show it trained on outcomes, never documents. Crypto rails bring the Travel Rule and sanctions screening: wallet addresses are tokenized, screening results are features, and the transaction-monitoring model's decision log is retained for the AML window (five years) and queryable by transaction — that log is a regulatory record, not an ML convenience. Provider payouts make the fleet-telemetry isolation rule concrete: a capacity-forecasting feature like region_free_gpu_hours_7d is fine; a feature that reveals one identifiable provider's utilization to another is a commercial breach, so cross-provider features aggregate to at least k=5 hosts.

And the deletion story: an account requests erasure. The job removes it from silver and gold, the feature store (both stores), the served-feature log, the ticket-embedding vector store, and expires older snapshots; the account goes on the inference exclusion list; the next monthly retrain of churn, ranking and fraud drops it from the weights. Ledger entries and AML records are retained under the legal-hold exception and documented as such. The dataset cards for the next models show the deletion-list version they honoured.

When to choose what

Next: what all of this costs, and how to keep the GPUs busy without setting money on fire — 12 · Cost, Scale & Feeding the GPUs.

✓ Check yourself

Answer from memory before opening each one.

1. An engineer says a trained model is not personal data because the rows are gone. What is the planning model the chapter uses instead, and what operations does it force you to support?

   Show answer

   Treat the model as a materialised view over its training set with an unknown refresh function. Whatever you would do to a view when its source changes, rebuild, restrict or retire, you must be able to do to the model. A model therefore inherits the obligations of its sources. See why models change the rules.

2. A support-ticket embedding table is built from raw ticket text. The team says vectors are anonymous. What do you do before embedding and how do you tag the table?

   Show answer

   Redact PII before embedding, because vectors can be inverted and probed for membership. Tag the vector table with the highest sensitivity tag of its inputs, and keep it under the same retention as the source text. See PII in training data.

3. A customer withdrew consent last week and appears in a training row from last year. Is it a violation to keep the row? What does the builder record?

   Show answer

   No, if consent as of the prediction time allowed that use, which is why the consent join is point-in-time. The builder records the purpose, the consent-policy version and the exclusion list version it applied, so the model card states which population the model was allowed to learn from. See consent and purpose limitation.

4. An account requests erasure and your fraud model retrains monthly. What does the deletion job touch, what covers the weeks until the retrain, and why is promising unlearning a mistake?

   Show answer

   The job removes the entity from silver, feature tables, both stores, the served-feature log and the vector store, then expires older snapshots so time travel cannot restore it. An exclusion list blocks predictions about that entity until the retrain drops it from the weights. Unlearning is a research area, so do not promise it to a regulator. See retention and the deletion problem.

5. Reproducibility wants snapshots kept for years and erasure wants them gone. How do you resolve the tension?

   Show answer

   Set explicit retention windows. Keep pinned snapshots only inside the window, retrain or retire models trained on older data, and check that no registered model still references a snapshot older than the window. See retention and the deletion problem.

6. Product wants differential privacy on the churn model and synthetic data in place of real data to satisfy privacy. Where is each justified and where is it theatre?

   Show answer

   Differential privacy fits released aggregates and small models where the accuracy cost is acceptable. Synthetic data fits development, vendor sharing and rare-class augmentation, but a generator overfit to a small population can reproduce real records, so it is no substitute for governance on the real training data. See differential privacy and synthetic data.

Takeaway

• A model inherits the obligations of its training data, so think of it as a materialised view you must be able to rebuild, restrict or retire.
• Classify every column and feature, propagate tags through lineage, and minimise at the feature boundary so the model gets a signal, not the identifier.
• Make the compliant path the default by turning policy into build failures in the feature registry and the training-set builder.
• Consent is a point-in-time feature of the entity, and the builder records the policy version it applied.
• You cannot delete from weights, so bound retention, propagate deletions on a clock, expire snapshots, exclude the entity at inference and use the retrain cadence as the deletion SLA.
• Generate dataset cards, model cards, lineage and a decision log from the pipeline, so an audit is a query.

Governance sets what the data plane may do, and cost sets what it can afford. → Cost, Scale & Feeding the GPUs

Further reading

• Regulation (EU) 2016/679, the General Data Protection Regulation — EU Official Journal, 2016. The primary text, including Article 5 on purpose limitation and storage limitation and Article 17 on erasure.
• Regulation (EU) 2024/1689, the Artificial Intelligence Act — EU Official Journal, 2024. The data governance duties for high-risk systems and the documentation they require.
• Model Cards for Model Reporting — Mitchell et al., FAT* 2019. A structure for documenting purpose, evaluation and limits.
• Datasheets for Datasets — Gebru et al., 2018. Questions a dataset's creator should answer, the basis for a dataset card.
• The Algorithmic Foundations of Differential Privacy — Dwork and Roth, 2014. The standard reference for what differential privacy guarantees and what it costs.
• Presidio — Microsoft, accessed 2026. Detection and redaction of PII in text before it enters a training set or an embedding.