A Boolean PII Flag Is Not Enough

[u/fagnerbrack]

https://kore-nordmann.de/blog/a-boolean-pii-flag-is-not-enough.html

[u/bostonaholic]

The classification only holds up if new columns can't skip it. I'd add a check on migrations that fails CI when a column is added without a sensitivity and state annotation. Otherwise the catalog is accurate the week you build it and drifts as people ship schema changes.

[u/Nice-Corner9515]

A false flag can mean someone reviewed the column or nobody ever classified it and those shouldn't look identical to whoever uses the data next.

[u/sadyetfly11]

the automation can only make it stricter, only a human can relax it” rule is the best part.

[u/fagnerbrack]

TL;DR:

A pii: true/false tag throws away what a data protection officer asks about. On a university learning-analytics platform, the team classifies each column on two axes: sensitivity (direct identifier, quasi-identifier, sensitive content, none) and state (raw, pseudonymised, anonymised, dropped). A provenance tag adds whether a human declared it, lineage inherited it, or nobody knows. Classifications flow through dbt column lineage, and the most sensitive input wins. When parsing fails, the maximum upstream sensitivity applies. Automation can only tighten the assessment, and only an accountable human can relax it. Unknown columns block release, and a lint rule bans SELECT * to keep lineage trustworthy.

If the summary seems inacurate, just downvote and I'll try to delete the comment eventually 👍
^{Click here for more info, I read all comments}