Rules and ethics
Part of Customer data quality CDP UK: how to clean before implementation
Customer data quality CDP UK: how to clean before implementation
Customer data quality work in a UK CDP: the accuracy duty under UK GDPR, the five pre-build checks and the order to run deduplication and survivorship rules.
What to take away
- Accuracy is an obligation under UK GDPR Article 5(1)(d), which makes cleaning a compliance task before it is a build task.
- Deduplication rules decide which record survives, and that is a business judgement rather than a platform default.
- A documented clean you can re-run beats a one-off tidy, because it gives you a baseline you can defend months later.
- Cleaning before implementation removes work that would otherwise reappear as emergency field mapping during the build.
- Every rule needs an owner, a date and a rollback, so someone who was not in the room can follow what happened.
Why accuracy sits at the front of implementation
The accuracy principle in the UK-retained GDPR requires personal data to be accurate and, where necessary, kept up to date. It also expects organisations to erase or rectify inaccurate records without delay, which turns ingestion into a compliance stage rather than a plumbing task.
Quality problems usually surface in the first weeks of a build: two records for one person, a stale email address, a consent flag on the wrong profile. The ICO's UK GDPR guidance and resources hub covers how accuracy interacts with retention and individual rights, so read it before you write a single rule.
Most teams file this under data engineering. It is closer to a decision log. Each rule needs an owner, a reason and a date, because the rule will later be read by someone who was not in the room.
The five checks worth running before ingestion
Run each check against every source system, not only the largest one.
Five pre-ingestion data checks
- Field inventoryformat, refresh rate, purpose
- Duplicate countstrongest key, then softer combinations
- Accuracy samplerandom records, trusted source, error rate
- Consent statepermissions, opt-outs, suppression survive merge
- Survivorship ruleswhich value wins per field
If you are choosing match keys for the first time, how to handle identity resolution in UK CDPs sets out the trade-offs without assuming a vendor.
Matching rules, and the order to run them
Deterministic matching on normalised keys goes first, because an exact email or mobile match is easy to explain and easy to reverse. Probabilistic matching on name, postcode and date of birth goes second, and it should route to a review queue rather than merge on its own. Record linkage methods are well documented; the false-merge risk is the part with commercial consequences.
Matching order and merge handling
- Normalise keys first
- Deterministic matching on exact email or mobile
- Probabilistic matching on name, postcode, date of birth
- Route probabilistic matches to review queue
- Never auto-merge on probabilistic match
A false merge does more damage than a duplicate. A duplicate inflates a count and can be cleaned later. A false merge attaches one person's purchases or complaints to another person's profile, and unpicking it means rebuilding history.
| Check | Evidence artefact | Typical owner |
|---|---|---|
| Field inventory | Source-to-field map with refresh rates | Analytics lead |
| Duplicate count | Counts per key, with the key named | Data engineer |
| Accuracy sample | Sample size, fields checked, error rate | Marketing operations |
| Consent state | Permission and suppression report | Privacy lead |
| Survivorship rules | Field-by-field precedence list | Business owner for the field |
What to record so the clean is repeatable
Version your rules the way you would version code. Each entry should carry a rule ID, the input field, the transformation, the date it took effect and the person who approved it.
Then schedule the clean to run again. Freshness decays as people change jobs and move house, and a one-off project leaves you no way to notice the drift.
Keep a short list of what you could not resolve: unmatched records, disputed merges, fields with no trustworthy source. That list belongs with the build, and the same discipline applies when implementing a customer data platform reversibly, where production activation stays switched off until the evidence holds.
Common questions
Does cleaning happen inside the CDP or before it?
Before. A platform can enforce rules, but it cannot tell you which record is right, and a supplier's default survivorship setting is not a business decision.
How do we know a record is accurate enough?
Set the standard per field. An email address can be tested against a delivery response; a job title usually cannot, so it carries a lower expectation and a shorter review cycle.
Can we clean after go-live?
Yes, but the cost changes. Cleaning after activation means correcting live segments, which risks messages reaching the wrong person while the fix runs.
Who should approve survivorship rules?
The business owner of each field, not the platform team. Marketing operations can implement the rule, but the choice of which value wins is a commercial one.
