Data Cleansing

Nobody notices bad data until it costs them something.

The same customer is in the system four times under slightly different names, so nobody can tell you how many customers you’ve got. Post goes to an address someone left in 2019. A report gives two different answers depending on who runs it. A mailshot goes out twice to the same person and once to somebody who asked to be removed.

Data cleansing is the process of detecting and correcting or removing corrupt, incomplete, inaccurate and duplicate records. We’ll work with you to sort yours out. ☎

When people usually come to us

Before a migration. Far and away the most common. There’s no sense carrying twenty years of duplicates into a new system — and a migration is the one moment when fixing it is straightforward. → database migration

When the reporting stopped being trusted. If the customer count is wrong, no reporting tool will fix it. → reporting and business intelligence

After merging two systems, or two businesses. Both had a customer list. Now you have one list with everybody on it twice.

When a data protection question gets asked and it turns out you can’t answer it — because the person is in the system three times and you’re not confident you’ve found them all.

When someone finally counts the cost of the post, the wasted calls, the chasing, and the time people spend working around it.

What we actually do

[IMG-1] — Before/after on real records: the duplicates, the variants, and the merged result. Redacted. This is an invisible service and the only way to show it is to show it.
Alt: Duplicate customer records identified and merged

Correcting and validating. Spelling, obvious errors, and values checked against a known list. Validation can be strict — rejecting any address without a valid postcode — or fuzzy, correcting records that partially match ones you already know are right.

Cross-checking against data you trust. Where there’s a reliable source, we use it rather than guessing.

Harmonisation. Making the same thing look like the same thing: St into Street, Ltd into Limited, Rd into Road, and the forty variations of your own company name that have accumulated.

Standardisation and deduplication. Eliminating duplicate and multiple records — including the near-misses that exact matching never finds, which is where most of the real duplication hides.

Address and postcode correction, so post arrives and geography-based reporting means something.

Cleansing is not validation, and this is the bit that matters

Cleansing fixes what’s already there. Validation stops it happening again — data gets checked as it’s entered and rejected if it doesn’t comply.

Once the cleansing is done, proper validation is essential, otherwise you’ll be paying to cleanse the same data again in three years.

So we don’t just clean. We look at how the bad data got in — the free-text field that should have been a list, the form with no postcode check, the import that’s been silently creating duplicates every month — and fix that too.

Cleansing without validation is mopping the floor with the tap running.

Data enhancement

We can also make data more complete, rather than just more correct — appending related information you don’t currently hold, such as phone numbers against an address, or filling gaps from a reliable source.

Useful before a marketing campaign, and useful when your records are accurate but thin.

Nothing changes until you’ve seen it

People are understandably nervous about letting somebody loose on their live data. So:

• We work on a copy. Always.
• We agree the rules with you first — what counts as a duplicate, which record wins when two disagree, what gets merged and what gets flagged for a person to look at
• You see a sample before anything runs at scale, on real records you recognise
• Everything is reversible. The original is kept, and you get a record of what changed.
• The grey cases come back to you, rather than being decided by an algorithm and quietly merged

Most projects start with a free look at a sample of your data, so you know what you’re dealing with before committing to anything.

A note on data protection

Accurate data isn’t only a commercial matter. UK GDPR expects personal data to be accurate and kept up to date, and not held longer than necessary.

In practice, a business that can’t reliably find every record for one person can’t properly honour a request to see or delete their data — and duplicated records are the usual reason. Cleansing tends to be the thing that makes that answerable.

We handle client data under a written agreement, and we’ll tell you plainly what we hold, where, and for how long.

Case studies

all case studies