Duplicate Records Deduping

According to MelissaData, an estimated 10 percent of the names and addresses in your average mailing list are duplicate records.

If 10 percent of your list is made up of duplicate records, you could be wasting a significant amount of money every time you mail. It can also be embarrassing. What if a donor or customer receives duplicate mailings from your organization? It comes across as sloppy on top of being a waste.

It doesn’t matter where the duplicate records come from – they are a costly problem.

Luckily, this problem has a solution. Deduping your data is one of the most critical steps in maintaining a quality database. Suppression can be a valuable savings and list cleaning tool as well.

What are the differences between deduping and suppression? How do the techniques work? Read on to learn more about suppression versus normal and priority deduping.

 

3 Types of Deduping

Deduping is simply the process of removing duplicate records from your data.

1. Normal Dedupe: Duplicate records are removed from within a single list and/or across multiple lists. Most of Badger’s customers use this simple deduplication type.

2. Priority Dedupe: Duplicate records are removed from multiple lists in priority order (i.e. which mailing list the duplicate records should be removed from first). All remaining, non-duplicate records will receive mail. For example, some nonprofit customers delete records from a list of smallest gift amounts first when they know those donors also appear on a list of higher gift amounts.

3. Suppression: A suppression list is a “Do Not Mail” list. It contains records that should be removed from all other lists. You may have an internal suppression list of people you don’t want to receive your communications. It can also include the Direct Marketing Association (DMA) Do-Not-Mail list, deceased suppression, and prison suppression.

Dedupe Criteria

When removing duplicate records from your data for a direct mail campaign, you have options that allow you to be as efficient and targeted as you would like. Here are your options for deduping criteria:

Individual (One Per Name): The mailing will contain only one mail piece per person at an address. For example, Jayne Smith at 123 Main St. would receive a mail piece, and John Smith at 123 Main St. would also receive a mail piece.

Residence (One Per Address): The mailing will contain only one mail piece per address or location. For example, Jayne Smith would receive the mail piece at 123 Main St., not John Smith (or vice versa). This method deletes the most duplicates. It is helpful when you want to reduce your mailing quantity or when you don’t have “clean” name data to accurately dedupe by another method.

Family (One Per Last Name at an Address): The mailing will contain one mail piece per different last name at an address. Jayne Smith, John Smith, and Tom Jones all live at 123 Main St. One of the Smiths would be deleted, so one Smith and the one Jones would receive the mailing.

Company (One Per Business Name at an Address): Taco Bell at 123 Main St. in Fort Atkinson, WI, and Taco Bell at 123 Main St. in Dubuque, IA, would both receive the mailing. Neither would receive more than one mail piece at that same location. This criterion is similar to one per residence, but would be used for scenarios when a restaurant is located within a gas station (same address, different businesses).

The Badger Group also has the capability to determine duplicate records based on other non-address fields (ex. by customer number).

Example of a Suppression Dedupe

Here is an example of how deduplication and suppression work together to remove data records from a mailing:

Duplicate Records Suppression Dedupe

After removing the suppression records from Lists 1, 2, and 3, the entire suppression list is removed.

Notice that records A in Lists 2 and 3 will both be deleted but record A is not found in the suppression list. These two A’s get deleted because A is found in List 1, making them duplicates. The same goes for record N in List 3, already found in List 2.

If the above example did not have a suppression list, duplicate records would still be removed as shown. The remaining non-duplicate records in List 1 – record G in this case – would remain.

 

All organizations struggle with data in some way. Poor data quality wastes revenue and potentially beneficial audience insights. Maintaining clean data can lead to greater success in your marketing or fundraising activities.

 

Want to learn more from our internal data and direct mail experts? Sign up for monthly email newsletters, Badger Blast and Nonprofit Insights.