Duplicates from data migration

Brian Mucha:

Elisabeth Platt:

We are slowly transitioning to using Luminate Online and have migrated our data from our previous online tool and also used the RELO connector to bring Raiser's Edge data in to Luminate. As part of this data integration process our duplicate records in Raiser's Edge skyrocketed. We now have numerous individuals in Raiser's Edge with only an email address and name. I was wondering how others manage duplicates between the systems when so little data is available to adequately know if two records are duplicates. My fear is that I will forever have around 1,000 duplicates and not know which ones are new and legitimate and which ones I've looked at multiple times. Thanks for any best practice advice for making de-duping more manageable and efficiant.

Boy, I wish we only had a 1000!


We have the RE connector set to only sync constituents that have an associated transaction. That way those low-info records don't get into RE to begin with.


In the Luminate duplication resolution tool you can mark a record as NOT a duplicate. That should keep it from being looked at multiple times. Not sure about the process on the RE or 3rd party side.


There is another recent thread about email sign-up forms that touched on this issue. If your survey sign-up only offers 3 fields, then this is where you wind up. If you add zip code to that form, then you can get the city and state. Suddenly you have a record with first name, last name, email address, city, state and zip code. Not bad!


I wrote a Jquery geocoding script to do that address lookup right on the sign-up form. I wouldn't be surprised if there was a plugin or service for RE that also does this.


If your 3-field records are from purchased or otherwise imported lists, then I'd say this is another reason why those are not a great idea. The few times we are forced to send to such a list, I push to do it from a 3rd party like mailchimp, and then gather the records of only those few that convert by donating or whatever.

Brian Mucha:

We're using the Connector for Datasync, which was written sometime in the last century.


We're in the process of switching over to the Omatic sync, where this is not an option. Everything syncs with Omatic. I'm not sure about RELO since pretty much everyone - including Blackbaud, amazingly enough - steered us away from it.


So I suspect we will be in exactly the same boat if we ever get this update done. Omatic does have a duplicate management tool called MergeOmatic, but I'm not really confident we have a definitive answer on what it's capabilities are.


Updating the datasync is turning into a giant pain in the butt. Between BB not endorsing their own product to Omatic delaying giving us a price for weeks, I'm starting to think it will never happen.