Any ideas for dealing with Massive duplicated recrods in Rasier'edge

Wilson Li · August 2016

Any ideas for dealing with Massive duplicated records in Rasier'edge

Gina Gerhard · August 2016

You may want to contact Zeidman Development for assistance.

Faith Murray · August 2016

If you can't or don't wish to enlist the aid of an outside firm, you can try the Duplicate Constituent Management tool found under Admin, or the mass merge utility plugin, available for purchase on an annual basis (read about it Here). But to warn you that the Duplicate constituent tool will take many hours of intense work if you have thousands of duplicate records. The plugin might be a better option.

Jennifer Lucarelli · August 2016

I've used both the PLUG IN (which we paid for) and the Mgmt Tool, which is free. I honestly like the tool better. It is tedious. But if you just devote a small chunk of time to it every day (or even once a week) it gets you where you want to be. It took me about a year to get our database clean after a merger. I used both tools. I felt like the plug-in did more harm than good. Not because it didn't work right, but because there were just too many variables.

Stephanie Crawley · August 2016

Wei Li:

Any ideas for dealing with Massive duplicated records in Rasier'edge

You might want to consider purchasing MergeOmatic from Omatic, which is a Blackbaud Partner. I wanted to purchase it this year, but I couldn't get approval to add it to the fiscal budget.

Catherine Burns · August 2016

Wei Li:

Any ideas for dealing with Massive duplicated records in Rasier'edge

We got MergeOmatic in January this year when they were offering a subscription offer, instead of paying for each merge, and have since merged over 16,000 duplicate records. I evaluated the Mass Merge tool and decided that MergeOmatic was better for our needs. It is still pretty new and new features have been added since we got it, including a couple of my ideas which have being implemented and released already. The best thing about it is you can use inclusion and exclusion records to search for and review duplicates so you can focus on your most important records first - e.g major donors and it finds duplicates for these records in your whole database. And this means you can stay on top of new duplicates being created too.

I created lots of inclusion and exclusion queries to separate inactive and active records, those with and without spouses, and those with and without solicit codes because we needed to take precautions to ensure they were merged correctly and I had volunteers helping. They queue records for merge and I reviewed them before merging.

Phillip Bicknell · August 2016

Wei Li:

Any ideas for dealing with Massive duplicated records in Rasier'edge

An ideal opportunity for a volunteer to become a lot more familiar with Raiser's. I should know - a list of 2000 records was one of my tasks when I was "just" a volunteer. Now I'm an employee

Annette Rurka · November 2017

Catherine Burns:

Wei Li:

Any ideas for dealing with Massive duplicated records in Rasier'edge

We got MergeOmatic in January this year when they were offering a subscription offer, instead of paying for each merge, and have since merged over 16,000 duplicate records. I evaluated the Mass Merge tool and decided that MergeOmatic was better for our needs. It is still pretty new and new features have been added since we got it, including a couple of my ideas which have being implemented and released already. The best thing about it is you can use inclusion and exclusion records to search for and review duplicates so you can focus on your most important records first - e.g major donors and it finds duplicates for these records in your whole database. And this means you can stay on top of new duplicates being created too.

I created lots of inclusion and exclusion queries to separate inactive and active records, those with and without spouses, and those with and without solicit codes because we needed to take precautions to ensure they were merged correctly and I had volunteers helping. They queue records for merge and I reviewed them before merging.

Catherine, I know that it's been a while since you posted this response but could you let me know if you compared Raiser's Edge Duplicate Constituents utility or Mass Merge Plug-In with MergeOmatic before deciding on using MergeOmatic? I'm researching the various tools available before we decide which one is the best to tackle our duplicate record backlog. Any feedback you can give on features, etc. would be greatly appreciated. Thank you, Annette

Jen Claudy · November 2017

Zeidmann Development offers a tool called Mergician, as well. I haven't used it, but I've used their Importacular product, and like it.

Catherine Burns · November 2017

Annette Rurka:

Catherine Burns:

Wei Li:

Any ideas for dealing with Massive duplicated records in Rasier'edge

We got MergeOmatic in January this year when they were offering a subscription offer, instead of paying for each merge, and have since merged over 16,000 duplicate records. I evaluated the Mass Merge tool and decided that MergeOmatic was better for our needs. It is still pretty new and new features have been added since we got it, including a couple of my ideas which have being implemented and released already. The best thing about it is you can use inclusion and exclusion records to search for and review duplicates so you can focus on your most important records first - e.g major donors and it finds duplicates for these records in your whole database. And this means you can stay on top of new duplicates being created too.

I created lots of inclusion and exclusion queries to separate inactive and active records, those with and without spouses, and those with and without solicit codes because we needed to take precautions to ensure they were merged correctly and I had volunteers helping. They queue records for merge and I reviewed them before merging.

Catherine, I know that it's been a while since you posted this response but could you let me know if you compared Raiser's Edge Duplicate Constituents utility or Mass Merge Plug-In with MergeOmatic before deciding on using MergeOmatic? I'm researching the various tools available before we decide which one is the best to tackle our duplicate record backlog. Any feedback you can give on features, etc. would be greatly appreciated. Thank you, Annette

Yes, I compared both. We had been using the Dupliicate Constituent Utility for a year or so with volunteers getting through a number of merges. But it found a lot of false possible duplicates where there was missing data which - e.g. if we only had a name and email address and the names weren't even a match but blank address matched blank address! So it was slow very going and not being able to adjust the criteria to suit our data was a major problem. I looked at the Mass Merge plug-in too but it had the same problems as the Constituent tool e.g we wanted to retain constituent attributes but needed to know what attributes came from the duplicate and which came from the survivor record. This was not possible with the mass merge tool but MergeOmatic retains information from the duplicate as a detailed note, and you can decided if you want to merge info or just keep it as the note. We decided not to merge attributes so only those from the survivor are on the attribute tab but if we ever want to know the attributes that were on the duplicate they are in the note.

We continue to use mergeomatic. We also use ImportOmatic and they are now integrated but the duplicate matching in MergeOmatic is still much better. We recently merged with another organisation and I had to import 15,000 records. Using ImportOmatic there were still too many false positives and it would have taken many hours to resolve each one individually as the data was imported so I turned off the duplicate check on ImportOmatic, imported them all, and then used MergeOmatic to find the most likely duplicates and merged them in bulk. It still took a while as there were about 4000 probably duplicates, but it also saved many hours work. Being able to deal with the most likely duplicates first and then work down to the less likely duplicates is also a big advantage.

We are now on top of our duplicate problem but I still use MergeOmatic before every major mailing to be sure no new duplicates have been created or found as other data is cleaned.

Donna Mainord · November 2017

Wilson Li:

Any ideas for dealing with Massive duplicated records in Rasier'edge

Hi Wilson,

I feel your pain. I worked on them as I could via the Dup. Constituent Management Tool. I had a student working in my office who was really good and I trained her to help me with the clean up. If you work at a university/college you might consider hiring a student to help you out. She was very cost effective and did a wonderful job.

P.S. She had been working with me for a few months doing data entry and was comfortable working in RE.

Any ideas for dealing with Massive duplicated recrods in Rasier'edge

Comments

Categories