Trouble pulling large exports
Comments
-
What exactly is the issue? Are you timing out before it exports? Or getting a specific error message? We are also hosted and I regularly export files of that size (just to give you hope that it is possible!)1
-
Hi Kim-
Start by exporting only the Constituent Ids of everyone in your entire database. If that goes prettty quickly, the most likely culprit is the complexity of your export. If you have a lot of calculated fields such as gift summaries it's going to take some time to complete those calculations.0 -
I had this problem several years ago as a self-hosted client, and found the only way to address this is to get out of the GUI. I've been told by Blackbaud that there is a memory cap on the standard interface we all use to tell RE what to do. I have no idea if this will help in a hosted environment.
We are not hosted, so we work around this by using scheduled Queue. As a hosted client, I'm not sure how you would do that. You could also try using RE Mail to see if it will output your data for you (but we found that too limiting). RE Mail allows RE to do it's processing behind the scenes out of the GUI. We also have to output the files a .csv or Excel 1997-2003 (but row limits create a problem with the older version of excel). While in theory, more current versions of Excel should work smoothly, I've run into bugs I can't explain or fix and Blackbaud hasn't helped solve our Excel issues yet.
Our files have grown even larger and eventually started causing trouble for Query. So, to work around all of this I simplified our gift coding structure and changed campaigns, appeals and fund to streamline the output. It was a lot of work, but today, it was worth the effort.
Good luck and I hope you can figure out something that works for you.0 -
You will need to use Queue and have the exports saved to Blackbaud's servers. Then download the files using an FTP client, such as FileZilla. Also, if the export is large enough, I would recommend scheduling the queue to run at night or over the weekend. I've tried to run a queue during work hours that would take nearly all day to finish, whereas running it after work hours will only take 30 minutes.
I have several large exports I run, most weekly, some nightly. I'm pretty much exporting our entire database twice - one for Reeher and another for a local data warehouse we use for reporting. There are times when the Queue will not finish processing for some reason. That might be due to system maintenance on BB's end. Mailing from Queue doesn't work in hosted environment.1 -
@Claudia Sauers My colleague recently exported our annual appeal mailing list and it took 20 hours. Have you utilized the Queue module to assist in large exports? Thank you.
1 -
@Kim Kruger - In my experience an export of this size while hosted can take upwards of 8+ hours, depending on how much summary information (calculations) needs to happen for each constituent. If you have multiple computers/accounts available you can break the export up into smaller pieces, and run the smaller sections simulanteously. As others have said, you could possibily utilize Queue to assist with larger exports as well.
0 -
@Kim Kruger:
I am really struggling pulling large exports (over 85,000 records) from Raiser's Edge. We just recently went hosted and I have not been able to successfully pull a large data export that our mail vendors need. I have done all of the Blackbaud suggestions such as requesting an extension for time out, making the queries static,reducing the fields that I could, etc. Nothing is working! If anyone else has any other suggestions to help I would appreciate it.If Queue isn't an option for you, which does have an additional cost, I definitely recommend exporting your data in smaller groups. I have done two things in the past:
- Break your audience up into multiple queries if you can. For instance, use your original query but update to only pull Organizations. Then save other versions that pull Individuals but only if their last name begins with A-M or N-Z. You may need more than two Individual queries, or don't even have Organizations in your query, but I think you get the idea. Once you're done, you'll need to manually copy and paste everything into a single file. A big pain, but I've had to use this method before.
- Another option is to export only contact information first, such as name, addressee/salutation, and address info. Then export Constituent ID along with other information needed on the output file - maybe some gift information. Keep exporting logical sets of data, including the ID for all of them. Familiarize yourself with doing joins in Access, or using VLOOKUP in Excel, because that is how you'll need to piece the files together in the end.
Once you get through this immediate project, I highly recommend talking with your direct mail vendor to understand why they need so much detail.
If Queue is an option, consider it. It was a few years ago that I had it available, but it never worked even in a Blackbaud-hosted environment. It was tested A LOT by me, and Blackbaud support did a lot of troubleshooting too, so it isn't like I tried it once and gave up. I think we had it at a very large organization prior to that, but we could never get it to work their either, so I am personally not convinced that it is worth it. I have NO idea if Blackbaud would let you try it to make sure it is successful or not, but worth asking.
Best of luck getting this done!!
Karen
0 - Break your audience up into multiple queries if you can. For instance, use your original query but update to only pull Organizations. Then save other versions that pull Individuals but only if their last name begins with A-M or N-Z. You may need more than two Individual queries, or don't even have Organizations in your query, but I think you get the idea. Once you're done, you'll need to manually copy and paste everything into a single file. A big pain, but I've had to use this method before.
Categories
- All Categories
- Shannon parent
- shannon 2
- shannon 1
- 21 Advocacy DC Users Group
- 14 BBCRM PAG Discussions
- 89 High Education Program Advisory Group (HE PAG)
- 28 Luminate CRM DC Users Group
- 8 DC Luminate CRM Users Group
- Luminate PAG
- 5.9K Blackbaud Altru®
- 58 Blackbaud Award Management™ and Blackbaud Stewardship Management™
- 409 bbcon®
- 2K Blackbaud CRM™ and Blackbaud Internet Solutions™
- donorCentrics®
- 1.1K Blackbaud eTapestry®
- 2.8K Blackbaud Financial Edge NXT®
- 1.1K Blackbaud Grantmaking™
- 527 Education Management Solutions for Higher Education
- 1 JustGiving® from Blackbaud®
- 4.6K Education Management Solutions for K-12 Schools
- Blackbaud Luminate Online & Blackbaud TeamRaiser
- 16.4K Blackbaud Raiser's Edge NXT®
- 4.1K SKY Developer
- 547 ResearchPoint™
- 151 Blackbaud Tuition Management™
- 1 YourCause® from Blackbaud®
- 61 everydayhero
- 3 Campaign Ideas
- 58 General Discussion
- 115 Blackbaud ID
- 87 K-12 Blackbaud ID
- 6 Admin Console
- 949 Organizational Best Practices
- 353 The Tap (Just for Fun)
- 235 Blackbaud Community Feedback Forum
- 124 Ninja Secret Society
- 32 Blackbaud Raiser's Edge NXT® Receipting EAP
- 55 Admissions Event Management EAP
- 18 MobilePay Terminal + BBID Canada EAP
- 36 EAP for New Email Campaigns Experience in Blackbaud Luminate Online®
- 109 EAP for 360 Student Profile in Blackbaud Student Information System
- 41 EAP for Assessment Builder in Blackbaud Learning Management System™
- 9 Technical Preview for SKY API for Blackbaud CRM™ and Blackbaud Altru®
- 55 Community Advisory Group
- 46 Blackbaud Community Ideas
- 26 Blackbaud Community Challenges
- 7 Security Testing Forum
- 1.1K ARCHIVED FORUMS | Inactive and/or Completed EAPs
- 3 Blackbaud Staff Discussions
- 7.7K ARCHIVED FORUM CATEGORY [ID 304]
- 1 Blackbaud Partners Discussions
- 1 Blackbaud Giving Search™
- 35 EAP Student Assignment Details and Assignment Center
- 39 EAP Core - Roles and Tasks
- 59 Blackbaud Community All-Stars Discussions
- 20 Blackbaud Raiser's Edge NXT® Online Giving EAP
- Diocesan Blackbaud Raiser’s Edge NXT® User’s Group
- 2 Blackbaud Consultant’s Community
- 43 End of Term Grade Entry EAP
- 92 EAP for Query in Blackbaud Raiser's Edge NXT®
- 38 Standard Reports for Blackbaud Raiser's Edge NXT® EAP
- 12 Payments Assistant for Blackbaud Financial Edge NXT® EAP
- 6 Ask an All Star (Austen Brown)
- 8 Ask an All-Star Alex Wong (Blackbaud Raiser's Edge NXT®)
- 1 Ask an All-Star Alex Wong (Blackbaud Financial Edge NXT®)
- 6 Ask an All-Star (Christine Robertson)
- 21 Ask an Expert (Anthony Gallo)
- Blackbaud Francophone Group
- 22 Ask an Expert (David Springer)
- 4 Raiser's Edge NXT PowerUp Challenge #1 (Query)
- 6 Ask an All-Star Sunshine Reinken Watson and Carlene Johnson
- 4 Raiser's Edge NXT PowerUp Challenge: Events
- 14 Ask an All-Star (Elizabeth Johnson)
- 7 Ask an Expert (Stephen Churchill)
- 2025 ARCHIVED FORUM POSTS
- 322 ARCHIVED | Financial Edge® Tips and Tricks
- 164 ARCHIVED | Raiser's Edge® Blog
- 300 ARCHIVED | Raiser's Edge® Blog
- 441 ARCHIVED | Blackbaud Altru® Tips and Tricks
- 66 ARCHIVED | Blackbaud NetCommunity™ Blog
- 211 ARCHIVED | Blackbaud Target Analytics® Tips and Tricks
- 47 Blackbaud CRM Higher Ed Product Advisory Group (HE PAG)
- Luminate CRM DC Users Group
- 225 ARCHIVED | Blackbaud eTapestry® Tips and Tricks
- 1 Blackbaud eTapestry® Know How Blog
- 19 Blackbaud CRM Product Advisory Group (BBCRM PAG)
- 1 Blackbaud K-12 Education Solutions™ Blog
- 280 ARCHIVED | Mixed Community Announcements
- 3 ARCHIVED | Blackbaud Corporations™ & Blackbaud Foundations™ Hosting Status
- 1 npEngage
- 24 ARCHIVED | K-12 Announcements
- 15 ARCHIVED | FIMS Host*Net Hosting Status
- 23 ARCHIVED | Blackbaud Outcomes & Online Applications (IGAM) Hosting Status
- 22 ARCHIVED | Blackbaud DonorCentral Hosting Status
- 14 ARCHIVED | Blackbaud Grantmaking™ UK Hosting Status
- 117 ARCHIVED | Blackbaud CRM™ and Blackbaud Internet Solutions™ Announcements
- 50 Blackbaud NetCommunity™ Blog
- 169 ARCHIVED | Blackbaud Grantmaking™ Tips and Tricks
- Advocacy DC Users Group
- 718 Community News
- Blackbaud Altru® Hosting Status
- 104 ARCHIVED | Member Spotlight
- 145 ARCHIVED | Hosting Blog
- 149 JustGiving® from Blackbaud® Blog
- 97 ARCHIVED | bbcon® Blogs
- 19 ARCHIVED | Blackbaud Luminate CRM™ Announcements
- 161 Luminate Advocacy News
- 187 Organizational Best Practices Blog
- 67 everydayhero Blog
- 52 Blackbaud SKY® Reporting Announcements
- 17 ARCHIVED | Blackbaud SKY® Reporting for K-12 Announcements
- 3 Luminate Online Product Advisory Group (LO PAG)
- 81 ARCHIVED | JustGiving® from Blackbaud® Tips and Tricks
- 1 ARCHIVED | K-12 Conference Blog
- Blackbaud Church Management™ Announcements
- ARCHIVED | Blackbaud Award Management™ and Blackbaud Stewardship Management™ Announcements
- 1 Blackbaud Peer-to-Peer Fundraising™, Powered by JustGiving® Blogs
- 39 Tips, Tricks, and Timesavers!
- 56 Blackbaud Church Management™ Resources
- 154 Blackbaud Church Management™ Announcements
- 1 ARCHIVED | Blackbaud Church Management™ Tips and Tricks
- 11 ARCHIVED | Blackbaud Higher Education Solutions™ Announcements
- 7 ARCHIVED | Blackbaud Guided Fundraising™ Blog
- 2 Blackbaud Fundraiser Performance Management™ Blog
- 9 Foundations Events and Content
- 14 ARCHIVED | Blog Posts
- 2 ARCHIVED | Blackbaud FIMS™ Announcement and Tips
- 59 Blackbaud Partner Announcements
- 10 ARCHIVED | Blackbaud Impact Edge™ EAP Blogs
- 1 Community Help Blogs
- Diocesan Blackbaud Raiser’s Edge NXT® Users' Group
- Blackbaud Consultant’s Community
- Blackbaud Francophone Group
- 1 BLOG ARCHIVE CATEGORY
- Blackbaud Community™ Discussions
- 8.3K Blackbaud Luminate Online® & Blackbaud TeamRaiser® Discussions
- 5.7K Jobs Board