Oh $%&@ - RE Data compromised

Brian Hoyt

Maybe you are lucky and didn't receive the email this morning at 10:30 AM EST / 7:30 AM PST but I sure did. Apparently Blackbaud hosting had a ransomware attack in May. You can see details from Blackbaud and news. My problem is this particular section:


I don't know that I agree with this statement. I feel I have to now notify everyone in my RE DB that their PII may have been compromised. What are your thoughts?

JoAnn Strommen

First I've heard of the issue. Whew!  Will check with CEO/CFO to see if they received email. 


That's a tough call. Not sure how they can be sure that info was not compromised. Don't know what we'd do in that situation. If you feel need to notify, perhaps just those who have credit card/bank info on their records. 


Best wishes.

Keri Barnhart

I was coming to the forums to see if anyone else was posting on the topic! I'm curious how extensive the databreach was - in the FAQ the sent us, one of the questions doesn't match up the the answer, and the question was about which services were impacted.


A close reading suggests that it was the kind of ransomware attack that the University of California had a few months ago, and not dissimilar to the ones that Atlanta and Baltimore faced a while back (but the software/approach/agent was different). Someone accidentally triggered it, and it started to spread through the servers, but they caught it before it had encrypted everything. Thankfully, as these attacks occur, infosec professionals have been providing best practices and ways to avoid locking up the entire server system.


I have always been wary/hesitant of cloud-hosted data because of the chances of the host servers being compromised and having no control - of course, it sucks if someone at our own server location triggers the ransomware attack, but at least then it's all on us! Silver lining? But here, it looks like it was only the backup data stored on blackbaud's own servers, not the data in the cloud. I'm not sure what that means regarding blackbaud's internal data, and they probably will not share that.


The notices are clear that any credit card, bank, and social security data has been stored differently than other fields. I hope that's true, as most of us have stored ACH and CC details to allow us to run recurring gifts... But we're a health non-profit and the mere fact that we need to record which donors are also clients is HIPAA-sensitive.


Blackbaud's resources for us include templates for notifying constituents of the databreach and possible exposure of PII. They say that they're monitoring the darkweb to see if any of the data gets shared out there and are doubtful about it (because what's the point in paying a ransom to destroy the data?), but it could be years before that happens. Sometimes the data wasn't actually scraped/removed by the ransomware attackers, only made inaccessible, and there is nothing to "destroy" on the hacker's side. I'm not really sure what happened here, and I don't think we'll be told, or that it matters with our response.

JoAnn Strommen

Well, now we got the email.  Reading through the resources and contacting other staff to determine plan of action needed.

Dariel Dixon 2

I think the first thing worth noting is that all these emails have been somewhat personalized.  I got the email this morning as well, but none of our RE data was involved.  I'm very fortunate in that regard, however some other data was part of the situation.  It's also possible that this breach didn't effect everyone.


This falls completely on Blackbaud.  I think it's worth looking into your business associate agreement with Blackbaud.  But currently, I'm just monitoring the situation.  I do find it interesting the method in which this information was disseminated. 

Duane Waite 3

Received the email early this morning, and I have a question I hope someone can help me with.

My version of the letter stated that an NXT backup was affected (i.e., copied, stolen), and that no credit card numbers or account numbers were stolen because they are encrypted. In my Database view, that is true for credit cards. However, donors who made donations through their debit card have a Bank Relationship record in their Relationships tab, and that relationship record contains unencrypted routing and account numbers for their checking accounts. But when I look at the gift's debit information in NXT, the account number is encrypted (well, tokenized).


So my question is, if the attacker only had access to the NXT backup, did s/he have access to these routing and account numbers? Any thoughts would be appreciated!

Austen Brown

The article on The Non-Profit Times goes into a bit more detail about the evidence Blackbaud has that makes them believe that the stolen data was destroyed; it's worth a read. 

Duane Waite‍ - The best person to ask is likely your Account Rep.  They should either be able to answer the question or point you in the direction of someone that can. 

 

Duane Waite 3

Called and emailed my account rep., no response as of yet.

But I did call the Incident number, and by luck of the draw, got straight through to a human voice. FYI, this is a contracted call service called Epic. The analyst was prepared with the information in the FAQ page, but after I described my issue, he said he would need to forward it to Blackbaud. He typed out my question, read it back to me to make sure it was correct, then forwarded it to Blackbaud. He said they would respond to him with an answer, and that he would call me and let me know what it was. If there was any misunderstanding or need for further answers, he would forward those back to Blackbaud and then call me once he gets further info. He was professional and polite, and I'm grateful I reached someone who was genuinely concerned about the problem we are facing. However, he is not an employee of Blackbaud, so Raiser's Edge is something he learned about yesterday.

The 21st century, amirite?

Rita Williams 2

We are most interested in whether or not other RE customers are notifying their constituents. Our State law has exclusions to the requirement of notifying affected persons, and is very specific about what qualifies as personally identifiable information. If we are to believe Blackbaud in that no SSNs, credit card #s or banking account #s were accessible to the criminals, then that leaves DOB, giving information, phone numbers and addresses as information that was potentially obtained. Do we risk our own reputation as a trusted non-profit fundraising organization because such information may have been obtained through our "third party service provider" (Blackbaud obviously doesn't want us to use their name!), but it does not fall under the definition of personally identifiable information?


We are an NXT customer also, so I will anxiously await to hear the response you receive Duane. If the bank routing number was compromised, that means we definitely have to notify our constituents.

Brian Hoyt

We were told by our legal similar to what Rita Williams‍ said. The one piece is birth date. Even so we are sending a notification to all constiuents in our RE DB an email today. Anyone we don't have email for we are sending letter. We are specifically naming Blackbaud, it is their fault not ours (other than trusting them) so we aren't shielding them in any way.


Brian Hoyt

Jessica Radomski

Hi all,


I'm having a hard time figuring out what information WAS compromised. Does anyone know the answer? I'm assuming all unencrypted fields apply here? DOB, email, phone, etc.


Thanks, 

Jessica

Huw Price

Morally I feel we need to notify people on our database, legally is a different issue.


There has been a data breach, if there are further repercussions( the data re-surfaces, peoples identifies cloned/used etc.) 

then data subjects could quite rightly come after you. 


Thoughts? 

Matt Page

Huw Price:

Morally I feel we need to notify people on our database, legally is a different issue.


There has been a data breach, if there are further repercussions( the data re-surfaces, peoples identifies cloned/used etc.) 

then data subjects could quite rightly come after you. 


Thoughts? 

Huw, if you're based in the UK then I think you will be legally obliged to under GDPR (and report the breach to the ICO). But I've not seen the email from Blackbaud as we're self-hosted.


Matt

Huw Price

Hi Matt

Thank you for your thoughts. It was system backups
that were stolen - all financial information would have been
encrypted by default...

This is where opinions may come into play: "Does
the data taken pose a significant risk to the identities and
security of the data subjects?" 

Until I find out the exact details of data taken
I'm unable to give a balanced opinion. 

I'm (still!) waiting for a response from
Blackbaud... 

Thank again. 

This
email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom
they are addressed. If you have received this email in error please
notify the system manager. This message contains confidential
information and is intended only for the individual named. If you
are not the named addressee you should not disseminate, distribute
or copy this e-mail. Please notify the sender immediately by e-mail
if you have received this e-mail by mistake and delete this e-mail
from your system. If you are not the intended recipient you are
notified that disclosing, copying, distributing or taking any
action in reliance on the contents of this information is strictly
prohibited.

WARNING: Computer viruses can be transmitted via email. The
recipient should check this email and any attachments for the
presence of viruses. St Leonards School accepts no liability for
any damage caused by any virus transmitted by this email. E-mail
transmission cannot be guaranteed to be secure or error-free as
information could be intercepted, corrupted, lost, destroyed,
arrive late or incomplete, or contain viruses. The sender,
therefore, does not accept liability for any errors or omissions in
the contents of this message, which arise as a result of e-mail
transmission.

The
School is a private company limited by guarantee and is registered
in Edinburgh with registration number SC016693.

The
company's registered office address is St Leonards School, St
Andrews, Fife KY16 9QJ.

The
School is also registered as a charity with
OSCR. 

The
charity registration number is SC010904.

Brian Hoyt

I'm having a hard time figuring out what information WAS compromised. Does anyone know the answer? I'm assuming all unencrypted fields apply here? DOB, email, phone, etc.

 

Anything in the RE DB, they copied the entire DB which is an SQL file that can be loaded up and searched if you know what you are doing. There shouldn't be CC or Bank info in RE, that is to say I have seen people put things they shouldn't be in it. The one question is the one in this thread about some routing info which may be stored somewhere else and integrated into NXT at view time rather than stored in RE DB.

Huw Price

What data has been taken? Depends what 'backup' they are talking about. Is it a backup of the physical database? (Rdbms files+data files) or just a "database backup" i.e. a backup that can be used to restore an original 'point in time' for your data within the target database structure. I am not sure whether backups are configurable in RE as to what data you wish to backup or whether it is a full backup, where all data fields are extracted. I have asked for a full disclosure from Blackbaud and will post the findings when I get them. 

Karen Blaney

Our notice from Blackbaud said a backup copy of our Research Point Database was part of the incident. Microsoft Azure, which is were our RE NXT is located is not part of the incident.

Stacey Brake

I am happy to find this discussion.  I wasn't able to access the community forum this morning and have had minimal luck reaching someone in support today.  Our organization also received the data breach email last week.  We have also struggled to find out exactly what information was affected as that will determine how (or if) we notify our constituents.We spoke with someone at the "hotline" number Blackbaud provided and gave them some questions we had to forward to Blackbaud.  As already mentioned, this is a third-party company fielding the calls and yes, all they do is regurgitate the same information Blackbaud emailed last Thursday.  We have reached out to our Account Executive as well but were directed to customer support as she did not have any information.  We also sent an email directly to Customer Success and received a response this morning.  They also repeated what was in the email Blackbaud sent last Thursday.  It's very frustrating.


We keep getting told that we need to look at our product database for a list of fields that are not encrypted to see what may have been compromised.  Does that mean I need to look at the tables in Query/Export for a list of those fields?  I'm assuming the product database they mention is our live database.  However, the knowledgebase articles I have found regarding "product database" seem to refer to self-hosted versions of Raiser's Edge.  Again, it's been frustrating trying to figure out where to look for specifics.  Am I missing something or is anyone else as confused as I am?

Dariel Dixon 2

Robyn Halbert‍ We haven't used ResearchPoint for our grateful patient solution, so we don't have any PHI in our product.  Granted, if you have an automatic feed from your EHR system, you might want to consider what fields the protected data is coming through as.  If you have a BAA with Blackbaud, you may want to consider taking a look at it.

Meg Finley

Have to agree with Stacey Brake - I'm very confused by what of my data was compromised.  And it doesn't help that my database manager has been out for over 2 months and this isn't my area of expertise.  I've asked for the 'data schema' which is basically the list of potential fields of data entry that could have been accessed by the cyber criminal but am waiting on Blackbaud to reply.  I don't know what of my own data is encrypted or not; thankfully we don't have any banking, health or other truly personal security info so I think we're okay.  But with donors across the US, I've been reading state law codes until my eyes are overstrained and blurry.  

If anyone has a shortcut idea of how to find the list of fields from your own database (our NXT backup was hacked) that are unencrypted, then please share!  I will be eternally grateful.

Meg - 

Jessica Radomski

I reached out to my Account Rep who created a case on my behalf. I have requested a backup copy of my database so I can see exactly what fields were compromised. This has yielded the best results so far but the path has been long and tedious. 


I swear if someone from Blackbaud tells me one more time that encrypted fields weren't compromised, I'm going to lose my mind.

Elissa Karim

Hi Meg,


List of encrypted fields from blackbaud's knowledgebase: https://kb.blackbaud.com/articles/Article/47633?_ga=2.180561831.1225762633.1595263300-1691160020.1594746434 (if you aren't using RE, you can search "what fields are encrpyted in [your solution] in the knowledgebase) So my understanding is all the other fields are not encrypted (essentially most of them), meaning most fields fall into the possibly compromised category.


I hope that is helpful. If anyone has any other understanding, please chime in.

Stacey Brake

Update:  Our Account Executive sent a link this morning to the same knowledgebase article that Elissa Karim just posted.  It was also confirmed that all other fields are not encrypted and were part of the data breach.


Customer Support also contacted me this morning and told me the best way to get lists of our fields:


Go to Configuration and select Fields

Select the category you want to see the fields for (Action, Constituent, etc.)

Right-click anywhere on the open white space to the right of the fields listed to Export to Excel.


This was posted on another thread by Duane Waite as well.  (I hope I am giving credit to the correct person!)  


Thank you everyone for all your comments and advice!

Keri Barnhart

https://kb.blackbaud.com/articles/Article/57547


Has anyone seen a better response than the linked KB for "What fields are encrypted in Blackbaud Net Community?" - because the question + answer don't seem to match on that page.


From what I understand:

- The ransomware affected full backups, not the delta of a backup (which would only be changed/new)

- The "full backup" of Raiser's Edge should be the same as the live Raiser's Edge on the day that it was backed up. I don't remember how often BB does backups of their hosted clients.

- The backup data that got caught up in the ransomware attack was from between February and May 2020 - May is when the ransomware attack occurred, but they may have had access to data as early as February, so I'm presuming that there were some stored backups from then.

- Only the fields specifically listed in https://kb.blackbaud.com/articles/Article/47633 (for RE/FE) were encrypted or stored elsewhere. Every single other field in RE was involved.


My assumptions:

- This was similar to the attack on the University of California earlier this spring.

- Blackbaud is trying to downplay the seriousness of the issue by not giving us an idea of how many organizations are affected. Knowing the scope of this would be really helpful for communicating with our donors!


I'm still trying to work out whether or not we need to disclose the databreach to our constituents based on the risk of harm, but I think that will require meeting with our legal team? It's unclear to me whether or not the risk etc is fully mitigated, though the Webcast from Ted Claypoole from Womble Bond Dickson was helpful in having a place to start from, regarding the laws.

Bill Connors

Cross-posted from the other thread on this topic in this community:  Stacey and others:  exporting the list of fields from Config will give you an IDEA of MOST of the fields, and perhaps the most important ones, but it certainly will not give you all the fields in RE -- just look at the Phone option, for example.  While this might also not be 100% perfect, here's what I posted in the RE group on Facebook on Friday that is likely much more comprehensive:  The easiest way to get a list of fields in RE is here: https://www.blackbaud.com/docs/default-source/how-to-documentation/raisers-edge-how-to/raisers-edge-user-guides-administration/import.pdf With that consider: (1) the list of encrypted fields that are not at risk per BB (https://kb.blackbaud.com/articles/Article/47633); (2) ADD your Attributes which won't be here (easy to print or PDF with File, Print from that screen); (3) REMOVE the fields for RE optional modules you don't have; and (4) MODIFY the list for any fields you use other than as intended. On this: IMHO, do *not* start exporting a ton of information from RE, especially the sensitive fields, to a spreadsheet to analyze, search, etc. the data or fields. Do not create a new security problem to solve another one: data in a spreadsheet on your computer is a security disaster waiting to happen when someone gets ahold of that spreadsheet, computer, laptop, etc. And deleting a file does not actually delete it. 


Original post above 7/17. 7/21 addition: Today I realized that with #2 in my post above I was thinking "RE 7" and "RE NXT database view." Blackbaud's emails reference "Blackbaud Raiser's Edge NXT" for some I've seen. Be aware that I don't know where or how Blackbaud stores and backs up RE NXT web view exclusive content, like Attachments (although there are other fields as well), and to my knowledge has not clarified whether this breach involves RE NXT database view only data or "all" of RE NXT. I also do not know of a source for a list of those fields. So, I would add to #2 above (5) ADD RE NXT web view-only fields if your organization is on RE NXT.

Stacey Brake

Thank you Bill Connors for the information.  Do you have a different link though?  I tried the one posted but it takes me to Facebook asking if I want to follow the link elsewhere.

Bill Connors

Hmm, sorry, they work for me.  But here they are directly:

https://www.blackbaud.com/docs/default-source/how-to-documentation/raisers-edge-how-to/raisers-edge-user-guides-administration/import.pdf

https://kb.blackbaud.com/articles/Article/47633


Also, Stacey, I just edited my earlier post, so please see the edit as well.

Stacey Brake

Thank you Bill Connors!

Julie Rae

This situation is very upsetting a I am wondering if anyone else in Australia received the notification?  

Bill Connors

Edit #2 7/22: Sorry for this additional edit, but BB had over 2 months to prepare for this announcement and I've had less than a 1 week to try to help you all in my "spare time." I realized last night that the Import Guide understandably only includes fields that can be imported into RE 7/the RE NXT database view. It does *not* include *all* fields, such as the constituent and proposal Media tabs. I still think it's the best place to start for the quickest, biggest list available for RE fields, but I need to point out it does not include every single field. If you want to be 100% thorough, you should (6) COMPARE the guide to your live copy of RE and ADD to the list fields in your system not in this guide, like Media.

Roslyn Miller 2

Julie Rae:

This situation is very upsetting a I am wondering if anyone else in Australia received the notification?  

Yes we have as well Julie.  Hello by the way.