Importing NetCommunity history onto NXT record?

Kevin Fletcher

Gang – As you might well know, NetCommunity both does and doesn't talk to RE, and one of the things that it doesn't do is bring email history into the Web View record.

I know DBV isn't going away soon, but I need to find a way to get that history out of the DBV record and into the WV record, and I'm thinking a Power App/Tile is going to be the best route, but that could also be because I'm a man with a hammer looking for a nail. If anyone has other ideas (using Import to use other data fields that WOULD come over such as attributes/custom fields, though that idea makes me twitch uncontrollably).

I've begun the process of exporting NetCommunity info out via query to CSV files; I'm guessing I'm about ⅔ of the way through our 90,000ish living alumni, and have roughly 10 million combined email records (i.e. record-level info on email received/opened/link clicked, etc.).

Any ideas about how to go about this would be GREATLY appreciated.

Glen Hutson

@Kevin Fletcher
Are you still actively using BBNC and this data set will be growing or pulling it over more for historical purposes? Do you need this for overall reports or more for individual review?

One option may be to load this data into a 3rd party data storage such as a Sharepoint List (edit: I I just reread and saw you said 10M records, so maybe not Sharepoint), Dataverse Table, etc making sure to include the System Record ID. You could then create a SKY Add-in to pull that data as a tile (or button-activated fly-out) to view it on an individual record.

You could use Query API to periodically add new info to that external data source.

However, that won't make it reportable directly from RENXT, but will still make it reviewable from their Constituent Record if someone is simply wanting to review their engagement history.

Kevin Fletcher

@Glen Hutson: Short answer is yes to everything ? … still actively using, but hoping especially after watching yesterday's NXT PUB that NXT email will serve our needs well by the time TWS is in effect.

We could well be looking at 15M records when all is said and done … but the active data needs may be more to distill full email history down to things like (spitballing off the top of my head) “No. of emails received in last 3/6/9/12 mos” | “% emails opened in that span” | “% clicked” … etc., on a tile on the record; if we'd need to dig down deeper, we could go into wherever that data lives.

Not sure Dataverse is an option for me, so I'm reading that Sharepoint lists can handle up to 30M records (though certainly understand that that might not be a great idea) … wondering if I can index on the system record ID to keep the list view to a manageable number (note that I've read one article and don't necessarily know what I'm talking about!)?

Ben Regier

@Kevin Fletcher
15M rows in a SharePoint list might be technically possible (especially if you don't need to run complex queries), but SharePoint isn't known for handling large row counts gracefully. I think you'll find yourself running into surprise limitations and will end up inventing more workarounds than you expect.

I use SharePoint lists for a lot of things, but if it were me, I'd keep away from it for a project of this size.

Alex Wong

@Kevin Fletcher
sharepoint list is going to have various limitation when there are that many data.

You should seriously consider dropping some data that may be too old to be relevent. Or potentially “ summarizing”.

If you must have all that data, and that you will be querying on it, then you will be better off investing into a true data warehouse type solution (i.e. Azure SQL or Fabric), which can be a “entry” point for you to consider getting all good data into it for better reporting over all.