Email Marketing and A/B Testing

Jamin Sartor · November 2013

I would like to learn about running A/B testing on emails for our upcoming holiday e-appeal campaign. Does anyone have experience using NetCommunity to A/B test and is willing to share some best practices/tips?

Jaime Lassman 2 · November 2013

I've done a little testing on my site using the Content Comparison part. It won't help with emails in the inbox, but the Content Comparison part is similar to the Targeted Content part in that you load the part with two other parts. Then the Content Comparison part will do the A/B testing on the page for you. It collects data and updates the results nightly. You can set a threshold so that when your goal is met, it continues to deliver A or B depending on which version was more successful.

I'm pretty sure it measures success by the click throughs to the page you determine in the Content Comparison part. Of course, you have to make sure your A and your B parts are linking to the same page. Once you set those parts inside the Content Comparison part and start the test, it locks the FT&I parts from being edited. You need to make sure you like A and you like B before testing them.

From my experience, I learned how infrequently the items I promote on my homepage are clicked. It was cool, but a bit of a bummer. It took far longer than I anticipated to determine a winner.

If you're set on the A/B testing for emails and not the website, you might consider using Google Analytics and setting up goals for the site with a funnel. If email A and email B both link to different pages, you could measure goal A against goal B and get some results that were meaningful without a whole lot of filtering or advanced searching.

Chris Martin · November 2013

NetCommunity's Email Campaign feature exists for this exact purpose. You can read up on it in the NetCommunity Email Guide. Campaigns allow you to choose a seed list and do A/B split testing and to track conversions and ultimately send the more effective email to the broader list.

Jamin Sartor · November 2013

Jaime - thanks for your response. I am looking for email testing specifically, so your advice about leveraging Google Analytics is a good point. I will look into that! And I like the idea of testing the website too...I will save that for another day for sure!

And Chris, I attempted to run a campaign last year, but I had an issue when I couldn't run a test on my emails as I was creating them. My understanding is that you can only test emails in messages, so you would have to create messages and then copy the HTML into campaigns once all the glitches are worked out. Is that the only limitation with campaigns?!

Thank you again!

Jamin Sartor · November 2013

Chris Martin:
NetCommunity's Email Campaign feature exists for this exact purpose. You can read up on it in the NetCommunity Email Guide. Campaigns allow you to choose a seed list and do A/B split testing and to track conversions and ultimately send the more effective email to the broader list.

Hi Chris, as I said in my last post: I attempted to run a campaign last year, but I had an issue when I couldn't run a test on my emails as I was creating them. My understanding is that you can only test emails in messages, so you would have to create messages and then copy the HTML into campaigns once all the glitches are worked out. Is that the only limitation with campaigns?! Any further insight you have on this would be great.

Suzanne Murray 2 · December 2016

Jamin Sartor:

Chris Martin:

NetCommunity's Email Campaign feature exists for this exact purpose. You can read up on it in the NetCommunity Email Guide. Campaigns allow you to choose a seed list and do A/B split testing and to track conversions and ultimately send the more effective email to the broader list.

Hi Chris, as I said in my last post: I attempted to run a campaign last year, but I had an issue when I couldn't run a test on my emails as I was creating them. My understanding is that you can only test emails in messages, so you would have to create messages and then copy the HTML into campaigns once all the glitches are worked out. Is that the only limitation with campaigns?! Any further insight you have on this would be great.

This has been my experience as well. I know it has been three years, but any chance you got an answer from BB about it? I contacted BBNC Support but they did not seem to understand why this was a headache/annoying.

What I do is compose emails in Messages, always, then only set up the Campaign when I have the code 90% baked.

What I would love is to be able to save Messages in folders.... which is what I was hoping Campaigns could do for me; be an easy sorting/reporting area for a "campaign" in the sense of year-end fundraising, or similar.

Email Marketing and A/B Testing

Comments

Categories