A/B Testing

A/B testing lets you try out and compare two Campaign ideas at once! It’s a useful tool for situations where you want to create two versions of the same Campaign and see which one does better.

ab-testing-blog.png

You can use this method as a way of comparing two unique versions of your Mailout to determine which one gets the best response. A/B testing allows you and your team to make careful, nuanced changes to your customer experiences, while collecting valuable data on the results.

To be more specific, comparison results can be used as a tool to better learn which specific elements of your Mailouts most strongly impact engagement with your customers.

What you’ll find in this document:

How Does A/B Testing work?

Turning on A/B Testing

Understanding Your Results

 

How Does A/B Testing Work?

A/B testing works like this: Whenever you create a Campaign, you have the option to toggle A/B testing on or off.

abtesting.png

This feature gives you the opportunity to create an alternate Subject Line for half of your Mailouts, or even to select a completely different Template if you want!

Half of your contacts will get Version A and the other half will get Version B. The open rates for each version are tracked independently, enabling you to see at a glance which version does better.

The two versions can be drastically different from one another, or virtually indistinguishable except for a few details—it’s all up to you!

 

Turning on A/B Testing

Step 1: Create a new Automated Series Mailout or Campaign. Enter your Campaign details, and your Subject Line. This Subject Line will be displayed on your A Template.

Step 2: Click the box that says "Use A/B Testing?"

abtesting2.png

Step 3: Next, set up the Campaign Details for Version B the same way you did for version A. You can choose here to use a different subject line and preview if you want. If you're only testing the template, make sure to copy it from version A.abtesting3.png

Step 4: Select your Contact List or Segment, and continue to the Template page. Remember, of this distribution List, half of the recipients will get Version A and the other half will get Version B.

Step 5: Select or create your Version A Template.

NOTE: If you want to use the same Templates for the testing, click on COPY TEMPLATE. If you are planning to use different Templates, click on SELECT TEMPLATE.

Step 6: Select or create your Version B Template. You will be asked to confirm that you wish to use a different Template for your Version B.

abtesting4.png

NOTE: If you choose to use the same Template for your Version B Mailout as the one being used for your Version A Mailout, you will see that a copy or duplicate version was made of the Version A Template.

Step 7: Finally, select your delivery method and … Voila! You are ready to publish your A/B tested Mailout.

 

Understanding your results

Once you have published your A/B testing Mailout and the stats start rolling in, you will find it very useful (and interesting) to see the results of how the two different Mailout versions performed.

Head to your Campaigns tab and click the “VIEW REPORT” button of your Mailout to see the results of both Mailouts, Template A and Template B.

abtesting5.png

By clicking on each Template, you can easily compare the individual results. By viewing the performance and the activity generated by each Template, you can get a clear snapshot of what works well—and doesn't work well—with your customers.

A/B testing is a powerful tool, and the more familiar you become with how to use it, the better it will help you to understand your customers and create future Campaigns that they will respond enthusiastically to.

Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request

Comments

0 comments

Article is closed for comments.