A/B testing lets you try out and compare two Campaign ideas at once! It’s a useful tool for situations where you want to create two versions of the same Campaign and see which one does better.
You can use this method as a way of comparing two unique versions of your Mailout to determine which one gets the best response. A/B testing allows you and your team to make careful, nuanced changes to your customer experiences, while collecting valuable data on the results.
To be more specific, comparison results can be used as a tool to better learn which specific elements of your Mailouts most strongly impact engagement with your customers.
What you’ll find in this document:
How Does A/B Testing Work?
A/B testing works like this: Whenever you create a Campaign, you have the option to toggle A/B testing on or off.
This feature gives you the opportunity to create an alternate Subject Line for half of your Mailouts, or even to select a completely different Template if you want!
Half of your contacts will get Version A and the other half will get Version B. The open rates for each version are tracked independently, enabling you to see at a glance which version does better.
Turning on A/B Testing
Step 1: Create a new Automated Series Mailout or Campaign. Enter your Campaign details, and your Subject Line. This Subject Line will be displayed on your A Template.
Step 2: Click the box that says "Use A/B Testing?"
Step 3: Next, set up the Campaign Details for Version B the same way you did for version A. You can choose here to use a different subject line and preview if you want. If you're only testing the template, make sure to copy it from version A.
Step 4: Select your Contact List or Segment, and continue to the Template page. Remember, of this distribution List, half of the recipients will get Version A and the other half will get Version B.
Step 5: Select or create your Version A Template.
NOTE: If you want to use the same Templates for the testing, click on COPY TEMPLATE. If you are planning to use different Templates, click on SELECT TEMPLATE.
Step 6: Select or create your Version B Template. You will be asked to confirm that you wish to use a different Template for your Version B.
NOTE: If you choose to use the same Template for your Version B Mailout as the one being used for your Version A Mailout, you will see that a copy or duplicate version was made of the Version A Template.
Understanding your results
Once you have published your A/B testing Mailout and the stats start rolling in, you will find it very useful (and interesting) to see the results of how the two different Mailout versions performed.
Head to your Campaigns tab and click the “VIEW REPORT” button of your Mailout to see the results of both Mailouts, Template A and Template B.
By clicking on each Template, you can easily compare the individual results. By viewing the performance and the activity generated by each Template, you can get a clear snapshot of what works well—and doesn't work well—with your customers.
A/B testing is a powerful tool, and the more familiar you become with how to use it, the better it will help you to understand your customers and create future Campaigns that they will respond enthusiastically to.