In our customer's panel, you can conduct an A/B test for a newsletter campaign. The main purpose of this functionality is to automatically compare the reactions of your recipients to e-mail creations in terms of selected criteria, in other words, checking which e-mail enjoyed greater recognition and interest. Remember that what speaks to you will not always speak to your audience!
Why it is worth conducting A/B tests?
They increase the effectiveness of mailing campaigns by examining the actual reaction of recipients to various variants of e-mail messages
They allow you to better understand the preferences of your recipients by checking their reaction and monitoring the results for specific creations
The winning creation increases the chances of involving other recipients in the received communication
They allow you to increase awareness in the context of your e-mail marketing activities and optimise them thanks to the analysis of the creations sent so far
Our A/B tests are automatic – you choose templates and set the success criterion, and we will show you the numbers, indicate the winning creation and send it to other recipients after the time you set. |
How to set up an A/B test for the newsletter?
1. Go to the Mailing tab in the main menu on the left and then click Create newsletter.
2. In the Audiences section, select a group to which you want to send the newsletter or create a new one. Click Next.
3. In the Email templates section, move the slider to On under Create an AB test to open the A/B test creator.
4. In the A/B test creator:
set the size of the test base – i.e. how many percent of recipients from the selected target group are to participate in the test mailing; you can change the value with the slider
Remember! The percentage of recipients from the test group depends on the size of the selected base. It should be relatively low with a large target group (because it will cover a large number of addressees anyway) and vice versa – it should be relatively high with a small target group (to cover the number of addressees sufficient to conduct a reliable study). |
create control groups (min. 2, max. 4) and assign templates to them – for example, if you create two control groups, the control base will be divided into two equal parts and the assigned messages will be sent to them; you can add up to four control groups
Click Choose a template and add a template for each control group.
If you hover your cursor over the name of the template you've added, you'll see a preview of it.
if you want our system to automatically send the best e-mail selected by the result of an A/B test to the rest of the recipients:
select Yes next to the Automatically send the best creation – thanks to this option, the system will assess itself which of the templates used in the test is most effective in the criterion chosen by you and, after the time indicated by you, will automatically send the best e-mail to the rest of the recipients
select a success criterion – choose which indicator is to be taken into account when automatically assessing which control group won the test; you can choose from:
- open rate (highest open rate)
- CTR (highest click rate)
- highest conversion
choose the time after which the system is to send the selected creation to the rest of the audience after launching the campaign – for example, if you set 12 hours, the winning creation will be sent to the rest of the audience 12 hours after the test is started
If you want to test:
Remember to test only one variable in one A/B test, e.g. in the case of CTR only the colour of the CTA buttons. Otherwise, you will get lost in the conclusions and you will not find out what influenced the reactions of your audience! |
if you do not want our system to automatically send the best template to the rest of the recipients, select No next to the Automatically send the best creation – in this case, you will decide which creation won
5. Click Next and go to the settings. Indicate the name of the campaign and its duration (start).
Also complete the details of the sender. What you enter will be displayed to recipients in their inboxes.
We encourage you to use the functionality of managing the list of senders. For more information, see the article How to manage the list of senders?.
Click Next.
6. A summary will appear in the last step. Check if everything is correct and click Finish.
How to activate a campaign with A/B test?
The newly created newsletter campaign is now visible in the campaign list in the main view of the Mailing tab. However, it is still inactive. To activate it, click on the red circle and the Activate button.
