Delivering smarter
customer communications
A/B testing is a method of testing such marketing variables as copy, image, offer or format. Comparing test cells to identify which one brings a better result. This is the only way to improve your marketing and increase activity instore and online.
This is the first of a two-part series on the topic of A/B testing for Direct Mail.
Part 2 Covers: Direct Mail elements that you should test and optimise.
The long-term benefits of continuous A/B testing in continuous improvement. A few small tweaks to artwork, mail format or a call to action can significantly improve the response rate and campaign ROI. An A/B test should be based on testing one variable at a time, such as an image (woman v/s man,) colour, or call-to-action. If you have enough data files, you can implement a multi-variable test.
HINT: A test that does not provide the initial desired results is not a failure, it is the first stage of an improvement process. Better to get less than expected results on a test rather than going full throttle on a campaign!
Whether you are testing one or multi-variables, you must prioritise the test variables to maximise the results. The Kissmetrics and HubSpot eBook “An introduction to using A/B testing for Marketing Optimization” details 10 Guidelines for Effective A/B Testing, that we have adapted for Direct Marketing Communication:
You cannot test an offer to a new audience, at the same time test a new print format and alter the usual landing day for your Direct Mail communication. Then sensibly measure the impact that each had on the success or failure of the campaign. How do you know which change most influenced the response? If you test one change at a time, you’ll have better metrics to draw conclusions from to adapt future campaigns.
Only change one element of your direct mail communication to test its impact. For example, change the image only, not the image and the copy.
It’s worth remembering that small changes can have a big impact on your response rate. Switching the colour of your call-to-action or a slight copy amend can improve performance.
Changing the format of your mailing without altering the copy or offer can both increase return and reduce cost. Remember, this is a test so a straight 50:50 split across is wise. Your print provider will be willing to help with various formats as part of an ongoing strategy to help your DM be a success.
Direct Mail is part of your whole marketing mix, so, you must consider how you are to measure your DM brings, what will be seen as a successful campaign? Website traffic, click through rate, sales conversion rate, lead? Are those leads responding to mail producing a higher conversion rate than those sent just email communication?
In any experiment, you must have the original version (Control) and create an alternative (Test). You compare results from the Test against the Control to see if the test produced comparably better or worse results. When setting up the test and control, all parameters including data segmentation should be the same EXCEPT for the one parameter that you are testing for.
List a series of key metrics that you want to test. How could they impact your marketing and business goals? How difficult is it to implement?
HINT: When starting out- start simple. Simple adjustments are often the best and easiest to make, and can produce improved results without large cost.
You need to test audience that is equal. Testing one variable on Millennials and Baby Boomers at the same time isn’t a good idea. Because may cause large differences in response rates. At least that you want to compare the performance of the same message/artwork, so you get clean results based on their profile and nothing else.
When comparing your test campaigns against a control campaign, you must run the tests at the same time. External economic or environmental influences can affect campaigns so the door-drop day must be at the same time.
HINT: Have you tried varying the ‘door-drop’ day of your campaign without altering other parameters to see if this makes a difference?
Which version will it be the winner? Before you begin your test, think about how significant your results should be before you to decide that changes should be made to your Direct Mail Communication. Set the statistical significance goal for your tests. e.g. what results above normal, would drive change.
Testing and optimising your Direct Mail is essential to the continuous improvement of your marketing ROI. It will only give you statistically reliable results if you follow strict controls during testing. At Romax, we help our clients to plan their direct marketing communications and readily implement A/B testing. Ask our Client Service team for advice about how you can implement testing in your next Direct Marketing campaign.
Resource: 3betmedia, Kissmetrics, HubSpot An introduction to using A/B testing for Marketing Optimization,
Romax Marketing & Distribution has more than 20 years of experience managing membership and client printing and communication services for organisations such as Southbank Centre, DKMSand PlusNet. Contact us for a bespoke Membership Marketing Solution: hello@romax.co.uk, +44 (0) 20 8293 8550.
