Written by Usually when someone hears the word “split,” it often has negative connotations. Splitting up with your partner, running from the scene of a crime, a banana split (if you really don’t like bananas) or having to do a LITERAL split. But if you’re a marketer or a designer, the word can take on a whole different meaning.
A/B testing, or “split testing,” is a way for a team to collect real data about their projects. Extremely simple in execution, to run an A/B test, one simply needs to create two different versions of the same piece of content with changes to a single variable (Kolowich). A single variable could be changing the typography or the color or even the placement of an element of a button on a website. This method is all about little, intentional changes to produce an effective result. When most companies A/B test they are looking for ways to:
- Increase their website traffic
- Increase their conversion rate
- Decrease their bounce rate
- Decrease cart abandonment
- Streamline their approach
What makes A/B testing so great is that it is easy for anyone to do and it seamlessly can be used for both printed and digital mockups. As long as you have a deliverable that people can see and a way to track results/metrics that’s all you need! It’s all about yes/no answers so there are no confusing gray areas. The results speak for themselves. Here’s how you get started on performing a test:
- Pick the variable you want to test
- Zero in on what goal you want to achieve
- Create the two versions of your design
- Split your audience into sample groups
- Run your campaign
- Analyze the data
- Make informed design decisions
It is important to note that when doing A/B tests, its important to only run one test at a time. It could become really confusing to users to keep seeing diverse solutions on a website and it will mess up your results. It’s not about changing everything, it’s about finding out which pieces are most important to the user. A split test takes time to garner results. Depending on the company you work for, it could take days to a month to produce useful insights. Don’t rush it, just trust in the process.
VAIO Case Study
To help give you a picture of what this looks like in practice, let’s take a look at what Sony did for their VAIO computers. In their original banner ad, Sony noticed that they weren’t achieving the conversion rates they were looking for. After analyzing the original control ad, it became apparent that it had two CTRs one enticing the personalize the computer, while the other tried to stress a promotional deal (Wishpond, 2019).
In order to understand which message worked best, they created two ads to run against the control. One focused on personalization while the other focused on promotion. At the end of the testing period, the ad with the personal CTA increased CTR by 6% and increased shopping cart ads by 21.3%. By focusing on the CTA, they were able to discover that people are more interested in an ad that includes them.
Microsoft Case Study
Staying within the computer theme, Microsoft was looking to increase CTR for a Dutch email campaign. Curious in seeing if color had a major impact on users, they tested a plain white ad versus an ad that followed the branding of Microsoft’s digital marketing (Wishpond, 2019).
After testing, they were surprised to find the aggressive ad with the orange background didn’t perform as well as they’d hoped it would. Although it was used the same look and feel of Microsoft’s suite of digital ads, in an email, it seemed that users had a hard time figuring out what was a picture and what wasn’t. By emphasizing the CTA in a simple purple box, Microsoft was able to more seamlessly direct users to their landing pages.
Have you ever tried an A/B test for a project? Did you get the results you expected? Let me know in the comments below.
Works Cited
Kolowich, L. (n.d.). How to Do A/B Testing: A Checklist You’ll Want to Bookmark. Retrieved April 10, 2020, from https://blog.hubspot.com/marketing/how-to-do-a-b-testing
Wishpond. (2019, January 18). 6 Landing Page A/B Test Examples to Improve Your Conversion Rate. Retrieved April 10, 2020, from https://blog.wishpond.com/post/98235786280/50-a-b-split-test-conversion-optimization-case-studies
Alex Piwtoratsky. 