Avoid Failed A/B Tests with these Simple Tips

Posted by Kristin Ravesloot on June 27, 2016

Did you know that even though you may be testing rigorously, you could unknowingly be making crucial statistical errors? And did you know that these errors could be throwing off your results and giving you false data? What’s worse is that your team is likely using this false data to optimize your website, increase conversions, and grow your company or department!

This is why valid, error-free A/B testing results are are so important.

Check out our 3-step checklist for performing accurate A/B tests and retrieving valid data so that you can build an overall successful optimization strategy for your business.

Step 1: Before Testing

• Analyze your Analytics tool data to help inform you of what to test.
• Pick 3 metrics e.g. Revenue, Conversion Rate & Bounce Rate that you want to improve.
• Validate the data and metrics chosen with a 2nd source, e.g. your internal backend solution to ensure that the data is telling the same story in both sources.

Step 2: During Testing

• Ensure that you set up click goals (the basic goals to measure click analysis).
• Ensure that your test is able to track the metrics you selected in step one – the ones that you want to improve.
• Monitor tests closely during early days. As the test progresses ensure you are thinking of ways to amplify results.

Step 3: After Testing

• Evaluate the results of your click goals and metrics.
• Check for anomalies in the data (spikes and dips) that may have been caused by large purchases or promotions.
• Most importantly, understand that if a test fails there is always something to learn from it!

If you want to learn more about how you can get the most out of testing and data, we’d love to hear from you!

Topics: Test and Optimization

Written by Kristin Ravesloot