Do you A/B test your site’s content, commerce and customer experience? You should, no matter how big your site. Especially in a down economy, the worst decision you can make is to abandon testing. If anything, now is the perfect time to look for ways to improve. And testing is the best way to find the improvements that work best.
But what if you site gets little traffic? Just the other day someone told me, “My site doesn’t get enough traffic to bother with testing. Why should I wait six weeks – or more – to get data when I need business results now?”
Fair question. Here are 3 reasons why testing matters, no matter the size of your traffic:
- Time isn’t the issue. Conversion rate is. If your test case performs dramatically better than your original, you’ll probably know it (relatively) quickly. You can feel comfortable a 15% increase in conversion wasn’t just a fluke in as few as 200-250 conversions[*]. Said another way, you only need so many conversions to prove what works. The bigger the increase in conversion, the fewer conversions you’ll need to know it’s working. And the bigger the increase in conversion, the less time it takes to collect the data.
- Meaningful changes take less time to show up. If you don’t see a significant lift in a timely period, it may mean the new version isn’t working dramatically better than the original. If that’s the case, the amount of time it takes to collect the data isn’t the problem; the lack of lift is. Put your focus on the areas where you’ll see greatest improvement. Not sure where to start? Check out the tips here, here and here.
- Sometimes the cure is worse than the disease. What happens if your changed page actually performs worse than the original? You could lose sales and not know whether it was the new version that hurt you or some external factor that would have impacted sales in either case. If you don’t run the two versions side-by-side, how do you know which works better?
Yes, you’ll still need to run your tests for a bit of time. Anything less than a couple of weeks worth of data could be influenced by many factors, including day of week, time of day, holidays, what have you. But don’t focus on that. Focus on what you’ll learn and the results you’ll see.
Simply put, the right answer is never, “don’t test.”
* – This is, admittedly, a rough figure that assumes an 85% confidence level. To check statistical significance for your tests, use Google Website Optimizer’s built-in tool or download the Teasley Stat Calculator [Excel spreadsheet] here.
Are you getting enough value out of your small business website? Want to make sure you’re ready for the mobile, social, local web? Subscribe to thinks today and get more than just news. Get understanding.