3 reasons why A/B testing is always the right answer (Guide to Small Business E-Commerce Strategy)
Do you A/B test your site’s content, commerce and customer experience? You should, no matter how big your site. Especially in a down economy, the worst decision you can make is to abandon testing. If anything, now is the perfect time to look for ways to improve. And testing is the best way to find the improvements that work best.
But what if you site gets little traffic? Just the other day someone told me, “My site doesn’t get enough traffic to bother with testing. Why should I wait six weeks – or more – to get data when I need business results now?”
Fair question. Here are 3 reasons why testing matters, no matter the size of your traffic:
- Time isn’t the issue. Conversion rate is. If your test case performs dramatically better than your original, you’ll probably know it (relatively) quickly. You can feel comfortable a 15% increase in conversion wasn’t just a fluke in as few as 200-250 conversions[*]. Said another way, you only need so many conversions to prove what works. The bigger the increase in conversion, the fewer conversions you’ll need to know it’s working. And the bigger the increase in conversion, the less time it takes to collect the data.
- Meaningful changes take less time to show up. If you don’t see a significant lift in a timely period, it may mean the new version isn’t working dramatically better than the original. If that’s the case, the amount of time it takes to collect the data isn’t the problem; the lack of lift is. Put your focus on the areas where you’ll see greatest improvement. Not sure where to start? Check out the tips here, here and here.
- Sometimes the cure is worse than the disease. What happens if your changed page actually performs worse than the original? You could lose sales and not know whether it was the new version that hurt you or some external factor that would have impacted sales in either case. If you don’t run the two versions side-by-side, how do you know which works better?
Yes, you’ll still need to run your tests for a bit of time. Anything less than a couple of weeks worth of data could be influenced by many factors, including day of week, time of day, holidays, what have you. But don’t focus on that. Focus on what you’ll learn and the results you’ll see.
Simply put, the right answer is never, “don’t test.”
* – This is, admittedly, a rough figure that assumes an 85% confidence level. To check statistical significance for your tests, use Google Website Optimizer’s built-in tool or download the Teasley Stat Calculator [Excel spreadsheet] here.
Are you getting enough value out of your small business website? Want to make sure you’re ready for the mobile, social, local web? Subscribe to thinks today and get more than just news. Get understanding.
Sphere: Related ContentRelated posts:
- The 3 reasons Amazon’s PayPhrase matters to your business.
- How much testing is too much testing?
- What A/B testing, P.F. Chang’s, a really good CEO and the hare have in common
- 4 sure e-commerce tips to boost conversion… for sure. (Small Business E-Commerce Link Digest – November 13, 2009)
Related posts brought to you by Yet Another Related Posts Plugin.
December 9th, 2008 at 12:03 pm
Tim, I couldn’t agree more that testing is the right way to gather data on improving website performance. I checked out the articles that you linked to, and they provided a good overview on how to choose the pages on which you should prioritize your testing.
I’m curious how you think about prioritizing types of media within individual pages. When you have the option of altering marketing copy, “buy now” links, positioning of contact details, product tour movies, etc, how do you decide where to start?
December 9th, 2008 at 8:45 pm
Jake,
I have the perfect answer for this. But I’m going to make you wait a day or two for it, because it’s such a great question, it deserves an equally good answer. Give me a couple days, and I will write it up the answer as a separate post. Thanks for the comment and keep me honest on the follow-up.
December 10th, 2008 at 7:10 am
Good to hear, Tim. I’ll look forward to reading the post.
December 17th, 2008 at 6:35 am
[...] between converting a sale and seeing your site visitors vanish. And, as we looked at last week, nothing works better than A/B testing to find out what your customers will respond to. But, Jake Stein asked a great question: How do you [...]
December 17th, 2008 at 6:36 am
Jake,
It took me a couple days, but here’s the answer you’ve been looking for. Let me know what you think.
January 7th, 2009 at 9:42 am
[...] Vox says so. Of course, you know what I suggest. A/B test to see for yourself what’s best for your business. A 10% lift is nothing to sneeze at. But [...]
March 20th, 2009 at 11:24 am
[...] testing? Whether it’s headlines or copy, images or information, calls-to-action or content, A/B testing is always the right idea. Maybe you’re wondering what to test [...]
March 23rd, 2009 at 12:28 pm
[...] So can great engineers. And when they work well together, truly amazing things can happen. But, not testing the work of either can lead you to failure. Falling into the trap of either/or is a poor decision. Making this an “x vs. y” [...]
May 13th, 2009 at 7:49 pm
[...] not impossible. By following best practices for developing your key performance indicators, testing where possible and segmenting your keywords into appropriate ad groups, you can attribute more [...]
May 20th, 2009 at 6:10 am
[...] know A/B testing is always the right answer, right? Why? Well, here’s everything you need to know. Whether you call it kaizen, continuous [...]
November 13th, 2009 at 7:33 am
[...] working on the right layout to show all this additional product information, don’t forget to A/B test your designs. Of course, while you’re at it, you also want to avoid Search Engine Land’s 6 deadly [...]
December 10th, 2009 at 9:17 am
[...] 3 Reasons Why A/B Testing Is Always the Right Answer [...]