Posts tagged as:

test design

Can You Market Test Your Time and Money?

by Tim on November 22, 2011

Test what works I’m no fan of linking to the New York Times (damn you, paywall), but this interview with Sasha Issenberg is well worth the read. Issenberg, who is writing a book about “the new science of [political] campaigns,” covers ground most Big Thinkers know well: measurement and testing. I’ve talked often about why testing is always the right answer. It’s interesting to see how political campaigns put these same tactics to use in measuring the effectiveness of their campaigns. The interview focuses on Rick Perry’s reelection campaign (the book looks at several campaigns), but I loved this quote:

“The findings from those 2006 tests dramatically changed how Carney prioritized the candidate’s time and the campaign’s money when Perry sought re-election again in 2010 and will inform the way he runs for president now.” [Emphasis mine]

Political campaigns, just like any marketing, can’t afford to waste money or resources (especially the candidate’s limited time). So testing where those resources and funds work best matters. The question is, are you testing how effectively you’re spending your money and your time? Don’t you think you should?

Are you getting enough value out of your small business website? Want to make sure your business makes the most of the local, mobile, social web? thinks helps you understand how to grow your business via the web, every day. Get more than just news. Get understanding. Add thinks to your feed reader today.

Or subscribe via email.

And while you’re at it, don’t forget to follow Tim on Twitter.

Tim Peter & Associates helps companies from startups to the Fortune 500 use the web to reach more customers, more effectively every day. Take a look and see how we can help you.

Technorati Tags: a/b testing, marketing, marketing best practices, measurement, metrics, online marketing, strategy, success, testing

Tagged as: marketing 101, marketing strategy, online business, online marketing, online marketing strategy, Strategy, test design, testing

{ 0 comments }

Do A/B Tests Worry You?

by Tim on February 22, 2011

in E-commerce

Anyone who’s spent much time around thinks Central knows how passionately I believe A/B testing is always the right answer. Seriously. And while A/B testing might seem scary if you’ve never done it before, it’s not as hard as you might think.

What’s that, you say? You’re not to sure what to test and how to do it? Well, never fear, Big Thinker. The following ideas will help you along the way.

For example, Linda Bustos and Anne Holland’s great 6 tips for e-commerce testing offers amazing lessons on how to test effectively and efficiently on your site.

And while A/B testing has its pitfalls when not done right, you can easily learn from others’ real-world experience. For instance, ProFlowers is one of the best converting sites on the Internet and conducts A/B tests all the time. Just take a look at what a well-executed A/B test looks like to help you along the way.

What if you’re just getting started with A/B testing and don’t know where to begin? In addition to Linda and Anne’s tips, take a look at the thinks method for what to test first.

And, finally, for more ideas, check out our Ultimate A/B Testing Case Study Roundup.

I suppose testing isn’t the right answer if you don’t know how to do it or don’t test the right things. But, thanks to Anne and Linda and ProFlowers, you don’t have to go it alone.

Or subscribe via email.

And while you’re at it, don’t forget to follow Tim on Twitter.

Technorati Tags:
a/b testing, call-to-action, conversion rate optimization, E-commerce, e-commerce, ecommerce, getelastic, landing page optimization, marketing best practices, online marketing, testing

Tagged as: a/b test, a/b testing, test design, testing

{ 1 comment }

Ultimate A/B testing case study roundup (Small Business E-commerce Link Digest – June 25, 2010)

by Tim on June 25, 2010

in del.icio.us links,E-commerce

It’s no secret here that I believe A/B testing is one of the the best ways to help your customer and quiet Avinash Kaushik’s HIPPOs (see Rule #10) (Full Disclosure: As the head of e-commerce, I can be something of a HIPPO myself in my day job. So, in some ways, I know the dangers even more. It’s easy to think you’re really smart when you’ve been doing something for a while – and really cool when a test a.) proves you wrong and b.) makes you a lot of money).

In fact, I’ve talked before about reasons why A/B testing is always the right answer and examples of what an A/B test looks like. But, I’ve also been collecting a number of other posts that make the case for A/B testing. Check ‘em out:

I’ve talked about how to choose what to A/B test first before. But, I have to say, I love this look at 8 A/B testing case studies from Visual Website Optimizer’s blog.
FOT (that’s Friend of Thinks) and guest poster Jake Stein of RJMetrics shows an A/B test he conducted for a landing page, complete with a full analysis of what worked best.
Sometimes, even with lots of data, you’re forced to make a challenging call. GetElastic offers a case study that didn’t quite go as planned. But, even though the GetElastic study didn’t pan out as expected, take particular note of what the team learned.
Evan Miller gives a quick overview of the math behind A/B testing that’s worth a read. Sure, you can just use calculators like the Teasley set to test your significance. And, admittedly, the math gets a little geeky, but if you’re serious about A/B testing, I’d take the time to check it out.
Also, if you’re really into the math, Cennyd Bowles takes a deep look at statistical significance and other testing pitfalls. Heavy, but surprisingly readable. I’d also call attention to my comment on the post for a counter-argument to Cennyd’s concern.
You don’t just have to A/B test your landing pages, of course. Jason over at A Smart Bear gives a great example (complete with cute hamsters!) of A/B testing on AdWords. Cool. (And, did I mention the HAMSTERS?!?)
And, finally, if all that isn’t enough for you, Smashing Magazine has their Ultimate Guide To A/B Testing. And it is simply smashing.

Whew! That’s a ton of resources for you to use as you plan your own A/B tests. And you should.

Did I leave something out? I’d love to continue to collect these case studies here. If you find a good one, do me a favor and post it in the comments.

Or subscribe via email.

And while you’re at it, don’t forget to follow Tim on Twitter.

Technorati Tags:
a/b testing, adwords, Avinash Kaushik, call-to-action, click, continuous improvement, conversion, conversion rate, conversion rate optimization, E-commerce, e-commerce, ecommerce, getelastic, google adwords, internet marketing, landing page design, landing page optimization, marketing best practices, measurement, metrics, page design, testing, tools