Why Split Testing Can Be Too Much Of A Good Thing

Author: kathyalice
Total views: 12
Word Count: 879
View PDF

Add2Netvouz

Split testing, actually any testing in general, can be enormously helpful in fine tuning your websites to be more effective sales channels for your business. The goal of the testing is to produce higher conversion rates, but the term conversion rate is used loosely here. For example, you can measure whether a user buys your product, clicks on your affiliate links or opts into your mailing list. Split testing can be a powerful too, however, like any other tool that produces hard measurements it can be abused and it's results misinterpreted.

What is the purpose of split testing?

If you are selling an ebook on how to improve your golf game, what is the best headline to engage your visitor's interest? Is it "Perfect Your Swing?" or "Lower Your Handicap?". Even if you are a diehard golfer, don't make the mistake of thinking you know the answer. You'll likely get it wrong. You will need to test different versions of your headline and let your site visitors tell you the answer. I once wrote a headline that started off with "Who else wants to ...". I had seen similar long headlines on many other sales pages. My rationale was that if that style of headline was working on other sales pages, it should work on mine. Fortunately I decided to test it, and what I found was that it performed poorly against my control headline! By listening to my site visitors through testing, I found out that they didn't like my "Who else..." headline at all.

How does split testing work?

Here is how split testing (also known as A/B testing) works, half of your site visitors see one version of your site (the A version), the other half sees the another (the B version). You monitor the conversion rate of the two versions. The version that has the highest conversion rate is what you use going forward. It's often an iterative process, you keep testing changes until the conversion rates don't change very much from version to version. You are not limited to only testing the headline of your site. You can test other prominent components of your site that play a strategic part in converting your customers. For example, here are some website components you may consider testing: the main image of your landing page, the call to action, and even the promotional copy.

So when it is not useful?

Split testing is not particularly useful if you change more than one component of your site at a time. If you change both the headline and the main image on your B copy and it performs better, what does that tell you? That the image performed better or the headline? Sure you can use the B copy of your site going forward, but what if the best combination was the B version of the image and the A version of the headline? The only way to know is to test each change individually. If you want to test multiple versions of your site components, you should explore using a tool that supports multivariate testing. Multivariate testing it just what it sounds like, a way to test multiple changes at a time.

Another trap that testers fall into is putting too much significance on minor differences in the results. If you have 50 visitors to your site and 3 of them convert on the A copy and 4 of them convert on the B copy, is that justification enough to decide the B copy is the best version going forward? Not really. If you have a low traffic site, it's best to be patient and wait for more traffic, or put some effort into promoting the site to get more traffic so you have a more statistically significant test. Don't keep changing your site day by day trying to take advantage of what could be the result of a complete whim of your site visitor.

Keep your mind open

Split testing is useful but variances in your conversion rates may have nothing to do with your split tests and you have to be careful not to draw the wrong conclusions. For example I was running a split test over a period of time for my headlines. I submitted two articles spaced apart by over a week to the article directories during my test. The first article had a higher conversion rate than the second. The likely cause is that my second article was not as well targeted to my desired audience as the first. Simply put, the people that found my second article interesting and clicked through to my site were not interested in what my site was offering. If I had changed my headlines in the midst of this I might have drawn the wrong conclusion. My choice of topic for my second article had a much larger impact that either of the headlines I was testing.

Split testing is a powerful tool but like all measurement tools it's results need to be taken into context so you can make wise decisions going forward.

About the Author

Kathy Alice has been in the technology industry for 20 years. She helps business owners and marketers bridge the technical gap. She loves finding innovative solutions and sharing them with online business owners.