What is Split Testing?

What's better – my dog or my cat?

What's better – my natural hair or my new ginger wig?

What's better – Convert Magazine or any of its subpar rivals? (though the answer to that one is pretty obvious)

If you compare one thing with another, that's split testing.

It's also known as A/B testing, which is basically exactly the same thing.

(If we're being irritatingly fastidious, the term ‘split testing‘ is typically used to refer to bigger comparisons, while ‘A/B testing‘ is often used to refer to very small comparisons, but I can already feel your mouse hovering ominously over the close button in boredom, so I'll shut up.)

We all split test every day. In our lives, in our heads, in our shopping baskets and in our bedrooms.

But if you're not split testing for your business, you should be.

Wait – I'm still confused! I'm an idiot and I need more half-baked examples

Don't worry, I'm an idiot too. We're all idiots here at Convert Towers. Stick around – you'll love it.

The bottom line is this: split testing compares two ideas in an objective, empirical way (instead of a speculative, creative, arbitrary, ‘my-idea-is-better-than-yours' way).

Let's take a familiar situation:

You: I think the ‘subscribe to our newsletter' button should be red.

Your colleague: I think it should be blue.

You: Okay, we're gonna have to settle this in the only way we ever settle things. Naked jelly wrestling.

(Cue jelly wrestling and an eventual – though admittedly arbitrary – winner)

Your colleague: See, I told you blue was better. Naked jelly wrestling is the best way to settle things.

You: No it isn't. Naked mud wrestling is a much more effective way of settling things.

Your colleague: Hah! Naked mud wrestling is better than naked jelly wrestling? You're insane! We need to get to the bottom of this once and for all. Set up the chess board!”

… And so on.

But this familiar situation is avoidable.

Next time you and your colleague are arguing about whether the ‘subscribe to our newsletter' button should be red or blue, you can use split testing (instead of asinine contests) to see which one is categorically best – with the help of actual data.

Split testing brings you to an objective, useful, practical conclusion, and it saves you the hassle of daily (and frankly repellent) jelly-based office clean-ups.

The real question is… why are you and your colleague doing naked jelly wrestling? You're both weird and pretty disgusting.

What's the point in split testing?

In case you've somehow missed this, split testing allows you to make helpful comparisons in order to increase your conversion rate.

Split testing allows you to work out which email template your customers prefer. Which cut-price product your customers like best. Which unbelievable offer your customers are more interested in.

You might think you understand your customers. But you don't. Until you undertake a statistical analysis of how they react to what you're speculatively and hopefully offering up to them, you don't know what they're into.

Yeah, I know – you love your neon green homepage, and you love how it's liberally dotted with pictures of your dead grandma. You especially love how there's a loop of Donna Summer's ‘I Feel Love' perpetually playing while your site visitors frantically search for their mute button.

You love it all.

But your customers might not.

You aren't converting all of your customers, and split testing allows you to find out why. Maybe your customers don't like the layout of your home page. Maybe they don't like its colour scheme. Maybe – more perilously – they can't find the ‘subscribe' button.

Or – even worse – the ‘buy now' button.

You need to find out –statistically and objectively – what's working and what isn't.

Split testing works, and everyone is doing it.

What can I conduct split testing on?

Basically anything (but let's use an email as an example):

  • The layout of an email
  • The wording of that email
  • The images
  • The colour scheme
  • The placement of the all-important ‘buy now' button

Basically, you're comparing different versions of the same email to see which version performs best – and the options for comparison are limitless. For a handy (and simple) visual introduction, check this video.

The more comparisons you make, the more you can get a handle on why your conversion rate is where it is – and how you can improve it.

But split testing isn't just useful for your emails. It's useful for your home page, your landing pages, your blog posts and every single facet of your online presence.

How do I start split testing?

Before you begin, make sure you know what you're trying to measure. If you don't know what you're measuring, you're wasting your time. There's not much point in using a thermometer to fit your new carpet.

What are you trying to measure? Landing page conversion rate? Why so few people are buying a certain product? How to get more people to sign up for your free course on your ‘PLEASE TAKE MY FREE COURSE' page?

If, for example, it's the product example (that's the second one I used for anyone struggling to follow), consider all of the variables which might influence that decision.

You've been sending out an email to promote that product, but very few people are buying. So here are some things you could try changing:

  • Where the ‘buy' button is
  • How big the ‘buy' button is
  • The subject title of the email
  • The wording of the email
  • Where you're displaying the price of the product
  • … and much more

There are endless options you should be considering.

What are the best tools for split testing?

You probably don't have time to manually create hundreds of different emails, web pages and landing pages. So you're gonna need some tools!

There are lots of options, and what you should use depends upon your budget, your goals and which tools you're already using. So I'm not gonna waste my time writing up an endless analysis of all the variables… but here are some good places to start.

That small list brings up two main thinking points.

The first is ‘I wonder which rigorous testing process Hubspot went through to decide their own tool should be ranked #1 on their own tool roundup list?'

The second is this – that there are different types of split testing tools. If you want a tool designed only for split testing and conversion rate optimisation, you can get it (VWO, ranked at #2 on Hubspot's list, is the best option).

But if you're already using an email marketing platform such as Mailchimp or Hubspot or whatever, you can use this same tool to perform some of your split testing experiments.

Should I split my lists 50/50?

You don't need to. But you do need to ensure you're using a big enough group of test subjects to arrive at a set of results which are helpful rather than hollow.

To conduct a split test, you need to send some of your customers in one direction, and some in another. Some get the red ‘buy now' button; some get the blue ‘buy now' button.

And that split can be 50/50. But it can also be 30/70, or 20/80, or any other weird ratio you can think of. Just make sure the sample size of each group is big enough that the results are significant and indicative.

You can even have more than two simultaneous comparisons, with smaller splits – 25/25/50, or 25/25/25/25 and… I'm sure you get the idea.

How should I measure my split tests?

You can measure according to click rate, conversion rate, open rate, buy rate and much more. Whatever you want to measure, you can measure it.

But again, it comes back to what you're trying to measure. If you're measuring how often a customer will buy a product when they're at the end of the funnel, split testing your email open rate based upon colour scheme won't do much good.

Be very specific.

Okay, that's cool, and I've learned more stuff. But that doesn't change the fact that I'm still an idiot. Give me a step-by-step process now or I'm gonna cry

Here are the three most important steps in your split testing process:

  1. What do I want to test?
  2. How exactly can I test it? What is the variable or variables which truly influence this thing?
  3. Run the test.

But that's just the simple version – Hubspot have explained it in a lot more depth than we could.

In short…

… that's why, for getting results and for knowing your audience, split testing is essential. There's no better way to get to the heart of giving your customers what they want – and how they want it.

Measurable, quantifiable and objective, split testing is incomparable.

Plot twist: we're split testing another version of this article where I say split testing is a waste of time. (Joke!!! It's awesome.)