Split testing sounds like a lot of work, especially when you are already hustling to get the most out of every minute you already have. Coming up with double the copy, the design, or concept and then figuring out what works feels daunting…but have no fear, this guide will not only take you through each step but will also show you how to test, what to test, and how to measure.

What is split testing all about anyway?

Split testing is when you conduct controlled, random experiments with the goal of testing and iterating to improve a specific metric, such as checkouts, form completions, or click-through.

With split testing…anything is fair game. You may find that a smiling man gives you more sales than a sultry woman or a red button fairs better than blue…but you never know until you test.

The typical process goes something like this.

Brainstorm a thousand ideas…implement one…and hope for results. Sometimes you get them, sometimes you don’t…so you make a decision and try something else. While it sounds like an intimidating process, one resounding success could mean thousands of more dollars for your business.

Before we test…let’s go over some rules.

While we (especially designers) like to go with our gut…our subjective intuition, that prefers blues to yellows, doesn’t count for much unless we have reason to back our choices. Mentioning your “split testing” gives you an automatic pass to the smart market/startup club…but just testing a whole bunch of random variables will only waste your time.

Here’s how to avoid wasting your time:

Create a test plan.

Decide what you will test, when you will test, and your goals for the test. These will inform your next steps including the length of time you run your tests.

Have some sort of measurement tool.

There are plenty of apps on the market that will help you run and analyze the results of your split tests. Our personal favorite is Optimizely…but a simple Google search will present you with a number of options that may work for you.

Create a hypothesis.

Testing isn’t just for fun. You need to have an idea as to what you are doing and the results it may produce or if the data it will uncover is worth your time.

Only test one thing at a time.

Each test should have one variable and a constant. Variables must be specific…this isn’t testing one whole design against another, it is testing the granular details like a blue button vs. a red button…or a verb in the headline vs. an adjective. You pick one item to test (ie. a blue button) and then measure the performance against your current site.

Make sure your test achieves statistical significance.

Split testing isn’t about asking your grandma and a few friends what they think. You need to test a large enough sample of random people so that you can draw conclusions. To achieve statistical significance you need a large sample size, so that very small differences will be detected as significant. This means that if you do get a significantly different answer, with a larger sample size you can be sure that the difference is real.

Document

Having answers isn’t enough…write the conclusions of each test down so you have documentation of the tests you have run and their results. Keeping it all in your head is not the way to go when it comes to numbers and tests.

Set goals and measurements for success

Testing is an ongoing process. Setting goals for what you want or expect to accomplish with each test is important when measuring success. If your goal is to increase purchases and you create a variable that gets more click throughs but doesn’t increase purchases…you can’t deem that variable a winner just because you saw an increase in something. If you decided you wanted to increase purchasing and your test didn’t do that….You must iterate and test again.

Think of measurements for success like this. If a doctor gives you a cure for baldness that fixes your toothache, but you still have a head as shiny and as bald as a bowling ball…and your end goal was hair, you wouldn’t consider taking that medicine a success.

Would you?

Now that parameters have been set, what exactly do you test?

You can and should test pretty much everything. You never know what surprising adjustment may give you results. As long as you have a constant and have followed the rules above, you can test everything from a color scheme to a word in a sentence (don’t drive yourself crazy though…).

Try these common things people have tested that have provided great results:

Calls to action

You didn’t build your website for kicks…the number one goal of your site is to get people to take action, so this should probably be something you test first. Whether you are using a button or plain text, test the size, color, position, and text of the call to action. And make sure you give whatever call to action you have plenty of breathing room. Whitespace and focus are important so that you can optimize the testing you are doing.

Visuals

Does a video work better than an image? Does an image of a woman work better than a man? One of the most interesting tests we have come across is the test of the professional image vs. the casual image. Business people tend to think a buttoned up image of yourself impresses potential customers but tests have shown that people gravitate toward regular, relatable guy or gal images. Never would have thought that right?

But then again, this is a perfect example of why you should test… just because a test has worked for some people doesn’t mean it will work for you (you probably don’t want to see a doctor performing surgery in her Halloween costume). Testing based on other people’s results is a guideline to get you started on discovering what optimizes your site best.

Forms

Do three fields convert better than five? Does asking for an email address vs. an email address + someone’s name convert better? These are things you should test. You may be thinking…well, it’s just a form, but a form is the gateway to your treasure trove…their information. Getting people to fill out that form is your lifeline so all focus should be on that action. The intricacies of a successful form can take on many shapes such as larger fields, assuring text (we promise to never spam you!), images, and offers. Don’t take for granted that just because you got them to click on your button, they will make it to the finish line.

Emails

Not only can you optimize when you send emails, subject lines, and length and design of emails, you can test how well they bring people back to your site. Setting up landing pages specifically for each email campaign is a great way to test the effectiveness of the call to actions in your emails.

This is just the beginning, however, there are a number of other things you can test and your priority list should be based on your specific goals and business.

Now that you have the parameters on how and what to test, what do you do with this information?

We live by the motto AT…or Always Test.

Testing is never done, but once you have conclusive answers to a question, you need to implement your results so that they become your new constant and you can iterate further.

You also have to quantify the ROI.

If changing a process will cost you thousands of dollars but will only have a marginally positive result, it may not be worth it to implement.

Perhaps you have to change your whole website or train employees on a new process only for a few dollars in return. Positive results are good, but you have to measure that against the cost to implement.

Also, remember that these results aren’t necessarily from click to money in your pocket, they may be a little more subtle. For example a more effective marketing campaign means you sales funnel is greasier. Testing what worked in an ad can make your website more effective when you match the content.

When done consistently, A/B testing, can improve your bottom dramatically. It’s easier to make decisions when you know what works and what doesn’t, and have evidence to support your iterations.

A few last tips to consider

Your answer should have a winning variation to show a lift with 95% statistical significance and 80% statistical power (industry standards when it comes to evaluating the validity of your results).

And remember, you have to measure the impact of your iteration based on your entire funnel, not just on the page you are testing. More clicks on a button because your text said “free bananas for everyone” when you are selling shirts that doesn’t result in more shirts sold is not a success.

Photo by: Sébastien Marchand