How to use A/B tests to optimize your website

How to use A/B tests to optimize your website

Your gut is a good guide for a lot of things—when to eat, what to eat, whether it’s a good idea to call your ex—but when it comes to the design of your website, it can be pretty unreliable.

Now, that’s not to say you have bad taste or don’t know your users. But questions of where to put your navigation, if a particular hero image is effective, and whether you should use a red button or a blue button—these are matters best decided not by your gut, but by quantifiable data.

In today’s post, we’re going to show you how to get that data using an incredibly useful and infinitely adaptable method called A/B testing. Let’s dive right in!

What is A/B testing?

A/B testing simply means pitting two versions of something against each another and seeing which comes out on top—kind of like a cockfight, only with less squawking. These tests let you compare your original site (variant A) to a version you’ve adjusted based on assumptions and/or research telling you it will work better (variant B). If B wins, you can do a little dance and make the modification permanent. If A wins, you should probably try something else.

What’s so great about A/B testing?

Rather than trusting your gut or waiting for someone to volunteer an opinion, A/B testing allows you to trial changes before committing to them. As Torque Magazine writes, “It enables data-backed decisions and a shift from ‘I think’ to ‘I know.’” When you know how your audience will react before you pull the trigger, you’re much, much more likely to make the right decision.

What does the A/B testing process look like?

You begin the A/B testing process the same way you would a ninth-grade science experiment: with a defined goal. What is it that you want to accomplish? Do you want to sell more products? Get more people to hire you? Increase your readership? Your goals will become the metrics you use to determine which variant works best for your site.

Like any scientist worth her salt, you also need to develop a hypothesis, or an “if x, then x, because x” statement. For example, if you’re testing the colour of your CTA, you might write, “If I change the colour of my CTA from red to green, then I will convert more browsers to buyers because green means ‘go.’”

With hypothesis in hand, you can start creating variations of the site elements you want to test. In this example, you’d create a green CTA button. You might also create blue and orange versions while you’re at it, but make sure you don’t go overboard. (Don’t create a version that’s both purple and sparkly, because if it wins, how are you to know if it’s effective because it’s purple or because it’s sparkly?)

Now all that’s left to do is run your experiments. There are lots of tools to help with this, and we’ll get to those in a minute, but first let’s talk a bit about the types of things you could be testing…

What are some example A/B tests you can run?

You can test virtually anything on your site, but you might want to start by looking at your analytics. Can you identify any weaknesses? Are there pages you think are underperforming? If your goal is to increase the number of sign-ups, you might want to run a few tests on your sign-up page: the length of the form, the types of fields, the privacy disclaimer, etc.

Here are a few of the more commonly tested elements you might consider:

  • Call-to-action buttons: placement, copy, colour, size
  • Navigation: sticky vs. static, primary vs. secondary
  • Layout: headlines, sidebars, footers
  • Content and copywriting: length, tone, typography
  • Forms: fields, length, privacy, social proof
  • Advertising: pop-ups, banner ads, content, level of annoyingness

ConversionXL recommends starting with the “low-hanging fruit” (presumably the colours and sizes of various elements) and working your way up from there. We think this is good advice. Don’t run before you can walk, etc.

A few recommended tools and plugins

Okay, now that you’ve got a handle on what goes into A/B testing, let’s look at some of the awesome tools and plugins you can use to pull it off. There are a bunch of people who have already written about this (see here and here and here and here) so we’ll keep it brief and play favourites. Here are our top three:

  • Google Analytics Content ExperimentsOur top pick because, odds are, you’re already using some of GA’s incredible functionality on your site. With installation and setup out of the way, you can get right down to testing—and not just A/B testing, but A/B/C/D/E/F/G/H/I/J testing (it allows for up to 10 variants to be tested at once). For the price—free!—it can’t be beat.
  • Optimizely: Ideally suited to beginner A/B testers, Optimizely makes it easy to set up experiments using its WYSIWYG (what you see is what you get) page editor. To get started, simply add their testing code to your site. There’s even a plugin to help you! It’s free for up to 50,000 visits a month, so you can learn the basics while you build your audience, and graduate to a more robust plan when you’re ready.
  • Nelio for WordPress: This one costs money, but it’s worth shelling out for if you run a larger site. Unlike the other options in this list, Nelio allows you to set multiple goals, as well as compare different themes, landing pages, custom post types, widgets and CSS variations. Nelio also has the advantage of storing all its data and calculations on an external server, meaning its extensive functionality won’t be a drag on your site. 

Wrapping up

A/B testing is about knowing your audience, plain and simple. Not “thinking” or “guessing” who they are or what makes them tick, but actually knowing. Equipped with real, quantifiable data, you’ll be able to make better decisions, provide more value, and maximize your site’s potential.

What are you waiting for?

1 Comment

  1. Intaj Mondal July 6, 2016 Reply

    Alanna, I must say its unique piece.Cool way to know audience and then need modification.Thanks

PSA: As of Sept. 29, 2017, you can no longer purchase themes from this website. We will continue to provide limited support, including bug fixes and security updates, until March 31, 2018. Thank you for everything. We'll miss you dearly.