Beginner's Guide to A/B Testing

Most websites profit from one thing: conversions. The more visitors you can convince to become a lead, purchase your product, subscribe to your service, or accept an offer, the more money you can make. Of course, the world of sales and marketing often feels daunting, like you're leaving some things to chance. If you've been approaching your conversions as a game of trial and error, it's time to look into A/B testing.

What Is A/B Testing?

The concept of A/B testing is straightforward. The process starts with a hypothesis about a specific element of your website or webpage and then turns to run a test to see if your theory is correct (or incorrect). For instance, you might wonder if using "Buy Now" will get you more sales than "Add to Cart" (or vice versa). To test your theory, you would use a split-testing tool that randomly directs half of your visitors to a version of the page with the former text (Test Group A) and the other half to a version of the page with the latter text (Test Group B).

Not convinced that A/B testing could be beneficial? Do you think it's too much of a hassle? Consider these statistics:

Improving your UX design through user testing can boost your conversion rate by 400%.
Over 60% of companies feel that using A/B testing helps them optimize their conversion rate.
One in every eight A/B tests drives significant change for organizations.

The fact is, running A/B tests is incredibly simple, fewer than half of the companies use dedicated software to do it, and it can reveal small changes that have significant effects on your website's conversion rate, making them extremely worthwhile. With that said, there are some best practices you'll want to follow to get the most accurate and meaningful results from the test.

Common Elements to A/B Test

If you're looking at your website, you won't need long to decide on a handful of elements that could use A/B testing. Here are the most common and helpful elements to split test.

Test Your CTA Buttons

Your call-to-action buttons may appear across your website, and they should all be consistent in text and color so that they stand out. However, you may end up asking yourself whether you could use a brighter color or a better contrast in their design. Maybe they should be a little larger? Perhaps they'd do better if you worded them differently?

Since your CTA button drives all of your conversions, it needs to be as effective as can be. Popular things to test with your CTA button would include:

The size, shape, or color of your CTA button
The placement of your CTA button on your website
The wording on your CTA button
The motion or hover effects of your CTA button

These elements help bring the eye to your CTA button and, ultimately, give people the nudge they need to click. So, maybe changing the text from "Book a Call" to "Schedule a Consultation" (or vice versa) will lead to the improvement you want to see.

Check Your Headings

While A/B testing the headings for each blog post you publish likely isn't feasible or even necessarily beneficial, it is worthwhile to test the headings on important pages, like your homepage and landing pages. These are pages visitors are bound to see on their way to converting. So, a good heading could make the difference between a busy visitor sticking around to learn more or else move on.

You may also test sub-headings or their position on the page. For instance, you may find that visitors prefer your H1 showing below your featured image rather than above it. Testing seemingly small things like this over time will help you continuously perfect and improve the performance of your website.

Try Different Images

Most websites overflow with graphics, but do your visitors like them? Whether corporate-style illustrations, stock imagery, or unique images of your products, team, or company, images are the first things that catch our eyes when we go to a website. It's worth checking to see if yours are doing their job in enticing visitors to stick around.

Besides swapping the images themselves, consider their size and placement. You may also keep the image the same but make adjustments, like editing text onto the image, cropping it into a shape other than a rectangle/square, using parallax effects, and so on. While testing all of those options will take time, it's an endeavor that can prove well worth it — especially for websites that revolve around graphics (think creative niches like art, fashion, and design).

How to Use A/B Testing

As you venture into the prospect of using A/B testing, a few questions might come to mind, like: How long should you run the tests for? Can you run tests simultaneously? By how much do the results need to differ to warrant a change? Let's explore these questions in this step-by-step guide.

Set a Goal

Before you pursue any change or research as a business, you need to set out a clear objective. Here, you'll need to identify the metric you're trying to improve. For instance, do you want more people to sign up for the newsletter you advertise in your blog posts? Are you looking to get more visitors to schedule a consultation?

Once you decide on a goal, you need to select a single element that you're going to split test and then come up with your Version B. You must keep everything else on your website the same, or it will be impossible to tell whether the Version B results are different because of your selected change or some other change.

Decide How to Measure

Setting your goal tells you what you're measuring, but how will you go about doing it? The exact metrics you're looking at may not always be the same. For instance, changing the wording on your CTA button means you're likely tracking click-through rate and the overall number of clicks that result in conversions (i.e., the conversion rate).

If you're testing something like headings or sub-headings on a page, you'll instead want to look at elements like average time spent on the page. This metric will tell you if the changes you made to the heading or sub-heading truly engage visitors more than the originals.

Choose Your Tools

There are some very advanced A/B testing tools out there that let you design and re-design pages using a drag-and-drop editor, but they can be pricey. The other end of the spectrum means designing and coding things manually; however, not everyone is a developer, nor do you necessarily have one at the ready.

Don't feel that A/B testing is out of reach if you don't want to commit to a super-advanced tool and don't have the resources to code things yourself. Plenty of free and affordable tools exist to help you run split tests with ease. Google Analytics even has an A/B test feature you can try out.

Best Practices of A/B Testing

Now, you probably have a good list of ideas regarding the elements you're able to test, so before you dive in, review these best practices.

Test What Matters

With a handful of things to potentially test on every page of your website, you want to start with the elements you feel will have the most impact. For many websites, this is the CTA button or the checkout page — something that directly fosters and affects a visitor's decision to convert.

Of course, if you feel that you need to work on an element lower in the funnel, like getting a visitor to visit your feature page, start there instead. Just don't start your tests on small aspects that may or may not garner results. For instance, image placement on a page is likely a low-priority thing to test (unless, as mentioned above, you're in a creative niche where graphics have significant power).

Consider Sample Size

One thing that holds many small websites back from using A/B testing, or at least using it to its full effect, is not having enough regular traffic to find it useful. To be effective, A/B testing requires a relatively large sample size. For instance, ten visitors, with five seeing version A and the other five seeing version B, isn't enough to tell you anything. Even one hundred visitors can't truly give you a clear picture of the problem.

When considering sample size, you'll want to look at the average number of visitors your website receives and make sure you have enough traffic coming in to make it worthwhile. You can always run an A/B testing campaign in conjunction with paid advertising campaigns if you don't have organic traffic, although it's best to use A/B testing to optimize your website before you pay to bring in traffic.

Schedule Your Tests Right

When and for how long you run your tests will come down to several factors. First, it depends on how much traffic your website is getting because you need to have enough visitors see each version before you can draw any conclusions. Second, it's determined by how you defined your goals in the first step.

A good rule of thumb is to determine how many conversions or events you want to occur with either version before you select one as superior. For instance, if you're testing your CTA button text, aim to get 100 conversions before you conclude which one is superior. You'll need to run your test for longer, though, if you want to get 400 conversions before making your conclusion. Remember that it's up to you how large of a sample size you feel is reasonable.

Avoid the Urge to Make Changes

While it goes against every rule of A/B testing, it's not uncommon for testers to get the urge to make changes mid-test. After all, you'll likely watch the metrics, and you may find that you suddenly re-think your versions or you decide there's a better alternative. In any case, you must not disrupt the test, or else you can't count on the results of the test.

If you really want to end the test, you'll need to scrap the results, make your desired changes, and start the test again. However, do your best to let the test run its course and then go with an additional split test after the fact if you still feel it's necessary. Whatever you do, remember the golden rule of A/B testing: For each test, only change one thing, and don't change it more than once.

Get the Results You're After

Are you considering A/B testing for your website? How about your email list? Scripted can help you get the results you're after with email copywriters who are experts at generating conversions. See for yourself! Explore our freelance writers today and give Scripted's membership plans a try.

Published by Ciara Antolini on Monday, November 1, 2021 in Industries, Guide, Marketing, Measurement.