Image
The Definitive Guide to

Multivariate and AB Testing

By Khalid Saleh & Ayat Shukairy

Introduction

Most websites continue to struggle with increasing their online conversion rates.

On average, an e-commerce store converts around 3% of its traffic into customers. The remaining 97% of the visitors leave the website without ever placing an order.

Other types of websites struggle with conversion as well. On average, a lead generation or subscription websites convert around 15% of its traffic into customers. The remaining 85% leave the website without ever placing an order.

If you are looking to increase your website conversion rate, you will have to conduct AB and multivariate testing. The problem is that most testing programs start without any planning and thus fail with the first 6 months.

We know this first hand because we have been helping online companies conduct testing for over 12 years. Our goal from this guide to help you with a good introduction to multivariate and AB testing. We will continue updating the different chapters in the book on regular basis, so if you have comments or feedback, please let’s know (@invesp).

Chapter 1

What is A/B testing (split testing)?

If visitors are not converting on your website, then obviously, there is a problem that is stopping them.

You can go ahead and ask your design team to create new designs, but the question remains: how do you know that the new designs will convert more visitors compared to the original design?

That is where AB testing comes in handy.

A/B testing (sometimes referred to as split testing) is the process of testing multiple new designs of a webpage against the original design of that page with the goal of determining which design generates more conversions.

The original design of a

Take Me To Chapter 1

Chapter 2

What is Multivariate Testing?

Editor note: Please note that we use button colors, fonts, headlines as a way to clarify the concept of testing. Succesful AB and multivariate tests will include more sophisticated changes to your page.

Multivariate Testing or MVT testing is the process of testing multiple variations of multiple elements on a webpage with the goal of determining the best combination of different elements on the page to increase conversions.

By using MVT testing software, you can test different variations of any element on your page (headlines, images, buttons, etc.) to measure their impact on your conversion rates. The following image displays an

Take Me To Chapter 2

Chapter 3

The Difference Between A/B Testing and Multivariate Testing

Both A/B testing and multivariate testing share the same concept: splitting visitors between several designs of your website or webpage to determine which of these designs generates more conversions.

So, which of these two methods is best suited to help you increase your website conversion rate?

AB Testing

AB Testing (or Split testing) allows you to test one or more variations for each page on your website against each other.

Let’s take few examples:

On an e-commerce website, you can create a new design of your product page to test whether the product image should be placed on the left or

Take Me To Chapter 3

Chapter 4

Comparing Multivariate & AB Testing Tools (updated for 2018)

It is time now to select an AB or multivariate testing platform to start running your tests. There are over thirty different tools available in the marketplace for you to choose from. So, you might ask:

What is the right split testing tool for my website? Will I affect the success of my conversion optimization program by choosing one particular tool?

The market is full of testing tools that range from simple free tools to sophisticated enterprise split testing software. This article will walk you through the pros and cons of different split testing tools available.

We will also suggest

Take Me To Chapter 4

Chapter 5

The Process of Creating a Successful A/B Test

You need people who can design proper test scenarios, analyze results accurately, and create meaningful follow-up experiments.

Poorly designed experiments might not provide concrete insights into why visitors behave a particular way on your website. You need criteria to determine, for example, which elements on a page you should test, which external and internal factors could affect the results, and in which ways to create the designs for new phases of your testing program.

As much as testing is essential to any conversion optimization project, it should only be conducted after the completion of equally essential stages of optimization work

Take Me To Chapter 5

Chapter 6

What Elements Should You Include in an A/B Test?

As you start the process of multivariate or AB testing, one of the critical questions you will face is: What elements of the page should change and test?

This article will suggest many elements that you should consider when creating a test. However, before you jump into selecting elements and creating variations for them, ask yourself: Why are you picking a particular item to test?

The process of selecting elements to test on a page is where real conversion optimization happens.

If you are looking for a long-term strategy, something that will produce a meaningful impact on your bottom line, then

Take Me To Chapter 6

Chapter 7

Multivariate and A/B Testing Best Practices

Conducting a multivariate test is exciting.

By using the right tool, you can quickly develop different designs for your webpage, split visitors between these designs and observe the conversion rates for each variation.

Done incorrectly, AB testing can result in a waste of money, misuse of staff-hours, and, even worse, a decrease in your conversion rates.

Here are ten best practices you must follow when conducting a multivariate test.

A/B testing best practices during the planning stage 1.  Set expectations correctly

Wrong expectations lead to disappointment and lost investment.

Many marketers start split testing because they read or watched a case

Take Me To Chapter 7

Chapter 8

16 mistakes that will kill your A/B testing (and what you can do about them)

AB testing mistakes

The scene has changed a lot since we first started doing conversion optimization back in 2006.

At that time, many marketers and C-suite executive were not convinced that there is a lot they can do about their website conversion rate. Things are much different nowadays with many companies attempting to deploy AB tests on their websites.

However, the majority of companies continue to primarily invest in visitor-driving activities and set lower budgets for converting visitors into customers.

Invesp’s 2017 state of the conversion rate optimization survey showed that majority of companies are allocating less than 8% of their digital marketing

Take Me To Chapter 8