Download PDF
The Definitive Guide to

Multivariate and AB Testing

By Khalid Saleh & Ayat Shukairy


Most websites continue to struggle with increasing their online conversion rates.

On average, an e-commerce store converts around 3% of its traffic into customers. The remaining 97% of the visitors leave the website without ever placing an order.

Other types of websites struggle with conversion as well. On average, a lead generation or subscription websites convert around 15% of its traffic into customers. The remaining 85% leave the website without ever placing an order.

If you are looking to increase your website conversion rate, you will have to conduct AB and multivariate testing. The problem is that most testing programs start without any planning and thus fail with the first 6 months.

We know this first hand because we have been helping online companies conduct testing for over 12 years. Our goal from this guide to help you with a good introduction to multivariate and AB testing. We will continue updating the different chapters in the book on regular basis, so if you have comments or feedback, please let’s know (@invesp).

Chapter 1

What is A/B testing (split testing)?

If visitors are not converting on your website, then obviously, there is something wrong that is stopping them. That is where AB testing comes in handy. AB testing (sometimes referred to as split testing) is the process of testing multiple variations or designs of webpage against the original page with the goal of determining which page generates more conversions.

Take Me To Chapter 1

Chapter 2

What is Multivariate Testing?

Multivariate Testing or MVT testing is the process of testing multiple variations of multiple ele-ments on a webpage with the goal of determining the best combination of elements to increase conversions.

Take Me To Chapter 2

Chapter 4

Comparing Multivariate & AB Testing Tools

It is time now to select an AB or multivariate testing platform to start running your tests. There are over thirty different tools available in the marketplace for you to choose from. So, you might ask:

What is the right split testing tool for my website? Will I affect the success of my conversion optimization program by choosing one particular tool?

The market is full of testing tools that range from simple free tools to sophisticated enterprise split testing software. This article will walk you through the pros and cons of different split testing tools available.

We will also suggest

Take Me To Chapter 4

Chapter 5

The Process of Creating a Successful A/B Test

You need people who can design proper test scenarios, analyze results accurately, and create meaningful follow-up experiments.

Poorly designed experiments might not provide concrete insights into why visitors behave a particular way on your website. You need criteria to determine, for example, which elements on a page you should test, which external and internal factors could affect the results, and in which ways to create the designs for new phases of your testing program.

As much as testing is essential to any conversion optimization project, it should only be conducted after the completion of equally essential stages of optimization work

Take Me To Chapter 5

Chapter 6

What Elements Should You Include in an A/B Test?

As you start the process of multivariate or AB testing, one of the critical questions you will face is: What elements of the page should change and test?

This article will suggest many elements that you should consider when creating a test. However, before you jump into selecting elements and creating variations for them, ask yourself: Why are you picking a particular item to test?

The process of selecting elements to test on a page is where real conversion optimization happens.

If you are looking for a long-term strategy, something that will produce a meaningful impact on your bottom line, then

Take Me To Chapter 6

Chapter 7

Multivariate and A/B Testing Best Practices

Conducting a multivariate test is exciting.

By using the right tool, you can quickly develop different designs for your webpage, split visitors between these designs and observe the conversion rates for each variation.

Done incorrectly, AB testing can result in a waste of money, misuse of staff-hours, and, even worse, a decrease in your conversion rates.

Here are ten best practices you must follow when conducting a multivariate test.

A/B testing best practices during the planning stage 1.  Set expectations correctly

Wrong expectations lead to disappointment and lost investment.

Many marketers start split testing because they read or watched a case

Take Me To Chapter 7

Chapter 8

16 mistakes that will kill your A/B testing (and what you can do about them)

AB testing mistakes

The scene has changed a lot since we first started doing conversion optimization back in 2006.

At that time, many marketers and C-suite executive were not convinced that there is a lot they can do about their website conversion rate. Things are much different nowadays with many companies attempting to deploy AB tests on their websites.

However, the majority of companies continue to primarily invest in visitor-driving activities and set lower budgets for converting visitors into customers.

Invesp’s 2017 state of the conversion rate optimization survey showed that majority of companies are allocating less than 8% of their digital marketing

Take Me To Chapter 8