Disclaimer: This section is a TL;DR of the main article and it’s for you if you’re not interested in reading the whole article. On the other hand, if you want to read the full blog, just scroll down and you’ll see the introduction.
- If you have been doing CRO for some time, you know it is not an easy task to produce consistent increases in your (or your clients’) website conversion rates.
- There are months in which your A/B tests produce significant increases in conversion rates. In other months, you find not-so-great results.
- To build our conversion optimization practice, we introduced the conversion framework. But the framework is not enough in itself. How you implement the framework is where the secret lies.
- Back in 2005, I led the largest eCommerce implementation for Motorola. When we launched the website, we had hundreds of thousands of people come, but less than 100 sales in the first month. This led to the beginning of Invesp.
- We started as a 2 man operation, as we grew, we brought in more specialists with the right skills but we needed a framework everyone can follow across every client project.
The Conversion Optimization System
- To successfully implement the framework, we created a 12-step process.
- This conversion optimization process is one we have tested across eCommerce websites, lead gen websites, SaaS, and affiliate websites. We have also tested this process in verticals such as consumer services, education, financial services, etc.
- The first step is to conduct a heuristic analysis. To conduct a heuristic analysis, the first step is to determine the 10 most common tasks a visitor takes on your website. This is usually done by four members of our team (more on this in the article).
- The second step is to conduct qualitative research. This is a crucial step because your site visitors get to tell you what’s working and what’s not working (more on this process in the article).
- The third step is to conduct quantitative research. This is where analytics come into play. Analytics provides a wealth of information on how visitors interact with the website and shows you where visitors are leaking (more details in the article).
- The fourth step is to conduct competitor analysis. As the name implies, you’re checking out the competition, but the rule of thumb applies here. Check them out, but don’t copy them (find out more in the article).
- Now it’s time to proceed to the 5th step which is creating your conversion roadmap. In this stage, your fix right away list should be sent to the developers to attend to, while your test list should be prioritized (more on this in the article).
- The 6th step is to identify the problem areas on a page. Leading up to this stage, you’ve conducted a heuristic evaluation, you’ve run some polls, etc. Now, you need to look at heatmaps and session recordings to evaluate individual pages you want to optimize (more on this in the article).
- The 7th step deals with you analyzing the page using the conversion framework. We utilize the Conversion Framework to do a 360 analysis of the page to determine some of the issues we are seeing on the page. The framework evaluates every element on the page from the following areas (see the article for more details).
- The 8th step is to prioritize the problems on the page. To determine what problems should be fixed first on the page, we use two factors: Potential conversion rate uplift. Implementation cost (find out more in the article).
- Now, it’s time to create a hypothesis. This is the 9th step. The hypothesis is a critical component to testing. Ultimately, you’ve done your homework and your research. You’ve selected the page and identified the problems on the page. Logically, the next step would be to come up with the solution (see the article for our template on coming up with the hypothesis).
- The 10th step is to implement the hypothesis using new designs. Creating a design that fulfills the hypothesis can be challenging sometimes. You must be careful that your new designs are driven by the hypothesis, otherwise, you will introduce variables you won’t be able to account for in your results (find out more in the article on how we control the variables for our designs to stay in line with our hypothesis).
- Now it’s time to conduct the A/B test. This is the 11th step. You split the page visitors between the old design and the new designs. You let your visitors judge the quality of the designs (see the article for more details on how we conduct A/b tests).
- The final step in the conversion framework process is to conduct post-test analysis. You ran your A/B test, it came to a conclusion, and you have a challenger that beat the original design. What is next? There are 3 questions to answer (see the article for the questions you need to ask yourself).
Here’s A Longer And More Detailed Version Of The Article
If you have been doing CRO for some time, you know it is not an easy task to produce consistent increases in your (or your clients’) website conversion rates.
There are months in which your A/B tests produce significant increases in conversion rates. Other months, you find not so great results. We struggled with this inconsistency 11 years ago, when we first started Invesp.
We knew that we had to solve this problem if we were looking to build our conversion optimization practice. Our first attempt to solve this problem was by introducing the Conversion Framework.
But the framework by itself is not enough.
The real secret to increases in conversion rates is in how you implement a framework on a website or a landing page.
For a very long time, this has been a secret we did not share.
Today, I am doing something different. I will be sharing with you the process we use to help some of the internet largest websites increase their conversion rates.
Let me first take you back to 2005.
I was a lead software architect for large companies such as American Express, Motorola, GoToWebinar and many others.
Back in 2005, I was leading the largest ecommerce implementation for Motorola. The project was a dream for any software developer. A team of 150 developers, the coolest technologies, and the company’s revenue depending on it.
While everyone was excited about working on the project, I was apprehensive that not enough visitors would come to our new website and the website would not generate any revenue.
After an endless number of hours at the office, and a price tag close to 50 million dollars, we finally launched the website.
As it turns out, I was wrong. I shouldn’t have worried about getting visitors.
Too many people came. As a matter of a fact, so many people came to the website, they crashed our 16 servers within a couple of hours.
We managed to recover and bring the servers back up.
But, I was also right. We should have been worried about the sales. While hundreds of thousands of people came, most of them did not place a single order. There was no money. As a matter of a fact, the website generated less than 100 orders in the first month.
Yes, 100 orders for a 50-million-dollar investment.
Not the best ROI, by no means.
That was the beginning of Invesp.
We launched our company with the goal of helping online businesses increase their website conversion rates.
We started as a two-man operation and slowly grew to a large scale.
As we began to hire new CRO specialists to work on landing pages, conversion rate optimization projects, and testing plans, we wondered how we could ensure our clients still got the best results out there. It is one thing if you are doing the work yourself. But as the team grows, it is impossible to have the founders working on every project.
Yes, we made sure we hired the right staff.
Yes, we made sure our staff had the skill set required to succeed in conversion optimization.
Yes, we provided our staff with consistent training on a regular basis.
But we had to be very honest with ourselves. We needed a framework and a process to ensure we continued offering top-notch services. Because, coming from a consulting background, we knew that even when the consultant is performing at a hundred percent capacity, there are instances where he or she misses a detail or makes the wrong assumption. And when that happens, clients get results below expectations.
In 2008, we introduced the Conversion Framework across all of our projects. From the very early days of Invesp, we did a lot of experimentation, but the Framework was our first attempt to encompass all the different elements, that impact whether a visitor converts on a website, into one central explanation.
After creating the Conversion Framework, we needed a process to implement it. As with any framework, only with an efficient process in place we could take full advantage of the structure we created.
This groundbreaking process we developed for our Framework helps our clients produce significant, repeatable and sustainable increases in conversion rates.
The Conversion Optimization System
To successfully implement the Framework, we created a 12-step process:
- Conduct Heuristic Analysis
- Conduct Qualitative Analysis
- Conduct Quantitative Analysis
- Conduct Competitive analysis
- Create a Conversion Roadmap
- Determine Problem Areas on a Page
- Analyze the Page through the Conversion Framework
- Prioritize the Problems on the Page
- Create Hypothesis
- Create New Designs
- Conduct A/B Testing
- Conduct Post-Test Analysis
Read this article very carefully and try to learn every little detail in it. The process we are about to reveal is one which we tested in different types of websites:
- Ecommerce websites
- Lead generation websites
- SaaS websites
- Affiliate websites
We have also tested this process in many different verticals:
- Consumer services
- Legal services
- Real estate
Step 1 – Conduct Heuristic Analysis
You know the old saying….
No chef has ever cooked a bad meal – but many consumers have had bad meals.
Designers think they designed the perfect website for your business. But your visitors think otherwise. If that were not the case, most websites would be converting 20% or 30% of their traffic into customers.
We typically use one of 7 different methods to determine CRO/usability problems on a website.
How do you conduct heuristic analysis?
Your first step is to determine the 10 most common tasks a visitor takes on your website.
If you have an e-commerce, your visitor might:
- Search for a product on Google, click on a link to your website, land on your product page, click on add to cart, and place an order.
- Click on a social media link, land on your blog, use navigation to locate a product.
- Type in your website address, land on your homepage, click on “items for sale” category.
- Land on a product page, click on the category page, viewing other products.
- Look for shipping and delivery information.
If you have a SaaS website, the common tasks might be:
- Search for a product on Google, click on one of your paid ads, land on a landing page, and then subscribe to your offer.
- Land on the home page, use navigation to look at different subscription plans you offer.
- Click on an email link you sent, land on your pricing page, starts comparing different subscription plans.
- Watch product demos, look at your product wiki.
- Sign up for a free trial and use a couple of the features on the website.
As you identify different tasks a user takes on the website, you should consider that some of these tasks lend themselves to be done in desktop, others on mobile devices, and a final group of tasks users will do regardless of the device.
When conducting the analysis, we sit as a team – usually three to four people – and start going through the different paths to complete the tasks we listed above. We do this for both mobile and desktop.
As we go through the tasks, we start grouping items into two lists:
- Fix right away: items where the website functionality simply breaks. There is no need for any type of testing in this case.
- Testing ideas: items we should test, according to conversion principles.
Step 2 – Conduct Qualitative Research
The next step in the process is crucial. This is the moment your site’s visitors get to tell you what works and what doesn’t on your website. You can read our detailed guide on conducting qualitative research.
How do you do that?
You ask them.
Sounds too simple. But all you have to do is start asking your website visitors about the experience of navigating around the site. In order for you to do that, you use one of several mechanisms (dependent on your budget and available resources):
- Launch website poll and target your website visitors.
- Conduct email surveys for those who converted on your website.
- Conduct one-on-one interviews with potential customers and ask them to navigate around the site as you watch them.
- Hold focus groups where you interview a group of potential customers and ask about their experience with your website.
A lot goes into conducting qualitative analysis. The level of effort ranges from couple of hours to couple of months. At a minimum, you should be able to launch a website poll or conduct an email survey.
Asking the right questions is also vital when you are conducting qualitative analysis.
You should ask three different types of questions:
- Motivators: what brought visitors to your website
- FUDs: what is stopping website visitors from converting
- Hooks: what persuaded visitors who converted to do so
As you start collecting data from your web visitors, you will again add items to the two previously collected lists.
- Fix right away
- Testing ideas
Step 3 – Conduct Quantitative Research
At this point, our two lists, fix right away and testing ideas, are becoming larger, but we still need to add a few more items to them.
The next stop will be your analytics. Analytics provides a wealth of information on how visitors interact with the website and shows you where visitors are leaking. You can read more details on how to use analytics for conversion optimization here.
If you are looking to fix your website conversion rate, I can only assume that you have at least one analytics software installed on your website. Most larger companies have two or three different programs installed.
So, how do you track what areas of the website should be fixed?
While analytics programs are very powerful, we need to match our approach to all the power they provide.
Let me start by highlighting what not to do when using analytics.
- If your analysis focuses on bounce and exit rates, then you are not doing a good job conducting your analysis.
- If you are spending most of your time reporting data (what are visitors doing on my website) vs. analyzing data (why are visitors doing this on my website), you are wasting your time.
Here is what you need to do:
- Start by setting up several funnels on your website, tracking how visitors navigate from one area of the website to another.
- Segment the behavior of your website visitors across different sources/mediums in these funnels. Your analysis should answer questions such as:
- Which pages does Google organic traffic exit the website at?
- How many pages per visit does Facebook organic traffic visits?
- What is the average time on site for direct visitors?
- How many sessions does it take a Google paid visitor to convert?
You may come up with many more questions like these. As you answer them, you start adding more items to your two lists, fix right away and testing ideas.
Step 4: Conduct Competitive Analysis
Now that you have focused on your website, it is time to check your competitors’ sites.
The rule of thumb here? Do not ignore your competitors. However, do not copy them either.
You have two goals in this step:
- find the areas where you can improve your website by learning from other competitors.
- find areas where your competitors are falling short, which you can focus on to provide better value.
To learn from the strengths and weakness of your competitors:
- Look at your top three to seven
- Navigate through their websites as if you were going to buy from them.
- Look at the design, copy, layout, and visitor flow.
- Analyze their value proposition, headline, images, banners, colors, size, types of product.
- Answer three questions:
- What are their strengths?
- What are their weaknesses?
- What are the areas or features that they are not providing to visitors?
If you have the budget, hire a user testing service and ask users who are not familiar with you or your competitors to answer these questions.
This analysis should give you a list of 10-20 items to add to your testing ideas list. You can read more about conducting CRO competitive analysis in this article.
Step 5: Create Your Conversion Roadmap
It is now time to sit down with your two lists and evaluate them.
The fix right away list should be sent to your developers right away, so they can get started on solving any functionality issues of your site.
The testing ideas list should now be prioritized. Among the items you collected, your challenge now is to determine which pages and elements you will test first.
When creating a conversion optimization program, you need a map to follow, to make sure you address your central conversion issues first. By examining your analytics, heuristic analysis, and qualitative research:
- Determine which areas of your site are leaking visitors.
- Determine the health of different sections of your website.
- Learn how visitors navigate from one section of the website to the other.
Look at your number of conversions to find out if you should start by optimizing macro or micro conversions:
- If you have more than 300 conversions, start at the bottom of the funnel, optimizing pages of your site’s macro conversion goal.
- If you have less than 200 conversions a month, start at top of the funnel, optimizing pages for micro conversions.
Remember you map is a dynamic document to guide your testing process, rather than a set-in stone plan. It should cover three to four months of testing, not longer.
Step 6 – Determine Problem Areas on a Page
So, great, you’ve decided based on the data you collected that page X is the page you will be optimizing.
The good news is you’ve done a lot of the work to help you identify problems on the page:
- You’ve conducted a heuristic evaluation, so you should review the problems you discovered on this page to determine if they have merit.
- You’ve conducted polls and know some of the concerns or questions visitors have regarding:
- information they look for and need (is it clearly displayed and organized on the page?)
- confusion about certain elements on the page
- confusion regarding certain processes to help them proceed
This information is excellent and will help you point out some of the problems on the page.
You have two great tools as well that can help you validate these problems and display how the visitor is navigating the page:
- Heatmaps: to help see the clicks and mouse movement throughout the page
- Video recording: to help you visualize how visitors are interacting with the page
Although you may have viewed these tools during the creation of the conversion roadmap, it is worthwhile to go back to them and look at them specifically for the page.
Typically, a problem analysis should be conducted in a group format. It is not an individual activity.
Why, you ask?
Well, two sets of eyes and insight are always better than one. Hearing other perspectives on the page is very valuable when determining what problems you will be trying to solve. But coming into a meeting that will put together some creative minds requires some homework and preparation.
Step 7 – Analyze the Page Using the Conversion Framework
How do you determine the problems on the page?
We utilize the Conversion Framework to do a 360 analysis of the page to determine some of the issues we are seeing on the page. The framework evaluates every element on the page from the following areas:
- Trust and confidence: Are there trust elements missing from the page?
- FUDs (fears, uncertainty and doubts): Are there elements on the page that increase visitors’ FUDs?
- Incentives: Can you add incentives so visitors act right away?
- Engagement: Can you add engagement elements to the page?
- Personas: Does the page address different types of visitor persona?
- Sales complexity: Does the page provide all the information the visitor needs to convert?
- Buying stage: Is the page designed for both ready-to-convert visitors as well as early-in-the-funnel visitors?
The purpose of the framework is to help determine if:
- An element is missing.
- An element is not clearly visible or identified.
- If it is present but not effective based on data.
- It is present and doing its job.
For example, I may look at the framework element of trust. Under trust, I will look at continuity. The continuity principal states that elements on the page must maintain the same continuous design, copy and messaging. For example, if you are using an image in an email to drive visitors to a landing page, then you should use the same image on the landing page.
As I evaluate the page, my job is to determine if the page has elements of continuity.
If the page lacks continuity, then we identified a possible problem on the page.
If the page meets the criteria for continuity, I move on to the next element under trust.
Problem analysis is taxing. But the benefit of getting the team together to analyze a page, although it may seem tedious, has tremendous impact and benefit to our customer. The type of solutions and problems we come up with because of these sessions is far greater than any individual can on their own.
How many problems can you identify on a page?
I have sat through problem analysis where we identified 50 to 100 problems on the page.
Once you’ve identified the problems, you prioritize them based on what you know can have a greater impact.
Focusing on conversion optimization requires the right mix of art and science. Although much of CRO is embedded in science, a CRO expert needs to have that eye to determine problems and solutions based on the data selected.
Step 8 – Prioritize problems on a page
Remember that we are talking about conversion problems on the page. We are not talking about instances where the page functionality breaks down. Any breakdown in functionality must be fixed right away.
To determine what problems should be fixed first on the page, we use two factors:
- Potential conversion rate uplift
- Implementation cost
Potential conversion rate uplift is the expected conversion rate uplift by running an A/B test. This value will range anywhere from 3% to 30%. It has been our experience that 90% of your A/B tests will generate an uplift anywhere within that range.
Implementation cost is the estimated cost to implement a test. This of course will vary from one website to the next.
The trick to calculating the implementation cost is to determine the number of hours it will take to implement an A/B test on your website.
These are the three different types of tests you would want to run:
- Element level changes: In these types of tests you are changing an element on the page. The best example is trying different headlines, different images, different call to action buttons (CTAs). Element level changes do not require a lot of time to implement and most of them will take less than 8 hours to complete. We usually assign a value of -5 to these types of tests.
- Page level testing: In these types of tests you are changing the layout of the page. These tests are more complicated to implement compared to element level testing. We estimate these tests to run anywhere from 3 to 5 days to complete the implementation. We usually assign a value of -10 to these types of tests. Typically, you do the following:
- Adding new elements on the page
- Removing elements on the page
- Changing the behavior of some existing elements
- Changing location of some existing elements
- Visitor flow testing: in these types of tests, you are changing how visitors navigate around your website. Good examples of these tests include going from a single-step checkout to a multistep checkout (or vice versa), and changing website navigation. These tests are the most complex to implement. It might take a good developer anywhere from 8 to 12 days to implement. We usually assign a value of -15 to these types of tests.
To prioritize problems on the page, we typically use a 25-point framework to assign different priorities to different problem points. We will be releasing this framework in an upcoming article so make sure you stay tuned!
Step 9 – Create a Hypothesis
The hypothesis is a critical component to testing. Ultimately, you’ve done your homework and your research. You’ve selected the page and identified the problems on the page. Logically, the next step would be to come up with the solution.
Here is our template for A/B test hypothesis statement:
Because we saw (qual & quant data), we expect that (X change) will cause (Y impact). We’ll measure this using (data metric)
So, based on the goal you are trying to achieve on the page, you need to theorize what will improve the page and the site conversions accordingly.
Let’s take an example.
You identified that there are no clearly defined benefits on a landing page for a paid campaign that you are running.
Data from the qualitative poll you ran tells you that visitors complained about the lack of information on the landing page. You also look at heatmaps and notice that some visitors hovered near or at description to find more information about the offer.
So, your goal: to make the information easier to find and scan on the page.
And your hypothesis: By providing visitors with greater accessibility to easy-to-read benefits, you will better persuade them with the information they are lacking to proceed, thus increasing conversions.
Step 10 – Implement the Hypothesis Using New Designs
We have come a long way and we are almost ready to start our testing. You have identified the possible conversion problems on the page; you prioritized these problems and finally you created a hypothesis statement on how to fix these problems.
In this next step, you implement the hypothesis by creating new designs.
Creating a design that fulfills the hypothesis can be challenging sometimes. You must be careful that your new designs are driven by the hypothesis, otherwise you will introduce variables you won’t be able to account for in your results.
The challenge will always be the number of variables that have or don’t have anything to do with the hypothesis.
There are no hard-set rules here. Make sure that every design you produce is focused on implementing the hypothesis and solving for the problem you identified. Avoid the temptation to make more changes.
Step 11 – Conduct A/B Testing
We are finally here. Now, you are ready to conduct your A/B test (or multivariate test). You split the page visitors between the old design and the new designs. You let your visitors judge the quality of the designs.
Here are a few items that you want to keep in mind:
- There are several A/B testing tools & technologies At a very high level, they all do the same job. Choose one that speeds up the process of implementing the test on your website.
- Examine carefully how many conversions your website gets in a month. The number of conversions impacts how fast you can implement a test.
- If you are looking to implement a full website testing program where you will be testing several pages, multiple tests, for few months, then you will need to have a minimum of 200 conversions per month.
- If you are focused on optimizing specific landing pages, then you need to determine the number of conversions each landing page generates. If you have 200 conversions or more, then you can start doing A/B testing.
- If you do not have enough conversions (less than 200 conversions per month), you can still conduct A/B testing, however you will be conducting micro testing as opposed to macro testing.
You can always request a free A/B testing account on FigPii which provides you with an easy-to-use A/B testing tool, heatmaps, and video recording.
Step 12 – Conduct a Post-Test Analysis
You ran your A/B test, it came to a conclusion, and you have a challenger that beat the original design. What is next?
Most companies would move to the next test and you will do that. But you have to do a post-test analysis.
Here are the 3 questions you will have to answer:
- What are the main assumptions we made for website visitors prior to launching the test?
- Did the results from the test validate our initial assumptions?
- What additional information did we learn about our website visitors from running this test?
As you answer these questions, you will begin to know your visitors at a deeper level. You will make new assumptions about them. You will learn that they are impacted by certain messages, copy and design in a powerful way. What should you do next? You create new tests to validate your new assumptions.