Starting Your Conversion Optimization Project The Correct Way- updated Dec 2023

This article has been updated to reflect the more current process in Dec 2023

Additional updates are coming to include #JTBD framework!

Marketing projects can be complicated. Conversion optimization projects are even more intense. They require a very diverse set of skills, a lot of time and patience. It isn’t enough to just have an “eye” or know a few CRO “best practices” (whatever that is).

You see, when conversion rate optimization first gained momentum, a lot of sites lacked basic usability fundamentals. Finding “low hanging fruits” and making UX fixes, was a treat and usually ended with big revenue and conversion rate increases and wins. As companies gained further understanding about usability, websites followed suit. So merely relying on usability and heuristics to get big wins in CRO no longer works, because a lot of companies have gotten through the basics.

Is anything left?

Of course. If CRO did not require a creative approach to real problems on a site, then, well we would be out of business. The days of template tweaking are long gone.

That means you have to pull out all the stops on a CRO project. The investment early on is always quite large and should be done in a very organized way. Ultimately, during the first couple of months, it’s about uncovering. What most CROs struggle with is collecting all that they uncover and want to optimize in a single place for that full spectrum view. According to IMI Content Team:

“It’s important to remember that CRO isn’t a “once and done” task but rather an ongoing process. There are always areas of improvement for your website and overall business. Having a well-defined CRO process helps you identify and implement optimization efforts with the biggest impact on your website.“

It’s no easy feat, but in this article, I’ll try to shed some light on how to uncover website problems, and how to organize them in a way that will help you have an ideal conversion roadmap to success and consistent gains.

Why am I writing about this?

Gif Source

My company and I are on a mission set the record straight about CRO and what it really takes to reach success. CRO, for too long, has been seen as merely an A/B testing program, or best practices.  Whereas CRO is science that encompasses many digital marketing disciplines and techniques.

When CRO is done incorrectly, it can cause more harm than good.

We’ve said it once in for all: A/B testing is simply not for amateurs. It requires expertise because it is so precise.

Additionally, brute force A/B testing will NOT generate consistent and repeatable increases in conversion rates. At a basic level, you must ensure that usability and conversion issues are fixed. Beyond that, you must re-engineer your business processes to convert one time visitors into raving fans.

The Ship Methodology 

The methodology we’ve followed can be summed up in the acronym SHIP:

• Scrutinize
• Hypothesize
• Implement
• Propagate

Each stage holds importance and must be done thoroughly. However, the greatest emphasis in terms of time should be focused on two stages: the scrutinize and propagate stages.

Scrutinize stage includes all the conversion and usability research that you need to do in order to ready yourself for correct conversion optimization. Propagate stage is the intense post-test analysis which feeds back into the scrutinize phase (because after all, it is an iterative process).

So let’s evaluate what scrutinize entails more carefully.

1. Identifying Visitor Paths and Experiences

Why do visitors come to your website?

• Every visitor comes to your site to:
  • Answer a question
  • Find a solution (product/service) to a problem
  • Compare options
  • Convert
  • Complete a task
• What task are they trying to accomplish?
• What behavior you’re trying to help and facilitate?

Visitors come to your site not only to consume content, sign-up or buy something. Some may come to track an order, to see the status of something, to update an account. When we talk about paths, we talk about trying to identify what would bring different people to your site, and what actions do they take if they come for that purpose.

Not every website visitor is coming to convert. Some visitors landed on your website by mistake. Others are coming in different buying stages from early stage (browsing) to action stage (ready to convert). It has been our experience that most websites are designed for people in the action stage. Yet for most websites, only 20% of their visitors are in the action stage.

There are 7 questions to ask about the different visitors to your site (after you’ve made a list and consolidated the top 5-10).

      1. What is the visitor intent?

Here you want to gauge intent as much as humanly possible. Analytics can aid you if you created segments that are event based after observing certain patterns (like visitors that convert are likely to visit X amount of pages or visitors that are coming to cancel usually do X). The questions below can also give you some understanding of intent.

1. What is the traffic source? This helps you understand what they experienced before and what they are expecting to see.
2. What keywords/messaging brought them to the website? If you can log out Google search console to see what keywords drive organic visits, and into your Adwords account to see what keywords and combinations of keywords drive paid accounts, that will help you gain some further understanding of their intent upon coming and whether or not you’re providing them with the information they need to move forward.
3. What is the nature of the visit?
   1. Informational
   2. Transactional

• Navigational (brand related)

1. What is the bounce rate?
2. What is the conversion rate?

4. What internal search queries are the visitors using?

1. Number of searches per month
2. What was the visitor doing prior to the search?

      2. What is the context of the visit?

Context helps you get into the visitor state of mind. What were they likely did? It will likely be more about constructing sound assumptions based on what you know, but some things can be pulled via analytics like:

1. What is the device used for the visit?
   1. What time of day? That can help you determine if it was at work or at home.
   2. Zip code analysis can likely let you know if they’re employed or not as well (again giving further into to the context)
2. What is the user doing before clicking?
3. What is the user mental state?
4. Where is the user?

“Does the device, time of day or location call for a more tailored approach to being there?

“Are you only there at the bottom of the funnel, when people are in buying mode? Or are you there across the full range of consumer needs, wants, and curiosities? Are you also considering the various contexts of those needs?”

    3. Where does the experience start?

Based on your analysis, where did they land? Did they see an ad first?

    4. Where does the experience finish?

Where is it likely that this visitor with this experience exited? Does it end with a purchase or a sign-up? Or another action? What is the last page and experience they have on your site? Is it likely to get them to come back?

     5. What is your visitor feeling entering the experience?

Feelings are a bit more difficult to gauge. But you know if a visitor is coming to cancel their subscription, they’re likely in a hurry, maybe frustrated, and anxious to get it done seamlessly. You can think about what their experience entails and consider the context to reach some semblance of their state of mind.

      6. How brand-aware is the visitor?

Brand awareness is critical. The more familiar a visitor is with your brand, the less work needs to be considered to establish “trust and confidence.” The idea being: if there is positive brand awareness, trust and confidence is there. Otherwise, you need to fortify their experience with more trust elements. Here’s the breakdown:

1. Brand Rejection
2. Brand Non-recognition
   1. People do not know that you exist!
   2. Is your brand clearly differentiated from competitors?
3. Brand Recognition
   1. People lean toward your product/service when given the choice between your product and one they have never heard of
4. Brand Preference
   1. Given a choice between two brands – customers will choose yours over someone else’s.
5. Brand Loyalty/ Advocacy
   1. Customers will choose your brand over all the other brands ALL the time

      7. Where is the visitor in the buying funnel?

AIDA describes the different buying stages. Of course, depending on your specific vertical, you can describe the buying stages in different ways.

KaraThurkettle explains:

“According to a research study conducted by Google, customers form a judgement about a website after only 50ms, while some form an impression in as soon as 17ms. After this small amount of time, users either bounce or engage.“

Sometimes it can be as simple as:

• Early
• Mid
• Late
• Returning

What you’ll end up with is a simple somewhat deconstructed journey map of the different experiences that drive visitors to your site.

2. Persona Development

While we don’t push for persona development for clients as strongly as we used to, we try to ask clients to at least give us market segments to better understand who their customers are. When clients cast a wide net on who represents their target market – we get nervous.

As we progress in our CRO programs and gain new insights into customer behavior, we continuously update market segment with. If you are starting the project with a pixilated image of your website visitors, the image of these visitors should become clearer in 6 months and even more clear within a year.

According to Shivangi:

“Personas help bring in more clarity on crafting tailored content that appeals to the target audience of the business. As you understand the buying behavior of this user persona, you can run campaigns with content specifically focussed on converting these users.“

This is helpful in that it really allows us to visualize who we are targeting better.

Gif source

For instance, a client had a segment of 45 – 60-year-old middle-class women. The more we optimized the more we learned what behaviors and changes worked for them. We also uncovered that there are more segments they need to address beyond this particular one.

Psychology plays a big role in persuasive optimization. That’s why understanding visitor temperaments and what makes them tick can improve conversion tremendously. That can’t be done without understanding who the visitor is in the first place.

Understanding more about the language you should use, and how to ensure it is congruent is helpful to gain visitor trust and confidence.

The Limbic part of the brain is described as the emotional part of the brain. To gauge how users feel, you need to tap into this system. The limbic system was developed by a German research group, Gruppe Nymphenburg. They artfully broke down the human emotional system that influences consumer behavior into three main systems:

The stimulation system: this is the part of the consumer that aims to discover new skills. This part of the brain is more curious and is triggered by things that are exciting and novel.

The dominance system: which is more focused on competitiveness and status achievement. The desire for this system is to win.

Finally, the balance system which is focused on stability and aversion of risk. Triggers for this system may include fears and anxieties but are closely related with feelings of harmony and comfort.

People fall into all of these various categories, but when they shop some feelings dominate others. The way you would display language and elements, scarcity and incentives, and more would vary depending on what system meets the needs of your visitor.

This intense review of behaviors and motivations is critical to better persuasion.

3. Benchmarking

Conversion benchmarking is often overlooked part of the process, yet, quite important since everything you are doing relies heavily on improvement in site performance. It is like walking into a doctor’s office and having the staff take a record of your vital signs. According to Dave Chaffey:

“When benchmarking conversion rate, we think it’s important to explain to marketing managers that they should go beyond headline conversion rates to segment conversion by different types of visitor.”

You are doing the same task for the website in this case. You want to see how much progress you have made after few months of optimization. There are 3 different methods that we use to conduct benchmarking on a project:

1. A/A tests for different website funnels
2. Conduct usability tests
3. Creating and tracking different funnels in analytics

We have already identified the different paths visitors take in the first step of the process. What’s critical that you conduct both A/A and usability tests for these paths. There is some debate in the industry around conducting A/A tests. But that is not the point here. Yes, there will be instances where a particular variation (or challenger) would win in an A/A test with a very level of statistical confidence. We have done thousands of A/B tests and we have seen our share of data anomalies. Our goal from an A/A test is to get a general conversion rate metric for a particular funnel within the website.

Analytics is another tool that you should use for benchmarking. You must also understand the challenges that come with using an analytics tool within a CRO project and the drawbacks that it might have.

It is also important to keep in mind is seasonality, and trends throughout the year that accounts for certain ups and downs. Measure how the site is performing, and measure against that after you’ve optimized.

An important note about SHIP

Throughout the process of Scrutinize, the very first step in SHIP (which accounts for about 70% of the time spent on optimization), you should be collecting the issues or “problems” that you are seeing on your website.

We classify the problems into several categories:

FRA – fix right away (a problem that doesn’t need to be testing but is proving an obstacle to usability and design)

RO – research opportunity (basically a problem that is an idea for a test based on a problem that can be traced to data collected throughout the process)

IT – instrument (an area on the site that needs to be tagged and tracked)

IF – investigate further (seems like a problem but you just aren’t sure so you need to “investigate further” to find out whether it is a viable problem or not)

4. Expert review – “the crying game”

Gif source

You have already identified visitors’ paths around your website and what they try to accomplish. During this step, you will sit with your team and walk through the paths and experiences visitors go through. When doing this, you must keep three things in mind:

• You are mimicking visitor behavior around your website. Do not make any assumptions about your website. You must complete this task with a clean slate. Put yourself in your visitors’ shoes and forget about why you designed things in a particular way.
• Conduct this analysis coming from different sources Google, email, facebook, etc)
• Conduct this analysis using different devices (mobile, desktop, etc)

And then, cry….

As you go through different paths on the website, you will discover a sudden urge to cry because of the overwhelming number of UX issues. Never fear…UX is here.

The conversion framework

When conducting an expert review for a website, we run 6 different methods:

1. PURE method analysis
2. Conversion Framework analysis
   1. Trust
   2. FUDs
   3. Incentives
   4. Engagement
3. 10 Heuristics
   1. Visibility of system status
   2. Match between system and the real world
   3. User control and freedom
   4. Consistency and standards
   5. Error prevention
   6. Recognition rather than recall
   7. Flexibility and efficiency of use
   8. Aesthetic and minimalist design
   9. Help users recognize, diagnose, and recover from errors

• Help and documentation

4. Expert walkthrough
5. Navigational stress test
6. Cognitive walkthrough

If you are doing this correctly, you will discover that your website while it might have been designed well, it does not look great when you look at it through the lenses of your website visitors.

Identifying usability issues, however, is the first step. Fixing those will be great for conversions, but the objective of conversion optimization is WOWing visitors.

5. Qualitative Research

How can you not love qualitative research? It is one of the most effective methodologies to uncover real issues, problems on the site. It is a critical phase which requires a lot of detailed research and thinking.

Conversion-focused qualitative research is very grounded in focusing on uncovering “problems,” but that is not what CRO is all about.

Some CROs get stuck in the qualitative research phase and can’t move beyond it. The pitfall of qualitative research is that it focuses you on problems you have on the website and sometimes you forget about new additions or features that you should expand to.

Once you’ve cleared out most usability issues, you should be innovating for greater engagement and retention on the site.

Some of the activities that can take place during this phase:

• Surveying and Polling visitors
• Customer representative interviews
• Reviews of customer service calls
• Reviews of support tickets

Reviewing all this data is meant to uncover issues. What’s important during this phase is to go ahead and include that information within a single document or file so you can have a single view of all the issues you are collecting.

How do you avoid being in the rut of just uncovering problems?

• conducting two or three different expert reviews that are focused on different areas (like uncovering vs. areas to innovate) can be very useful.
• Also, the hypothesis and type of solution that you put forth for each issue you uncover can make a difference in terms of innovation.

6. Quantitative Research

The tools accessible in this phase are great. But we don’t recommend installing too many tools to track data. You can get deep insights with better analytics tools, but it’s simply not always necessary.

Analytics

Of course, Google Analytics takes the cake. It’s powerful, it’s free, and although not always accurate, It can provide valuable insights. When using google analytics on a conversion optimization project, you must ensure that it is configured it correctly. You must also remember that you are looking for general data trends as opposed to specific metrics and numbers.

With Google Analytics, you want to ensure that you have event tracking for key areas that visitors visit and click on. You want to set up funnels and segment to view how different visitors from different sources interact with a page and through a particular funnel.

What we find very powerful as well is pairing some of the insights taken from the qualitative research with the insights derived from Google analytics.

Heatmaps

A great tool that again shows how visitors interact with a web page or a class of pages (product or category pages). But they can either be pretty documents to stare out, or you can derive something meaningful from them.

At a basic level, heatmap analysis is straightforward. You simply look at a Heatmap to see what area of the page is most click and least clicked elements. You then determine if specific goals or elements are not being clicked and investigate why and if it’s necessary to maintain those elements on the page (or should they be rearranged). Finally, how deep they scroll and whether it’s because they aren’t engaged or everything they need to proceed is above the fold.

All of these types of insights can help you identify and uncover further problems or potential areas of improvement for the site.

Video Recording of User Sessions

Video recording tools capture hundreds or thousands of videos, and for anyone that can be one too many videos to watch. But there are interesting insights that can be uncovered, and possibly even identify different ways that visitors interact with the site.

For instance, one client of ours, we discovered through video (which was supported by heatmaps and analytics) that visitors use the shopping cart as a wishlist. So, we actually created a wishlist or “save for later” section within the cart so they don’t delete items all together and have the chance to come back to them after they’ve made an initial purchase. This lead to great success for our client and couldn’t have been identified as a solution if we hadn’t seen this particular issue.

7. Competitive Analysis

Know thy enemy! Competitive analysis for conversion optimization is only as useful as the information you collect. Whatever your marketing or CRO objectives may be, collect the information accordingly. If my goal is to improve my email marketing strategy, then I need to sign up for the email and experience the frequency, type of emails they send out, and the tone of voice in these emails.

If my goal is to improve CRO, it would be worthwhile to conduct a heuristic on your competitors’ websites to see what are they doing, and how are they funneling the site visitors differently than your own strategies.

The reason why we align competitive analysis with a goal is that it can open up a can of worms. You can drown in information and different areas of focus, which may lead you astray from what you are actually trying to achieve, and in the end, produce an unfocused document with too much information that causes analysis paralysis. Any activity you do should have a goal and structure behind it so you keep your eye on the ball at all times.

Usability Testing

Marisa Peacock explains:

“The goal of usability testing is to identify any usability problems, collect quantitative data on participants’ performance (e.g., time on task, error rates), as well as determine user satisfaction with the website. … Making it so they don’t leave, makes testing websites a necessary task.”

In the book Thinking Fast and Slow, the author Daniel Kahneman talks about the human mind and the idea of System 1 and System 2 of thinking. System 1 is where the majority of human thinking happens, which is the intuitive, subconscious choices we make on a daily basis. System 2 however, is about the slower, more analytical aspects. Most websites are created with system 1 in mind. They’re intuitive, all elements are placed where people typically expect them. However, if you want to get your visitors to become more analytical and selective in their thinking, you need to shock them with an element that pushes them to the system 2 way of thinking.

When you conduct usability testing, some aspects should be system 1. Clear and easy. However, if for example, you need visitors to use a widget, then you need to take them to system 2 by highlighting that element and drawing them to it in different ways.

Usability testing helps you evaluate different aspects of your site in terms of:

• • Learnability
  • Efficiency
  • Memorability
  • Errors
  • Satisfaction

The quality of software is often dependent on several factors, and so is your website and the widgets you have on your website.

Hypothesize

The hypothesize phase includes:

• Classify and prioritize
• Hypothesize
• Design

After you’ve conducted your research in the Scrutinize phase, you end up with several different issues

But the question is what next?

Categorize

You have of course 4 tracks you need to work on and consider:

FRA – just go ahead and list these as bugs for your developers to work through

IT – again work for the developer, ask them to add tagging and events as needed

RO – this phase still needs prioritization which will be covered in this section

IF – this is work for your marketing team to validate some of these problems through other means of research. Once validated, they either will fall into the FRA or RO sections

Prioritize

At this point, we recommend prioritizing all the research opportunities to see which problem should be tackled as a test first. We have a prioritization sheet that helps evaluate issues. Our prioritization differs from others because

• Separate problem identification from possible solutions
• Use 18 different factors to weight each line item
• Assign different priorities for different line items

Hypothesize

Image Source: Wikipedia

Testing is much like the scientific method. It’s important when conducting any test to construct a sound hypothesis and follow a highly scientific model for greater positive statistical outcomes. Otherwise, the validity threats of AB testing could be extremely grave.

According to Michael Aagaard:

“A solid test hypothesis goes a long way in keeping you on the right track and ensuring that you’re conducting valuable marketing experiments that generate lifts as well as learning.”

“A Hypothesis is a proposed explanation for a phenomenon.  The scientific method requires that one can test it.”

– Wikipedia –

Simple hypothesis model:

• Because we saw (data/feedback)
• We expect that (change) will cause (impact)
• We’ll measure this using (data metric)

It’s important to note that a research opportunity can have multiple test ideas. While a testing idea can have multiple hypotheses

For example RO: Qualitative data shows that customers don’t trust the brand

Testing idea: highlight testimonials on the page to enhance visitor trust

Hypotheses

• Testimonial location
• Testimonial design
• Details within testimonial
• Images with testimonial

Wireframing and Design  

Coming up with the solution and wireframing for them should be like most conversion rate optimization activities, a group project. The team should gather and sketch their solutions to a given hypothesis for a greater outcome.

Implement and Propagate

Implement

Now that the hypothesis and solution designs have been decided upon, it’s time to conduct a test. There are some key testing statistics and threats that need to be considered before beginning this process.

It’s also key to remember to do a thorough QA of the test before launching it.

And finally, never leave a test without monitoring it daily. You never know what could happen and you need to be watching closely so as not to lose time and effort.

Propagate

Once a test concludes, we analyze the experiment by asking 3 What’s:

• What were results
• What did we learn
• What are our next steps

When analyzing results, we segment the data to compare the different sources of traffic and types of visitors before fully concluding results.

Finally, regarding the “what our next steps” question:

Iterate: take our learnings and apply elsewhere on the site

Pivot (invalid hypothesis): review the test solution, hypothesis, and problem to see what went wrong

Investigate further: the test left us with questions that require qual or quant research.

Another Research Opportunity (new test): the test introduced other possible test ideas

Design and innovative thinking

Conversion optimization and this process in particular, if adopted correctly, can truly take the organization to the next level. Rather than have separate teams, create teams that are designed for growth and innovation. Create an environment where change is embraced and encouraged. Design an organization where everyone is always testing and innovation is happening across every department.

SHIP: Scrutinize, hypothesize, implement, propagate. It’s not easy, and it shouldn’t be easy. But if you want to get a greater, more optimized experience, you need to follow this methodology.