Creating A Conversion Roadmap: How to Prioritize Conversion Problems on Your Website

Most teams waste time testing low-impact ideas. This playbook helps you focus your CRO roadmap on what actually lifts conversions.

The Four Phases of a CRO Program (The SHIP model)

A successful conversion optimization project follows the same rhythm:

• Scrutinize stage: You conduct various activities to identify potential testing areas within a platform or campaign.
• Hypothesis: Create a hypothesis for each testing idea identified during the “scrutinize” phase.
• Implement: Validate your hypothesis through an A/B or usability test.
• Propagate: You look for insights, share them within the organization and across marketing channels and verticals.

Think of it as a continuous loop rather than a one-time project. Each phase feeds into the next, creating a compounding effect on your site’s performance and learning.

1. Scrutinize — Gather Data and Identify Issues

Your goal here is to uncover why visitors aren’t getting just where they drop off. Most marketers jump straight to testing without understanding the root cause. The Scrutinize phase fixes that by combining data, observation, and empathy to find what’s holding users back.

Think of your website as a sales conversation. When conversions stall, something in that conversation is breaking down. The Scrutinize phase is where you identify which part.

Here’s how to do it step-by-step:

Step 1. Start with quantitative data (the “what”).

Use our analytics to spot friction points.

• In GA4, check where users drop off in your funnel: product pages with high exits, CTAs with low click-throughs, or forms with high abandonment rates.
• Use server-side or event-based tracking to fill gaps left by cookie loss — especially for checkout or login flows.
• Add scroll-depth and click tracking to see which sections lose attention.

Bugs, usability issues, and conversion blockers

I will take a quick detour to discuss the various items we uncover during a website conversion rate audit.

1. Website bugs: areas where the website is broken and not functioning as expected. You should classify the bugs you identified by priority (P1: blockers, P4: small items). Bugs are typically put into the “fix it right away” bucket.
2. Website usability issues: issues where website usability is not ideal. Usability issues are typically classified as either “fix it right away” or “research opportunities” buckets.
3. Conversion blockers: issues that relate to persuading visitors to convert. Conversion issues are typically classified in the “research opportunities” bucket.

 

Many conversion consultants focus their effort on identifying bugs and usability issues on a website and rarely pay close attention to conversion issues. Without getting into a philosophical debate, I firmly believe there is a vast difference between creating a user-friendly website and a high-converting site.

Usability issues:

1. Fixing usability issues on a website makes it user-friendly.
2. Usability issues focus on making the site easier to use
3. Usability issues focus on top-of-mind issues for website users

Conversion issues:

1. Fixing conversion issues on a website creates a user-friendly website (Similar to fixing usability issues)
2. Conversion issues focus on making the site more persuasive
3. Conversion issues focus on psychological items that persuade visitors to convert

Every usability issue is a conversion issue, but not the other way around.

Beyond problems – adding new website features

CROs focus heavily on issues that prevent visitors from converting on a website. However, your research during the scrutiny phase should go beyond that to include potential new features you can implement on the platform to enhance conversions further.

Understanding your initial AB test plans: the research opportunities list

It is not enough to identify a list of research opportunities for a website. You will need to go a step further by determining your initial test hypothesis for each item on the list. The initial test hypothesis identifies how we think the particular issue could be tested/fixed.

A hypothesis is a predictive statement about a possible change on the page and its impact on your conversion rate.

Now, notice how I used the term “initial hypothesis.”

An initial hypothesis is the first stab our team takes on how we should address a potential problem on the website/webpage.

Example of an initial hypothesis:

“Adding social proof will enhance visitor trust in the website and increase conversions”

We use the initial hypothesis to assess each line item on our list of research opportunities. When we decide to implement a particular test, we further develop our initial hypothesis into a

“concrete hypothesis.” Let’s look at the initial hypothesis above and see the concrete hypothesis for it:

Based on qualitative data collected through online polling, we observed that website users do not trust the brand and are unaware of how many users use it. Adding social proof on the homepage will increase visitors’ trust and improve conversion rates by 10%.

A concrete hypothesis

• states how we identified the issue
• indicates the problem identified on the page
• states the potential impact of making the fix (we set an actual numerical goal for the test)

Now that you have your research opportunities list and each item has an initial hypothesis attached, we are ready to prioritize the items on the list.

Prioritizing your AB testing ideas (the research opportunities list)

CRO practitioners have used several frameworks to classify testing ideas. I will cover some common frameworks and suggest using a new model.

Karl E. Wiegers explains:

 “When setting priorities, you need to balance the business benefit that each function provides against its cost and any implications it has for the product’s architectural foundation future evolution.”

Pie by Widerfunnel

The pie framework evaluates items based on three different factors: potential, importance, and effort.

1. Potential: Not all of your testing ideas will have the same potential to increase conversion rates. A good example of this is testing different CTA colors on a page (with little to no real impact) compared to changing the page layout to emphasize the value proposition.

While the potential is significant, it is very subjective. What I might think will have a significant impact on the bottom line, someone else might think it will have no effect.

2. Importance: How important is the page you are optimizing?

The Widerfunnel team defines Importance as follows:

“How valuable is the traffic to the pages? Your most important pages are the ones with the highest volume and the costliest traffic. You may have identified pages that perform terribly, but if they don’t have significant volume of expensive traffic, they aren’t aren’tg priorities.”

According to Chris Goward:

“How do I figure out where to test first?”It’s a smart question to ask. You can’t test all pages at once. With limited time and resources to commit and, most importantly, limited traffic to allocate to each test, test prioritization is an important part of your conversion optimization plan.”

3. Ease: How easy is it to implement the test? Widerfunnel defines ease as:

There are two elements to consider when it comes to ease:

• Technical implementation
• Organizational or political barriers

Both of these elements impact how easy it is to implement a testing idea; however, they are somewhat subjective.

Hotwire framework

Pauline Marol, the Lead Product Manager for optimization at Hotwire, introduced this model back in 2015.

Here are the elements in this framework:

• Main Metric
• Location
• Fold
• Targeting
• New Information
• Benchmarking
• Conversion Veins
• Strategic Topic
• Mobile
• Opaque

What I like most about this model is the specificity it provides for the potential problem and potential fix. It also brings into focus how strategic a test is to the overall business objective. According to Pauline Marol: 

“Prioritization is a critical part when building an efficient, impactful testing program. Creating a framework to make smarter choices and thinking deeply about the key factors will make your program successful in the long term.”

PXL framework from CXL

The PXL framework uses the following criteria to score items on your research opportunities list:

• Is the change above the fold? Changes above the fold are noticed by more people, increasing the likelihood that the test will have an impact.
• Is the change noticeable in under 5 seconds? How fast does a user notice the change?
• Does it add or remove anything? The idea is that “removing distractions” or “adding key information: will have more impact.
• Does the test run on high-traffic pages? Changes to “high traffic page(s)” will have a larger impact on the bottom line.
• Designed to increase user motivation: does the introduction of your solution increase visitor motivation? Highly motivated visitors are more likely to convert.
• How was the problem identified?
  • User testing
  • Quantitative data
  • Qualitative data (polls, surveys, etc.).
  • Mouse tracking, heat maps, or eye tracking
• Ease of implementation

The PXL framework is a step in the right direction. Peep listed three main advantages to using it:

1. It makes any “potential” or “impact ” rating more objective
2. It helps to foster a data-informed culture
3. It makes the “ease of implementation” rating more objective

The central theme is the effort to establish a more accurate model for scoring different line items on your list.

I believe that there is room for improvement for the following reasons:

1. All prioritization frameworks use a mix of points to score the identified issue and its potential fix. But a single problem could have multiple possible solutions. And each potential solution should be assessed with a different score.
2. A potential problem could be identified in various ways, which give it a higher score. But does identifying a problem in 4 ways make that problem a more significant issue compared to a problem identified in 3 different ways? At some point, adding more ways to how a problem is detected should not give the problem more weight.
3. There is not enough focus on how a potential solution could impact visitors’ view of the website. PXL does an excellent job of motivating, but there are other criteria we should consider when evaluating a possible solution.
4. The models do not distinguish well enough between the different types of pages (top, middle, and bottom of the funnel).
5. Finally, the models use “high traffic pages” to score a problem. Since analytics provides a specific metric for identifying a high-traffic page, we should use it.

Invesp Prioritization framework

Let me state this again: a particular problem can have multiple initial hypotheses for how to fix it. Some of our team members like to call these testing ideas. This is important because each problem and each initial hypothesis is listed as a separate line item in our prioritization framework.

You can address a single research opportunity in different ways, each requiring different time to implement and having a different impact on the bottom line.

Next are the criteria we use to evaluate each research opportunity:

1. What is the potential Impact?
2. Is the problem above the fold?
3. Was the problem discovered through expert review?
4. Was the problem identified through qualitative research?
5. Was the problem identified through user testing?
6. Was the problem identified through analytics?
7. Was the problem identified through heatmaps/Video recording?
8. Known top competitor feature?
9. Does the test idea enhance primary conversion goal?
10. Is implementing the change easy (politically, LOE)?
11. Adding or removing an element?
12. Does the test idea increase visitor trust?
13. Does the test idea reduce FUDs?
14. Does the test idea incentivize visitors to act right away?
15. Does the test idea increase engagement?
16. What type of change?
17. What type of page?
18. The percentage of page views

Not all of these items carry the same weight.

Let’s discuss some of these elements:

1. Potential Impact: We do think that this is a subjective measure. However, we did not delete it. The value of the impact is high (score of 4), mid-impact (score of 2), and low impact (score of 1).
2. Is the problem above the fold? Where did you uncover the problem? The higher the problem is on a page, the more likely visitors will notice it.
3. How did you discover the problem? Most CROs will conduct a mix of activities to uncover issues on a site. Each of these activities should be called out. These are
4. Expert review (some might refer to it as heuristic analysis)
5. Analytics data
6. Heatmap/session
7. Qualitative data
8. User testing

Most conversion blockers are uncovered through a combination of the above elements. So, we decided to take a weighted average as follows:

If an item is discovered through one method                     score of 3

If an item is identified through two methods                      score of 6

If an item is identified through three methods                    score of 9

If an item is identified through four or five methods        score of 12

4. Known competitive feature: there are instances where you want to test an item because you observed that one of your competitors have implemented. By no means should you understand this as a way to copy your competitors, but you should not ignore them either.
5. Does the test idea enhance the page’s primary conversion goal? Is your testing idea going to increase that conversion goal?
6. Is implementing the change easy (politically, LOE)?
7. Adding or removing an element: Your test can do several things at the same time:

Add or remove an element: scored at 3

Replace an element: scored at 2

Change the location of an element: scored at 1

Emphasize an element: scored at 1

8. Evaluation of visitor-centric factors that would impact how a visitor perceives a page:

• Does the test idea increase visitor trust?
• Does the test idea reduce FUDs?
• Does the test idea incentivize visitors to act right away?
• Does the test idea increase engagement?

Two or more elements give you a score of 6. One gives you a score of 3.

9. Type of change: This focuses on the testing idea at hand. Are you:

• Focused on an element level (changing a single element), scored at 1
• Focused on page level (changing layout or several elements on the page), scored at 2
• Focused on visitor flow around the website, scored at 3

10. Type of page: How is the page classified in your funnel?

• Top of funnel page scored at 1
• Mid-funnel page scored at 2
• Bottom of the funnel page scored at 5
• Global Change scored at 5

11. % of page views: By looking at the % of unique page views the page gets compared to the overall website page views, you can select one of the following values:

Is there a perfect prioritization framework?

I do not think there is a perfect framework out there. We continue to tweak this framework as we work on different projects. On a recent project, our team identified close to 200 research opportunities. We ran it through a previous version of the framework and looked at the results. Some of the top items were confusing to us. It did not make sense for us to start with them. We went back and reworked some of the formulas. I am sure that someone can take the same framework and tweak it further. According to Chris Goward:

“There are no standard rules for which pages are best to prioritize. Your website lives in a unique target market, including factors like your competition, seasonality, and internal cultural environment; all of these affects how your site is used and should be optimized. The priority rating you give each of your potential test pages will depend on this unique business environment.

It is a step in the right direction. We built on existing frameworks and added elements that impact prioritization for our different projects.

You can download the Excel file for the prioritization framework from here.

As always, if you have suggestions, please do let us know

Frequently Asked Questions

What is Conversion Rate Optimization (CRO)?

• Answer: Conversion Rate Optimization, or CRO, is the practice of increasing the percentage of website visitors who take a desired action, such as making a purchase or filling out a form. It involves analyzing user behavior and implementing changes to improve the user experience and boost conversions.

What is a Conversion Funnel?

• Answer: A conversion funnel is a series of steps that a website visitor goes through before completing a desired action. It usually starts with attracting visitors to the site and ends with them taking action, such as making a purchase.

What is A/B Testing?

• Answer: A/B testing is a method where you compare two versions of a webpage or app against each other to determine which one performs better in terms of conversions.

How Do I Prioritize My A/B Testing Ideas?

• Answer: Prioritizing A/B testing ideas involves evaluating the potential impact, ease of implementation, and relevance of each test. Frameworks like PIE (Potential, Importance, Ease) or the Invesp Prioritization Framework can help in this process.

What are FUDs in CRO?

• Answer: FUDs stand for Fears, Uncertainties, and Doubts. These are psychological barriers that can prevent a visitor from converting. Identifying and addressing FUDs can significantly improve your conversion rate.

What Does ‘Above the Fold’ Mean

• Answer: ‘Above the fold’ refers to the portion of a webpage that is visible without scrolling. Elements placed here are more likely to be seen by visitors and therefore have a higher impact on conversions.

How Long Does It Take to See Results from CRO?

• Answer: The time it takes to see results from CRO can vary widely depending on the changes made and the metrics you’re tracking. However, a good rule of thumb is to allow at least a few weeks for meaningful patterns to emerge.