23+ Questions To Ask Before Hiring a CRO Agency

Hiring the right conversion optimization agency can transform your business.

Hiring the wrong one can drain months of time and tens of thousands of dollars, with nothing to show for it.

After nearly two decades in the conversion optimization space, I’ve seen both stories play out. About 40% of my time is spent talking to companies evaluating CRO partners — some ask the right questions, most don’t.

The truth is, conversion optimization isn’t just about running A/B tests. It’s about understanding human behavior, building a culture of experimentation, and aligning insights with business growth. The agency you choose will directly shape how those principles are applied to your website.

This updated 2025 guide breaks down the 23+ most important questions to ask before hiring a CRO agency along with what a good answer (and a bad one) sounds like.

Whether you’re new to CRO or looking for a better long-term partner, these questions will help you separate firms that talk about optimization from those that actually deliver it.

Types of CRO Partners in 2025

Before determining the questions to ask before hiring a CRO agency, you need to have a better idea of the options you have, i.e., the type of CRO partners you can choose from. 

When conversion rate optimization first became mainstream, your options were limited. You had a handful of specialists, a few tool vendors, and maybe a freelancer or two.

However, by 2025, the CRO landscape will look very different. AI-driven analysis, privacy regulations, and deeper integration between analytics, UX, and experimentation have expanded the scope of what a “CRO partner” can look like.

Below are the five main types of CRO partners you’ll encounter today:

• Independent consultants: Solo specialists offering strategy, audits, or light testing support.

• Full-service marketing agencies offering CRO: Traditional agencies that integrate CRO alongside SEO, PPC, and analytics.

• Testing tool vendors with service arms: Platforms that provide add-on professional services for setup and test execution.

• Specialized CRO firms (for example, Invesp): Agencies dedicated exclusively to conversion optimization and experimentation.

• AI-augmented consultancies (emerging): New-age firms combining human CRO expertise with AI-driven data analysis and hypothesis generation.

Pros and Cons of Each CRO Partner Type

Cost of conversion optimization companies

I wrote a detailed post about the cost of hiring conversion optimization agencies. Keep in mind that a good consultant should increase your annual online revenue from 30% to 50%.

CRO consultants will charge anywhere from $40 per hour to $250/hour. A specialized conversion optimization company will most likely charge a monthly retainer of $2,000 to $30,000 per month. Many of them will also require a multi-month commitment.

25 questions to ask before hiring a CRO agency

These are the specific questions you should ask before hiring a CRO firm:

1. How do you determine what pages you will optimize?

Choosing which page to optimize first and creating a conversion roadmap will impact your project. Remember that optimizing any page requires analysis, development of a test plan, implementation of that test plan, and finally waiting for testing results.

This could translate into 2 to 4 weeks’ worth of work per page.

If the CRO firm chooses the wrong page, time and money are wasted.

What you need to focus on is understanding the agency’s process to determine what to work on and how to prioritize different conversion-related work.

There should be no shortcuts here. Without following a process or providing a generic answer, such as using best practices, the firm is essentially making random guesses.

If they are randomly guessing from the start, avoid them.

A top-tier conversion agency will use a mix of the following activities to come up with a conversion roadmap for the site:

1. Expert reviews & heuristic analysis
2. Jobs to be done interviews (JTBD)
3. User research (qualitative analysis, including on-site polls)
4. Data analysis ((quantitative analysis)
5. Usability testing
6. Competitive analysis

2. How do you determine what changes you will make on a page?

This question helps you understand how the conversion company determines what changes they should make on the page. Again, you are digging deep to understand the process the conversion firm uses.

You want to validate:

1. The process they use to determine which elements will be changed and tested
2. How will that process be applied to your particular website

Best practices are great in theory, but difficult in practice.

How does that impact you?

Relying on good practices will not take you anywhere. We have seen this firsthand with over 3,000 A/B tests. What works for one website may or may not work for your website. Different conversion optimization designs and processes are effective for websites within the same industry.

You can find several renowned methodologies in the CRO space. For example, Invesp’s Conversion Framework and Widefunnel’s LIFT model are well-known conversion models in the industry.

It is beneficial when a company claims to follow a specific process, but you should always request a brief demonstration of how that process would be applied to your website. The CRO company should select a page on your site and utilize its conversion process to identify weak areas on that page.

3. What type of user research (qualitative research) will you conduct on the project?

User research explains the “why” question on your website and informs your assumptions about your visitors. Multivariate and A/B tests validate these assumptions.

Good conversion optimization requires the firm to conduct some level of user research to understand its customers.

CRO qualitative research is a vast and complex field, including:

• Jobs to be done interviews
• Online surveys
• Focus groups
• One-on-One interviews
• Website exit surveys

You should avoid companies that ignore doing qualitative research, regardless of the excuse they might give.

Optimizing a website for conversion without conducting qualitative research is flying blind.

4. What type of data (quantitative) research will you conduct on the project?

Quantitative research answers the “what” question on your website:

1. What are visitors doing on your website?
2. What pages are performing well, and what pages are underperforming?
3. What areas of your website are leaking visitors?

Analytics software provides a wealth of information for quantitative research.

You should assess the firm’s ability to provide deep analysis and insights beyond exit and bounce rates. Assess the CRO firm processes towards:

• Data segmentation
• Funnel and goal creation
• Heatmap analysis

5. What is the most challenging part of doing a conversion optimization project?

Increasing a website conversion rate is a complex problem. Otherwise, most websites will convert at 20 or 30%. Solving the conversion problems for a website requires a careful analysis that generates focused solutions with a higher probability of success.

You should ask this particular question to the team members who will work on your project. Listen carefully to their answer.

Do they find the analytics assessment challenging, or is it a qualitative research approach? Perhaps it is identifying problems on a page or creating a successful A/B test.

Your follow-up question is, “What are you doing to mitigate these challenges on my particular project?”

6. What was the worst conversion optimization project you worked on?

Most companies think of conversion optimization as a one-time effort, a campaign they conduct once, and they are done with it. Needless to say, that mentality is counterproductive.

As a result, a company that has been doing conversion optimization long enough must have run into a bad project now and then. If the company claims they never did, they are lying or have not been doing CRO long enough.

This is not unique to conversion optimization. It is the nature of doing business. Large and well-established consulting firms in various industries often have unsuccessful projects.

Assess the answer of the company about their worst project by analyzing:

1. The reasons the firm gives for the bad project. Are they blaming the client? Even with lousy clients, who do exist, a consulting firm should have a process in place to manage expectations and implementation.
2. How the firm handles bad projects.

7. How many experiments do you launch per month?

It goes without saying that the more tests you can launch on a website, the higher the chances are of increasing your website’s conversion rate. This, however, should NEVER come at the expense of following all the required and necessary steps to create a fruitful A/B test.

There is a delicate balance between launching a limited number of tests and an excessive number of tests.

You must understand how often the CRO company plans on launching tests on your website. You will also have to allocate your resources to conduct a test. If the company is running just a few tests, your development team would not be involved. These few tests are typically small and have a limited impact on your bottom line.

Different companies take different approaches to the number of tests they launch:

• Some CRO companies conduct one test every four to six weeks
• Some CRO companies conduct two tests per month
• Some CRO firms conduct three or more tests per month

Conducting one test every four to six weeks is too slow. Following this approach, you will perform between 8 and 12 tests per year. In our experience, that will generate a lower ROI.

Conducting more than three tests per month means that tests are not carefully analyzed, nor are results studied to determine marketing insights. This approach of throwing things at the wall rarely works.

Conducting two tests per month is the ideal target, where careful analysis is applied to test creation and results. In this approach, you will launch 20-24 tests per year. An expert CRO team should easily generate a 35 to 65% uplift in conversions in this scenario.

8.  Do you conduct concurrent split tests?

Our recommendation to conduct multiple tests per month comes with the caveat that:

1. The tests are not run concurrently, or
2. If tests are run concurrently, each test runs in a separate swim lane.

If tests run in separate swim lanes, traffic from one test intersects traffic from the other test. This rarely happens on large, complex websites where visitors jump from one side to the next.

9. How do you handle mobile website optimization?

Mobile traffic accounts for 35 to 50% 70% of online visitors to most websites. Examining your website analytics will help you understand:

1. What percentage of your website visitors are coming from mobile devices
2. What is the conversion rate for mobile visitors compared to desktop visitors

Mobile website optimization requires a different set of processes compared to desktop optimization. Visitors on mobile devices typically progress through two stages: the research phase and the action phase. Presenting mobile visitors with the right information will have a significant impact on your bottom line.

When evaluating different conversion rate optimization companies, ask them for examples of their mobile device optimization. Also, understand their mobile optimization process.

Stating that mobile device optimization is similar to desktop optimization is a red flag!

10. Who will handle implementing the test on our website?

Understanding who will handle implementing the A/B tests ensures you will allocate the required resources to guarantee the project’s success. It will also provide you with insights into the types of tests the firm plans to introduce during the project.

AB tests require development time. The amount of time depends on the complexity of the AB testing that you will run on the site. Most AB tests are implemented using front-end developers. These tests use JavaScript to manipulate how a page appears on the site. Some CRO agencies ask you to handle the development of the AB tests yourself. Some clients prefer this approach since it helps them control the project’s cost.

This, however, is not a good idea.

I have seen it fail one too many times for two reasons:

1. Your development team gets busy.
2. AB test development is different from regular development. It requires an entirely different skill set.

Look for an agency that will handle the AB test development for you. That will save you a lot of time/effort/money.

11.  What type of resource commitment do you require from my team?

Most conversion optimization services will require some level of commitment from you. You should meet with the company regularly to see their progress, provide feedback, and analyze results.

When you have a winner for an AB test, then your development team should implement that winner on your site. Please also account for some development time on your side.

12. What additional fees should I expect to pay in addition to the consulting fees?

Expectations must be set correctly. That will ensure the success of any conversion optimization project.

The conversion consulting agency will require you to install and use conversion optimization software to collect visitor data and launch the tests.

Some tracking and A/B testing software can be expensive.

This will largely depend on your website and the number of visitors it gets. If your website gets more than 100,000 visitors per month, the cost for the testing software can run you up to $500 per month. Even more, some software will require an annual contract commitment.

A recent company seeking conversion rate optimization services had 500,000 monthly visitors to their website. They were not allocating any money for the testing software. They had been negotiating with several optimization firms. No one bothered to explain to them that the cost of the testing software would run them close to $20,000 per year.

13. What type of education do you require from your conversion optimization staff?

Conversion optimization is an ever-changing field. If online marketing is on the cutting edge of marketing, conversion rate optimization is on the cutting edge of online marketing.

Effective conversion optimization companies should invest in and require their staff to be educated. At a minimum, a conversion rate optimization consultant should stay up-to-date with the latest in the conversion optimization field and analytics.

In addition, since conversion optimization impacts different aspects of a website, an expert conversion optimization consultant should have a well-rounded knowledge of SEO, PPC, and social media.

The education a firm requires from its staff is a great insight into how much it wants to see its clients succeed.

14. Can you guarantee an increase in conversion rate?

This is an easy one. If anyone guarantees results for conversion optimization services, run away from them as fast as you can.

Many factors impact the result of conversion work. Among these factors, several are not within the control of the conversion rate optimization firm.

Take, for example, the quality of traffic coming to your website. If the site receives a substantial number of visitors organically from search engines, a change in the quality of that traffic (such as an increase or decrease in rankings) will directly impact your conversion rate.

The conversion company also does not control some market factors. A major ad campaign of one of your direct competitors will impact your conversion rate.

In our experience, companies that guarantee conversion optimization results fall in one of two.

• Companies that are trying to make quick money and will not be able to deliver on their promises
• Companies that guarantee an uplift define the uplift as for any success on any test they launch. This includes conducting a test that generates a 0.00001% increase in conversions!

Both practices are somewhat deceptive.

15. How do you determine the success of the project?

Website owners conduct conversion rate optimization to increase their conversion rate and revenue. But as we clarified in the previous question, no reputable CRO company can guarantee the results of their work.

If you set expectations correctly from the start of the project, you will save yourself many headaches and heartburn.

When we talk to clients, we define the success of a project according to three factors:

1. The percentage of successful tests: In our experience, up to 70% of the tests we conduct generate a conversion uplift.
2. The detailed and well-thought-out process we demonstrate first-hand to the client in every interaction with our team.
3. The learning process that the client team receives from watching and interacting with our team on the project.

16. Why should we hire you vs. another?

Why is one firm different than another one?

Differences could include process, experience, deliverables, and cost.

All of these are valid differences that will have a direct impact on the success of your project. Ultimately, your goals are to:

• Assess how their firm distinguishes itself from competitors
• Validate the claim of the company on its differentiators

17. What is the makeup of the team working on my project?

CRO agencies employ various approaches to structuring their teams. At a minimum, your project team should include the following people:

• CRO specialist
• UX designer
• Front-end developer
• Quality assurance analyst

Smaller agencies might not have a project manager staffed. However, if you are going to run multiple experiments simultaneously, you will need someone to make sure the work is running smoothly.

18. How many projects does the team working with me handle

I’m often asked about the number of clients we handle at any given time. We limit the number of our projects to 15 at the same time. But that is our approach. We employ a high-touch approach to managing CRO projects. A better question is how many projects does a team handle at the same time?

Top-tier CRO agencies work in pods. A pod typically handles 3-4 projects at the most. That allows the CRO pod to focus on the clients and deliver the best results for them.

19. How often do we meet? Do you have weekly or monthly standing meetings?

There are two sides to this.

• For you as a client, meetings are important. You get to know what the team is working on. You provide feedback. You learn. I can go on about the benefits of meetings.
• For the agency, meetings eat up time that could be used in doing work (for you or other clients).

While I appreciate that agencies try to manage the time spent on a project, I believe that successful CRO projects require regular meetings. My recommendation is to expect at least a meeting every two weeks, preferably once a week.

20. What methods of communication will we use on the project?

In addition to the standing meetings, how else will you communicate with the CRO agency? Avoid email at all costs. It isn’t easy to track. Instantaneous chat programs are okay, but they are not a good idea if you run a high-velocity testing program. A good project management tool (there are hundreds of them) provides a central repository of all the work on the project.

22. How fast do you launch the first AB test on my site?

While conversion research is important, some companies spend six to eight weeks conducting research. Their first AB test is not launched until the end of the second month. I don’t think you need to wait that long to launch your first A/B test. Our goal is to launch the first A/B test within 2 weeks of the project start (at the latest, 3 weeks).

23. How much time do you spend on my project?

The agency’s goal is to spend as little time as possible on your project while generating the most wins for you. It is a win-win strategy.

At least, in theory.

CRO projects require a lot of time. There are too many resources involved.

Agencies that try to cut corners end up launching simple tests (test the color of a button or the text on a CTA). While some of these experiments yield results, it is not a winning long-term strategy.

Trying to measure the activity of a CRO agency by looking at the number of experiments they produce in a month is also not a good idea. You can create 10 experiments in a month in less than 10 hours.

I would like to understand how many hours the team I will be working with will dedicate to my project.

What if the agency doesn’t tell me how much time they spend on my project?

You can do simple math and figure this out.

Ask two questions:

How many people work in the agency?

How many clients do they handle at the same time?

Let’s say that an agency has 25 resources. Most likely, 20 of them will be working on projects. Let’s also assume that the agency handles 50 clients simultaneously.

Therefore, the time spent on these 20 resources is dedicated to working on 50 projects.

Let’s say a full-time resource spends 120 hours a month doing actual CRO work. That means that the company pays 20 resources * 120 hours = 2,400 hours working on client projects.

How much time do they spend working with each client?

2400 hours/ 50 clients =48 hours at best.

24. How do you incorporate AI or machine learning in analysis or experimentation?

AI is changing CRO from manual analysis into a faster, insight-driven process. Top firms now use machine learning to spot patterns, prioritize test ideas, and analyze qualitative data at scale.

For example, clustering models can surface hidden friction points like visitors who apply multiple filters having a 40% lower conversion rate than average. Multi-armed bandit algorithms reallocate traffic dynamically, helping tests reach valid results 30–50% faster. And natural language models can sort thousands of survey responses by themes like “trust,” “pricing doubts,” or “delivery anxiety” in minutes instead of weeks.

Ask your prospective agency all the pertinent questions related to AI:

1. Don’t just ask if they use AI, but how they apply it and whether they can show concrete examples of better decision-making through it.
2. How exactly do they use AI in their CRO workflow—data analysis, hypothesis generation, or traffic allocation?
3. Can they show a real example of an AI-driven insight leading to a successful test?

25. How do you address consent, cookie restrictions, and data privacy in user research?

With privacy laws tightening and browsers phasing out third-party cookies, CRO agencies must now collect and analyze data responsibly without losing insight quality. The best firms use first-party tracking, server-side analytics, and anonymized qualitative research to stay compliant while maintaining accuracy.

They also rely on consent management platforms (CMPs) to record user permissions and test privacy-friendly setups, like cookieless analytics and masked session recordings.

Good agencies treat compliance as part of UX optimization, even A/B testing consent banners for clarity and opt-in rates.

Here’s what you should ask a CRO agency before hiring:

• How do they handle user consent and data retention in tests or recordings?
• Can they still run meaningful analysis in cookieless or limited-data environments?