Conversion rate optimization is a science.

This means you should be able to reproduce a process, step by step, through a time-tested system, to achieve the increase in conversions you aspire for your website.

Sounds good, right? That is why we created the Conversion Framework.

The goal of the Conversion Framework is to remove the guesswork from the conversion optimization process. It provides anyone interested in optimization with a specific methodology to produce consistent results.

In the following chapters of this guide, you will discover the Conversion Framework and its different elements. This framework is the process we developed and deployed on hundreds of websites and thousands of successful A/B and multivariate tests to help increase conversion rates.

The framework puts the buyers at the heart of your optimization effort.

It forces you to think of the buyers’ needs, wants, motivations, and fears. By focusing on the buyers, you will ultimately persuade more of them to become your customers.

What is the Conversion Framework and Why Do You Need It?

The Conversion Framework is a 7-principle model that explains why visitors convert or don’t on a website. It provides a structured, repeatable way to improve conversion rates by focusing on trust, fears and doubts, incentives, engagement, visitor temperament, buying stage, and sale complexity.

Since the early days of Invesp, we recognized the need to follow a proven methodology that would consistently deliver results for our clients. Our methodology is ever-evolving and growing, but the objectives remain the same and still guide the Conversion Framework we developed in 2007.

At its core, the conversion framework helps you evaluate:

• Trust (T): Does the visitor feel confident your site is credible and safe?
• FUDs (F): Are fears, uncertainties, or doubts causing hesitation?
• Incentives (I): What motivates the visitor to act now instead of later?
• Engagement (E): Does the site create connection and long-term value?
• Persona temperament (P): How do different visitor personalities make decisions?
• Buying stage (B): Where is the visitor in their decision-making journey?
• Sale complexity (S): How simple or complex is the purchase itself?

By running through these questions, you can spot where the page persuades, where it falls short, and what changes are most likely to lift conversions.

The 7 Principles of the Conversion Framework Explained

The Conversion Framework is built around seven principles:

1. Trust and confidence (T)
2. FUDs (F)
3. Incentives (I)
4. Engagement (E)
5. Visitor persona temperament (P)
6. The buying stage of visitors when they come to the site (B)
7. The sale complexity (S)

With these seven elements, you can find out how visitors interact with your website and whether they are persuaded or discouraged to convert.

We like to break down the factors in the Conversion Framework into two different groups: website-centric and visitor-centric factors.
Website-centric factors relate to your website and how information is presented on it through design, layout, copy, and flow. These factors include:

• Trust and confidence (T)
• FUDs (F)
• Incentives (I)
• Engagement (E)

As you delve deep into CRO, you will discover that there are many techniques that you can use to accommodate and enhance website-centric factors.

Let’s take an example of increasing visitors’ trust and confidence in your website. How can you do that? You can use a strong value proposition, social proof, continue messaging and many other factors. As a matter of a fact, there are over 50 things you can implement on your website with the goal of enhancing trust.

How about reducing FUDs (fear, uncertainty, and doubts)? Again, there are many things you can do on a website to reduce FUDs. But it all starts with understanding the major concerns and anxieties visitors have when they visit your website or your competitors’ websites. When you have a list of the top FUDs visitors have, you address them through copy and design.

How can incentives increase your conversions? Let me first state that incentives go beyond discounting your offer. Selling your products or services at a lower price is rarely a good long-term strategy. The goal of incentives is to encourage visitors to act right away, as opposed to postponing the purchase, by thinking “I will come back to this website next week.” You can encourage your visitors through urgency and scarcity, for instance, instead of offering discounts.

Finally, there is engagement. If you are looking for a long-term relationship with your customers, you must move beyond a single conversion, into customer retention. Engagement involves surpassing customers’ expectations, by creating Aha moments. That is easier said than done.

Visitor-centric factors relate to the website visitor’s mindset and nature of the purchase. These factors include:

1. Visitor persona temperament (P)
2. The buying stage of visitors when they come to the site (B)
3. The sale complexity (S)

You do not have control over these factors. That does not mean you ignore them. You have to accommodate them in your design, copy, and visitor flow.

By their nature, visitor-centric factors are more difficult to design for. They are more complex. That is the reason many conversion optimization consultants focus on website-centric factors. These tend to be easier to deal with.

Most marketers talk about personas and market segments. They know it all very well in theory. However, they fail in implementing it when they are designing their website or marketing campaigns.

Tech companies often fail at marketing because they take a fundamentally flawed approach to designing marketing strategy and campaigns. Subpar marketing leads to “me-too” uninspiring content on corporate blogs with insignificant social shares and superficiality from the target audience’s viewpoint. Tech companies struggle to create strategic messaging to describe their products and use undifferentiated messages, jargon and superlatives. The end result is that marketing drains significant resources on paid acquisition campaigns that have no market segmentation or designated landing pages.

Check Myk Pono’s How to Design Marketing Campaigns: The Importance of Market Segmentation

Not every visitor who comes to your site is ready to buy. Actually, studies by Invesp show that less than 20% of website visitors are in the action stage. Most websites focus just on these visitors, and ignore the remaining 80% who are interested in the products and services, but still unenthusiastic to make a purchase. However, visitors early in the conversion funnel should be addressed as well through copy and design.

Sale complexity, the final element of the Conversion Framework, deals directly with the nature of the item you are selling and how your prospect makes the decision of buying from you. To put things into perspective, consider the difference between selling t-shirts online versus enterprise software that costs millions of dollars. Your website and campaigns must account for both the complexities of selling your products as well as the many steps your visitors go through before making the conversion decision.
The following chapters in this guide go into details for each of Framework elements.

As you apply these principles to your website or campaign, you begin to see their impact. By implementing the science of online testing, you can measure the impact of any change you make on your bottom line.

Understanding the Conversion Rate Uplift Formula

The Conversion Framework Uplift Formula translates the different factors of the Conversion Framework in a formula you can apply to different pages on your website.

CRU = {3T-(2F-(I+E))] *B}^ (P * C)

This is not a mathematical formula.

This formula rather attempts to decipher human behavior on the web, recognizing which factors persuade visitors to convert on a website as well as which ones make them abandon the site.

Here are some notes you should keep in mind as you study this CRU formula:

• Each element in the Conversion Framework is broken into several sub-elements. The seven elements of the framework translate into close to 300 sub-elements to evaluate on a website.
• Trust (T) has the most impact on a visitor’s decision to convert. This makes sense. If the visitor does not trust the website, he will not buy from it. As you start analyzing your pages to determine what should be fixed on them, the first thing you look at is trust and its subcomponents such as value proposition, continuing in messaging, social proof and many others.
• FUDs (fears, uncertainties, and doubts – F) play a determining role in reducing visitors’ trust of the website by increasing anxieties.
• The best way to deal with FUDs is through incentives (I) and engagement (E).
• The buying stage (B) visitors come at impacts whether they will convert. Only 20% of your website visitors are ready to convert. The remaining ones are early on the buying process. That does not mean you should ignore them, you just need to design for them.
• The sale complexity (C) has a direct impact on the purchase decision. Some items are more of a considered purchase, while others are quick decisions. The more complex the sale, the more difficult it is to close.
• The visitor persona (P) temperament impacts how fast a visitor converts on your website.

Using the Conversion Framework during a CRO program

During one of our webinars, we presented the use of the Framework elements across different classes of webpages on a CRO program we ran for one of our clients. The video below will give you insights into how to use the Conversion Framework elements to see greater ROI.

So, how do you apply the Conversion Framework during a CRO project?

You go through several steps as you are optimizing your website for conversions. At a very high level these steps are:

1. Conducting Heuristic Analysis
2. Conducting Qualitative Analysis
3. Conducting Quantitative Analysis
4. Conducting Competitive analysis
5. Creating Conversion Roadmap
6. Identify Problems
7. Use the Conversion Framework for assessing problems
8. Prioritize Problems on the Page
9. Create Hypothesis
10. Create New Designs
11. Conduct A/B Testing
12. Conduct Post-Test Analysis

As you can see, the Conversion Framework is used in step 7 of a CRO system. Increasing a website conversion rate comes with a lot of complexities. If that were not the case, most websites would be converting at 20% and 30%, but average website conversion rates remain quite below this mark.

The next few chapters will introduce you to each one of the elements of the Conversion Framework and provide many examples on how to apply them while conducting a CRO project.

The post The Conversion Framework: 7 Principles to Increase Conversion Rates appeared first on Invesp.

Everywhere you look, you see ads. Amid this chaos, word-of-mouth marketing (WOMM) helps you cut through the noise by doing what algorithms can’t—building trust. 

Whether it’s a quick recommendation in a WhatsApp group, a 15-second TikTok from a micro-influencer, or a glowing review in a niche subreddit, WOMM drives more credible, more lasting influence than traditional media ever could.

This article breaks down the importance of word-of-mouth marketing in 2025, the channels where it’s most active, and the latest data that proves why it’s still your most valuable growth driver. You’ll also find actionable strategies—backed by examples and tools—to help you scale WOMM for your brand.

What Is Word of Mouth Marketing (WOMM)?

Word-of-mouth marketing (WOMM) occurs when customers organically promote a brand, product, or experience through conversations, posts, reviews, or recommendations. It can be spontaneous or sparked by a strategy (like a referral program or community campaign).

In 2025, it will not be just friends chatting that moves the needle. Every time you see referrals in private messages, niche forums, shared user-generated content (UGC), and everyday micro-influencer posts, you can safely say that it’s a part of WOMM. 

Key channels for WOMM in 2025

In 2025, word-of-mouth marketing will spread through more nuanced and fragmented channels than ever before. It’s no longer just face-to-face chats or viral tweets—it’s happening across invisible networks, niche communities, and personal-feeling content formats.

Here are the key channels where WOMM is most active today:

• Dark Social: Think WhatsApp, Instagram DMs, Slack, and email—places where people share product links privately. This is where most recommendations happen, but they’re hard to track. (More on this later.)
  
• Online Communities: Reddit threads, Discord servers, and closed Facebook groups often host authentic, high-intent conversations about brands, especially in niches like skincare, productivity, or sustainability.
  
• User-Generated Content (UGC): Organic posts, videos, and reviews from real customers serve as modern-day word-of-mouth, especially in DTC and creator-led spaces.
  
• Micro-Influencers: Smaller creators with strong community ties outperform larger influencers in trust and engagement. They often act as super-referrers.
  
• Reviews & Testimonials: From product pages to Trustpilot, customer reviews influence discovery and reinforce purchase decisions. It’s one of the most scalable forms of WOM.
  
• Offline Triggers: Thoughtful packaging, real-world experiences, and physical brand moments (like events or pop-ups) help spark conversations, both in person and online.

Word of Mouth Marketing Statistics to Know in 2025

Here are some statistics that prove the power of word-of-mouth in 2025: 

People Trust People More Than Ads—By a Long Shot

A massive 88% of consumers say they trust recommendations from friends and family more than any other type of advertising. 

Think about it: when someone you know says, “This worked for me,” you don’t question it like a paid ad or influencer post. You believe them. You might even go look the product up immediately.

That’s what makes word-of-mouth marketing so powerful. It moves people to act because it feels personal. Brands that focus on creating genuinely great experiences and then make it easy for people to talk about them get the kind of exposure money can’t replicate.

Word of Mouth is Still the #1 Way People Discover New Brands

A 2023 survey found that 36% of U.S. internet users said word of mouth was their leading source of brand discovery, beating out social media ads (32%) and mobile app ads (21%).

This means that, despite the rise of paid performance channels, people still trust people more than algorithms. 

For marketers, this is a reminder: while paid reach gets you in front of people, trust gets them to act. If people aren’t discussing your product on group chats, dinner tables, or Slack threads, you’re missing the channel that drives the highest-intent traffic of all.

Referred Customers Don’t Just Convert—They Multiply

Word-of-mouth becomes a growth loop, not just a one-time win.

A Harvard Business Review study analyzing data from over 41 million customers of a cashback app found that customers who joined through a referral not only purchased more. They also went on to refer 30% to 57% more new customers than users acquired through other channels.

This turns word-of-mouth into a compounding asset. That’s the kind of retention plus acquisition combo most ad campaigns can’t touch.

Referred Customers Stick Around Longer and Spend More

According to multiple studies, customers who come through referrals stay 37% longer and deliver 16% more lifetime value than those acquired through paid ads or cold outreach. 

In other words, they don’t just buy once and bounce. They buy again, engage more often, and are more likely to refer others.

This happens because they enter your brand ecosystem with built-in trust. Someone they know already gave you a vote of confidence, and that shortcut to credibility pays off long after the first purchase.

The takeaway? If you want higher-value customers, don’t just spend more on ads. Build systems that encourage your best customers to bring in more people like them.

Word-of-Mouth Marketing in 2025: Key Trends and How to Capitalize on Them

In 2025, word-of-mouth marketing is unfolding quietly across DMs, Discords, and creator-driven content. 

As consumer trust in polished ads declines, brands are finding smarter ways to spark organic advocacy in the spaces people pay attention to. 

Here are the key trends shaping modern WOMM and how to emulate them to make your business more successful and inspire customer loyalty. 

Micro-Influencers and Nano Communities Are More Trusted Than Celebrities

In 2025, the most influential voices are small, specific, and deeply embedded in their communities. There’s a reason micro-influencers (10K–50K followers) and nano-influencers (<10K followers) are outperforming traditional celebrity endorsements across every primary metric: trust, engagement, conversion, and ROI.

The numbers tell a clear story.

87.7% of all TikTok creators are nano-influencers, and they generate an average engagement rate of 10.3%, nearly triple that of megastars.

Engagement rates decline across all influencer tiers (2021–2024), but nano (1K–10K) and micro (10K–50K) creators consistently outperform larger accounts, maintaining the highest average engagement through 2024. 

63% of shoppers say they’re more likely to buy a product if it’s recommended by a social media influencer they trust. 

But how do you capitalize on micro-inflencer WOMM in 2025? 

Working with micro-influencers shouldn’t feel vague or hard to measure. Here’s how to turn it into a focused, repeatable channel that drives both reach and revenue.

1. Use Modash to shortlist 20–30 creators who already post about your category

Instead of working with whoever slides into your DMs, start with a focused shortlist. 

Use a tool like Modash to filter for creators with 5K–50K followers, at least 3% engagement, and existing content related to your niche—like “mineral sunscreen” if you’re in skincare, for example. 

Finding nano influencers in your niche and based on other filters (Source)

This approach helps you find creators whose audience already trusts them on the topic you’re trying to sell. 

2. Send 10 of them your best product, no strings attached

Reach out with a simple note offering to send them your hero product. Don’t attach any posting requirements or approval steps. Just ship the product, include a handwritten message that reflects your brand tone, and trust the quality to speak for itself.

You’ll often find that 4–5 of those 10 will post something anyway, and organic, unsolicited content performs better than most paid collaborations.

This kind of no-ask gifting is called product seeding—you send your product to creators without asking them to post. It’s personal, low-pressure, and often more effective than paid campaigns. 

Graza, a DTC olive oil brand, did precisely this. They shipped bottles to food creators they genuinely liked and thought aligned with their brand values, added handwritten notes, and didn’t ask for anything. 

The posts came anyway. Small content creators who received the product shared their experiences, generating authentic user-generated content (UGC).

The result? Graza sold out during their first week in business, bringing in $100,000 in revenue. Within three months, they surpassed $500,000 in revenue, all without spending on traditional advertising.

3. Give each creator a trackable URL to measure real impact

The biggest mistake in micro-influencer marketing is not tracking correctly. Every creator should get a unique, UTM-tagged link using Bitly or Switchy. It can look like:

brand.com/lip-oil?utm_source=instagram&utm_medium=influencer&utm_campaign=jenna

This link shows how many clicks each person made, whether they shared it via DMs, Stories, or in a private group.

Without this, all your traffic shows up as “direct,” and you’ll never know which creators helped you grab more eyeballs. 

Most Shares Happen in Dark Social, Not Public Feeds

Word-of-mouth in 2025 will live in private, untrackable channels: DMs, text messages, WhatsApp, Slack, and even Discord. Beyond exchanging messages, these platforms are your customers’ most-used sharing surfaces, and they don’t leave visible traces in traditional analytics.

Analytics expert Steve Lamar recently ran a deep attribution experiment to understand how platforms pass referral data.

By redirecting clicks through uniquely structured subdomains, he managed to track what GA4 misses. Here’s what he found:

• 100% of clicks from TikTok profiles and apps like WhatsApp, Discord, Slack, and Mastodon were attributed as “direct” traffic—completely untrackable via default Google Analytics.
  
• 75% of clicks from Facebook Messenger were also misattributed.
  
• Even mainstream platforms like Instagram (70%), LinkedIn (86%), and Pinterest (88%) mostly passed incomplete or no attribution data.

Dark traffic on major social networks (Source)

This means that even if people actively share your brand, you won’t see it in your dashboards unless you track differently.

But despite its untraceability, dark social is modern-day word-of-mouth. When a friend DMs a product link, shares a TikTok, or mentions your brand in a private Slack channel, they recommend it and influence purchase decisions.

But since it shows up as “direct traffic,” most brands don’t realize what’s working.

So, since you can’t see every share, how do you capitalize on dark social WOMM? Here are some quick tips that can help: 

1. Track smarter with share-optimized URLs. 

Many brands lose visibility because all shared links look the same in their analytics (“direct traffic”). 

The fix? Use custom links with tags. Create trackable URLs using UTM parameters (small bits of info added to the end of a URL) so you know where the click came from.

For example, instead of sharing website.com/product-a, use website.com/product-a?utm_source=whatsapp&utm_campaign=drop1.

This helps you see if someone clicked the link from WhatsApp, email, or Instagram bio, even if it was shared privately.

Pro tip: You can use Bitly for branded, shortened links or Switchy for link retargeting and deeper analytics. 

2. Make it easy to share products via WhatsApp, Messenger, or DMs 

After someone buys a product, gets a discount code, or finishes a quiz, they’re in a high-emotion state. This is when they’re most open to sharing something they love (or just scored) with a friend.

To make the most of this, brands need to remove friction. The best way to do this is by adding “Send to a friend” or “Share on WhatsApp” buttons on post-purchase confirmation pages, wishlists, and quiz result screens.

A great example of this in action: Fashionette, a luxury fashion retailer in Europe, integrated WhatsApp sharing directly into a giveaway flow.

They launched a giveaway campaign where users could click a Facebook ad and join a sunglasses giveaway by clicking “Chat with us” on WhatsApp. 

Once in the chat, users get real-time responses and can easily share the offer with friends, creating a conversational entry point for word-of-mouth.

Users click “Chat with us” on the ad to enter the giveaway via WhatsApp, making it easy to join and share the offer with friends in just a few taps (Source)

This shift to real-time, one-on-one messaging paid off. Fashionette’s WhatsApp flows converted 7.4 times higher than their most engaged email segments, driving a 25% increase in average order value.

3. Watch for sudden traffic spikes—then trace the source

Not all dark social traffic is invisible. Sometimes, the signal is a sharp rise in direct traffic to a specific product or blog page, primarily when you haven’t run paid campaigns or sent emails.

This often means someone shared your link in a WhatsApp group, Discord server, or Slack workspace. But instead of showing up as a referral, it shows up as “direct traffic.”

Here’s what to do: 

• Set up daily traffic alerts in your analytics platform. For example, in Google Analytics 4, you can use “Custom Insights” to trigger alerts when a page’s traffic exceeds a certain threshold—say, 200% of its weekly average. (This guide helps you set up custom insights in GA4.)

Setting up custom metrics in GA4 to track a sudden traffic spike (Source)

• Cross-check those spikes with your marketing activity. Identify this by asking yourself:
  • Did you publish a TikTok, YouTube Short, or Instagram Story in the past 24 hours?
  • Was your product linked in a newsletter, Substack issue, or X thread?
  • Did someone post about you in a Reddit community (search via site:reddit.com yourbrand)?
  • Did a micro-influencer quietly tag or mention you in a private story or channel?

• Correlate timestamped events. Once you notice a traffic spike, dig into the timestamps in your analytics and compare them to when your content—or any third-party content—went live. Let’s say your product page for a “hydrating mist” usually gets around 120 visits daily. On May 5th, you notice 670 visits to that page within two hours, all showing up as “direct traffic.” That’s a red flag for dark social. You check TikTok and discover that a creator with 18,000 followers posted a GRWM (get ready with me) video that morning featuring your mist. That’s likely your referral source—just hidden from plain view in your analytics.

Private Communities Are Becoming Brand Powerhouses

In 2025, brands are seeing more word-of-mouth action inside closed groups than they do on open platforms. Think, Slack channels, Discord servers, subscription-based Substacks, even private Facebook or Geneva groups.

According to a 2024 report by TINT, 40.9% of consumers said they planned to increase their participation in online communities, marking a 9% year-over-year growth. 

But more importantly, it’s working:

• 73.6% of consumers say they purchase more frequently because of a brand’s online community
  
• 75% agree that communities improve the customer experience
  
• 57% report better brand SEO tied to community discussions
  
• 90% of community managers say user suggestions directly inform product improvements
  
• And in some cases, over 30% of total revenue can be traced back to the influence of a brand’s community

So, how do you tap into this in 2025? Here are some quick, actionable tips: 

• Find where your customers already gather. Instead of building your community from scratch, start by identifying where high-intent conversations are already happening. You can use platforms like Common Room to surface mentions of your brand or product across Slack groups, Discord servers, or Reddit threads you care about. 

• Don’t show up to promote, show up to participate. Community members can spot a sales pitch from a mile away. Join as a peer instead of leading with a CTA or product link. Offer value: answer questions, host an AMA (Ask Me Anything), or share honest behind-the-scenes updates from your product or team.
  
• Seed early access or limited drops inside these groups. When members feel they’ve discovered something first, they share it naturally.

If you enter these groups thoughtfully, participate sincerely, and offer value early, the word-of-mouth that follows is more powerful and longer-lasting than any shoutout on a public feed.

Short-Form UGC Is the Most Believable Form of Marketing

Short-form UGC (think 10 to 60-second videos shared by real users) is now one of the most trusted and influential content types in marketing.

Whether it’s a quick “unboxing” on Instagram Stories, a GRWM (get ready with me) TikTok, or a 15-second “before and after” reel, short-form UGC feels real, spontaneous, and relatable. And that’s precisely why people believe it.

And the numbers back this up.

• 73% of consumers say they prefer short-form videos when researching products or services, because they feel more to-the-point and trustworthy.
• Short-form videos generate 2.5x more engagement than long-form, making them more likely to be liked, shared, or saved.
• And when brands use short-form UGC in paid ads, the results are hard to ignore: 4× higher click-through rates and 50% lower cost-per-click compared to traditional branded content.

As we mentioned earlier, a low-pressure seeding campaign is one of the most effective ways to spark this kind of UGC. You send your product to a curated list of creators without any obligations to them and simply trust that those who genuinely love it will talk about it.

Conclusion: Why Word of Mouth Marketing Should Be a Priority in 2025

At a time when paid ads are getting more expensive and less effective, WOMM gives you scalable, high-intent growth without burning budget. It turns your best customers into your best marketers. And in 2025, it’s happening where brands aren’t looking: inside DMs, private Discords, WhatsApp groups, and short-form creator videos.

If you’re serious about scaling WOMM in 2025, here’s where to focus:

• Track private sharing behavior through better attribution (UTMs, GA4 custom insights, link tagging)
  
• Encourage advocacy with smart referral, loyalty, and creator seeding programs
  
• Optimize your site for post-purchase sharing, especially on mobile and messaging-first channels
  
• Test how user-generated content impacts CRO on PDPs, review sections, and post-purchase flows
  

At Invesp, we help brands turn passive word-of-mouth into measurable conversion lifts by testing how, where, and why your best customers are talking.

Want to see how CRO and WOMM overlap on your site? Start a conversation with our team.

The post The Importance of Word Of Mouth Marketing – Statistics and Trends 2025 appeared first on Invesp.

We’ve got exciting news; Invesp is now an official Shopify Plus Partner!

If you’re running an e-commerce brand, you already know that Shopify Plus is more than just an online store builder. It’s a powerful platform designed for high-growth brands that need customization, flexibility, and serious scalability.

At Invesp, we’ve spent the last two decades helping e-commerce brands maximize conversions, increase revenue, and create better shopping experiences. Now, as an official Shopify Plus Partner, we’re taking that expertise even further.

What Does This Partnership Mean for You?

Shopify Plus is built for brands that are scaling fast; but growth brings complexity. You need to keep conversions high, optimize user experience, and ensure every experiment you run leads to real business impact. That’s where we come in.

Here’s how this new partnership benefits you:

1. More Advanced A/B Testing & Experimentation

One of the biggest challenges Shopify store owners face is platform limitations on testing and customization. But with Shopify Plus, we can unlock deeper insights and run more advanced A/B tests—whether it’s optimizing checkout, personalizing customer experiences, or refining product pages.

This means we can test bigger ideas, make data-backed changes faster, and drive more revenue for your business.

2. Exclusive Shopify Plus Tools at Your Disposal

Being a Shopify Plus Partner gives us access to exclusive tools like:

• Custom checkout experiences – Optimize every step of the buying process to reduce cart abandonment.
• Shopify Functions – Customize discounts, promotions, and backend logic to fit your business needs.
• Enhanced automation – Set up workflows that streamline operations and improve efficiency.

With these tools, we can go beyond just “running tests” and create a more profitable, seamless experience for your customers.

3. Faster Implementation, Less Guesswork

When you’re optimizing an online store, speed matters. As a Shopify Plus Partner, we have direct access to Shopify’s resources, priority support, and insider knowledge—which means we can move faster and get results quicker.

Instead of spending weeks (or months) trying to implement conversion strategies, we can roll out tests, analyze results, and make high-impact changes without delays.

4. A CRO Agency That Truly Understands Shopify Plus

Not all CRO strategies work the same way on every platform. Shopify Plus has its own ecosystem, features, and best practices. Now that we’re an official partner, we’re deeply integrated into the Shopify Plus world; which means we can help you avoid common pitfalls and implement strategies that actually work.

Whether it’s increasing average order value (AOV), optimizing your checkout process, or improving mobile conversions, we bring two decades of CRO experience combined with Shopify Plus expertise to drive real, measurable results.

What’s Next?

If you’re already on Shopify Plus (or considering upgrading), this is the perfect time to take your conversion rates to the next level.

• Are you struggling with abandoned carts?
  Want to see how custom checkout experiences can increase conversions?
  Need expert guidance on running A/B tests on Shopify Plus?

Let’s talk. As a Shopify Plus Partner, we’re here to help you turn more visitors into customers, more clicks into purchases, and more tests into wins.

Book a call with us today and let’s grow your Shopify Plus store together!

The post Invesp Is Now a Shopify Plus Partner: Here’s What That Means for Your Business appeared first on Invesp.

Retail therapy plays a significant role in impulse buying, as individuals often make purchases to improve their mood, especially during rough days. This emotional aspect of shopping can trigger a dopamine release, similar to other pleasurable activities, and marketers exploit these emotions to encourage consumer spending.

Impulsive personas are the apple of the eye of marketers and business owners.

The credit-card-in-hand shoppers get undivided attention in copy, design, and layout of websites, landing pages, and ads, as well as in brick-and-mortar displays and services.

So much is carefully crafted to address impulsive buying: scarcity and emergency elements, benefits displayed in bullet points, big and bright CTAs, smooth and fast check-out processes.

But who are the impulsive buyers shaking up online and offline commerce? Are they mostly married or single? Do they usually act impulsively while online shopping or in-store browsing? Do they really regret their purchases? Do they spend more on unplanned shopping trips? How much do they generally spend in each impulsive purchase? What is the state of mind they normally experience during impulsive buying?

Let’s get to know this darling persona better with our infographic. You will be able to identify trends and general traits to address impulsive prospects in your business.

What Is Impulse Buying?

Impulse buying refers to the act of making unplanned and spontaneous purchases, often driven by emotions, desires, or a sudden urge. It involves buying items without careful consideration or prior intention, and the decision to make the purchase is typically made on the spot.

Impulse buying, also known as impulse purchasing, is characterized by a lack of premeditation, with individuals succumbing to the immediate desire for a product or service. This behavior is often associated with the psychological concept of instant gratification, where individuals seek immediate satisfaction or pleasure without considering the long-term consequences.

84% of all shoppers have made impulse purchases

This high percentage suggests a significant influence of emotional and psychological factors on spending habits, as consumers often prioritize immediate satisfaction over rational decision-making.

It also sheds light on the prevalent phenomenon of impulse buying in today’s consumer culture. Common examples of impulse purchases include clothing and groceries, with trends showing increased impulse buying during occasions like the holiday season. This data underscores the pervasive nature of impulse buying behavior, where individuals make unplanned and spontaneous purchases.

The prevalence of impulse buying in the market emphasizes the need for retailers to understand and capitalize on these tendencies, while consumers may benefit from recognizing and managing their own impulses to make more mindful purchasing decisions.

40% of all the money spent on e-commerce is attributed to impulse purchases

The data highlights the substantial impact of impulsive behavior in shaping the financial dynamics of e-commerce, emphasizing the role of immediate desires and spur-of-the-moment choices in driving online sales. Managing impulse buying is crucial to save money and make more mindful purchasing decisions. Businesses in the digital marketplace can leverage this information to tailor their strategies and capitalize on the prevalence of impulse buying in the online shopping experience.

54% of U.S shoppers acknowledge spending $100 or more on impulse buys

More than half of U.S. shoppers—54 percent—have admitted to spending $100 or more on an impulse buy, including 20 percent who have spent at least $1,000. By understanding the extent to which consumers are willing to spend impulsively, businesses can optimize their approaches to capitalize on this behavior, whether through strategic promotions, targeted advertising, or creating a shopping environment that encourages spontaneous purchases.

Compulsive shopping, however, is a dangerous progression from impulse buying, with detrimental impacts on finances and personal well-being.

8 out of 10 impulse buys are made in a brick-and-mortar store

This indicates the continued significance of physical retail spaces in driving spontaneous consumer purchases. Optimizing in-store experiences and strategically placing products can be pivotal in capitalizing on impulsive buying behavior. Creating visually appealing displays, offering limited-time promotions, and ensuring an easily navigable layout can enhance the likelihood of impulse purchases.

Impulsive shopping, often driven by emotional spending, is a common behavior where people buy things to reward themselves or cope with negative feelings. Strategies to manage this behavior include undertaking a no-spend challenge.

While e-commerce continues to grow, the tangible and immediate nature of brick-and-mortar environments provides a unique opportunity for businesses to leverage human psychology and boost spontaneous buying, emphasizing the need for a well-crafted in-store strategy to complement online efforts.

Single shoppers make 45% more impulse buys compared to married shoppers.

Marketing strategies and product placements targeting single individuals may yield higher returns in terms of impulsive purchases. Understanding the distinct consumer buying behavior of single shoppers versus married ones enables businesses to tailor their promotional efforts accordingly.

Only 13% of buyers engage in planned impulse buying on a shopping trip.

While a majority of buyers may adhere to predetermined shopping lists during planned trips, the 13% engagement in impulse buying suggests that there is still room to influence purchasing decisions through targeted marketing, appealing displays, and strategic product placements.

Recognizing and capitalizing on this minority’s openness to spontaneity becomes crucial, urging businesses to craft nuanced approaches that balance the predictability of planned trips with opportunities to stimulate impulse purchases and enhance overall sales.

46% of men regret impulse buys compared to 52% of women.

Understanding the varying degrees of post-purchase regret can inform targeted interventions, such as personalized post-purchase communication or specialized return policies, aimed at mitigating buyer’s remorse. By recognizing and responding to these gender-specific tendencies, businesses can refine their customer engagement strategies, enhance customer satisfaction, and potentially influence brand loyalty based on distinct consumer behaviors among men and women in the aftermath of impulse purchases.

52% of millennials were more likely to make impulse purchases

The revelation that 52% of millennials exhibit a higher propensity for impulse shopping than any other generation underscores the distinct impulse-buying behavior prevalent among this demographic. For businesses, understanding and catering to the preferences of millennial impulse buyers is paramount.

Crafting marketing strategies that resonate with the spontaneity of their impulse spending tendencies, providing visually engaging displays, and promoting limited-time offers can be instrumental in capturing their attention during unplanned purchases.

Recognizing the unique dynamics of millennial impulse buyers allows businesses to tailor their product offerings and promotional efforts effectively, capitalizing on the spontaneity inherent in their purchasing behavior to drive increased sales and engagement.

The average shopper will make an average of 3 unplanned purchases out of every 10 store visits they make.

All this highlights the prevalence of impulsive spending habits and impulsive buying behavior among consumers. Understanding this aspect of shopping behavior is crucial in developing effective strategies to capitalize on unplanned purchases.

Creating an in-store environment conducive to spontaneous buying, strategically placing enticing products, and leveraging promotional tactics can harness the impulse buying tendencies of customers.

Acknowledging and adapting to the frequency of unplanned purchases in the design of marketing campaigns and store layouts allows businesses to optimize their overall sales strategy, tapping into the dynamics of impulsive spending to enhance customer satisfaction and drive increased revenue.

The Psychology of Impulse Purchases

A combination of mental and emotional triggers often drives impulse purchases. Research suggests that the brain’s reward system is activated when a consumer sees a product or receives a promotional message, releasing feel-good chemicals such as dopamine. This can lead to a sense of instant gratification and pleasure, making it more likely for the consumer to make an impulse purchase.

Additionally, emotional states such as stress, anxiety, and boredom can also contribute to impulse buying. Consumers may use shopping as a coping mechanism to relieve negative emotions or to satisfy their temptation to buy. Understanding these mental and emotional triggers can help individuals develop strategies to resist impulse purchases and make more intentional purchasing decisions.

Understanding the mental and emotional triggers

Why We Engage in Impulse Shopping

Impulse shopping is a multifaceted behavior influenced by various factors. One primary reason is the thrill of the hunt. The excitement of discovering a new product or snagging a limited-time offer can be exhilarating, leading to spontaneous purchases. Social and cultural factors also play a significant role. For instance, societal norms and peer pressure can drive individuals to make unplanned purchases to fit in or keep up with trends.

Marketing tactics are another powerful influence. Retailers use strategies like scarcity, urgency, and visually appealing displays to entice consumers into making impulse buys. The combination of these elements creates an environment where impulse shopping thrives, making it essential for consumers to be aware of these influences to manage their spending habits better.

Types of Impulse Purchases

Impulse purchases can be categorized into several types, each with its own characteristics and triggers:

1. Pure Impulse Buying: This is the most spontaneous form of impulse buying, where a consumer makes an unplanned purchase without any prior intention. An example could be grabbing a candy bar at the checkout counter.
2. Reminder Impulse Buying: This occurs when a consumer is reminded of a need or desire upon seeing a product. For instance, seeing a display of sunscreen might remind someone to buy it for an upcoming beach trip.
3. Suggestion Impulse Buying: In this type, a consumer makes a purchase based on a suggestion or recommendation, often influenced by marketing tactics. An example could be buying a new gadget after seeing an influencer’s review.
4. Planned Impulse Buying: This involves a consumer planning to make a purchase but not deciding on the specific product until they see it. For example, someone might plan to buy a gift but only choose the actual item when they find something appealing in the store.

Understanding these types of impulse purchases can help consumers recognize their buying patterns and make more informed decisions, ultimately aiding in saving money and reducing unplanned expenditures.

States of Mind When Making an Impulsive Purchase

When making an unplanned purchase, 50% of females express greater excitement, whereas only 47% of males share the same level of enthusiasm.

When intoxicated, 13% of males exhibit impulse buying behavior, while only 5% of females engage in impulse purchases under the same circumstances.

Infographic by- Invesp Conversion Optimization Services

To Publish this Image on your Blog or Website . Copy this code

84% of all shoppers have made impulse purchases

Impulse purchases represent almost 40% of all the money spent on e-commerce

More than half of U.S. shoppers—54 percent—have admitted to spending $100 or more on an impulse buy, including 20 percent who have spent at least $1,000

8 out of 10 impulse buys are made in a brick-and-mortar store

Single shoppers make 45% more impulse buys than married shoppers

Buyers are 13% less likely to make impulse buys on a planned shopping trip. They are 23% more likely to impulse buy on an unplanned shipping trip

Only 46% of men said they regretted an impulse purchase compared to 52% of women

52% of millennials were more likely to make impulse purchases than any other generation.

The average shopper will make an average of 3 unplanned purchases in 4 out of every 10 store visits they make.

States of Mind When Making an Impulsive Purchase

Khalid Saleh

Khalid Saleh is CEO and co-founder of Invesp. He is the co-author of Amazon.com bestselling book: “Conversion Optimization: The Art and Science of Converting Visitors into Customers.” Khalid is an in-demand speaker who has presented at such industry events as SMX, SES, PubCon, Emetrics, ACCM and DMA, among others.

The post The State of Impulse Buying (Statistics & Trends 2025) appeared first on Invesp.

Although Quora existed for few years now, recently I started using it more regularly. Perhaps opening Invesp’s office in Istanbul, Turkey was a major factor there.

And yes, I am a big fan!

Spending my time between our US and Istanbul offices is an amazing adventure. At the same time, it comes with its own set of challenges. That is where Quora comes in handy. I usually ask questions trying to understand more about Turkish culture, economy, business practices and yes, even good restaurants to eat at.

The website has also a healthy section on conversion optimization where those who are looking for advice post their questions. As I went through that section, I noticed that some of the questions were repeated again and again.

So, in this blog post I will take the 8 most common  questions on conversion optimization I see in Quora and answer them.

1. What Is the Best Strategy for Increasing My Website Conversion Rate?

You see this question asked in many different forms. Those who run an e-commerce website ask about getting more orders. Those who run lead generation website ask about ways to increase the number of leads from their website.

The problem is that many of the answers focus on tactics. Rarely do you find a thread that goes into strategy. That is fine, if you are looking for quick fixes. However, it comes with a problem.

Conversion optimization tactics may or may not work for your website.

There is a place for tactics but without a well through out strategy, you are most likely wasting your time.

Each web page in your website represents an opportunity. If you are working with an unlimited budget, unlimited resources and unlimited time, then you have all the time to test every little detail on your website.

If you are like the rest of us, then you want to start fixing these pages that present the most impact on your bottom line.

How do you find the pages with the highest potential? 

To increase your website conversion rates, you should use analytics to understand how visitors are flowing from one area of your website to the next.

In order for you to get to the big “Yes” – a macro conversion – the visitor must go through a series of smaller “Yeses”, or micro conversions. Before visitors place an order with your website, fill out a contact form, or subscribe to your service, they go through a series of steps and follow different paths.

Our task as marketers is to identify all the possible funnels which visitors go through and to determine which of the funnels are working well and which funnels are leaking visitors so that we can focus on them.

The next question to think about is what changes should you make on a particular web page?

I will tell you what you should NOT do first. You do not randomly change things because best practices tell you to do so. Persuading visitors to convert requires us to anticipate the information they need and to present that information to visitors when they need it.

So, how do you identify conversion problems on a webpage?

At Invesp we use the Conversion Framework to determine the factors that impact  visitor behavior on a web page. These factors are grouped into two main categories:

1. Website-centric factors
2. Visitor-centric factors

Website centric factors are the factors that relate to your website and the signals it sends to your visitors. The main categories under website-centric factors are:

1. Trust factors: Remember that if a visitor does not trust you, he will not interact with you.
2. FUDs: How do you identify and minimize the fears, uncertainty, and doubts that stop website visitors from doing business with you?
3. Incentives: How do you incentivize a visitor to interact with you immediately?
4. Engagement: How do you engage visitors so you move from a one-time interaction to a long term relationship?

Visitor-centric factors are the factors that relate to your website visitors, the nature of their purchase and where they are in the buying funnel. You have less control over these factors, however, you must account and prepare for them.

The main categories under visitor centric-factors are:

1. The visitor persona: How do you address the different types of visitors’ temperaments?
2. How do you design your website to address each of the different visitor buying stage?
3. The third factor is the complexity of the sale. Selling an item that costs $10 is a lot different than selling an item that costs $2,000

We typically evaluate every page (or class of pages) on your website using the Conversion Framework with the goal of determining possible conversion problems on that page.

A conversion rate optimization specialist can identify 50 to 150 problems on a single webpage.

You then prioritize these problems and focus on the top 3 to 5 problems by creating new designs to fix them.

But you do not stop there. You conduct A/B or multivariate testing by splitting your current website visitors between your original design and the new designs and watch which of these design generates more conversions.

Quora question: What’s the best ways to increase your conversion rate honestly?

2. Tools to Perform a Conversion Optimization Audit of My Site?

As of now, there are no tools that help you conduct a conversion optimization audit of a website. There are, however, tools that help you understand how visitors behave on your website which, in turn, will help you in the process of conducting a CRO audit for your website.

Some might think that I am splitting hairs. But a tool that is designed to conduct an audit will be designed in a completely different way compared to a tool that tracks how visitors behave on the website.

Having said that, here is a list of tool categories which we think every website owner should have installed on their website:

Analytics: Needless to say that analytics is a must have for every website. Any self-respecting analytics tool will help you track the performance of different pages on your website (how many visitors they are getting, what is the bounce and exit rates, etc). The main dominant player in this field is Google Analytics. And while GA is free (there is an enterprise option), you are giving your data to Google.

Suggested analytics tools:

• Google Analytics
• Omniture
• Clicky
• Kissmetrics
• Piwik

Heatmap: Heatmap software record the clicks visitors make on any page of your website. They also provide a scroll map which shows how much visitors scroll when they view different pages of the website.

Suggested heatmap tools:

• Crazyegg
• Hotjar

Video recording: Similar to heatmap software, the tools in this category record the interaction between website visitors and the different pages on the website.

Suggested video recording tools:

• Inpsectlet

Website survey: the tools in this category ask website visitors to answer a small set of questions with the goal of gaining insights from these visitors on what is working and what is not in your website design.

Website survey tools:

• Hotjar
• Q4

A/B testing tools: these tools allow you to conduct A/B and/or multivariate testing on your website so you can determine which design generates more results for you. We have written a lengthy article on selecting the best A/B testing tool for your website. You can find it here.

A/B testing tools include:

• Google analytics content experiments
• Omniture
• Optimizely
• VWO

Quora question: Which tools do you use for conversion rate optimization?

3. Does the CRO Pay Per Performance Model Work?

Short answer:

We have not seen it work in the long run and we have been doing conversion optimization for over 11 years.

Long answer:

The idea of a pay per performance tests sounds good at the surface. The client will only pay for increases in conversion rate which shifts the burden and the risk of the project to the CRO firm. For the CRO firms there are two main reasons this model is appealing. First, if the project is successful, then the reward for success will be higher than charging a fixed fee. Second, this model works well for companies that are just starting out.

Here is the problem with this model. The success of any conversion optimization project depends on many factors. But the main two factors are:

1. How many tests do you deploy in a given time period and,
2. How sophisticated are these tests?

Most CRO firms end up relying on the client’s development team to implement the test. And here the problem lays in. The CRO firm is responsible and will be rewarded for the success of the project, but it will have to rely on the client to implement the tests.

In order for the CRO firm to deploy more tests, the firm must have access to the client’s website and should be able to make changes to it. The need for the backend of the website and its database becomes more prevalent, as the tests get more sophisticated.

But most clients, if not all, will not give unlimited access to the CRO firm because that assumes two things:

1. The CRO firm has good handle on the technology used by the client and,
2. The CRO firm has the technical knowledge to be trusted with making change on the website.

For any medium to large website both conditions are typically a no go.

So, most CRO firms end up relying on the client’s development team to implement the test.

And here the problem lays in. The CRO firm is responsible and will be rewarded for the success of the project, but it will have to rely on the client to implement the tests.

Some might say that the client is incentivized to see the project succeed and is motivated to implement tests faster. In our experience, that might have been the case but as the project progresses, development teams get busy and CRO work is pushed further behind and is given less priority.

As a result, these projects have an inherited conflict of interest built into them.

We believe that in order for a project to succeed, everyone must have skin in the game. For the client that will include paying a fee.

Now, there are instances where we agreed to lower our fees in return for an additional fee which the client pays, if we meet certain performance criteria.

Quora question: Which conversion optimization companies work on pay per performance model?

4. Running Multiple Overlapping A/B/n tests Simultaneously on One Page?

As you talk to different companies, you will hear differing opinions on this matter. It has been in our practice that multiple A/B tests on larger websites where 100% of the traffic is included in an A/B test can and will cause data pollution.

While we recommend running multiple tests at the same time, we also highly advice that:

1. The tests are not run concurrently, or
2. If tests are run concurrently, each test runs in a separate swim lane.

If tests run in separate swim lanes, traffic from one test does intersect traffic from the other test. This rarely happens on large complex websites where visitors jump from one swim lane to the next.

Good examples of concurrent tests include: running separate tests on desktop and mobile websites, running concurrent tests targeted for different countries, and running concurrent tests targeted for different sources and media of the traffic.

Quora question: Conversion optimization: Running multiple tests simultaneously on one page?

5. What Does “Confidence Level” Mean in Conversion optimization Lingo?

Statisticians have been debating confidence level for over 90 years.

Confidence level refers to the frequency which an observed interval contains a parameter (a test result). So, if confidence levels are constructed across many separate data analyses of different experiments, the proportion of such intervals that contain the true value of the test result will match the given confidence level.

The term confidence level is one of the least understood, yet most used terms in conversion optimization.

Confidence level does help in persuading people to test, but you need to understand how it works and affects your data.

Regina Nuzzo reports:

According to one widely used calculation, a P value of 0.01 corresponds to a false-alarm probability of at least 11%, depending on the underlying probability that there is a true effect; a P value of 0.05 raises that chance to at least 29%. So Moty’s finding had a greater than one in ten chance of being a false alarm. Likewise, the probability of replicating his original result was not 99%, as most would assume, but something closer to 73% — or only 50%, if he wanted another very significant result. In other words, his inability to replicate the result was about as surprising as if he had called heads on a coin toss and it had come up tails.

In calculating financial risks, the banking sector is sometimes required to see a 99% confidence level. The FDA also requires a 99% confidence level in some cases of drug testing.

Our goal is not to confuse you.

So, what does confidence level mean?

We recommend you think of the confidence level (p-value) as the probability of having a result as opposed to the potential frequency of that result.

So, what confidence level should you choose?

We recommend a 95% confidence level, but if an experiment runs for multiple business cycles and it repeatedly shows higher than 85% confidence level, you should definitely consider it.

Quora question: What does “confidence” mean in conversion optimization lingo?

6.  How Do I Decrease the Bounce Rate of My E-Commerce Website?

Let’s start by defining what a bounce rate is. Bounce rate is the percentage of your visitors who land on a web page from an external traffic source and leave the website without visiting a second page.

So, if a web page gets 100 visitors from an external traffic source (google organic traffic for example) and of those 100 visitors, 35 visitors decide to leave the website without ever visiting a second page on the website, then the bounce rate for the page is: 35/100 = 35%.

A common mistake we see with different websites is considering bounce rate for the website as a KPI (key performance indicator). For most websites, it is not.

A KPI should have a direct business impact. Bounce rate and exit rate are important metrics to track on your website. They rarely have an impact on a firm bottom line.

The conversion rate for a website or the average order value, on the other hand, are business KPIs. They impact your bottom line.

All these things make me doubt the general understanding of this question. An e-commerce website should focus on increasing its conversion rates, visits per value or average order value. Optimizing for bounce or exit rates is like chasing a ghost and will have no direct impact on the bottom line.

7. What Do I Need to See During My A/B Testing to Be Sure That One Variation Is the Winner?

There are three different criteria which must be met before you can call a winner for an A/B test.

1. You need a  statistically significant winner. Statistical significance will provide you with an indication on how solid your test is. The higher the confidence level for the challenger, the less likely it is that the challenger is a false positive. For most tests, a confidence level of 95% is acceptable. There are however instances where the challenger reportedly remains at 80% confidence level or a bit higher. In these cases, you can still call the test with the challenger being a winner, knowing full well that there is a segment of the visitors which will likely convert better with the control (original design).
2. Both the control (original design) and the winning challenger (variation) must have recorded a minimum number of conversions: the goal from this criterion is to ensure that your challenger is NOT a false positive and that testing data stabilized. The minimum number of conversions will vary from one website to the next and will depend on the number of conversions your website gets per month. With smaller websites, we usually ask for a minimum of 100 conversions per design. For larger websites, the minimum can be as large as 3,000 conversions.
3. The test should have run for at least one week. Website visitors shop differently on different days of the week. At a minimum, you want the control to run against the challengers for at least one day in the week. If you want to be more cautious, you can increase the minimum time required to two weeks.

FOR MOST TESTS, CONFIDENCE LEVEL OF 95% IS ACCEPTABLE

CLICK TO TWEET

Another thing we recommend in cases where the test includes many challengers against the control is to run the winner of the test against the control in a head to head test. This new test will help ensure that your winning challenger actually beats the original.

Quora question: What do I need to see during my A/B testing to be sure that one variation is the winner?

8. What’s a Typical Conversion Rate for Online Retailers?

In the early 2000’s, Shop.org reported the average e-commerce website converted 3% of its traffic into customers. FireClick Index data from 2015 show that an average e-commerce conversion rate  is close to 7.2% globally.

Conversion data for fashion and apparel, from Fireclick index:

Conversion data for catalog websites, from Fireclick index:

Conversion data for specialty websites, from Fireclick index:

Conversion data for outdoors and sports websites, from Fireclick index:

Conversion data for software websites, from Fireclick index:

AVERAGE E-COMMERCE CONVERSION RATE IS ALMOST 7.2% GLOBALLY

CLICK TO TWEET

Back in 2010, outdoor and sporting stores converted less than 0.5% of the traffic they receive. In 2015 these stores were converting 4.8% of their visitors into customers. This is close to 900% increase in conversion rates. 

The purpose of this data is to help you compare your site to the average conversion rates of your industry. But you shouldn’t necessarily feel better, if you are within your industry average because most online conversion rates are scant compared to brick-and-mortar rates.

Quora question:What’s a typical conversion rate for online retailers?

The post 8 QUESTIONS ABOUT INCREASING YOUR WEBSITE CONVERSION RATES FROM QUORA appeared first on Invesp.

As you focus on increasing your online conversions, you will have to spend time digging through and analyzing your analytics data. CRO is about looking at different Google Analytics metrics and putting them together to determine which pages to optimize and which hypothesis to pursue.

Let’s take a look together at some of the central features of your website data.

In this chapter, we explain the entrance, bounce rate, exit rate, average time on page, and traffic sources. Learn how to differentiate between bounce and exit rates, and find out how to interpret the behavior of your traffic segments.

This data will give you accurate directions to take on your site’s conversion optimization. Remember, conversion rate optimization is about looking at different metrics and putting them together to determine which pages to optimize and which hypothesis to pursue.

As we have mentioned before, while we have no preference for any of the analytics programs available on the market, we do recommend you use two different programs when tracking your website. Some web analytics software you might consider are Google Analytics, Clicky, and Omniture.

1. What Is An Entrance?

An entrance is an event of a visitor entering a website from an external source. Among the several terms associated with an entrance event, you can find “entrance page,” “entrance source,” and “entrance keyword.”

• Entrance page or landing page is the first page a visitor lands on when he enters a website.
• Entrance source is the source of traffic that drives a visitor to the website, either a paid campaign, organic search result, social media, or any other type of external source.
• Entrance keyword is the keyword the user searches for organically or in PPC before he clicks on a website. This keyword is critical in determining the visitor intent which directly impacts the decision to stay on the website or leave.

In October of 2011, Google announced that search data would be encrypted via SSL for logged users to Google (personal accounts on Google). This HTTPS query protects your personal search data from getting abused by third parties. While this protected users, there was a secondary impact of this decision. Google itself could no longer see this data, so it would not be able to report on it. That meant a good percentage of the keyword data would disappear from analytics reporting. While Google estimated that this would impact only 10% of the queries, most webmasters reported that up to 60% of the searches did not provide keyword data.

You can see the keyword data at a landing page level in Google Analytics by going to Behavior and selecting the Landing Pages report. From there, click on any of your top landing pages.

Here’s an example of entrances in Google Analytics;

2. What Is A Bounce Rate?

Bounce rate is the percentage of visitors who enter the website at a landing page but leave it without ever navigating to a 2nd page on the site. Google Analytics defines bounce rates as:

Bounce rate is the percentage of single-page visits or visits in which the person left your site from the entrance (landing) page. Use this metric to measure visit quality – a high bounce rate generally indicates that site entrance pages aren’t relevant to your visitors. The more compelling your landing pages, the more visitors will stay on your site and convert. You can minimize bounce rates by tailoring landing pages to each keyword and ad that you run. Landing pages should provide the information and services that were promised in the ad copy.

Based on this definition, the bounce rate is calculated as follows:

Bounce rate = Number of single page visits / Total number of entrances to that page

Analytics software reports the bounce rate for single pages as well as website average, usually calculated by averaging the bounce rate for all pages of the site. High bounce rates (more than 50 to 60%) indicate a problem in your online marketing. Assuming that your website performance is not an issue, then visitors are leaving your website because the landing page is not providing the information they need. In this case, you need to assess:

• Messaging on the source of traffic
• Continuity of messaging on the landing page
• Page relevancy to the search term

A word of caution here! Website bounce rate can be misleading most of the time. Taking an average bounce rate of thousands of pages is too generic to measure the health of your website.

How do you use bounce rate in optimizing your website?

You can choose between two approaches when using bounce rate for CRO:

1st Approach:

Focus on pages with bounce rate higher than the website average bounce rate. For example, if a website has an average bounce rate of 52%, pages with a bounce rate higher than that should be considered first for optimization. We do not like nor recommend this approach.

2nd Approach:

Establish a mark for the highest adequate bounce rate on each type of page. Evaluate pages with a bounce rate above this mark.

• For the home page of a website, we assess bounce rates above 40%.
• On a lead generation website, we focus on pages with a bounce rate of 30% or higher.
• In an e-commerce website, we look at category pages with a bounce rate higher than 20%.

Take into account that optimizing pages with high bounce rates only makes sense if these pages receive a large number of visitors. Optimizing a landing page that has a bounce rate of 80% will not make any sense if that page only gets 100 entrances per month. We recommend the following guidelines:

• With smaller websites, focus on pages with a high bounce rate if they receive a minimum of 1,000 entrances to that particular page.
• With mid to large websites, focus on pages with a high bounce rate if they receive a minimum of 10,000 entrances to that particular page.

To find the bounce rate overview of your website, or specific pages, go to reports, click on behavior, then click on overview. You’ll see the overview of your website’s bounce rate, average time on page, exit rate, etc.

If you want a specific bounce rate for a particular page, you can click on it below, or you go to behavior again, click on site content, click on landing pages, then you’ll see every page on your site and you can find out the bounce rate for a particular landing page.

Here’s an example of bounce rate in Google Analytics;

3. What Is An Exit Rate?

Exit rate is the percentage of visitors who leave a website from a particular page after visiting, at least, one other page of the site.

This theoretical definition does not always coincide with how analytics programs calculate and display exit rate.

Google Analytics, for example, uses the following formula to calculate the exit rate on a page:

Single page exit rate = Number of page exits / Number of page views

The number of page views used in the formula includes single page visits. By doing so, exit rate as displayed in Google includes bounce rate.

You might be tempted to start optimization by picking pages with exit rates higher than the site average.

A better approach, however, is first to establish a standard acceptable exit rate for different pages on the site based on their function. Then, select for optimization only the pages that meet the following criteria:

• Pages exit rate is higher than the established acceptable exit rate based on the type of page.
• The number of unique page views is higher than x [where x is dependent on the size of the site]

For some websites, x is 500 page views, for others x is 50,000 page views.

To find the exit rate overview of your website, or specific pages, go to reports, click on behavior, then click on overview. You’ll see the overview of your website’s exit rate.

Here’s an example of the exit rate in Google Analytics;

4. What Is The Difference Between Bounce And Exit Rate?

Bounce and exit rates might sound similar to each other, but they measure different visitor behaviors on a website. Bounce rate deals with a visitor’s first impressions of a website. It measures the effectiveness of a landing page design and copy to persuade visitors to remain on the site and take a particular action on the landing page. Exit rate, on the other hand, is used to detect breakdowns in process flow in the website.

In simpler terms, a bounce rate refers to the first page a visitor enters and then leaves the site.

An exit rate refers to the last page a visitor visits before he leaves the site.

Here’s a visual to drive home the point.

Bounce Rate;

Exit Rate;

5. What Is the Average Time On a Page?

Average time on page calculates how long, on average, visitors stay on a particular page. Of course, you want to see visitors spend more time per visit on your site. In most cases, more time equals more interest and a higher chance of a conversion.

To find the average time on page overview of your website, or specific pages, go to reports, click on behavior, then click on overview. You’ll see the overview of your website’s average time on page.

6. What Are Traffic Sources?

Websites receive visitors either directly, through search engines, or via referring websites. Each of these sources drives unique quality of visitors based on specific products or services. The breakdown of the traffic between the several sources varies for each website. The following figure shows how different sources of traffic drive disparate percentages of visitors to particular websites.

How different sources of traffic drive different % of visitors to the website

While we have been speaking of conversion rates in general terms here, we rarely do that on our projects.

At a minimum, conversion rates should be tracked at the level of the different sources of traffic. General and un-segmented conversion rates do not tell us a whole lot about how well a website is performing.

Typically, the breakdown of conversion rates based on the traffic source shows a higher conversion rate of direct traffic compared to referring websites or search engines. Direct visitors already recognize the brand, thus having a higher chance of converting.

7. Putting Google Analytics Metrics Together To Determine Optimization

• For each of your site’s traffic sources, examine how well the traffic converts. Sometimes traffic from Google converts better than traffic from Bing. If traffic from a search engine converts better than others, it might be worthwhile to increase PPC spending with that engine. But that is not enough.
• Also, examine the average time visitors spend on your site based on the traffic source. Remember that a higher average time means higher interest and a higher chance of conversion. If visitors from Google spend twice as much time as visitors from Yahoo, probably they are more interested in your product or service.
• Next, examine the bounce rates for each of your site traffic sources. Higher bounce rates for a source of traffic compared to other sources means that your landing page does not match the type of customers coming from that source.
• The following step is to ask yourself why there is a difference in conversion rates among the traffic sources. Are visitors using different keywords to get to your website from these different engines? Is your ranking different in search results? Analyze and determine why traffic from one source converts better compared to another source. After you come up with some possible reasons, assess what you can do to increase conversions for each of the different traffic sources.

Let’s examine the following example:

In this example, Bing is the worst engine for converting traffic. Most companies would decide not to spend any money on PPC if they see this data.

However, you should also examine the average time a visitor from Bing spends on the site. You will notice that Bing visitors spend significantly more time on the site. But why aren’t they converting? Somehow the site is not giving visitors from Bing what they are looking for, but these visitors are still willing to spend the time.

Also, notice that Bing has the highest bounce rate. That means spontaneous visitors from Bing are not able to find what they are looking for immediately after landing on the site, in contrast to visitors from Yahoo and Google.

8. Evaluating A Homepage Using Key Metrics

You can check in the video below the use of web analytics for evaluating a homepage. During one of our webinars, we explain the key metrics and how their data help you decide on your CRO focus.

Watch below as we talk about typical bounce rates and exit rates, page value, and percentage of page views. Learn how to cross references when assessing the pages of your website for conversion optimization.

Video – Evaluating Your Homepage For Conversion Rate Optimization by Invesp

Additional Resources

7 Little Known Tips to Cut Down Your Bounce Rate

Landing Page Optimization Tactics For Different Sources Of Traffic

How To Convert Different Types Of Visitors On Your Website

Landing Page Optimization Checklist: What To Do When  Your Landing Pages Are Underperforming

The post Google Analytics Metrics That Impact Conversion Rate Optimization appeared first on Invesp.

As a digital marketer, you’re likely always on the lookout for new methods and strategies to improve your website’s performance and user experience. One such technique is multivariate testing, a powerful approach that helps you optimize various aspects of your website simultaneously. In this article, we’ll explore the multivariate testing basics, providing an overview of this valuable digital marketing strategy and showing you how it can become an integral part of your website optimization techniques.

Key Takeaways

• Learn the fundamentals of multivariate testing and how it can enhance your digital marketing efforts.
• Understand the importance of multivariate testing in optimizing user experiences and conversion rates.
• Discover the differences between multivariate testing and other common testing methods, such as A/B testing.
• Gain insight into the multivariate testing process, from identifying variables to creating variations and analyzing results.
• Brush up on best practices for executing successful and effective multi-variate tests.

Disclaimer: Please note that we use button colors, fonts, and headlines as a way to clarify the concept of online testing. Successful AB and multivariate tests will include more sophisticated changes to your page.

What is Multivariate Testing?

Multivariate Testing, or MVT testing, is a testing method where multiple variations of multiple elements on a webpage are combined to determine the best combination of elements on the page to increase conversions.

It’s a key testing method in conversion rate optimization.

By using a multivariate testing tool, you can test different variations of any element on your page (headlines, images, buttons, etc.) to measure their impact on your conversion rates. The following image displays an example of how MVT testing software works.

In this example, the online testing software tests different variations of the page headline, image, and the “call to action” button:

• The original headline is tested against three other possible headlines for a total of four possible headlines on the page.
• The original image is tested against two other possible photos for a total of three possible pictures on the page.
• Three different buttons are tested against the original button on the page, for a total of four possible buttons on the page.

As a visitor arrives at a page, the multivariate testing tool picks a particular element (one of the four headlines, one of the three images, and one of the four buttons to display.)

Your team does not have to create all of the 48 designs; the multivariate testing software will swap the different variations of the landing page, create the designs automatically, and create all 48 possible variations. The following image shows four of the 48 possible designs the testing software can generate.

The total number of testing variations (also called challengers) depends on the number of elements you will test on a page (headline, image, buttons, etc.) and the number of variations you will test for each of these elements.

You can calculate the total number of challengers in a multivariate test by multiplying the number of different variations of each of the elements.

For a webpage in which we will be testing (N) number of elements, we calculate:

Total number of page variations = Number of variations of 1st element  x Number of variations of the 2nd element x Number of variations of the 3rd element x …x Number of variations of the Nth element

The number of page variations can grow very fast. Some testing software allows you to tens of thousands (sometimes millions) of variations of a single page.

Multivariate Test Examples

Let’s take the product page from Jambys as an example:

 On this page, you can test different:

• Variations of headlines.
• CTA texts.
• Variations of offers.

Let’s take an example from Bite:

 On this page, you can test:

• different variations of the headline
• displaying a different visual
• CTA text

A/B Tests vs. Multivariate Tests

Suppose you’re just getting started with running online tests; you’d probably ask yourself, what is multivariate testing, and how does it differ from other testing methods, such as A/B testing?

Let’s dive in and explore the importance of this approach in the digital marketing realm.

While A/B testing involves comparing two versions of a web page or app to determine which one performs better, multivariate testing goes a step further. It compares multiple combinations of elements on a single page, allowing marketers to analyze the interactions between these elements and understand their combined effects on key performance indicators (KPIs), such as conversion rates, click-through rates, and engagement levels.

RELATED ARTICLE: A/B Testing Vs Multivariate Testing: When To Use Multivariate Testing Or A/B Split Testing

This method often proves more efficient than A/B testing, as it enables the testing of numerous variables simultaneously. Consequently, multivariate testing can reveal subtle yet significant differences in the performance of various element combinations – insights unachievable through simple A/B comparisons.

The importance of multivariate testing lies in its ability to provide actionable insights into the relationships between various webpage elements, which are crucial for developing highly effective marketing strategies. By employing this data-driven approach, marketers can make strategic decisions backed by solid evidence rather than relying solely on intuition or experience.

How to create a successful multivariate test.

Multivariate testing software allows marketers to create and start simple tests of landing pages or web pages in a few hours.

But that is the easy part.

Many companies ultimately fail when designing successful test scenarios, assessing results, and creating meaningful follow-up tests.

Poorly designed experiments can take years to conclude. Even worse, they might not provide accurate insights into what elements convert more visitors into customers, weakening the learning curve.

Imagine a case where you plan to test different headlines on a page. You start by coming up with ten different possible variations to the headlines. Which of these ten possible headlines should you test against your original headline? Why not test all of them? Why not test variations of images, buttons, and layouts?

You will likely find yourself relying on guesswork to determine which versions to include in the test. The same logic, of course, applies to all elements you want to test on a web page.

Without being judicious with test scenarios, you might attempt to test millions of combinations.

Testing is an essential component of any conversion rate optimization project. However, it should not be the only component. Testing should only occur after the conclusion of other equally critical optimization stages, such as persona development, the voice of customer research (including polls and surveys), heuristic evaluation, usability testing, site analysis, and design and copy creation. Each element provides a building block for a highly optimized website that converts visitors into customers.

To create a successful test, you must go through the following steps:

• Evaluate the page, looking for possible problems in it
• Prioritize the issues identified on the page in terms of their impact on your conversion rate
• Create a hypothesis of how to fix some of the top issues on the page and how your fix will affect your conversion rate.
• Assert the validity of your hypothesis through multivariate or AB testing.
• Analyze the test results to determine the test hypothesis‘s correctness.
• Create a new test based on the test result.

The results from running multivariate testing

While MVT testing is powerful in helping online businesses increase conversion rates, the results you will achieve from running a single test may vary.

You can choose different approaches to design and create your multivariate test:

1.  Element level testing:

In this type of testing, you test different variations of an element on the page. For example, you test different headline variations or several images. The goal of an “element level test” is to measure that element’s impact on your conversion rate.

Element-level testing is considered the easiest type of testing. It requires the least amount of effort. In most cases, element-level testing has minimum impact on your website conversion rates.

2. Page-level testing:

In this type of testing, you test multiple page elements simultaneously. For example, you can test different landing page layouts and/or a different combination of elements and so on. Page-level testing requires more effort from the development team to implement and generates a higher impact on your conversion rates than element-level testing.

Carefully designed page-level testing can produce anywhere from 10% to 20% increase in conversion rates.

3. Visitor flow testing:

In this type of testing, you test several navigation paths for visitors within your website. For example, an e-commerce website might test single-step vs. multi-step checkout. Another example is to test different ways visitors can navigate from category pages to product pages.

Visitor flow testing can get complicated quickly. It typically requires more effort from your development team to implement. Done correctly, this type of testing will have a higher impact on your conversion rates than page-level testing.

Full Factorial or Fractional Factorial MVT. Which is best?

When people talk about multivariate testing, they usually refer to full factorial testing. In this type of multivariate test, the traffic is distributed equally among every variation.

Let’s say, for instance, you’ve got 10 variations based on the number of variables you’re testing, and the page has 1000 unique visitors. When you do your math, an equal distribution of traffic here means every variation gets 100 visitors.

Now, because each variation gets the same amount of traffic, this test type is best for determining which particular variation performed best.

Much more than finding out the winning variation, it allows you to single out the element in the variation that had the most impact in improving the conversion rate.

It’s important you know that in a winning variation, not all elements perform equally. The position of the testimonials might have had the most impact on the winning variation, while the headline pulled no weight.

Partial or Fractional Factorial Multivariate Testing.

The name gives it away. Unlike the full factorial testing that requires all of the variables to get traffic to drive results, with fractional factorial, only a subset of the variations gets the traffic.

The other variations don’t get traffic, while their conversion rates are inferred from the ones that got the traffic. 

This mvt testing type requires some hard maths for conversion rate inference and assumptions for the variation that didn’t get traffic.

Professional tip: I’ll advise you to always go with the full-factorial multivariate test. This provides you with data that is better than inferences.

Do’s and Don’ts Of Multivariate Testing

Do’s:

1. Decide which sections should be included in the test:

Not every element has the same impact on the conversion rate. 

Suppose you included a headline, a testimonial section, and a footer; you might come to realize that the footer section has little to no impact on the conversion rate of that page or user engagement. 

It’s an important factor in the conversion impact of elements and sections.

2. Preview every combination:

This is a mistake even mature experimenters make at times. They forget to preview the product of the element variations. 

This is important because you don’t want to have a variation where the header reads 20% off while the call-to-action button reads free samples. Both messages in this variation are incompatible. Previewing helps you detect these errors and remove them.

3. Estimate the traffic for significant results:

Having ten variations for a page that gets a hundred visitors will take much time to achieve statistical significance.

To avoid running tests that the results would be invalid before it’s ready, learn to estimate the amount of traffic that will be required.

Here’s a simple method to use. Use your web analytics to get an idea of the traffic that page gets. Secondly, know how many sections/elements you want to test.

The total number of page variations = Number of variations of 1st element x Number of variations of the 2nd element x Number of variations of the 3rd element x …x Number of variations of the Nth element.

Now, divide the number of traffic by the variations. If the number of traffic you get is small, then a multivariate test might not be a good fit for that page.

Don’ts

1. Don’t include a lot of sections or elements:

The more elements and sections you test, the more variations you get. This is a big deal because when you test many elements that increase the number of variations, you’ll need a lot more traffic to get statistical significance.

The Pros Of Multivariate Testing

1. Ability to test more variations.

2. You better understand the impact of individual elements on conversion rate.

3. Save time because you don’t have to conduct individual A/B tests.

The downsides of multivariate testing

If you are not careful with planning your multivariate tests, you will end up with weak-quality tests that take too long to implement and produce neither results nor insights.

You must always remember that testing (AB or multivariate) is only one component of conversion rate optimization work.

We have seen many companies that entirely relied on testing software without doing an in-depth analysis of what they were testing. Our 2007 article on the case against multivariate testing points out this example:

Let’s do some simple math.

Say you want to test six different elements on a page (headers, benefits list, hero shots, call to action, etc).

For each element, you will choose four different options. This means you will have a total of 4^6 = 4,096 possible scenarios that you will have to test.

As a general rule of thumb [being more aggressive], you will need around 200 conversions per scenario to ensure the data you are collecting is statistically significant. This translates into 4,096 * 200= 819,200 conversions.

If your website converts around 1%, you will need 819,200 * 100=81,920,000 visitors before you start gaining some confidence in your test results.

If testing 4,096 variations sound difficult, imagine how complicated matters will get by adding variation in campaigns, offers, products, and keywords. Yes, running that many test variatins is not unheard of for many larger websites.

Possible Problems When Creating A Multivariate Test.

1. Be aware of creating the test without paying close attention to the hypothesis behind it:

A faulty hypothesis means your test won’t yield good insights that will impact your business’s bottom line.

Also, creating a multivariate test without a hypothesis results in failure.

There are different templates for creating a hypothesis online, but you can use the one below;

Changing (the element being tested) from ___________ to ___________ will increase/decrease (the defined measurement).

2. Be mindful of the number of variables you are testing and their dependency on one another:

You don’t just test any element because they’re available on the page; you need to understand how each element impacts the others.

Also, remember that the more elements you’re testing increases the number of variations. This means you’ll want to test fewer elements if the traffic and conversions that page receives aren’t much.

3. Be aware of the length of time it will take to complete the test.

Before launching any test at all, it’s good practice to calculate how long it’ll take for the test to achieve statistical significance.

It means having a good grasp of certain numbers like;

• sample size
• number of visitors to the page.

Here’s the formula you can use;

Expected experiment duration = sample size/number of visitors to the tested page.

4. Traffic Allocation:

As the possible number of variations goes up, you’ll need more traffic to complete the test on time.

In the case of an A/B test, you could easily split the traffic 50-50 between the control and variation; for a multivariate test, it’s not the same. Because you’ve got more variations to test, it’ll take longer for the test to be completed because the traffic won’t be evenly split; it could be 5% to a variation, 10% to another variation, etc.

Professional tip: Calculate the sample size per variation before running the multivariate test. If the traffic on the page you want to test won’t be good enough to achieve statistical significance, I suggest you go for an A/B test.

5.  Complexity in analyzing results:

A/B tests are simpler to understand, especially in analyzing the result. You have a hypothesis driving the A/B test; you can easily deduce why the control won or didn’t. 

This is not the case for a multivariate test. A variation has many elements working simultaneously, so analyzing the result requires mental gymnastics because you need to explain how the individual elements interact. 

Multivariate Testing FAQ

What is the main difference between multivariate testing and A/B testing?

While A/B testing involves comparing two versions of a single element, multivariate testing examines multiple variables simultaneously to determine the best-performing combination of elements. A/B testing may be more suitable for simpler experiments, while multivariate testing offers deeper insights in more complex scenarios.

How can I choose which variables to test in a multivariate experiment?

Focus on elements with the greatest potential impact on user behavior or conversion rates, such as headlines, images, and call-to-action buttons. Analyze your website’s analytics to identify areas with room for improvement or elements generating the most engagement.

Do I need specialized software or platforms to set up a multivariate test?

Yes, there are various tools available for conducting multivariate tests, such as Google Optimize, Optimizely, and Visual Website Optimizer. These tools help create a controlled test environment, allocate traffic, and gather accurate data, streamlining the testing process.

How do I determine if my multivariate test results are statistically significant?

Statistical significance indicates the likelihood that the observed test results are not due to chance. Most multivariate testing tools calculate statistical significance automatically. A general guideline is to aim for a confidence level of at least 95%.

What are some best practices for effective multivariate testing?

Best practices include selecting the right metrics to evaluate, understanding sample size and test duration, avoiding common pitfalls, and ensuring test validity. Analyze your website analytics, choose variables and variations strategically, ensure proper test environment setup, and leverage the resulting insights for a data-driven marketing strategy.

The post Understanding Multivariate Testing in Digital Marketing appeared first on Invesp.

Let’s say you’re hosting a dinner party and must decide between spaghetti and tacos. A quick vote settles the issue—easy, right? Now, imagine you want to test whether garlic bread or salad pairs better with spaghetti or if salsa or guacamole steals the show with the tacos. 

Suddenly, your testing and experimenting get a lot more complex with multiple combination possibilities. 

This is the difference between A/B and multivariate testing. A/B is like choosing between two dishes—simple and focused. Multivariate goes deeper, testing how combinations of elements work together.

This article will explore when to use each method to help you make smarter, data-driven decisions. Let’s find out! 

Understanding Multivariate Testing vs A/B Testing 

Both A/B testing and multivariate testing are popular methods for testing hypotheses in marketing, web design, product development, and user experience optimization. 

While both help you improve performance through data-driven decision-making, they differ in complexity, scope, and applicability. 

What is A/B Testing?

A/B testing, or split testing, is a simple method for comparing two versions of a single variable to determine which performs better. You can use it to test a hypothesis about a single element on a webpage, app, or marketing campaign. 

For example, a company might want to compare the effectiveness of two different call-to-action (CTA) button colors on a landing page.

Airbnb regularly uses A/B testing to optimize its website and app interfaces.

In 2018, the holiday rental company used A/B testing to optimize a pricing feature that offered hosts personalized price suggestions based on demand, competition, and location. The test revealed higher booking rates for hosts using the feature, leading to its global rollout.

Key advantages of A/B testing: 

• Simplicity: A/B testing is straightforward to set up and analyze.
• Clear results: With only one variable being tested, results are easy to interpret.
• Low traffic requirements: A/B tests can be effective even with a lower traffic volume.

What is Multivariate Testing?

Multivariate testing (MVT) takes A/B testing a step further by testing multiple variables simultaneously. Instead of testing just one change, MVT tests combinations of changes across multiple elements of a webpage or marketing campaign. The goal is to understand how different combinations of elements impact the overall performance.

For example, a business could simultaneously test different combinations of headline text, images, and button designs to determine which combination delivers the highest conversion rate.

Key advantages of multivariate testing: 

• Comprehensive analysis: MVT allows businesses to test a range of variations simultaneously, offering a deeper understanding of the interactions between different elements.
• Efficiency with high traffic: MVT is ideal for websites or platforms with high traffic; the more data you have, the more reliable the results are.
• Better insights: MVT can show how different elements interact with each other, revealing hidden opportunities for optimization that A/B testing alone might miss.

Difference Between Multivariate Testing vs. A/B Testing 

To truly determine which testing tools to use, you must first discover their differences. Here’s an overview of the common differences between multivariate and A/B testing. 

• Scope of testing: This refers to the number and types of elements being tested in an experiment and how broad or narrow the focus of the test is.
  • A/B testing: Focuses on testing a single element, such as a button color or headline copy.
  • Multivariate testing Tests multiple elements simultaneously, such as the headline, button color, and image placement, to understand how combinations of changes affect performance.

• Complexity:
  It refers to the ease or difficulty of designing, implementing, and analyzing the test.

• A/B Testing: Simpler to set up and analyze, especially for beginners. For instance, changing the text of a call-to-action button and comparing its performance with the original button is straightforward to implement and evaluate. It’s ideal for beginners or teams with limited resources.
  
• Multivariate testing: More complex due to the more variations and combinations involved. For example, if you’re testing three headlines, three button colors, and two images, there are 18 combinations to analyze. This requires advanced tools and statistical analysis to determine which combinations are most effective.

• Traffic requirements:
  It refers to the amount of user activity (e.g., website visits, clicks, or impressions) needed to achieve statistically significant results.

• A/B testing: Works well with lower traffic volumes, as you only test one variation at a time. For instance, if you have 1,000 visitors to your website daily, you can run an A/B test and get reliable results relatively quickly since only two groups are being compared.
• Multivariate testing: Requires a higher traffic volume to ensure that the test reaches statistical significance for each variation and combination. Let’s say you’re testing four combinations, each needing at least 1,000 visitors. In that case, you’ll require at least 4,000 visitors to gather meaningful insights. This makes MVT better suited for high-traffic websites or apps.

• Result analysis:
  This entails the process of interpreting the data collected during the test and understanding which variation or combination performed best.

• A/B testing: Results are usually clear-cut because only one variable is being tested.  For example, if Version A of your landing page converts at 20% and Version B converts at 25%, you can quickly identify the winner.
• Multivariate testing: Results require more advanced analysis to understand which combinations of elements are most effective. For instance, if you’re testing multiple elements, you’ll need to understand the individual performance of each element and how they interact. In this case, you’ll need specialized software and statistical models to analyze the results accurately.

• Duration:
  Here, duration refers to how long a test needs to run to collect enough data for statistically significant results.

• A/B testing: Typically faster, as it tests fewer combinations. For instance, an A/B test with two versions of an email subject line might take a few days to conclude.
• Multivariate testing: Due to its complexity and the number of variations, it may take longer to reach conclusions. For instance, testing multiple variations on a homepage might take weeks or even months, depending on the traffic volume and the number of combinations being analyzed.

When to Use A/B Testing Tools vs. Multivariate Testing Tools

Now that you know the differences, it’s time to choose an option to run your experiments. Choosing between A/B testing and multivariate testing tools depends on your goals, resources, traffic, and the complexity of the changes you want to test. Here’s a closer look at when and why you’d use each testing tool.

When to Use A/B Testing Tools

A/B testing is like a “trial by duel”—you pit two versions of a single element against each other and see which one wins. It’s straightforward, quick, and ideal for specific use cases.

• Simple scenarios: A/B testing is perfect when you want to simultaneously test one variable or change. Examples include:
  • Testing the color of a CTA button (e.g., red vs. green) to see which gets more clicks.
  • Comparing two different headlines for an email campaign.

• When there’s limited traffic: A/B testing is the better option for businesses with low website traffic. Because it only compares two variations, it requires fewer visitors to reach statistically significant results.

When you need quick and precise results: A/B tests work well when you need quick and actionable results. Since you’re testing just one change, it’s easy to interpret the outcome and implement the winning variation.

For example, during the 2008 U.S. presidential election, Obama’s campaign used A/B testing to optimize donation pages and email subject lines for quick, precise results. Simple changes like testing headlines, images, and workflows boosted engagement, added 2.88 million email signups, and generated $60 million in donations, showing how A/B testing comes in handy when you want outcomes quickly and clearly.

• You require low complexity in the test setup: If your team is new to testing or doesn’t have advanced analytics expertise, A/B testing is easier to set up and manage. You don’t need sophisticated web analytics tools or data science knowledge—just a hypothesis and a clear goal.

Now, let’s see the other side. 

When to Use Multivariate Testing Tools

Multivariate testing, on the other hand, is like playing chess—it involves evaluating multiple web page elements simultaneously and understanding how they interact. This approach is suited for more complex scenarios.

• Complex scenarios: If you’re making several changes simultaneously (e.g., modifying headlines, images, and button colors), multivariate testing is the right choice. It helps you determine which combination of elements works best.

Pro tip: Use multivariate testing when you suspect that two or more elements on a page interact with each other. For example, a headline and an image might work well together, but they might not perform as effectively separately.

• High-traffic websites: Multivariate testing requires a large traffic volume to split users across all the variations and still produce statistically significant results. It’s a numbers game—the more variations, the more visitors you need.

• When trying to understand the interaction between multiple variants or elements: One of the most significant advantages of multivariate testing is that it reveals how different elements interact. For example:
  • A CTA button might perform well with one headline but poorly with another.
  • Testing combinations of layout, color scheme, and content placement can uncover synergies that A/B testing can’t.

Over to You!

Both A/B and multivariate testing methods have their strengths, but choosing the right tool depends on the scenario, traffic volume, complexity, and long-term goals. 

A/B testing is a practical choice for businesses looking for quick wins. For those with the resources and traffic to handle complex analyses, multivariate testing offers invaluable insights into how multiple changes interact. 

The post The Difference Between A/B Testing and Multivariate Testing appeared first on Invesp.

It is time now to select an AB or multivariate testing platform to start running your tests. There are over thirty different tools available in the marketplace for you to choose from. So, you might ask:

• What is the right split testing tool for my website?
• Will I affect the success of my conversion optimization program by choosing one particular tool?

The market is full of testing tools that range from simple free tools to sophisticated enterprise split testing software. This article will walk you through the pros and cons of different split testing tools available.

We will also suggest criteria to help you select the software that best fits your needs, considering ease of use, the agility of results, cost of running, testing support, operational issues, and testing analytics.

The different types of AB testing tools available

There’s no shortage of conversion rate optimization tools in the market. Cancelled 

CRO tools can help you understand the performance of your website by showing you how visitors use it. Simply put, these tools collect data about your visitors and website, therefore helping you identify problematic areas on your website that are making visitors struggle.

Before I give you a list of the best CRO tools out there, it’s important to mention that these tools fall into three categories:

1. Web analytics CRO tools
2.  Behavior analytics CRO tools
3.  CRO testing tools
4. Web analytics CRO tools: Show you what’s happening on your site

You can’t improve or optimize your site’s conversion rate if you don’t know what’s happening on it, can you?

This is where Web analytics CRO tools come in.

Web analytics CRO tools help you understand your site in terms of numbers. They offer you quantitative data to answer questions like:

1. How long are visitors staying on my site?
2.  Which landing pages have the highest bounce rate?
3.  Which stage of the conversion funnel am I losing visitors?

Knowing the answers to the above questions is important as it gives your optimization efforts a high chance of being successful.

1. Behavior analytics CRO tools: Shows the behavior of your website visitors

To improve your website’s conversion rate, you need to provide an excellent experience that is in line with what your visitors want. So, how do you know what visitors want?

Well, one way of doing that is by looking at how they behave on your website.

Unlike web analytics tools, user behavior analytics tools can show you exactly what visitors are doing on your website by collecting qualitative data. You can rely on behavior analytics tools to answer the following questions:

1. What’s causing visitors to leave the site without purchasing?
2.  Are visitors finding what items they are looking for?
3.  Which website areas are frustrating visitors?

Without knowing the answers to the above questions, your optimization efforts are going to be fruitless.

2. CRO testing tools: Allow you to compare and measure website changes

Testing is an important piece of the conversion rate optimization puzzle. The process usually involves testing two or more design versions of a web page to determine which version has more conversions.

Before you permanently implement changes on your website, you need to measure and see if they make your website better or not. This is why you need to have a reliable A/B testing tool.

RELATED ARTICLES: Top 10 A/B Testing Tools Used By CRO Experts in 2021

CRO testing tools can be used to test different web elements such as the position of a call to action button, web form size/placement, landing pages, website copy, website navigation, checkout process of an eCommerce site, pricing models, etc.

Now that you know how these tools work, here’s a list of the best conversion rate optimization tools:

• Google Analytics 360
•  FigPii
•  Heap
•  Adobe Analytics
•  Crazy Egg
•  FullStory
•  Istanpage
•  VWO

Some of the above tools have similar features and only differ in price plans. However, our CRO team relies on FigPii when it comes to understanding user behavior, and it is the only tool on that list that has a recommendation feature that pinpoints problem areas you should fix right away on your site. We also use Google Analytics as our main tool for tracking traffic data.

Classifying A/B testing tools based on price

Split testing tools vary in price from free of charge to tools that can cost tens of thousands of dollars.

1. Tools priced based on site metrics

These tools are priced based on specific website metrics. Different tools use different metrics to set the price, including:

1. The number of tested visitors in a time period (typically a month)
2. Number of the website’s monthly visitors
3. Number of monthly page views
4. The number of calls to the testing tool.

These tools will vary in price and range anywhere from $50/month to tens of thousands of dollars.

2. Free AB testing tools

These tools provide the customer the ability to conduct split testing free of charge, but they offer limited sets of features, including a smaller set of segmentation, targeting and reporting functions. The leading player in this space is Google Optimize which continues to add more features on a regular basis.

Classifying AB testing tools based on testing technology (client side vs. server side)

There are two different types of ways testing tools are used on a website:

1. Client-side testing tools

In these tools, the testing scripts are executed by the browser of the website visitor. When a visitor comes to a webpage, the testing software javascript is executed, and the particulars of the test are loaded on the page.

Client-side testing tools typically come with a point-and-click visual editor that allows users to manipulate a web page.

In theory, these tools require limited involvement from the user’s technical team. Creating new designs with these tools is much faster compared to their counterparts.

Client-side testing tools are great for creating new designs for one page, but they are less suited to conducting more complex testing, such as visitor flow tests. The cornerstone “visual editor” of these platforms breaks when modifying websites that rely heavily on JavaScript or dynamic content.

These tools come with a low-cost monthly subscription starting at $50 per month. The subscription cost will vary based on the number of visitors a website receives in a month.

Amongst client-side testing tools, you can find:

• Optimizely
• Visual Website Optimizer
• Convert
• Omni convert

2. Server-side testing tool

In these tools, the testing script runs on the testing platform as opposed to the visitors’ browser. The visitor’s browser gets the final version to be displayed.

There are several ways testing is implemented using server-side tools. Adobe tests and tags, for example, rely on the markings made by users on specific areas of the tested webpage. Variations of the marked areas are then created in the testing software by the optimization or development team. Each new test will require involvement from the development team.

Server-side testing tools frequently involve custom configuration to retrieve meaningful visitor tracking data. They also require organizations to have a full-time dedicated resource(s) to manage the software and to analyze the results.

While many of these tools do not come with an easy-to-use “visual editor” to make quick changes on a webpage, they do provide detailed reporting and targeting features.

Monthly plans for these tools start around $1,000 and vary based on the number of page views a customer uses in a month. In addition to the software cost, we estimate a $100,000 annual in-house cost of running tests on enterprise testing packages.

Server-side testing platforms are useful in conducting sophisticated testing such as multi-page testing, mobile application testing or visitor flow testing. However, they are overkill when doing quick A/B testing for a webpage design due to the substantial cost associated with the deployment of each test. The leading player in this field is Adobe Test and Target. 

Selecting the right testing tool for your website

With many tools available, the following criteria can help you choose the best split-testing software to meet your requirements.

1. Ease of use: How fast can you deploy a test?

We evaluate the ability of a testing tool to deploy tests quickly as a way of measuring its ease of use.

When you are conducting a conversion optimization program, your goal is to implement tests efficiently and not to have technology as a barrier to your efforts of increasing conversion rates.

Testing tools vary tremendously when it comes to ease of use. Some of the more sophisticated tools require complicated test setups, which can take days. Client-side testing tools outperform any of their counterparts in this area.

2. What is the cost of running the AB testing software?

The cost of running testing software varies from zero to thousands of dollars per month.

In addition to the cost of running the software itself, there might be additional required investments. Some split testing software is too complicated and will require having full-time staff to manage in addition to involvement from your development team. That, of course, will increase the cost of running your testing program.

3. Testing support

Sooner or later, you will need some help from the testing software company. Many smaller companies ignore this one area when they are doing the initial assessment of the testing packages. We highly recommend evaluating the different alternatives available.

4. Operational issues (scale, performance, and high availability)

How long does it take the testing software to load up a particular design? Does the testing software use CDN (content delivery network) to deliver designs to visitors who live in different parts of the world?

We highly recommend evaluating the response time for each of the different software packages. Avoid any testing package that takes longer than 500 milliseconds to deliver a design.

5. Testing analytics

In addition to reporting the conversion rate for each variation, some split testing platforms will also report the following metrics for each of the variations:

• Revenue per visit
• Bounce rate
• Exit rate
• Visitor type (new vs. repeat)
• Traffic source
• Traffic medium

This data is essential to understand the impact of the new designs on all aspects of the visitor experience on your website. We recommend evaluating what metrics the testing software reports in a test as that can impact your final analysis.

6.  What multivariate approach does the software use?

This point is particular in evaluating multivariate testing software. When launching a multivariate test, the software will display the original and new variations to different visitors to determine the winning design. To select the winning design, the software will use either full factorial or fractional factorial testing:

Full Factorial Testing

The testing software will test all of the different combinations of elements and their alternatives. So, if a test has four elements and each element has three different combinations in it, the testing software will test all possible 3^4= 81 designs.

Fractional Factorial testing (Taguchi method)

The testing software will select a subset of all possible combinations of the different elements and their alternatives. So, in our example above, the testing software will fully test a smaller set (less than 30) to determine the winner. Fractional factorial testing allows tests to run faster since they do not fully test all possible combinations. Critics of this method point out that it is less accurate compared to full factorial testing.

The debate between full factorial vs. fractional factorial has been going on for years.

Most MVT testing software uses full factorial in determining the winner of an experiment or a mix between the two different methods. While this might be a sticking point for some conversion experts, for 90% of online businesses, it is not a critical point in selecting which software to use.

The post Comparing Multivariate & AB Testing Tools appeared first on Invesp.

The scene has changed a lot since we first started doing conversion optimization back in 2006.

At that time, many marketers and C-suite executive were not convinced that there is a lot they can do about their website conversion rate. Things are much different nowadays with many companies attempting to deploy AB tests on their websites.

However, the majority of companies continue to primarily invest in visitor-driving activities and set lower budgets for converting visitors into customers.

Invesp’s 2017 state of the conversion rate optimization survey showed that majority of companies are allocating less than 8% of their digital marketing budgets to conversion rate optimization.

If your company is starting with A/B or multivariate testing, the following guide will help you avoid 16 common mistakes we have seen companies fall into during their first years of testing.

1. Testing the wrong page

If you have a website with a large number of pages, a large number of visitors and a large number of conversions, then the possibilities are endless.

For an e-commerce website, you can start your conversion program by focusing at the top of the funnel pages, such as the homepage or category pages. You can also start at the bottom of the funnel pages, such as the cart or checkout pages.

If you are a lead generation website, you can start by optimizing your homepage, landing pages, or contact pages.

When you choose the wrong page, you invest time and money in a page that might not have a real impact on your bottom line. Surprisingly, many companies continue to pay little attention to the importance of selecting the right page to perform a split test on.

We covered creating a  prioritized research opportunities list in a previous chapter.

Our research opportunities list for different CRO projects contains on average anywhere from 150 to 250 items on it. This list is very powerful in guiding your CRO work. You will no longer pick random pages and random items to test. Every test you create is packed up by research.

2. Testing without creating visitor personas

Testing gives your visitors a voice in your website design process. It validates what works on your site and what does not. But, before you start testing, you must understand your visitors at an intimate level to create split tests that appeal to them.

We discussed the process of creating personas in several of our webinars and wrote about it in our book Conversion Optimization: The Art and Science of Converting Prospects to Customers.

Most companies have a decent knowledge of their target market. The challenge is how to convey that knowledge into actionable marketing insights on your website.

Personas play a crucial role in this process.

Let’s say you are an e-commerce website that sells gift baskets online. You have worked with your marketing team to define two different segments of your target market:

• B2C segment: white females, age 38 to 48, college educated, with annual income above $75,000. Your average order value for this segment is $125.
• B2B segment: corporate clients, with the purchase decision made by an executive. These companies are generating between 10 to 50 million dollars in annual revenue. Your average order value for this segment is $930.

These two segments provide a typical format of market segmentation and raise the central question of marketing design.

How can you design your website to appeal to these two distinctly different segments?

Creating personas will help you identify with each of the segments at more detailed level. At the end of the persona creation process for this website, you could end up with eight different personas. Let’s take one of them as an example:

As you can see, Suzan resembles your target market. However, she removes the abstract nature of the marketing data. As you start designing different sections of the website, you will be thinking of Suzan and how she would react to them.

You might be thinking to yourself, “this is all great, but how does that impact my testing?”

Proper and successful testing uses personas in creating design variations that challenge your existing baseline (control).

How would you create a homepage test when you are thinking of Suzan?

• She is a caring persona, so you can test different headlines that appeal to her
• She is looking for unique gifts, so you can test different designs that emphasize the uniqueness of the products you offer
• Price is an essential motivation for Suzan, so you can test different designs that emphasize pricing

3. Testing without a hypothesis

A testing hypothesis is a predictive statement about possible problems on a webpage, and the impact that fixing them may have on your KPI.

In our experience, there are two different types of issues companies run into when it comes to creating hypotheses for different tests:

1. Many testers dismiss hypothesis as a luxury. So, they create a test that generates results (positive or negative), but when you ask them about the rationale behind the test, they cannot explain it.
2. Others understand the importance of hypothesis but they create meaningless statements that do not have a real impact on testing.

Getting disciplined about creating test hypotheses will magnify the impact of your test results.

How do you come up with a test hypothesis in the first place?

Each conversion problems on your website should include three different elements in it:

1. A problem statement
2. How the problem was identified
3. An initial hypothesis on how to fix the problem

As discussed previously, you will have to solidify your initial hypothesis and create a concrete hypothesis. 

Of course, you can always ignore the process, throw things at the wall, and pray that one of your new designs will beat the control. And yes, it might work some of the time. But it will not work most of the time. And it will indeed never work if you are looking for repeatable and sustainable results.

4. Not considering mobile traffic

Many websites are getting a higher percentage of their website traffic on mobile devices. Most of our European clients are reporting anywhere that 50 to 70% of their traffic arriving at the site using a mobile device.

You can only expect these numbers to grow over the next few years.

So, what should you do?

1. Determine the percentage of your website traffic that uses mobile devices for browsing
2. Determine the top ten devices visitors are using to browse your website
3. Evaluate the behavior of mobile traffic compared to desktop traffic for all of the website funnels

These three steps should give you a lot of action points to take on your website.

Let’s see an example:

For a CRO expert, the data above provides a wealth of information. When examining the flow from category to product pages, you will notice that Bing paid traffic outperforms all other types of paid traffic. It only comes second to the email campaigns. On the other hand, Facebook paid traffic underperforms compared to other channels.

If you consider the category to product flow, mobile traffic outperforms desktop for all traffic sources and mediums. This indicates that category page design for mobile is acceptable to users. However, notice the drop in mobile performance when visitors get to product pages.

Things get even worse when we start evaluating mobile checkout. The numbers are telling us that the mobile checkout has much higher abandonment rates compared to desktop.

The real investigation begins here. Why is it that mobile checkout underperforms?

Is it a problem with the website design or usability? Or is it the case that visitors use their mobile devices to browse the website and then when they are ready to place an order, they use their desktops?

5. Mobile vs desktop experience: the aftermath of testing

What brings visitors to your website on a mobile device is vastly different from what brings them to your website on a desktop. And that is the reason you should run separate mobile and desktop tests on different pages of your website. That is a no-brainer.

But, how do you handle instances where a winning design for a mobile test on a particular page is very different from the winning design for a desktop expierement for that same page?

If you are lucky, your technical team is able to serve two completely different experiences to website visitors based on the type of device they are using to navigate. Having two different designs (mobile and desktop) for the same page can cause major development problems. And as a result, we have seen companies use a single layout for both desktop and mobile.

That does not solve the issue, it ignores it. So, do not do that. Discuss your options with your technical team and figure out a way to handle this issue.

6. Not running separate test for new vs. returning visitors

Returning visitors are loyal to your site. They are used to it with all of its conversion problems!

Humans are creatures of habit. In many instances, we find that returning visitors convert at a lower rate when we introduce new and better designs. This has been explained in the theory of momentum behavior.

For this reason, we always recommend testing new website designs with new visitors.

Before you test new designs, you need to assess how returning visitors interact with your website compared to new visitors. If you run Google Analytics, you can view visitor behavior by adding a visitor segment to most reports.

Let’s examine how visitors view different pages on your website.

After you log in to Google Analytics, navigate to Behavior > Site Content > All Pages

Google will display the page report showing different metrics for your website.

To view how returning vs. new visitors interact with your website, apply segmentation to the report:

Select “New Users” and “Returning Users”

Google will now display the same report segmented by the type of user:

Notice the difference for this particular website in terms of bounce and exit rates for returning visitors compared to new visitors.

Most websites show anywhere from 15% to 30% difference in metrics between the two visitor segments.

If the difference in metrics between new and returning visitors is less than 10%, then you should scrutinize your design to understand why returning visitors react the same way as new visitors.

Next, create a test specifically for new visitors. Most split testing platforms will allow you to segment visitors.

If your testing software does not support this feature, then switch to something else!

7. Ignoring different traffic sources

Visitors land on your website from diverse traffic sources and mediums. You will notice that visitors from different sources interact with your site in different ways.

Trust is one of the first and most significant influences to whether a visitor is persuaded to convert on your website or not. One of the subelements of trust is continuity.

Continuity means you must maintain a consistent messaging and design from the traffic source/medium to the landing page.

Creating the same test for different traffic sources ignores that visitors might see different messaging or designs before landing on your website.

To assess how traffic sources can impact your website, follow these three steps:

1. Understand how different traffic sources/mediums interact with your site
2. Analyze reasons for different visitor interaction (if any)
3. Create separate tests based on the traffic sources/mediums

Let’s see how this is done in Google Analytics.

First, generate the traffic sources/mediums report. To do this, navigate to Acquisition > All Traffic > Sources/Mediums

Google will generate the report for you. Now, it is difficult to assess all your traffic sources. We recommend evaluating either your top ten or the traffic sources that drive more than 50,000 visitors to your website.

As you analyze the report, examine the following metrics:

• Bounce rate
• Exit rate
• Pages/Session
• The conversion rate for different goals

Ask questions such as:

• Which traffic sources are driving the highest conversions?
• Do you see high bounce rates for paid traffic?
• Which traffic sources are driving the lowest bounce and exit rates?

These steps help in determining if there is a difference in visitor behavior for different traffic sources.

Next, you need to determine the causes of such different behavior.

This will require examining each traffic source:

• What are visitors seeing before landing on your website?
• Do you have control over the display/messaging that visitors see before landing on the website?
• Can you change your landing page to maintain continuity in the experience?

8- Trying to do too much in one test

This is one of the mistakes we fell in during the first year we conducted A/B and MVT testing. Our clients wanted to see large-scale experiments. Small tests did not convince them. And instead of explaining to them what we were trying to accomplish, we created extensive tests where we changed too many elements at the same time.

Most of our testing produced excellent results.

As a matter of fact, in 2007, 82% of our testing generated an uplift in conversions and 78% of our test produced more than 12% increase in conversion rates.

These are amazing results. So, what was the problem?

Since we were making too many changes on a page for every test, we could not isolate what exactly was generating the uplift. So, our team could only make assumptions. Every test generated an increase in conversions due to seven to nine different factors.

This approach might be okay if you are looking to do two to three tests and get done with testing (we do not recommend it).

But if you are looking for a long-term testing program that takes a company from 2% to 9% conversion rate, that approach will undoubtedly fail.

Our approach twelve years later looks tremendously different. Our testing programs are laser sharp focused.

Every test we perform relies on a hypothesis. The hypothesis can introduce several changes at the same time to the page, but research backs all changes. A single hypothesis about enhancing social proof could be implemented by deploying a new headline, new image, and a new layout. Yes, there are several elements but they all support the overall hypothesis.

Let’s take a recent evaluation we did for a top IRCE 500 retailer. Their product pages suffered from high bounce rates. Visitors were clicking on PPC ads, getting to the product pages, checking out prices for different products, and then leaving. Visitors were price shopping.

Instead of doing a single test for the product pages, we did five rounds of testing:

1. A split test focused on the value proposition of the website
2. A split test focused on price-based incentives
3. A split test focused on urgency-based incentives
4. A split test focused on scarcity-based incentives
5. A split test focused on social proof

What were the results of running these tests?

The website increased revenue (not conversions!) by over 18%.

9- Running A/B tests when you are not ready

Everyone is talking about A/B and multivariate testing. The idea of being able to increase your website revenue without having to drive more visitors to the website is fantastic.

But A/B testing might not work for every website. Certainly, multivariate testing is not for every site.

Testing might not work for you in two instances:

• when you do not have enough conversions
• when you do not have the mindset required to run a successful CRO program.

1. You do not have enough conversions

If you do not have enough traffic coming to your website or enough conversions, then split testing will not work for you.

A small A/B test that has one challenger to an original design requires that the particular page you plan to test to generate a minimum of 500 conversions per month. If that particular page is getting less than 500 conversions, then your tests might run for too long without concluding.

We typically do not start A/B testing with a client unless the website has 500 conversions per month. Our standard program requires that clients cross the 800 conversions per platform (desktop or mobile).

Multivariate testing requires more conversions and more traffic compared to AB testing. Do not consider MVT testing unless your website has 10,000 conversions per month.

2.  You do not have the mindset needed to run a successful conversion program

While the first reason why a conversion program fails is straightforward to figure out (you look at your monthly conversions), the second problem is more difficult to assess.

The truth is that not every organization or business is ready for testing.

Testing requires you to admit that visitors may hate your existing website design.

Testing requires you to admit that some designs which you hate will generate more conversions/sales for you.

Testing requires surrendering the final design decision to your visitors.

At the surface, every business owner or top executive will say that they are focused on their revenue. But after running over 500+ conversion optimization projects with over 10,000+ spit tests in them, we can simply state that this is not the case.

We have seen business owners reject the results of testing that generated 32% uplift in conversions with 99% confidence because they liked the original design and could not bear the idea of changing it.

We have seen executives reject the results of testing that generated 25% uplift in conversions with 95% confidence because they hated the winning design.

We have also seen testing programs fail because, while the CEO of the company was committed to testing, the team was not sold on the idea.

For many companies, split testing requires a complete culture shift. To make sure its results will have a direct and significant impact on your bottom line, everyone – and we do mean everyone – must be completely committed to it.

10- Calling the test too soon

You run the test and, a few days later, your testing software declares a winning design. Everyone is excited about the uplift. You stop the test and make the winning challenger your default design.

You expect to see your conversion rate increase. But it does not.

Why?

Because the test was concluded too quickly. Most testing software declares winners after achieving a 95% confidence level regardless of the number of visitors and conversions reported for different variations.

Many split testing platforms do not take into account the number of conversions the original design and the variations recorded. If the test is allowed to run long enough, you will notice that the observed uplift slowly disappears.

So, how do you deal with this?

1. Regardless of the setup in the split testing platform, make sure you adjust it to require a minimum of 500 conversions for the original design and the winning challenger.
2. Run your tests for a minimum of seven days so that your test will run on every day of the week to account for fluctuations might happen between different days (We typically run tests for a minimum of two weeks depending on the business cycle for the particular business).

11- Calling the test too late

You have no control over external factors when you run a split test. These factors can pollute the results of your testing program.

There are three different categories of external factors that could impact your experiments:

• General market trends: a sudden downturn in the economy, for example
• Competitive factors: a competitor running a massive marketing campaign
• Traffic factors: change in the quality of organic or paid traffic

All of these factors could impact the results of your testing at no-fault of the testing program itself. For this reason, we highly recommend limiting the time span of any split test for no longer than 30 days.

We have seen companies require tests to run for two to three months trying to achieve confidence on a test. In the process, they pollute their testing data.

In fixed horizon testing, you should calculate the number of visitors required to go through the test before launching any experiment. In this approach, you will use your current conversion rate, significance level, and the expected uplift to determine required sample size.

A final note on significance levels: Remember that achieving 95%significance is not a goal set in stone. Significance levels provide a general trend line that the test results are positive and consistent.

12- When technology becomes a problem

The purpose of testing is to increase your conversion rates and your revenue. Developers sometimes struggle to focus on this fact, especially when they get fascinated by a particular piece of software that complicates test implementation.

As a goal, most split tests should not take longer than three to five days to implement.

Smaller tests, with few changes to a page, might take a few hours to implement.

You must keep in mind that the first two split tests you deploy on your website will take a little longer to implement as your development team gets used to the split testing platform you selected.

However, if you notice that, over a six-month period, all of your tests are taking over a week to ten days to implement, then you MUST assess the cause for the delay:

• Is the testing platform too complicated and an overkill for the type of testing you are doing?
• Is your website or application code not developed with proper standards which are causing the delays?
• Does your development team have a good handle on implementing tests or are they struggling with every test?
• Are you creating and deploying complicated tests that require significant time to develop?

Golden rule: A testing program should deploy two to six tests per month.

13- Running successful simultaneous split tests

Launching simultaneous split tests expedites your testing program. Instead of waiting for each test to conclude, you run several tests at the same time.

If you are running simultaneous split tests in separate swimlanes, then this will not cause you an issue.

However, if the same visitors are navigating through your website and seeing different tests, you could be cannibalizing your testing data.

Imagine the scenario of running a test on the homepage, with three challengers to the original design. A visitor might view any of the following configurations:

At the same time, you run a test on the product pages, with two challengers to the original design. A visitor might view any of the following configurations:

In this scenario, as the visitor navigates from the homepage to the product pages, he can see any of the following combinations of designs:

Your two separate and straightforward tests end up impacting each other.

So, does that mean that you should never run simultaneous split tests?

No, we run split tests simultaneously on projects with the following guidelines:

1. we target different traffic and device types to split the traffic.
2. After we declare a winner for a test, we run the winner against the control in a head-to-head match in a period where no other tests are running.

14- Missing the insights

The real impact of conversion optimization takes place after you conclude each of your split tests. This impact is by no means limited to an increase in conversion rates.

Yes, seeing an increase in conversion rates is fantastic!

But there is a secret to amplifying the results of any test by deploying the marketing lessons you learned across different channels and verticals.

Let’s put this in perspective.

We worked with the largest satellite provider in North America helping them tests different landing pages for their PPC campaign. The testing program was very successful, generating significant increases in conversion rates.

As we implemented the testing program, their digital marketing director asked if we could use the same lessons we learned from split testing in their newspaper advertising.

This was a new challenge.

Would offline buyers react the same way to advertising as online buyers?

There was only one way to find out. We had to test it.

Each test was built using a hypothesis. We applied that hypothesis to both online tests as well as the newspaper advertising. We ran three different tests both online and offline.

Each test generated an uplift in conversions in both online and offline.

But things did not stop there.

We then applied the same lessons to mailers which the company sent to residential addresses.

Again, we saw uplifts in conversions.

If you follow a conversion optimization methodology, then you will be able to take lessons from your testing and apply them again and again.

15- Your follow up tests

Not every test will generate an increase in conversions. As you conclude your experiments, you will have to decide on the next steps for each test. I have seen clients and teams get frustrated because they are not able to increase conversions on a particular page. So, they move on to other sections on the website.

The post-test analysis should determine one of four possible next steps:

1. Iterate on the test: conduct further tests on the page, fine-tuning the test design or trying different implementation to the original hypothesis.
2. Test new research opportunities: test results point to a new hypothesis that should be tested on the page.
3. Investigate further: test results show that you need to dig for deeper insights before you determine next steps.
4. Pivot: Your testing data and analysis clearly indicate that you had the wrong hypothesis to start with. Time to look for other issues on the page.

16- Not documenting everything

A conversion optimization program is documentation intensive. You should record every little detail.

You must document:

• Your qualitative analysis research and findings
• Your quantitative analysis and conclusions
• Every page analysis you conduct
• Every hypothesis you make
• Screen captures of every design you deploy
• Testing data
• Post-test analysis

Many companies do not pay close attention to the importance of documentation. They discover its importance when they come back to the testing results a few months later. When there is no documentation or when it is scattered in across multiple emails, they struggle to remember why they made a particular change. But it is too late.

From the very start of any CRO project, decide on the method you will use to document everything and use it thoroughly.

The post 16 mistakes that will kill your A/B testing (and what you can do about them) appeared first on Invesp.