{"id":14705,"date":"2021-03-17T14:42:21","date_gmt":"2021-03-17T19:42:21","guid":{"rendered":"https:\/\/www.invespcro.com\/blog\/?p=14705"},"modified":"2023-08-17T07:34:43","modified_gmt":"2023-08-17T07:34:43","slug":"expert-advice-on-developing-a-hypothesis-for-marketing-experimentation","status":"publish","type":"post","link":"https:\/\/www.invespcro.com\/blog\/expert-advice-on-developing-a-hypothesis-for-marketing-experimentation\/","title":{"rendered":"Expert Advice on Developing a Hypothesis for Marketing Experimentation\u00a0"},"content":{"rendered":"<span class=\"span-reading-time rt-reading-time\" style=\"display: block;\"><span class=\"rt-label rt-prefix\">Reading Time: <\/span> <span class=\"rt-time\"> 11<\/span> <span class=\"rt-label rt-postfix\">minutes<\/span><\/span>\n<p><span style=\"font-weight: 400;\">Every marketing experimentation process has to have a solid hypothesis.&nbsp;<\/span><\/p>\n\n\n\n<p><span style=\"font-weight: 400;\">That\u2019s a must \u2013 unless you want to be <\/span><span style=\"font-weight: 400;\">roaming in the dark and heading towards a dead-end in your experimentation program.<\/span><\/p>\n\n\n\n<p><span style=\"font-weight: 400;\">Hypothesizing is the second phase of our <a href=\"https:\/\/www.invespcro.com\/blog\/creating-a-conversion-roadmap-how-to-prioritize-conversion-problems-on-your-website\/\">SHIP optimization<\/a> process here at Invesp.<\/span><\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/www.invespcro.com\/blog\/images\/blog-images\/Hypothesize.jpg\" alt=\"\" class=\"wp-image-14727\"\/><\/figure>\n\n\n\n<div class=\"blog_img\"><\/div>\n\n\n\n<p><span style=\"font-weight: 400;\">It comes after we have completed the research phase.&nbsp;<\/span><\/p>\n\n\n\n<!--more-->\n\n\n\n<p><span style=\"font-weight: 400;\">This is an indication that we don\u2019t just pull a hypothesis out of thin air. We always make sure that it is based on research data.&nbsp;<\/span><\/p>\n\n\n\n<p><span style=\"font-weight: 400;\">But having a research-backed hypothesis doesn\u2019t mean that the hypothesis will always be correct. In fact, tons of hypotheses bear inconclusive results or get disproved.&nbsp;<\/span><\/p>\n\n\n\n<p><span style=\"font-weight: 400;\">The main idea of having a hypothesis in marketing experimentation is to help you gain insights \u2013 regardless of the <a href=\"https:\/\/www.invespcro.com\/ab-testing\/results-analysis\/\">testing<\/a> outcome.&nbsp;<\/span><\/p>\n\n\n\n<p><span style=\"font-weight: 400;\">By the time you finish reading this article, you\u2019ll know:&nbsp;<\/span><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><span style=\"font-weight: 400;\">The essential tips on what to do when crafting a hypothesis for marketing experiments<\/span><\/li>\n\n\n\n<li><span style=\"font-weight: 400;\">How a marketing experiment hypothesis works&nbsp;<\/span><\/li>\n\n\n\n<li><span style=\"font-weight: 400;\">How experts develop a solid hypothesis<\/span><\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\"><b>The Basics: Marketing Experimentation Hypothesis<\/b><\/h2>\n\n\n\n<p><span style=\"font-weight: 400;\">A hypothesis is a <\/span><b>research-based<\/b> <b>statement<\/b><span style=\"font-weight: 400;\"> that aims to <\/span><b>explain<\/b><span style=\"font-weight: 400;\"> an observed trend and create a <\/span><b>solution<\/b><span style=\"font-weight: 400;\"> that will improve the result. This statement is an educated, testable prediction about what will happen.<\/span><\/p>\n\n\n\n<p><span style=\"font-weight: 400;\">It has to be stated in declarative form and not as a question.<\/span><\/p>\n\n\n\n<p><span style=\"font-weight: 400;\">\u201c<\/span><i><span style=\"font-weight: 400;\">If we add magnification info, product video and making virtual mirror buttons, will that improve engagement?<\/span><\/i><span style=\"font-weight: 400;\">\u201d is not declarative, but \u201c<\/span><i><span style=\"font-weight: 400;\">Improving the experience of product pages by adding magnification info, product video and making virtual mirror buttons will increase engagement<\/span><\/i><span style=\"font-weight: 400;\">\u201d is.<\/span><\/p>\n\n\n\n<p><span style=\"font-weight: 400;\">Here\u2019s a quick example of how a hypothesis should be phrased:&nbsp;<\/span><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><span style=\"font-weight: 400;\">Replacing ___ with __ will increase [conversion goal] by [%], because:<\/span><\/li>\n\n\n\n<li><span style=\"font-weight: 400;\">Removing ___ and __ will decrease [conversion goal] by [%], because:<\/span><\/li>\n\n\n\n<li><span style=\"font-weight: 400;\">Changing ___ into __ will not affect [conversion goal], because:<\/span><\/li>\n\n\n\n<li><span style=\"font-weight: 400;\">Improving&nbsp; ___ by&nbsp; ___will increase [conversion goal], because:&nbsp;<\/span><\/li>\n<\/ul>\n\n\n\n<p><span style=\"font-weight: 400;\">As you can see from the above sentences, a good hypothesis is written in clear and simple language. Reading your hypothesis should tell your team members exactly what you thought was going to happen in an experiment.<\/span><\/p>\n\n\n\n<p><span style=\"font-weight: 400;\">Another important element of a good hypothesis is that it defines the variables in easy-to-measure terms, like who the participants are, what changes during the testing, and what the effect of the changes will be:&nbsp;<\/span><\/p>\n\n\n\n<p><b>Example<\/b><span style=\"font-weight: 400;\">: Let\u2019s say this is our hypothesis:&nbsp;<\/span><\/p>\n\n\n\n<p><i><span style=\"font-weight: 400;\">Displaying full look items on every \u201ccontinue shopping &amp; view your bag\u201d pop-up and highlighting the value of having a full look will improve the visibility of a full look, encourage visitors to add multiple items from the same look and that will increase the average order value, quantity with cross-selling by 3%<\/span><\/i><span style=\"font-weight: 400;\">.<\/span><\/p>\n\n\n\n<p><b>Who are the participants<\/b><span style=\"font-weight: 400;\">:&nbsp;<\/span><\/p>\n\n\n\n<p><i><span style=\"font-weight: 400;\">Visitors.&nbsp;<\/span><\/i><\/p>\n\n\n\n<p><b>What changes during the testing<\/b><span style=\"font-weight: 400;\">:&nbsp;<\/span><\/p>\n\n\n\n<p><i><span style=\"font-weight: 400;\">Displaying full look items on every \u201ccontinue shopping &amp; view your bag\u201d pop-up and highlighting the value of having a full look\u2026<\/span><\/i><\/p>\n\n\n\n<p><b>What the effect of the changes will be:<\/b><i><span style=\"font-weight: 400;\">&nbsp;<\/span><\/i><\/p>\n\n\n\n<p><i><span style=\"font-weight: 400;\">Will improve the visibility of a full look, encourage visitors to add multiple items from the same look and that will increase the average order value, quantity with cross-selling by 3%<\/span><\/i><span style=\"font-weight: 400;\">.<\/span><\/p>\n\n\n\n<p><span style=\"font-weight: 400;\">Don&#8217;t bite off more than you can chew! Answering some scientific questions can involve more than one experiment, each with its own hypothesis. so, you have to make sure your hypothesis is a specific statement relating to a single experiment.<\/span><\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><b>How a Marketing Experimentation Hypothesis Works<\/b><\/h2>\n\n\n\n<p><span style=\"font-weight: 400;\">Assuming that you have done <a href=\"https:\/\/www.invespcro.com\/blog\/the-conversion-framework\/\">conversion research<\/a> and you have identified a list of issues (<a href=\"https:\/\/www.invespcro.com\/blog\/things-ui-ux-designers-need-to-know-about-conversion-rate-optimization\/\">UX<\/a> or conversion-related problems) and potential revenue opportunities on the site. The next thing you\u2019d want to do is to <\/span><a href=\"https:\/\/www.invespcro.com\/blog\/creating-a-conversion-roadmap-how-to-prioritize-conversion-problems-on-your-website\/\"><span style=\"font-weight: 400;\">prioritize the issues<\/span><\/a><span style=\"font-weight: 400;\"> and determine which issues will most impact the bottom line.<\/span><\/p>\n\n\n\n<p><span style=\"font-weight: 400;\">Having ranked the issues you need to test them to determine which solution works best. At this point, you don\u2019t have a clear solution for the problems identified. So, to get better results and avoid wasting traffic on poor test designs, you need to make sure that your testing plan is guided.&nbsp;<\/span><\/p>\n\n\n\n<p><span style=\"font-weight: 400;\">This is where a hypothesis comes into play.&nbsp;<\/span><\/p>\n\n\n\n<p><span style=\"font-weight: 400;\">For each and every problem you\u2019re aiming to address, you need to craft a hypothesis for it \u2013 unless the problem is a technical issue that can be solved right away without the need to hypothesize or test.&nbsp;<\/span><\/p>\n\n\n\n<p><span style=\"font-weight: 400;\">One important thing you should note about an experimentation hypothesis is that it can be implemented in different ways.&nbsp;&nbsp;<\/span><\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/www.invespcro.com\/blog\/images\/blog-images\/Share.jpg\" alt=\"\" class=\"wp-image-14729\"\/><\/figure>\n\n\n\n<div class=\"blog_img\"><\/div>\n\n\n\n<p><span style=\"font-weight: 400;\">This means that one hypothesis can have four or five different tests as illustrated in the image above. <a href=\"https:\/\/www.linkedin.com\/in\/khalidh?miniProfileUrn=urn%3Ali%3Afs_miniProfile%3AACoAAAA8G9QBsf9hlhNCLb7XN_Xn5vqw2WhsODE&amp;lipi=urn%3Ali%3Apage%3Ad_flagship3_detail_base%3Bbs2EsfxnScilvHjX9w476g%3D%3D&amp;licu=urn%3Ali%3Acontrol%3Ad_flagship3_detail_base-actor_container&amp;lici=RAtqxmHLSsSsj7LK3PoQEw%3D%3D\">Khalid Saleh<\/a>, the Invesp CEO, explains:&nbsp;<\/span><\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p><i><span style=\"font-weight: 400;\">\u201cThere are several ways that can be used to support one single hypothesis. Each and every way is a possible test scenario. And that means you also have to prioritize the test design you want to start with. Ultimately the name of the game is you want to find the idea that has the biggest possible impact on the bottom line with the least amount of effort. We use almost 18 different metrics to score all of those.\u201d<\/span><\/i><\/p>\n<\/blockquote>\n\n\n\n<p><span style=\"font-weight: 400;\">In one of the recent tests we launched after watching video recordings, viewing heatmaps, and conducting expert reviews, we noticed that:&nbsp;&nbsp;<\/span><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><span style=\"font-weight: 400;\">Visitors were scrolling to the bottom of the page to fill out a calculator so as to get a free diet plan.&nbsp;<\/span><\/li>\n\n\n\n<li><span style=\"font-weight: 400;\">Brand is missing&nbsp;<\/span><\/li>\n\n\n\n<li><span style=\"font-weight: 400;\">Too many free diet plans \u2013 and this made it hard for visitors to choose and understand.&nbsp;&nbsp;<\/span><\/li>\n\n\n\n<li><span style=\"font-weight: 400;\">No value proposition on the page<\/span><\/li>\n\n\n\n<li><span style=\"font-weight: 400;\">The copy didn\u2019t mention the benefits of the paid program<\/span><\/li>\n\n\n\n<li><span style=\"font-weight: 400;\">There was no clear CTA for the next action<\/span><\/li>\n<\/ul>\n\n\n\n<p><span style=\"font-weight: 400;\">To help you understand, let&#8217;s have a look at how the original page looked like before we worked on it:&nbsp;<\/span><\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/www.invespcro.com\/blog\/images\/blog-images\/ogg.jpg\" alt=\"\" class=\"wp-image-14732\"\/><\/figure>\n\n\n\n<div class=\"blog_img\"><\/div>\n\n\n\n<p><span style=\"font-weight: 400;\">So our aim was to make the shopping experience seamless for visitors, make the page more appealing and not confusing. In order to do that, here is how we phrased the hypothesis for the page above:&nbsp;<\/span><\/p>\n\n\n\n<p><em><span style=\"font-weight: 400;\">Improving the experience of optin landing pages by making the free offer accessible above the fold and highlighting the next action with a clear CTA and will increase the engagement on the offer and increase the conversion rate by 1%.<\/span><\/em><\/p>\n\n\n\n<p><span style=\"font-weight: 400;\">For this particular hypothesis, we had two design variations aligned to it:<\/span><\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/www.invespcro.com\/blog\/images\/blog-images\/two-variations.jpg\" alt=\"\" class=\"wp-image-14731\"\/><\/figure>\n\n\n\n<div class=\"blog_img\"><\/div>\n\n\n\n<p><span style=\"font-weight: 400;\">The two above designs are different, but they are aligned to one hypothesis. This goes on to show how one hypothesis can be implemented in different ways. Looking at the two variations above \u2013 which one do you think won? <\/span><\/p>\n\n\n\n<p><span style=\"font-weight: 400;\">Yes, you&#8217;re right, V2 was the winner.&nbsp;<\/span><\/p>\n\n\n\n<p><span style=\"font-weight: 400;\">Considering that there are many ways you can implement one hypothesis, so when you launch a test and it fails, it doesn\u2019t necessarily mean that the hypothesis was wrong. Khalid adds: <\/span><\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p><i><span style=\"font-weight: 400;\">\u201cA single failure of a test doesn&#8217;t mean that the hypothesis is incorrect. Nine times out of ten it\u2019s because of the way you&#8217;ve implemented the hypothesis. Look at the way you&#8217;ve coded and look at the copy you&#8217;ve used \u2013 you are more likely going to find something wrong with it. Always be open.\u201d&nbsp;<\/span><\/i><\/p>\n<\/blockquote>\n\n\n\n<p><span style=\"font-weight: 400;\">So there are three things you should keep in mind when it comes to marketing experimentation hypotheses:&nbsp;<\/span><\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li><span style=\"font-weight: 400;\">It takes a while for this hypothesis to really fully test it.<\/span><\/li>\n\n\n\n<li><span style=\"font-weight: 400;\">A single failure doesn&#8217;t necessarily mean that the hypothesis is incorrect.<\/span><\/li>\n\n\n\n<li><span style=\"font-weight: 400;\">Whether a hypothesis is proved or disproved, you can still learn something about your users.<\/span><\/li>\n<\/ol>\n\n\n\n<h2 class=\"wp-block-heading\"><b>How experts develop a solid hypothesis<\/b><\/h2>\n\n\n\n<p><span style=\"font-weight: 400;\">I know it\u2019s never easy to develop a hypothesis that informs future testing \u2013 I mean it takes a lot of intense research behind the scenes, and tons of ideas to begin with. So, I reached out to six <a href=\"https:\/\/www.invespcro.com\/blog\/category\/cro\/\">CRO<\/a> experts for tips and advice to help you understand more about developing a solid hypothesis and what to include in it.&nbsp;<\/span><\/p>\n\n\n\n<p><b>Maurice<\/b><span style=\"font-weight: 400;\">&nbsp;<\/span><span style=\"font-weight: 400;\">says that a solid hypothesis should have not more than one goal:&nbsp;<\/span><\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/www.invespcro.com\/blog\/images\/blog-images\/Screen-Shot-2021-03-17-at-12.16.20-292x300.png\" alt=\"Maurice Beerthuyzen's portrait\" class=\"wp-image-14708\"\/><\/figure>\n\n\n\n<h4 class=\"wp-block-heading\">Maurice Beerthuyzen \u2013 CRO\/CXO Lead at <a href=\"https:\/\/www.clickvalue.nl\/\">ClickValue<\/a><\/h4>\n\n\n\n<p>\u201cCreating a hypothesis doesn&#8217;t begin at the hypothesis itself. It starts with research. What do you notice in your data, customer surveys, and other sources? Do you understand what happens on your website?<\/p>\n\n\n\n<p>When you notice an opportunity it is tempting to base one single A\/B test on one hypothesis. Create hypothesis A and run a single test, and then move forward to the next test. With another hypothesis.<\/p>\n\n\n\n<p>But it is very rare that you solve your problem with only one hypothesis. Often a test provides several other questions. Questions which you can solve with running other tests. But based on that same hypothesis! We should not come up with a new hypothesis for every test.<\/p>\n\n\n\n<p>Another mistake that often happens is that we fill the hypothesis with multiple goals. Then we expect that the hypothesis will work on conversion rate, average order value, and\/or Click Through Ratio. Of course, this is possible, but when you run your test, your hypothesis can only have one goal at once.<\/p>\n\n\n\n<p>And what if you have two goals? Just split the hypothesis then create a secondary hypothesis for your second goal. Every test has one primary goal. What if you find a winner on your secondary hypothesis? Rerun the test with the second hypothesis as the primary one.\u201d<\/p>\n<\/blockquote>\n\n\n\n<p><b>Jon <\/b><span style=\"font-weight: 400;\">believes that a strong hypothesis is built upon three pillars:<\/span><\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/www.invespcro.com\/blog\/images\/blog-images\/Jon.jpeg\" alt=\"Jon McDonald's portrait\" class=\"wp-image-14707\"\/><\/figure>\n\n\n\n<h4 class=\"wp-block-heading\"><a href=\"https:\/\/www.linkedin.com\/in\/jonnymac\/\">Jon MacDonald<\/a> \u2013 President and Founder of <a href=\"https:\/\/thegood.com\/\">The Good<\/a><\/h4>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Respond to an established challenge \u2013 The challenge must have a strong background based on data, and the background should state an established challenge that the test is looking to address. Example: &#8220;Sign up form lacks proof of value, incorrectly assuming if users are on the page, they already want the product.&#8221;<\/li>\n\n\n\n<li>Propose a specific solution \u2013 What is the one, the single thing that is believed will address the stated challenge? Example: &#8220;Adding an image of the dashboard as a background to the signup form&#8230;&#8221;.<\/li>\n\n\n\n<li>State the assumed impact \u2013 The assumed impact should reference one specific, measurable optimization goal that was established prior to forming a hypothesis. Example: &#8220;&#8230;will increase signups.&#8221;<\/li>\n<\/ol>\n\n\n\n<p>So, if your hypothesis doesn&#8217;t have a specific, measurable goal like &#8220;will increase signups,&#8221; you&#8217;re not really stating a test hypothesis!&#8221;<\/p>\n<\/blockquote>\n\n\n\n<p><span style=\"font-weight: 400;\">Matt uses his own <\/span><a href=\"https:\/\/corvuscro.com\/hypothesisbuilder\/\"><span style=\"font-weight: 400;\">hypothesis builder<\/span><\/a><span style=\"font-weight: 400;\"> to collate important data points into a single hypothesis.&nbsp;<\/span><\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<figure class=\"wp-block-image alignleft\"><img decoding=\"async\" src=\"https:\/\/www.invespcro.com\/blog\/images\/blog-images\/Matt.png\" alt=\"Matt Beischel's portrait\" class=\"wp-image-14709\"\/><\/figure>\n\n\n\n<h4 class=\"wp-block-heading\"><a href=\"https:\/\/www.linkedin.com\/in\/ACoAAAG_6bMBwwBHl82c44N_ou9w2kQMjmnB2b8\/?lipi=urn%3Ali%3Apage%3Ad_flagship3_profile_view_base_recent_activity_details_shares%3B7pGGydWfSoepiVsniHCjjA%3D%3D\">Matt Beischel<\/a> \u2013 Founder of <a href=\"https:\/\/corvuscro.com\/\">Corvus CRO<\/a><\/h4>\n\n\n\n<p><span style=\"font-weight: 400;\">Like Jon, Matt also breaks down his hypothesis writing process into three sections. Unlike Jon, Matt sections are:<\/span><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><b>Comprehension<\/b><\/li>\n\n\n\n<li><b>Response<\/b><\/li>\n\n\n\n<li><b>Outcome<\/b><\/li>\n<\/ul>\n\n\n\n<p>I set it up so that the names neatly match the &#8220;CRO.&#8221; It&#8217;s a sort of &#8220;mad-libs&#8221; style fill-in-the-blank where each input is an important piece of information for building out a robust hypothesis. I consider these the minimum required data points for a good hypothesis; if you can&#8217;t completely fill out the form, then you don&#8217;t have a good hypothesis. Here&#8217;s a breakdown of each data point:<\/p>\n\n\n\n<p><b>Comprehension<\/b> \u2013 Identifying something that can be improved upon<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Problem: &#8220;What is a problem we have?&#8221;<\/li>\n\n\n\n<li>Observation Method: &#8220;How did we identify the problem?&#8221;<\/li>\n<\/ul>\n\n\n\n<p><b>Response<\/b> \u2013 Change that can cause improvement<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Variation: &#8220;What change do we think could solve the problem?&#8221;<\/li>\n\n\n\n<li>Location: &#8220;Where should the change occur?&#8221;<\/li>\n<\/ul>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Scope: &#8220;What are the conditions for the change?&#8221;<\/li>\n\n\n\n<li>Audience: &#8220;Who should the change affect?&#8221;<\/li>\n<\/ul>\n\n\n\n<p><b>Outcome<\/b> \u2013 Measurable result of the change that determines the success<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong><a href=\"https:\/\/www.invespcro.com\/blog\/behavioral-segmentation\/\" title=\"Behavior Change\">Behavior Change<\/a><\/strong>: &#8220;What change in behavior are we trying to affect?&#8221;<\/li>\n\n\n\n<li><b>Primary KPI:<\/b> &#8220;What is the important metric that determines business impact?&#8221;<\/li>\n\n\n\n<li><b>Secondary KPIs:<\/b> &#8220;Other metrics that will help reinforce\/refute the Primary KPI&#8221;<\/li>\n<\/ul>\n\n\n\n<p>Something else to consider is that I have a &#8220;user first&#8221; approach to formulating hypotheses. My process above is always considered within the context of how it would first benefit the user. Now, I do feel that a successful experiment should satisfy the needs of BOTH users and businesses, but always be in favor of the user.<\/p>\n\n\n\n<p>Notice that &#8220;Behavior Change&#8221; is the first thing listed in Outcome, not primary business KPI. Sure, at the end of the day you are working for the business\u2019s best interests (both strategically and financially), but placing the user first will better inform your decision making and prioritization; there&#8217;s a reason that things like personas, user stories, surveys, session replays, reviews, etc. exist after all.<\/p>\n\n\n\n<p>A business-first ideology is how you end up with dark patterns and damaging brand credibility.&#8221;<\/p>\n<\/blockquote>\n\n\n\n<p><span style=\"font-weight: 400;\">One of the many mistakes that CROs make when writing a hypothesis is that they are focused on wins and not on insights. <\/span><b>Shiva<\/b><span style=\"font-weight: 400;\"> advises against this mindset:<\/span><\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/www.invespcro.com\/blog\/images\/blog-images\/Shiva.jpeg\" alt=\"Shiva Manjunath's portrait\" class=\"wp-image-14710\"\/><\/figure>\n\n\n\n<h4 class=\"wp-block-heading\"><a href=\"https:\/\/www.linkedin.com\/in\/ACoAAAT8hmIBTwJc2ajtIV5sPO_xNlks4Nh2OQA\/?lipi=urn%3Ali%3Apage%3Ad_flagship3_profile_view_base_recent_activity_details_shares%3B7pGGydWfSoepiVsniHCjjA%3D%3D\">Shiva Manjunath<\/a> \u2013 Marketing Manager and CRO at <a href=\"https:\/\/www.gartner.com\/en\">Gartner<\/a><\/h4>\n\n\n\n<p>\u201cTest to learn, not test to win.<\/p>\n\n\n\n<p>It\u2019s a very simple reframe of hypotheses but can have a magnitude of difference. Here\u2019s an example:<\/p>\n\n\n\n<p>Test to Win Hypothesis: If I put a product video in the middle of the product page, I will improve add to cart rates and improve CVR.<\/p>\n\n\n\n<p>Test to Learn Hypothesis: If I put a product video on the product page, there will be high engagement with the video and it will positively influence traffic<\/p>\n\n\n\n<p>What you\u2019re doing is framing your hypothesis, and test, in a particular way to learn as much as you can. That is where you gain marketing insights. The more you run \u2018marketing insight\u2019 tests, the more you will win. Why? The more you compound marketing insight learnings, your win velocity will start to increase as a proxy of the learnings you\u2019ve achieved. Then, you\u2019ll have a higher chance of winning in your tests &#8211; and the more you\u2019ll be able to drive business results.\u201d<\/p>\n<\/blockquote>\n\n\n\n<p><b>Lorenzo <\/b>&nbsp;says it\u2019s okay to focus on achieving a certain result as long as you are also getting an answer to: \u201cWhy is this event happening or not happening?\u201d<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/www.invespcro.com\/blog\/images\/blog-images\/Lorenzo.jpeg\" alt=\"\" class=\"wp-image-14711\"\/><\/figure>\n\n\n\n<h4 class=\"wp-block-heading\"><a href=\"https:\/\/www.linkedin.com\/in\/ACoAAAE7APABnu_MVT_M_uJdzLAs51fw5h0ht58\/?lipi=urn%3Ali%3Apage%3Ad_flagship3_profile_view_base_recent_activity_details_shares%3B7pGGydWfSoepiVsniHCjjA%3D%3D\">Lorenzo Carreri<\/a> \u2013 CRO Consultant<\/h4>\n\n\n\n<p>\u201cWhen I come up with a hypothesis for a new or iterative experiment, I always try to find an answer to a question.<\/p>\n\n\n\n<p>It could be something related to a problem people have or an opportunity to achieve a result or a way to learn something.<\/p>\n\n\n\n<p>The main question I want to answer is \u201cWhy is this event happening or not happening?\u201d<\/p>\n\n\n\n<p>The question is driven by data, both qualitative and quantitative.<\/p>\n\n\n\n<p>The structure I use for stating my hypothesis is:<\/p>\n\n\n\n<p>From [data source], I noticed [this problem\/opportunity] among [this audience of users] on [this page or multiple pages]. So I believe that by [offering this experiment solution], [this KPI] will [increase\/decrease\/stay the same].<\/p>\n<\/blockquote>\n\n\n\n<p><b>Jakub Linowski<\/b> says that hypotheses are meant to hold researchers accountable:<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/www.invespcro.com\/blog\/images\/blog-images\/Jakub.jpeg\" alt=\"Jakub Linowski's portrait\" class=\"wp-image-14712\"\/><\/figure>\n\n\n\n<h4 class=\"wp-block-heading\"><a href=\"https:\/\/www.linkedin.com\/in\/ACoAAABh8fgBiCLvRZXcaSCzceWXkOWPShklKjY\/?lipi=urn%3Ali%3Apage%3Ad_flagship3_profile_view_base_recent_activity_details_shares%3B7pGGydWfSoepiVsniHCjjA%3D%3D\">Jakub Linowski<\/a> \u2013 Chief Editor of <a href=\"https:\/\/goodui.org\/\">GoodUI<\/a><\/h4>\n\n\n\n<p>\u201cThey do this by making your change and prediction more explicit. A typical hypothesis may be expressed as:<\/p>\n\n\n\n<p><em>If we change (X), then it will have some measurable effect (A). <\/em><\/p>\n\n\n\n<p>Unfortunately, this oversimplified format can also become a heavy burden to your experiment design with its extreme reductionism. However you decide to format your hypotheses, here are three suggestions for more flexibility to avoid limiting yourself.<\/p>\n\n\n\n<p><b>One Or More Changes<\/b><\/p>\n\n\n\n<p>To break out of the first limitation, we have to admit that our experiments may contain a single or multiple changes. Whereas the classic hypothesis encourages a single change or isolated variable, it\u2019s not the only way we can run experiments. In the real world, it\u2019s quite normal to see multiple design changes inside a single variation. One valid reason for doing this is when wishing to optimize a section of a website while aiming for a greater effect. As more positive changes compound together, there are times when teams decide to run bigger experiments. An experiment design (along with your hypotheses) therefore should allow for both single or multiple changes.<\/p>\n\n\n\n<p><b>One Or More Metrics<\/b><\/p>\n\n\n\n<p>A second limitation of many hypotheses is that they often ask us to only make a single prediction at a time. There are times when we might like to make multiple guesses or predictions to a set of metrics. A simple example of this might be a trade-off experiment with a guess of increased sales but decreased trial signups. Being able to express single or multiple metrics in our experimental designs should therefore be possible.<\/p>\n\n\n\n<p><b>Estimates, Directional Predictions, Or Unknowns<\/b><\/p>\n\n\n\n<p>Finally, traditional hypotheses also tend to force very simple directional predictions by asking us to guess whether something will increase or decrease. In reality, however, the fidelity of predictions can be higher or lower. On one hand, I\u2019ve seen and made experiment estimations that contain specific numbers from prior data (ex: increase sales by 14%). While at other times it should also be acceptable to admit the unknown and leave the prediction blank. One example of this is when we are testing a completely novel idea without any prior data in a highly exploratory type of experiment. In such cases, it might be dishonest to make any sort of predictions and we should allow ourselves to express the unknown comfortably.\u201d<\/p>\n<\/blockquote>\n\n\n\n<h2 class=\"wp-block-heading\"><b>Conclusion&nbsp;<\/b><\/h2>\n\n\n\n<p><span style=\"font-weight: 400;\">So there you have it! Before you jump on launching a test, start by making sure that your hypothesis is solid and backed by research. Ask yourself the questions below when crafting a hypothesis for marketing experimentation:<\/span><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><span style=\"font-weight: 400;\">Is the hypothesis backed by research?<\/span><\/li>\n\n\n\n<li><span style=\"font-weight: 400;\">Can the hypothesis be tested?<\/span><\/li>\n\n\n\n<li><span style=\"font-weight: 400;\">Does the hypothesis provide insights?<\/span><\/li>\n\n\n\n<li><span style=\"font-weight: 400;\">Does the hypothesis set the expectation that there will be an explanation behind the results of whatever you\u2019re testing?<\/span><\/li>\n<\/ul>\n\n\n\n<p>Don\u2019t worry! Hypothesizing may seem like a very complicated process, but it&#8217;s not complicated in practice especially when you have done proper research.<\/p>\n\n\n\n<p>If you enjoyed reading this article and you&#8217;d love to get the best CRO content \u2013 delivered by the best experts in the industry \u2013 straight to your inbox, every week. Please subscribe <a href=\"https:\/\/invesp.us10.list-manage.com\/subscribe\/post?u=1b3cdbfb9475f7867454d8024&amp;id=d943ed0f69\">here<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p><span class=\"span-reading-time rt-reading-time\" style=\"display: block;\"><span class=\"rt-label rt-prefix\">Reading Time: <\/span> <span class=\"rt-time\"> 11<\/span> <span class=\"rt-label rt-postfix\">minutes<\/span><\/span>Every marketing experimentation process has to have a solid hypothesis.&nbsp; That\u2019s a must \u2013 unless you want to be roaming in the dark and heading towards a dead-end in your experimentation program. Hypothesizing is the second phase of our SHIP optimization process here at Invesp. It comes after we have completed the research phase.&nbsp;<\/p>\n","protected":false},"author":54,"featured_media":14743,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[36],"tags":[],"class_list":["post-14705","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-cro"],"_links":{"self":[{"href":"https:\/\/www.invespcro.com\/blog\/wp-json\/wp\/v2\/posts\/14705","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.invespcro.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.invespcro.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.invespcro.com\/blog\/wp-json\/wp\/v2\/users\/54"}],"replies":[{"embeddable":true,"href":"https:\/\/www.invespcro.com\/blog\/wp-json\/wp\/v2\/comments?post=14705"}],"version-history":[{"count":1,"href":"https:\/\/www.invespcro.com\/blog\/wp-json\/wp\/v2\/posts\/14705\/revisions"}],"predecessor-version":[{"id":97039,"href":"https:\/\/www.invespcro.com\/blog\/wp-json\/wp\/v2\/posts\/14705\/revisions\/97039"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.invespcro.com\/blog\/wp-json\/wp\/v2\/media\/14743"}],"wp:attachment":[{"href":"https:\/\/www.invespcro.com\/blog\/wp-json\/wp\/v2\/media?parent=14705"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.invespcro.com\/blog\/wp-json\/wp\/v2\/categories?post=14705"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.invespcro.com\/blog\/wp-json\/wp\/v2\/tags?post=14705"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}