{"id":10100,"date":"2017-11-16T18:14:29","date_gmt":"2017-11-16T23:14:29","guid":{"rendered":"https:\/\/www.invespcro.com\/blog\/?p=10100"},"modified":"2024-09-03T14:53:11","modified_gmt":"2024-09-03T14:53:11","slug":"aa-tests","status":"publish","type":"post","link":"https:\/\/www.invespcro.com\/blog\/aa-tests\/","title":{"rendered":"What Is An AA Test And Why You Should Run AA Tests"},"content":{"rendered":"Reading Time: <\/span> 10<\/span> minutes<\/span><\/span>

Did you know 80% to 90% of A\/B tests do not produce any statistically significant result? Only<\/span> 1 out of 8<\/span><\/a> A\/B tests show a significant difference.<\/span><\/p>\n

Done incorrectly, some marketers have begun to question the value of <\/span>A\/B testing<\/span><\/a>\u2014their A\/B test reports an uplift of 20%. Yet, the increase reported by the AB testing software never seems to translate into improvements or profits.<\/span><\/p>\n

The reason? \u201cMost winning A\/B test results are illusory.\u201d <\/span>(Source: Qubit)<\/span><\/a><\/p>\n

Furthermore, most arguments that call for running A\/A testing consider it a sanity check <\/span>before you run an A\/B test<\/span><\/a>.<\/span><\/p>\n

In this post, we\u2019ll examine arguments around that (for and against it) and suggest other ways to look at A\/A tests and why we run them regularly on our <\/span>CRO projects<\/span><\/a>.<\/span><\/p>\n

Let\u2019s dive right in!<\/span><\/p>\n

What is an AA test?<\/span><\/h2>\n

An A\/A test is essentially an A\/B test where both variations are identical. Instead of comparing different versions of a webpage or marketing material to see which performs better, an A\/A test compares two identical variations against each other.\u00a0\u00a0<\/span><\/p>\n

In theory, AA tests are designed to help marketers examine the reliability of the <\/span>A\/B testing tool<\/span><\/a> used to run them, aiming to find \u201cno difference between the control and variant.\u201d<\/span><\/p>\n

\"What
AA Testing<\/figcaption><\/figure>\n

Now, since you\u2019re running the original page against itself (or multiple versions of itself), it will be logical that visitors will react the same way to all the different test recipes. Thus, the A\/B testing software will not be able to declare a winner.<\/span><\/p>\n

The theory is that the difference in <\/span>conversion rates between variations<\/span><\/a> will not reach statistically significant results.<\/span><\/p>\n

Other versions of AA tests include running an AABB test. In this case, you will have the control, an identical variations of the control, a challenger, and an identical copy of the challenger.<\/span><\/p>\n

Then, you will <\/span>run your A\/B test<\/span><\/a> as you usually do with an original and a challenger, but you will also add two sanity check versions to measure the accuracy of the testing software on the results.<\/span><\/p>\n

How does AA testing work?<\/span><\/h2>\n

Running an AA test is much like running AB tests, except in this case, the two groups of users randomly chosen for each variation are given the same experience.<\/span><\/p>\n

Here\u2019s a quick breakdown of how AA testing works:<\/b><\/p>\n