Any experiment that involves later statistical inference requires a sample size calculation done BEFORE such an experiment starts. A/B testing is no exception. Calculating the minimum number of visitors required for an AB test prior to starting prevents us from running the test for a smaller sample size, thus having an “underpowered” test.

I will take you back to 2007 to one of the very first CRO projects we conducted for a large online retailer. The website received a massive amount of traffic (100k per day). Since it was one of our first projects, and the team was a bit trigger happy with AB testing, our first test included 30 different designs to run against the control. The headache of implementing all of these designs was overwhelming. None the less, we launched the test after about six weeks of implementation.

Seven of our challengers beat the original design within five days. The team …

Both user testing and a/b testing are important techniques used during conversion optimization. They aren’t poles apart as far as their purpose is concerned. You can say that they are indeed two sides of the same coin.

The ultimate purpose of user testing and a/b testing is to improve the overall on-site experience for users and to improve the conversion rates of websites. However, one must understand the 8 differences between the two and decide when to pick the right test based on its objective. So, let’s find out what is the purpose of each of these tests.

Image source: Twoface

I have meant to write this post for a long time. There has been a lot written about one-tailed vs. two-tailed tests. However, most of the articles approach the topic from a purely statistical perspective providing many formulas but do not show how to do the calculations. Others articles approach the issue from a high-level business perspective. So, here is my attempt to delve deep into the topic. Some sections in this article are heavy on stats and formulas. If you are not interested in these, you can skip them! Special thanks to our data sciences team …

Conversion optimization, much like most SEM activities, touches all verticals and various channels. Each company has its own unique experience, and we always emphasize, no two projects are ever the same. It’s that uniqueness which ensures CRO will succeed for that organization. Nonetheless, projects are brought together with a common thread: the documentation and deliverables that are produced as a result of the research and effort are pretty standard.

Much to my own chagrin, most of our potential clients say that what most CRO agencies say they offer sounds pretty much the same. Obviously, as an agency co-owner, I …

There are hundreds of case studies and examples on AB testing, explaining what makes it highly useful for conversion optimization. While AB testing is important, it is a mere component of the overall CRO process. What’s critical to keep in mind is not dive into AB testing right away until you know everything about interpreting, analyzing, and understanding test results.

AB testing isn’t foolproof and like anything in statistics, results can simply be wrong. But the more you know about what makes a test valid, and basic statistical concepts, the more likely it is that you will not face …

Image source

Test this, test that…and always be testing.

As I listened to one speaker after the next, I wondered why a client would hire them in the first place. If you are a CRO agency, and the best advice you can provide to attendees in a conference is to test this and test that, then I am not sure why these companies would hire you in the first place.

All you need to do is start AB testing, find that one fantastic test that will take you from absolute anonymity to an overnight web success…that is what brings most …

Wild guesses and opinions often only result in delusional outcomes. Though opinions are by far the most widely accepted truth, they rarely are factual. It is a better practice to lean on deductive reasoning and logic instead of guesswork. But how do you arrive at what is labeled as a fact or a valid statement? Experimentation is the key.

Businesses realize the importance of scientific experimentation and testing. Many product-based enterprises have imbibed experimentation in their organizational culture. The reason – innovation cannot result without experimentation. Moreover, though there are millions of ideas that spur every minute, every hour, it …