A/B testing, also known as split testing or bucket testing, is a controlled experiment used in marketing, product development, and user experience (UX) design to compare two or more variations of a web page, app, or marketing campaign. The goal of A/B testing is to determine which variation performs better in terms of a specific metric or key performance indicator (KPI).

Marketing companies want to run successful campaigns, but the market is complex and several options can work. So normally they tun A/B tests, that is a randomized experimentation process wherein two or more versions of a variable (web page, page element, banner, etc.) are shown to different segments of people at the same time to determine which version leaves the maximum impact and drive business metrics.

The companies are interested in answering two questions:

Would the campaign be successful?

If the campaign was successful, how much of that success could be attributed to the ads?

With the second question in mind, we normally do an A/B test. The majority of the people will be exposed to ads (the experimental group). And a small portion of people (the control group) would instead see a Public Service Announcement (PSA) (or nothing) in the exact size and place the ad would normally be. The idea of this analysis is to analyze the groups, find if the ads were successful, how much the company can make from the ads, and if the difference between the groups is statistically significant.

License

Free for both personal and commercial use. No need to pay anything. Just need to make attribution.

This work is licensed under a Creative Commons Attribution 4.0 International License

Tags