by letthedataconfess

For example: If you own a company and want to increase the sales of your product. You may divide the products into two parts – A and B. Here A will remain unchanged while you make significant changes in B’s packaging. Now, on the basis of the response from customer groups who used A and B respectively, you try to decide which is performing better.

Alternative Hypothesis: There is a difference between the control and variant groups, i.e. your A/B test is true.

If the experiment turned out to be statistically significant, we reject the null hypothesis and accept the alternate hypothesis. To reject the null hypothesis, the p-value should be less than the confidence level ( significance level) i.e. 0.05

Control Group is the one which receives the original version of the product.

Test Group is the one which receives the product with some significant changes that the company is planning to do.

- The control and the test group are formed using random sampling from the population. Random sampling is important in hypothesis testing because it eliminates sampling bias, and it’s important to eliminate bias because you want the results of your A/B test to be representative of the entire population rather than the sample itself.
- We must also determine the minimum sample size for our A/B test before conducting it so that we can eliminate under coverage bias, which is the bias from sampling too few observations.

The two–sample t–test is one of the most commonly used hypothesis tests. It is applied to compare whether the average difference between the two groups.

- A/B testing is used by both startups and giant businesses to increase the traffic and conversion.
- A/B testing is majorly used when testing incremental changes, such as UX changes, new features, ranking, and page load times.

Join our Wishlist

Event Over