A/B/n Testing

Validate your hypothesis

An A/B/n test is an experiment that compares at least 3 different versions of something and measures which one is more effective. These different versions are often called the control and the tests, or the control and the variants. The n in A/B/n refers to any number of variations beyond the A (control) and B (test) variations. This can mean A/B/C (1 control and 2 variations), A/B/C/D (1 control and 3 variations), etc.

Let’s look at a simple example:

  • An e-commerce company may want to test two new product page designs against the current product page design. Some people in the company hypothesize that one of the new product page designs may lead to a higher percentage of users that make a purchase. Before they roll out one of the new product page designs, they would run an A/B/n (in this case, A/B/C) test to measure whether or not this is the case. For the A/B/n test experiment, the e-commerce company may show 10% of their users the new product page B, 10% of their users the new product page C and the remaining 80% of their users would see the standard product page (control group, or A). Over time, the company will measure the conversion rate on each page in order to understand which product page actually drives the most purchases before making any decision.

A/B/n tests can be run on much more than just page designs though. For example, you may want to test different versions of a search algorithm, recommendation engine or user onboarding flow. Typically, there will be a key set of metrics that you are measuring to understand the impact of the control and test variants.

We use cookies to ensure you get the best experience on our website.

Privacy Policy