/

/

A/B Test

A/B Test

A/B Test

An A/B test is a controlled experiment that serves two versions of an experience to a randomly split audience and compares a chosen outcome to see which performs better.

Updated

Category

Marketing Automation

Also known as

A/B testing; split test

The randomization is what makes it trustworthy: it isolates the change from everything else that varies. The common failures are calling a winner before the result is statistically significant, and running so many small variants that nothing reaches significance.

In practice

Decide the metric and the sample size before you start, and let the test run its course. A test stopped early because an early lead looks good is guessing with extra steps.

Something out of date?