An A/B test is a controlled experiment that serves two versions of an experience to a randomly split audience and compares a chosen outcome to see which performs better.
Updated
Category
Marketing Automation
Also known as
A/B testing; split test
The randomization is what makes it trustworthy: it isolates the change from everything else that varies. The common failures are calling a winner before the result is statistically significant, and running so many small variants that nothing reaches significance.
In practice
Decide the metric and the sample size before you start, and let the test run its course. A test stopped early because an early lead looks good is guessing with extra steps.
Something out of date?