Web design & UX

A/B testing is a method in web design and SEO where two versions of a website or element (e.g., buttons, headings) are compared to see which performs better. Users are randomly assigned to a variant, and metrics such as conversion rates and bounce rates are analyzed.

For a test to be worthwhile, it needs enough participants and a single, consistent change. If you swap out the headline, color, and button all at once, you won't know what effect it had. And it needs patience: a test that's stopped after two days usually only measures chance.

Here's a calculation example: With 500 visitors per variant and an initial conversion rate of 3 percent, a difference from 3,2 to 3,4 percent is statistically insignificant. Only with around 2.000 visitors per variant does an increase from 3 to 4 percent become statistically significant. At 200 visitors per day, this translates to a runtime of just over three weeks.

Back to the glossary