Conversion
4 min read
Stop A/B testing your way to a worse site
Small tests on a weak page produce small wins on a weak page. Here is how we decide when to optimise and when to rebuild.
Traffic first, then tests
A test needs enough conversions to reach significance in a reasonable window. Below a few hundred a month you are not running an experiment, you are reading noise and calling it a decision.
Ask what the page is arguing
Before testing a button colour, read the page aloud. If the argument is confused, no amount of testing will fix it, because every variant inherits the same confusion. Rewrite first, test second.
The tests worth running
Changes that alter the offer, the proof, or the amount of work asked of the visitor. Those move numbers. Micro-copy and colour tweaks mostly measure how patient your team is.