Testing & Iteration
How long should I run a test?
Long enough to cover full business cycles, usually at least a couple of weeks, and decided before you start. The temptation is to stop the moment a variant looks good, which is how people ship changes that were random. Set the win condition first, then honor it.
Next moves
- Test one meaningful thing at a time, and make it big enough to matter.
- Decide the win condition before you launch the test.
- Below real traffic volume, prefer judgment and qualitative feedback over statistics.
Bring your traffic numbers. They decide whether testing is a real tool for you yet or a distraction.