Visitors needed for a reliable A/B test. The Sample Size Calculator takes baseline conversion rate, minimum relative lift to detect, confidence level, statistical power and returns visitors needed per variant plus total visitors for the test, variant rate you are testing for. Results update as you type, and the formula is shown under the result so you can repeat it in your own spreadsheet.
Marketing metrics are most useful as trends and comparisons between channels. Measure every campaign the same way, over the same period, and judge it against your gross margin rather than against revenue alone. Use the worked example below to check the maths against your own figures.
How the Sample Size Calculator works
Small lifts on low conversion rates need surprisingly large samples: detecting a 10% relative lift on a 3% rate at 95% confidence takes around 50,000 visitors per variant. Decide the sample size before the test starts and do not stop early.
Worked example
With the example values (baseline conversion rate of 3%, minimum relative lift to detect of 10%, confidence level "95%", statistical power "80%"), the visitors needed per variant is 53,151; total visitors for the test 106,302, variant rate you are testing for 3.30%. Change any figure above and the result updates immediately; use Copy results to paste the summary into a note or email.
Related Calculators
Frequently Asked Questions
How is visitors needed per variant calculated?
n = [zα√(2p̄(1−p̄)) + zβ√(p1(1−p1) + p2(1−p2))]2 ÷ (p2 − p1)2.
Which figures do I need?
Baseline conversion rate, minimum relative lift to detect, confidence level, statistical power. Take them from the same period and the same set of accounts or reports so the ratio is consistent, and check the example values as a guide to the units expected.
How often should I track this metric?
Weekly for live campaigns, monthly for channel comparisons, and always over the same period for spend and results so the figures line up.
