A/B test significance calculator
Test whether the difference between two conversion rates is statistically significant.
A clear answer. Here’s how.
Uses a pooled two-proportion z test: z = (p̂B − p̂A) ÷ √(p̂(1 − p̂)(1/nA + 1/nB)) with p̂ the combined conversion rate. The confidence interval for the difference uses the unpooled standard error. Relative uplift is the difference divided by A’s rate. The normal approximation needs at least about 10 conversions and non-conversions in each variant. It assumes independent visitors and a test planned in advance.
See a worked example
Worked example
A fixed demonstration, separate from the values you enter above.
- Visitors in variant A
- 1000
- Conversions in variant A
- 200
- Visitors in variant B
- 1000
- Conversions in variant B
- 250
- Confidence level
- 95%
- Alternative
- B differs from A (two-sided)
Example result
- Variant A conversion rate
- 20.00%
- Variant B conversion rate
- 25.00%
- Absolute difference (B − A)
- 5.00 percentage points
- Relative uplift
- 25.00%
- z statistic
- 2.6774
- P-value (two-sided)
- 0.00741965
- 95% confidence interval for the difference
- 1.35% to 8.65%
- Result
- Significant at 95% confidence
Pooled two-proportion z test; the interval uses the unpooled standard error. Decide the sample size in advance and read the result once: stopping the moment p dips below the threshold inflates false positives.