A/B Test Calculator – Free Statistical Significance Checker
Enter visitors and conversions for version A and version B. See which one won, by how much, and whether the result is statistically significant or could just be luck, plus how many visitors you need.
What Does "Statistically Significant" Mean?
It means the difference between A and B is large enough, with enough data, that it is unlikely to be random chance. At 95% confidence, there is under a 5% chance the result is just luck.
| Test | A | B | Verdict at 95% |
|---|---|---|---|
| Small test | 10 / 300 | 15 / 300 | Not significant: too little data |
| Bigger test | 60 / 2,000 | 85 / 2,000 | B wins |
| Tiny difference | 300 / 10,000 | 310 / 10,000 | Not significant |
A/B Testing Tips
Test one change at a time
Headline, button text or image. If you change everything, you will not know what worked.
Do not stop early
Checking every day and stopping when B looks ahead gives false winners. Set the sample size first.
Test big ideas
Small tweaks need huge traffic to prove. A new offer or page layout shows clearer results.
Track conversions properly
Know your baseline with our conversion rate calculator.
Test ads too
Compare two ad creatives by CTR. Check rates with our CTR calculator.
Test subject lines
Score them first with our email subject line tester, then A/B test the top two.
Make Decisions With Data
Testing beats guessing. Learn analytics, landing pages and ad testing hands-on in our small-batch classes in Coimbatore.
Frequently Asked Questions
How do you know if an A/B test is significant?
Compare the p-value with your confidence level. At 95% confidence, a p-value below 0.05 means the difference is statistically significant.
What test does this calculator use?
A two-proportion z-test (two-sided), the standard method for comparing conversion rates between two versions.
How many visitors do I need for an A/B test?
It depends on your conversion rate and how big a difference you want to detect. The tool estimates visitors needed per version for 80% power.
What confidence level should I use?
95% is the common standard. Use 90% for low-risk changes and 99% when a wrong decision would be costly.
Can I stop the test when it becomes significant?
It is better not to. Decide the sample size or end date in advance and check at the end, or you risk false winners.
Is this A/B test calculator free?
Yes. It is free, needs no login, and works in your browser.
More Free Digital Marketing Tools
All free, no sign-up. See all tools
