An A/B test calculator that tells you if your winner is real, the chance B beats A, and how many sends or visitors you need.
Find out if your test has a real winner, or how many sends or visitors you need before you start.
Your numbers
For emails, count clicks, not opens: Apple's Mail Privacy Protection hides who opened.
The answer
Type your numbers to see the answer.
What you type stays in this browser. The math runs on your device.
This free A/B test calculator tells you if your test has a real winner.
Type in the sends or visitors and the clicks or sales for each version. It says whether B beat A, how sure you can be, and how many more you need if it's too early.
Here's what an A/B test is, how to read the result, and how big your test needs to be.
What is an A/B test?
An A/B test shows 2 versions of something to 2 random groups, and counts which one does better.
It could be 2 subject lines, 2 headlines or 2 prices. Everything else stays the same, so the difference you see comes from the change.
Is my result real?
A small difference can come from chance. The calculator works out how likely that is.
At 95% confidence, "B wins" means a difference this big would come from chance less than 5% of the time. That's the p-value under 0.05.
It also shows the chance B beats A, as a plain probability, and the range B's rate is likely in.
How many sends or visitors you need
The smaller the lift you want to find, the bigger the test. Here's how many each version needs at 95% confidence and 80% power:
| Your rate now | To find 10% better | 20% better | 30% better |
|---|---|---|---|
| 2% | 80,682 | 21,109 | 9,798 |
| 5% | 31,234 | 8,158 | 3,780 |
| 10% | 14,751 | 3,841 | 1,774 |
| 20% | 6,510 | 1,683 | 772 |
So a list of 2,000 can find a big difference in clicks, but not a small one. Plan the size before you start.
4 A/B testing mistakes
Most tests that "found a winner" and then flopped made 1 of these:
- Stopping the first day B looks ahead, before the planned size
- Changing 3 things at once, so you can't tell which one worked
- Counting opens, which Apple's Mail Privacy Protection hides
- Calling it with a handful of clicks in each version
What to test
Start with what most people see:
- Email subject lines. Send 2 to 2 halves of your list and count clicks.
- Landing page headlines. Split your visitors and count sign-ups.
- YouTube titles. YouTube Studio's Test & compare tries up to 3 titles and thumbnails, and keeps the one with the most watch time.
A/B Test Calculator FAQs
Questions about A/B testing? Here's what to know.
An A/B test shows 2 versions of something to 2 random groups and counts which one does better, with everything else the same.
Statistically significant means a difference this big would rarely come from chance: at 95% confidence, less than 5% of the time. The A/B test calculator tells you if your result gets there.
How many people an A/B test needs depends on your rate and the lift you want to find. At a 5% rate, finding a 20% lift takes about 8,158 per version at 95% confidence and 80% power.
Test email clicks, not opens. Apple's Mail Privacy Protection prevents senders from seeing if a message was opened, so open counts aren't reliable.
An A/B test should run until each version reaches the size you planned, and ideally over full weeks, so weekdays and weekends are both in it.
The chance B beats A is the probability that B's true rate is higher than A's, given your numbers. The A/B test calculator works it out from both rates.
Yes, the A/B test calculator is free, with no sign-up. The math runs on your device.
What you type in the A/B test calculator stays in your browser, so it's there when you come back. Nothing is sent to a server.
The A/B test calculator is for creators, founders and marketers testing subject lines, headlines, pages and prices, who want to know if a winner is real.
To pick what to test, the headline analyzer scores headlines against real A/B tests, and the UTM builder tags each version's links so you can count the clicks. See all my free tools.
Navid Moazzez
AI business strategist & AI OS builder
Navid Moazzez helps creators and founders master AI and build their own AI Operating System (AI OS) to automate their business and life.
Navid.me is reader-supported. When you buy through links on this site, I may earn an affiliate commission. Learn more.
More free tools
Bluesky
Apple Books
Amazon
Free AI newsletterThe most actionable AI newsletter for founders
Every week, get proven AI strategies, curated tools, and step-by-step systems to grow your audience, create better content, and build a profitable creator business.
No fluff, no filler, no BS. Just five minutes each week that might level up your online business and life.
P.S. Sign up now to get free access to my ultimate AI tools guide for creators.
