A/B test sample size calculator
Plan an approximate sample for a two-sided conversion-rate test at 5% significance and 80% power with equal allocation. See the formula, change the inputs and save your results.
On this page 6 sections
The short answer
A/B test sample size uses baseline conversion rate, minimum detectable relative lift, eligible observations per day across both arms to estimate approximate observations per arm. Change the example inputs to your own figures. The result is a planning calculation, not an industry benchmark or a prediction.
Key points before you start
Use this tool alongside the saas growth guide. Plan an approximate sample for a two-sided conversion-rate test at 5% significance and 80% power with equal allocation.
Your numbers
Defaults are illustrative inputs, not industry benchmarks. Use one consistent reporting period.
Results
Editable CSV worksheet
Save your marketing measurement plan
Keep a worksheet for your inputs, assumptions and next actions. You can also print the calculation directly from your browser.
How to read this
- This is a planning approximation, not a guarantee of significance. It assumes independent observations, equal allocation, a fixed analysis horizon and no multiple-testing adjustment. Add time for delayed outcomes.
Which inputs do you need?
| Input | Example value | What to check |
|---|---|---|
| Baseline conversion rate | 5% | Use the value from the same reporting period as the other inputs. |
| Minimum detectable relative lift | 20% | Use the value from the same reporting period as the other inputs. |
| Eligible observations per day across both arms | 500 | Use the value from the same reporting period as the other inputs. |
Before entering numbers, choose the unit of analysis. An account, a user and a paying subscription are different objects. Counting users in one field and accounts in another can produce a precise answer to the wrong question. Record the start and end dates beside your source export so another person can reproduce your work.
What formulas does the calculator use?
Absolute detectable difference in percentage points
delta = baseline * lift / 100This output is expressed as a number.
Approximate observations per arm
perarm = baseline > 0 && baseline < 100 && delta > 0 && baseline + delta <= 100 ? Math.ceil(2 * Math.pow(1.96 + .841621, 2) * (baseline / 100) * (1 - baseline / 100) / Math.pow(delta / 100, 2)) : NaNThis output is expressed as a number.
Approximate collection days
days = daily > 0 ? Math.ceil(perarm * 2 / daily) : NaNThis output is expressed as a number.
Percent fields use whole percentages: enter 5 for five percent. The formula divides by 100 where a decimal rate is needed. Values in the formulas correspond to the labelled inputs above; earlier outputs can be used by later formulas.
Worked example
The defaults are a constructed scenario, not results from a named company or survey. With the example inputs above, the calculation produces:
| Output | Example result |
|---|---|
| Absolute detectable difference in percentage points | 1 |
| Approximate observations per arm | 7,457 |
| Approximate collection days | 30 |
Change one assumption at a time and watch the main result. Then test a conservative case by reducing the expected benefit or increasing the associated cost. If a decision works only at the most optimistic settings, investigate the uncertain input before committing the budget.
How should you interpret the result?
- This is a planning approximation, not a guarantee of significance. It assumes independent observations, equal allocation, a fixed analysis horizon and no multiple-testing adjustment. Add time for delayed outcomes.
A formula describes the assumptions entered into it. It cannot establish that a channel caused a sale, that historical retention will continue, or that a projected cost is achievable. Compare the output with your own previous cohorts before using a broad market comparison.
For a management review, save the result together with the source date, segment, owner and planned action. Recalculate when the underlying input changes. Keep a separate copy of the original scenario so the team can explain the difference between the plan and the observed outcome.
Continue the analysis
Use the metrics guide to align definitions, browse all calculators for adjacent calculations, and keep a measurement worksheet beside the model. The pricing hub and growth hub cover decisions that often change these inputs.
Apply a/b test sample size calculator in a working review
Record the source and unit of every input before using the result. Change one assumption at a time to understand which inputs matter most. Keep outputs that describe money, time and percentages distinct, and preserve undefined cases rather than converting them into plausible-looking zeroes.
For this topic, involve the experiment owner and the analyst responsible for design integrity and work from hypothesis, assignment rules, metric definition and decision record. The relevant unit is the prespecified eligible user or account cohort. State the question the review should resolve before choosing a chart, an asset or a tool. If participants disagree about the unit or scope, resolve that disagreement before combining their evidence.
Evidence to prepare
Check the design before interpreting a result. Assignment, exclusions, outcome timing and stopping rules can change the meaning of an apparently precise statistic. Separate practical effect from statistical evidence and keep guardrails beside the primary outcome.
| Review field | What to record |
|---|---|
| Topic | A/B test sample size calculator |
| Decision | The specific action this explanation should help you choose |
| Working evidence | hypothesis, assignment rules, metric definition and decision record |
| Unit and scope | the prespecified eligible user or account cohort |
| Responsible people | experiment owner and the analyst responsible for design integrity |
| Remaining uncertainty | The missing fact that could change the decision |
Two situations that can change the interpretation
When experiment groups have unexpected sizes
A tracking failure affecting one variant can create an apparent conversion lift even when the user experience did not improve.
Use this check: Check assignment, eligibility, logging and exclusions before interpreting outcome differences. Do not repair the result by silently dropping inconvenient observations.
The focused diagnostic guide provides the correction process and a working evidence sheet.
When an A/B test stops at the first positive result
A test that runs until it wins is not equivalent to a test evaluated once at its planned sample and time window.
Use this check: Compare the stopping behavior with the statistical design chosen before launch. A small p-value does not establish practical importance or rule out design problems.
The focused diagnostic guide provides the correction process and a working evidence sheet.
Record the decision and the limit
A higher signup rate is not automatically a better activation path if the removed step helped users reach a useful workflow. Review the complete sequence and the relevant customer outcome. A bundled product change can be evaluated as a bundle without claiming to isolate every component.
Keep the conclusion beside the evidence that supports it. Record what the team will do, who owns the next action and which event or date will trigger a review. If the underlying definition, audience or product behavior changes, revisit the conclusion rather than assuming the old result still applies. A clear limit is useful information; it tells the next reader where additional investigation is required.
Use the complete topic collection for related methods and the category field guides when the product’s buying situation or implementation requirements change how the method should be applied.
A reproducible sensitivity exercise
The a/b test sample size calculator tool provides a related numerical exercise. Its current default inputs are constructed examples, not industry observations. Under those defaults, the output labelled Approximate observations per arm is 7,457 in the tool’s displayed units. The table changes one input at a time and leaves the others at their defaults.
| Input changed | Default input | Alternative input | Approximate observations per arm after change |
|---|---|---|---|
| Baseline conversion rate | 5 | 6 | 6,149 |
| Minimum detectable relative lift | 20 | 24 | 5,179 |
| Eligible observations per day across both arms | 500 | 600 | 7,457 |
The alternative inputs are sensitivity cases, not recommended targets. A result marked not defined means the proposed combination does not satisfy the model or produces an undefined ratio. Keep that state visible. If the output changes sharply after a small input change, investigate the uncertain input before using the model to justify a larger commitment.
Compare the model’s scope with the concept on this page. The calculator may represent one particular application rather than every use of the term. Record the reporting period, currency where relevant, and the source of the real values you enter.
Editable CSV worksheet
Save your marketing measurement plan
Keep a worksheet for your inputs, assumptions and next actions. You can also print the calculation directly from your browser.
Frequently asked questions
How does this a/b test sample size calculator work?
It evaluates the formulas shown on this page in your browser. Plan an approximate sample for a two-sided conversion-rate test at 5% significance and 80% power with equal allocation. Inputs are not sent to a calculation server.
Are the default values SaaS industry benchmarks?
No. They are example inputs chosen to demonstrate the calculation. Replace them with your billing, CRM or finance records before making a decision.
Why does a result show n/a?
The calculation is undefined or an input is outside its allowed range. Check for an empty field, a zero denominator or an impossible percentage before interpreting the result.
Can I save or share my calculation?
Use Print or save results to create a local PDF with your browser. Review the inputs before sharing and remove confidential customer or company information.
The saas-marketing.net editorial team Research and editorial
We research, write and maintain every page on this site. The library explains marketing decisions through practical frameworks, explicit assumptions and references. Corrections can be requested through the contact page.
Published September 17, 2026. Last updated .