# When experiment groups have unexpected sizes > Observed assignment proportions differ materially from the planned allocation. Diagnose the cause, choose a bounded correction and verify assignment integrity under the prespecified experiment design. Source: https://saas-marketing.net/guides/growth-test-has-sample-ratio-mismatch/ Topic: SaaS Growth Marketing Type: guide Published: 2026-09-17 Last updated: 2026-09-17 Publisher: SaaS Marketing (saas-marketing.net) License: CC BY 4.0. Quote or republish with attribution and a link to https://saas-marketing.net/guides/growth-test-has-sample-ratio-mismatch/ ## Short answer Observed assignment proportions differ materially from the planned allocation. Start with this check: Check assignment, eligibility, logging and exclusions before interpreting outcome differences. The corrective action is to pause the conclusion and investigate the mechanism causing the imbalance. ## Key takeaways - Check assignment, eligibility, logging and exclusions before interpreting outcome differences. - Pause the conclusion and investigate the mechanism causing the imbalance. - Do not repair the result by silently dropping inconvenient observations. - Review assignment integrity under the prespecified experiment design. --- Observed assignment proportions differ materially from the planned allocation. The useful response is a diagnosis that changes a decision, not another report describing the symptom. Use this play with the experiment owner and the analyst responsible for design integrity. The working evidence should include hypothesis, assignment rules, metric definition and decision record, with private or sensitive details removed from any shared example. ## Confirm the problem in the actual workflow Check assignment, eligibility, logging and exclusions before interpreting outcome differences. Start with one representative case and follow it from the original action to the reported outcome. Identify where the observed behavior first differs from the intended process. A screenshot of a final dashboard can be useful, but it may hide the source record, a delayed update or a decision made elsewhere. Keep the unit of analysis explicit: the prespecified eligible user or account cohort. The same label can conceal different populations or stages. Before comparing two results, check that they describe the same kind of work and have had a comparable chance to complete it. ## Separate the visible symptom from the cause Check the design before interpreting a result. Assignment, exclusions, outcome timing and stopping rules can change the meaning of an apparently precise statistic. Separate practical effect from statistical evidence and keep guardrails beside the primary outcome. The symptom in this case is specific: observed assignment proportions differ materially from the planned allocation. Ask which piece of evidence would distinguish an operating failure from a measurement failure or a mismatch in the original plan. If the evidence is unavailable, record the missing source and its owner instead of treating the preferred explanation as established fact. ## A situation to work through A tracking failure affecting one variant can create an apparent conversion lift even when the user experience did not improve. This is an illustrative situation, not a reported client case. Record the equivalent evidence and assumptions for your own workflow. ## Choose the smallest useful correction Pause the conclusion and investigate the mechanism causing the imbalance. Keep the change narrow enough that the responsible people can implement and inspect it. If a correction changes several things at once, describe it as a combined operating change; do not later claim that one small element caused the whole result. Assign the correction to the experiment owner and the analyst responsible for design integrity. Agree which artifact will show that the work is complete. An owner without an observable acceptance condition can close a task while leaving the original problem unresolved. A detailed checklist without an owner creates the opposite problem: the evidence requirement exists, but nobody is accountable for producing it. ## Preserve the important limitation Do not repair the result by silently dropping inconvenient observations. This condition belongs beside the recommendation because it can change the decision. It should not disappear when the plan becomes a short presentation or a status update. A higher signup rate is not automatically a better activation path if the removed step helped users reach a useful workflow. Review the complete sequence and the relevant customer outcome. A bundled product change can be evaluated as a bundle without claiming to isolate every component. ## Verification worksheet | Review item | What to record for this issue | Owner | Evidence | | --- | --- | --- | --- | | Observed symptom | Observed assignment proportions differ materially from the planned allocation. | | | | Diagnostic test | Check assignment, eligibility, logging and exclusions before interpreting outcome differences. | | | | Proposed correction | Pause the conclusion and investigate the mechanism causing the imbalance. | | | | Guardrail | Do not repair the result by silently dropping inconvenient observations. | | | | Review measure | Assignment integrity under the prespecified experiment design | | | Download a working copy and follow the [worksheet instructions](/resources/#using-worksheets). Keep unknown facts visible rather than filling gaps with guesses. ## Decide whether to keep, revise or stop the change Review assignment integrity under the prespecified experiment design after the agreed observation period. Keep the correction when the intended behavior is verified and the guardrail remains acceptable. Revise it when the diagnosis was useful but the intervention did not resolve the cause. Stop and reassess when new evidence shows that the original problem was framed incorrectly. Record what changed in hypothesis, assignment rules, metric definition and decision record. This gives the next review a stable starting point and prevents a definition change from being mistaken for a performance improvement. ## Related methods and next steps - [SaaS Growth Model: Build the Equation Before Tactics](/guides/saas-growth-model/) - [Growth experiment velocity: measure learning capacity](/research/experiment-velocity-benchmarks/) - [Growth experiment brief template](/templates/experiment-brief/) - [Design a test that can answer the question](/courses/growth-experimentation/02-design-a-valid-test/) Return to the [saas growth topic guide](/saas-growth/), browse its [complete resource collection](/topics/saas-growth/), or use the [working resource library](/resources/). The [primary reference](https://docs.statsig.com/) provides relevant platform or methodological context; the diagnosis and example here are original editorial guidance. ## Frequently asked questions ### What is the first diagnostic check? Check assignment, eligibility, logging and exclusions before interpreting outcome differences. Inspect the actual working record or customer path rather than relying only on a summary report. ### What should change after the diagnosis? Pause the conclusion and investigate the mechanism causing the imbalance. Record the owner and the evidence needed to verify the correction. ### What limit should the team keep visible? Do not repair the result by silently dropping inconvenient observations. A local improvement does not establish a universal benchmark or guarantee a commercial result. ### How should the correction be evaluated? Review assignment integrity under the prespecified experiment design using a consistent unit and observation window. Keep the original evidence and record any measurement changes.