Most of these are not arithmetic errors. The numbers are computed correctly and answer a question nobody asked. Here is how to recognise your own manuscript in each one.
Statistical mistakes in a manuscript rarely announce themselves as mistakes. They show up as a feeling: the result seems weaker than the paper claims, a reviewer keeps asking for something you thought you already gave them, or a number in the abstract does not sit right when you say it out loud. This guide is organised around those symptoms rather than around textbook names, because that is the direction the diagnosis actually runs. You notice the symptom in your own draft first and only learn the name of the problem afterward.
Seven situations follow. Each one starts with what you can see in your manuscript, explains what is actually happening underneath, and says what to do about it.
This is the gap between statistical significance and practical significance, and it is one of the most common statistical mistakes in manuscripts with large samples. A p-value answers whether an effect is distinguishable from zero given the data you collected. It says nothing about whether the size of that effect is large enough to matter to a patient, a policy, or a budget. With enough observations, almost any nonzero difference reaches significance.
Suppose a trial with 4,000 participants per arm finds that a new formulation lowers systolic blood pressure by 0.6 mmHg (p = 0.01). The result is real in the sense that it is unlikely to be sampling noise. It is not large enough to change a treatment decision. A reviewer who has seen this pattern before will ask you to state the effect size and its clinical interpretation, not just the p-value.
The fix: report the magnitude of the effect, in the units a reader in your field would recognise, next to every p-value. Then say in one sentence whether that magnitude is large enough to matter. If it is not, say so. A paper that reports a small effect honestly survives review better than one a reviewer catches doing this for them.
A p-value and a confidence interval are not two ways of saying the same thing. The p-value tests a single hypothesis, usually that the true effect is zero. The interval shows the range of effect sizes consistent with your data, which is almost always the thing a reader actually wants. Reporting p = 0.03 alone leaves out whether the true effect is plausibly tiny or plausibly large, and both can produce the same p-value.
This becomes a real statistical mistake, not just a stylistic preference, when the confidence interval is wide enough that the point estimate is not trustworthy on its own. An odds ratio of 2.1 with a 95% confidence interval of 1.02 to 4.3 is barely distinguishable from no effect at one end and a strong effect at the other. Reporting "OR = 2.1, p = 0.04" without the interval hides that the data are far less informative than the point estimate suggests.
The fix: report an effect size with a confidence interval for every primary and secondary outcome. Interpret the width of the interval in the text, not just its bounds. A wide interval is a finding about how much you do not yet know, and saying so directly is more convincing than hoping the reviewer does not check.
This is error-rate inflation from multiple comparisons, and it is one of the statistical mistakes that reviewers check for first in any paper testing several outcomes at once. Each individual test carries a 5% chance of a false positive at the conventional threshold. Run twenty independent tests on variables with no true effect, and you expect roughly one significant result by chance alone, at exactly the same threshold you would report as a discovery.
Suppose a study measures 30 biomarkers against a single outcome and reports the four that reached p < 0.05, without mentioning the other 26. A reviewer who does the arithmetic expects roughly one and a half false positives from chance alone at that threshold, and four survivors out of thirty tests is close enough to that expectation to be unpersuasive on its own.
The fix: decide the correction method before you see the results, not after. A Benjamini-Hochberg false discovery rate correction is usually more appropriate than a Bonferroni correction when you are screening many related outcomes, because Bonferroni is conservative enough to bury real effects. State the number of comparisons tested up front, including the ones that did not reach significance, and report both raw and adjusted values.
A non-significant result means the data could not rule out zero. It does not mean the data showed zero. These are treated as interchangeable in a lot of discussion sections, and it is one of the statistical mistakes most likely to draw a major revision request, because the two claims require different evidence.
A small study with wide confidence intervals around the null is uninformative, not reassuring. Suppose a pilot study of 40 patients finds a risk difference of 3 percentage points between two treatments (95% CI: -8 to 14, p = 0.6). Writing "there was no difference between groups" overstates what the study can support. The honest claim is that the study did not have the power to detect a difference of the size that would matter clinically, and the interval is wide enough to be consistent with either a meaningful benefit or a meaningful harm.
The fix: when a result is not significant, look at the confidence interval before you decide what to claim. If it excludes effects large enough to matter, you can say the study found no meaningful effect. If it does not, say the study was underpowered to detect an effect of that size, and report the interval so the reader can judge for themselves.
Cox regression assumes that the hazard ratio between groups is constant over the entire follow-up period. When that assumption fails, an unweighted average hazard ratio can be reported with a confidence interval that never captures anything true, because the actual effect is changing sign or magnitude over time and no single number describes it.
This shows up in survival analyses where a treatment has an early benefit that fades, or a surgical intervention carries short-term risk in exchange for long-term benefit. A Cox model fit without checking proportional hazards will report a single hazard ratio that averages over both periods and describes neither one accurately.
The fix: check the assumption, most commonly with scaled Schoenfeld residuals, and report that you did. If the assumption fails, options include splitting follow-up time into intervals with separate hazard ratios, adding a time-varying covariate, or switching to a model that does not require the assumption, such as an accelerated failure time model. Naming which option you used, and why, is usually enough to satisfy the reviewer.
Testing baseline characteristics for significance in a randomised trial answers a question you already know the answer to. Randomisation guarantees that any baseline difference between groups arose by chance, so a significance test on those variables is testing the null hypothesis that the coin flip was fair. It was, by construction. What actually matters is not whether a baseline difference is statistically distinguishable from zero, but whether it is large enough to confound the outcome.
Suppose a trial randomises 200 patients and the treatment group turns out five years older on average, with a baseline imbalance test giving p = 0.11. Reporting "groups were balanced (p > 0.05)" and moving on treats a non-significant test as proof of no confounding, when a five-year age gap can matter substantively regardless of what the test says.
The fix: report baseline characteristics descriptively, without p-values, and adjust for any variable that is plausibly related to the outcome regardless of whether its baseline imbalance was significant. Adjustment, not testing, is what handles residual confounding from an imperfect randomisation.
Splitting a continuous variable into two groups, such as age above or below 65, or a biomarker above or below a threshold, throws away information about how far each observation is from the cutoff. Two patients aged 64 and 40 are coded identically as "under 65," even though they may differ more from each other than either does from a 66-year-old on the other side of the line. This is a common statistical mistake because it feels like simplification when it is actually a loss of statistical power, sometimes substantial.
The loss is worst when the cutoff is chosen after looking at the data to find the split that produces the strongest result. That practice inflates the false positive rate well beyond the nominal 5%, because the analysis is implicitly testing many possible cutoffs and reporting only the best one.
The fix: keep the variable continuous in the primary model, using a flexible functional form such as a spline if the relationship is not linear. Dichotomised versions can appear in a table for readability, but the inferential model should use the full variable. If a cutoff is clinically meaningful and was chosen in advance, say so explicitly and state that it was pre-specified.
When a mean and a median disagree by a wide margin, the distribution is skewed, and reporting the mean alone as the summary statistic misleads the reader about what a typical observation looks like. Cost data, hospital length of stay, and many biomarker concentrations are right-skewed: most values cluster at the low end, and a small number of large values pull the mean upward.
Suppose a study reports mean hospital stay of 8.4 days. If the median is 4 days, most patients in the study left well before the reported average, and the mean is being driven by a smaller group of long, complicated admissions. A reader who takes 8.4 days as the typical experience has been misled by an accurate number used in the wrong role.
The fix: report the median and interquartile range alongside, or instead of, the mean for any variable known or suspected to be skewed, and say so if the two summary statistics diverge. Where a treatment comparison is the goal, consider whether the analysis itself should model the skewed distribution directly rather than testing means computed from it.
Not every question here has one correct answer. Whether to report a Bonferroni or a false discovery rate correction, and how conservative to be about it, depends on the field and on how costly a false positive is relative to a missed true effect. Statisticians disagree on this in good faith, and a reviewer who insists on one specific method as the only acceptable choice is overstating the consensus. What a reviewer can reasonably insist on is that you picked a method before seeing the results and can defend the choice, not that you picked the one they would have picked.
The common thread across all seven situations above is smaller than it looks. In every case, a single summary number is being asked to answer a question it was not built to answer: a p-value standing in for effect size, a non-significant test standing in for evidence of no effect, a mean standing in for a typical value. The fix is almost always to report the fuller picture the summary number was compressed from, and to say in plain language what it does and does not support.
What to check across methodology, statistics, writing, and formatting before you submit.
The reasons editors decline manuscripts before review, and how to avoid each one.
How the process works, who decides what, and what reviewers are actually asked to do.
Structure the response letter, disagree well, and handle contradictory reviewers.
The Reviewer 2 Generator reads an abstract free. The free assessment reads a whole manuscript and tells you where it stands. For a full review, PeerGenius runs a panel of specialist reviewers over the paper and returns their objections before an editor sees it.
PeerGenius is built by Neon Peach, LLC, a product studio.
Run the panel over your manuscript and get the objections before an editor sees them.
Get free assessment