The methods that catch impossible statistics in published papers were developed separately, by different people, for different problems. Each is narrow. None is a general-purpose checker.
This is a plain description of what each one proves, what it needs, and where it breaks.
statcheck — p-values against their test statistics
Proves: that a reported p does not follow from the reported statistic and degrees of freedom.
Needs: a test statistic, its degrees of freedom, and a reported p.
Introduced by Nuijten and colleagues in 2016, statcheck recomputes p from the statistic and flags mismatches — with special attention to decision errors, where the reported and recomputed values straddle the significance threshold.
Where it breaks: the original R implementation parses results only in strict APA format in running text. It cannot read tables, and it has a documented tendency to misclassify squared quantities — an R² can be read as a chi-square statistic, producing a confident false positive. A 2024 critique put its sensitivity around 0.52. Anything outside psychology’s reporting conventions is largely invisible to it.
GRIM — means against sample size
Proves: that a reported mean cannot arise from N integer observations.
Needs: the mean, N, integer data, and N × items < 10^decimals.
The mean of N integers is a multiple of 1/N. For 20 participants, 3.43 is
unreachable. Covered in detail in
the GRIM test explained.
Where it breaks: non-integer data (inapplicable), large N (uninformative),
unreported missing data (false positives), and averaged multi-item scales, where
the granularity is 1/(N × items) and naive implementations get it wrong.
GRIMMER — standard deviations against integer structure
Proves: that a (mean, SD, N) triple is unreachable for integer data.
Needs: mean, SD, N, integer measurements.
The extension of GRIM to variability. For integers, the sum of squares must be an
integer sharing the parity of the sum, since x² ≡ x (mod 2). Combined with the
mean constraint, this eliminates a large space of reported SDs.
Where it breaks: the same integrality assumption as GRIM, and it becomes uninformative as N grows. It also inherits GRIM’s failure — if the mean is already impossible, the SD test is moot, and a good implementation says so rather than double-reporting the same problem.
SPRITE — reconstruction, and the bounds you get for free
Proves (the bounds): that an SD exceeds what the response scale physically permits.
Needs: mean, SD, N, and the scale minimum and maximum.
SPRITE’s full form reconstructs candidate datasets consistent with the reported
summary statistics — useful for seeing what the data would have to look like.
But its most immediately usable core is a hard inequality: for values in
[min, max] with mean μ, the variance cannot exceed (μ − min)(max − μ).
On a 1–7 scale with mean 4.0 and N = 20, the maximum possible SD is about 3.08. A reported 3.50 is impossible — and note that GRIMMER may pass the same value, because it knows nothing about the scale. The two tests fail on different things, which is exactly why running both is worth it.
Where it breaks: you need to know the true scale bounds, which papers do not always state.
The Carlisle baseline test — randomised trials
Indicates: that the arms of a randomised trial are implausibly similar (or implausibly different) at baseline.
Needs: a baseline characteristics table with means, SDs and group sizes for several variables.
Under proper randomisation, the p-values comparing baseline characteristics between arms should be uniformly distributed. Carlisle showed in 2017 that fabricated or improperly randomised trials betray themselves when those p-values cluster near 1 — arms that match far too well.
Where it breaks: this is a heuristic, not a proof. Stratified randomisation and correlated baseline variables both mimic the signature. It needs several variables to say anything at all.
Benford and terminal-digit analysis
Indicates: very little, in this domain.
Genuinely powerful in forensic accounting, where values span orders of magnitude. Research data usually does not: bounded scales, rounded reporting and narrow ranges all violate Benford’s law innocently. Terminal-digit heaping often just means someone rounded.
Worth computing. Never worth citing alone.
Which to run
For a psychology or social-science paper with Likert measures: statcheck + GRIM + GRIMMER + SPRITE bounds. These four overlap least and prove the most.
For a randomised clinical trial: add the Carlisle baseline test over the Table 1 characteristics.
For anything with printed group descriptives and test statistics: add a descriptive recomputation — recompute the t or F from the group means, SDs and sample sizes. Remember that a mismatch may reflect a legitimately adjusted model, so treat it as a question rather than a finding.
The gap this leaves
Every method above exists as a separate implementation, most of them R packages, most requiring you to extract the numbers by hand first. A 2025 review of the field concluded plainly that no unified detection framework exists — the techniques work, but nobody has assembled them into something an editor or reviewer can actually use.
That gap is what Statlint was built to close: all seven checks in one pass over pasted text or an uploaded PDF, field-agnostic, with proven impossibilities kept strictly separate from heuristic flags.