What a Statistical Audit Checks Before Submission

Most rejected papers are not rejected because the research was worthless. They are rejected because a reviewer found a methodological problem the authors had not caught—an untested assumption, a specification that could not bear the weight of the claim, a result that fell apart under an obvious alternative analysis. By the time that objection arrives in a review, months have passed and the fix is far more costly than it would have been beforehand. A statistical audit exists to find those problems first: an independent, expert review of the analysis before it is submitted, by someone who reads it the way a demanding referee will.

This guide explains what a statistical audit checks, why the independence matters, and when it is worth commissioning. It reflects how we approach independent review in our Statistical & Methodological Audit practice—a service designed not to redo your work, but to stress-test it while you can still act on what it finds.

What a statistical audit is—and isn’t

A statistical audit is an independent examination of the methods, analysis, and results of a study to assess whether the conclusions are properly supported by the evidence. It is not a redo of your analysis, and it is not editing. It is a diagnostic: a methodologist who was not involved in the work goes through it specifically looking for weaknesses—the places where the analysis could be challenged, and where it would not hold up.

The value comes precisely from that independence. The researchers who did the analysis know what they intended, and that knowledge quietly fills in gaps a fresh reader would stumble over. They are also, understandably, invested in the result. An independent auditor brings neither of those—they see only what is actually on the page, and they have no stake in the finding surviving. That is the same perspective a journal referee brings, which is exactly why catching it first is so valuable.

Checking the assumptions

Every statistical method rests on assumptions, and the most common flaw an audit finds is assumptions that were never tested—or were violated and ignored. A regression assumes certain things about its errors; a t-test assumes certain things about the data; a panel model assumes things about unobserved heterogeneity. When those assumptions fail and the analysis proceeds anyway, the results can be biased or the inference invalid, however clean the output looks.

An audit checks that the assumptions behind each method were actually examined, that appropriate diagnostic tests were run, and that violations were addressed rather than overlooked. This is frequently where the most consequential problems hide, because the software produces a plausible-looking result whether or not the assumptions hold—the numbers never announce that they cannot be trusted.

Checking the specification and the method

Beyond assumptions, an audit examines whether the chosen method was the right one for the question and the data, and whether the model was specified correctly. Was the method appropriate to the kind of claim being made—descriptive, causal, or predictive? For a causal claim, does the design credibly support it, or is there an endogeneity problem left unaddressed? Are the control variables sensible; is the functional form justified; are there methodological choices that quietly shape the result? These are the questions a good referee asks, and an audit asks them while there is still time to respond.

Checking robustness

A single result, however striking, is rarely convincing on its own. A strong analysis shows that its findings survive reasonable alternative choices—different specifications, different samples, different ways of measuring the key variables. An audit assesses whether these robustness checks were done and, importantly, whether the headline result actually holds up under them. A finding that appears only under one specific set of choices and vanishes under equally reasonable alternatives is fragile, and a good auditor will surface that fragility. Reviewers increasingly expect robustness to be demonstrated rather than asserted, and an audit is where that expectation gets met before submission.

Checking reproducibility

A result that cannot be reproduced from the data and code is a serious liability, and reproducibility has become a growing expectation across fields—some journals now require code and data. An audit checks that the analysis can actually be reproduced: that the code runs, that it produces the reported numbers, and that there are no discrepancies between what the code does and what the paper says was done. This kind of check catches an entire class of errors—transcription mistakes, wrong variables, results that don’t match the tables—that are embarrassingly common and entirely avoidable, and that are far better found by an ally than by a referee or, worse, after publication.

Checking the reporting

Finally, an audit examines whether the results are reported accurately and honestly—whether the tables match the text, whether the limitations are acknowledged rather than buried, whether the conclusions stay within what the evidence supports or quietly overreach. Overclaiming—stating conclusions stronger than the analysis warrants—is one of the most common reasons careful reviewers push back, and it is easy to do unintentionally when you are close to your own work. An independent reader catches the gap between what was shown and what was claimed.

When a statistical audit is worth it

An audit is not needed for every piece of analysis, but it earns its place when the stakes are high: a submission to a strong journal, a thesis heading to defence, a grant application, or any result that will be acted upon or scrutinised closely. It is especially valuable when the methods are complex, when the claim is causal, or when a great deal rides on the finding being correct. In those cases, the cost of an independent review is small against the cost of a rejection, a failed defence, or a published error.

The deeper point is that an audit is not a sign of weak work—it is a mark of confidence in it. The strongest researchers actively want their analysis stress-tested before it goes out, because they would rather hear a hard question from an ally than from a referee. A statistical audit is simply a structured way to get that hard question early, while you can still give it the answer it deserves.