How to Conduct a Systematic Review and Meta-Analysis

When a body of research has grown large enough that individual studies pull in different directions, a well-conducted systematic review and meta-analysis can cut through the noise—summarising what the evidence collectively shows and, often, becoming one of the most cited papers in a field. But the same method, done carelessly, produces a misleading average that looks authoritative and is not. The difference is entirely in the rigour and transparency of the process.

This guide explains the distinction between a systematic review and a meta-analysis, walks through the main stages, and covers the standards—protocol registration, transparent reporting, bias assessment—that separate a trustworthy synthesis from a suggestive one. It reflects how we approach evidence synthesis in our Meta-Analysis & Evidence Synthesis practice.

Systematic review vs meta-analysis

The two terms are often used together but mean different things. A systematic review is a structured, comprehensive, and reproducible method for identifying, appraising, and summarising all the studies relevant to a clearly defined question. Its defining feature is method: an explicit protocol, a systematic search, and pre-specified criteria, all designed to minimise bias and to be reproducible by another researcher.

A meta-analysis is the statistical step—often, but not always, part of a systematic review—that combines the quantitative results of the included studies into a single pooled estimate. Not every systematic review includes a meta-analysis: if the studies are too different to combine sensibly, a narrative or structured synthesis may be more honest than a pooled number. The systematic review is the rigorous process of finding and appraising the evidence; the meta-analysis is one way of summarising it quantitatively when that is appropriate.

Start with a protocol

The single most important step for credibility happens before any data is extracted: writing and registering a protocol. A protocol specifies, in advance, the research question, the inclusion and exclusion criteria, the search strategy, and the planned analysis. Registering it—on a platform such as PROSPERO for many fields, or the Open Science Framework—creates a timestamped, public record of what you intended to do before you saw the results.

This matters because it guards against one of the most insidious problems in synthesis: decisions made after seeing the data that quietly shape the conclusion. If inclusion criteria or analyses can be adjusted once results start to appear, a review can be steered—consciously or not—toward a preferred answer. A registered protocol makes any departure from the plan visible and accountable. Reviewers and editors increasingly expect it, and its absence is a real weakness.

Search systematically

The search is what makes a review systematic. Rather than gathering the studies you happen to know, you search comprehensively across multiple databases using a documented, reproducible strategy, so that another researcher could run the same search and find the same records. A search that misses a large part of the literature—or that is not documented—undermines everything downstream, because the pooled estimate can only reflect the studies that were found.

Guarding against publication bias begins here. Studies with statistically significant or striking results are more likely to be published than those with null findings, so a search confined to published work can over-represent positive effects. A thorough search therefore considers grey literature and unpublished studies where feasible, precisely to counter this tilt. The whole process is then documented in a flow diagram—the PRISMA flow—showing how many records were identified, screened, and ultimately included, and why others were excluded.

Assess risk of bias

Not all studies deserve equal weight of trust. A core stage of any systematic review is assessing the risk of bias in each included study—evaluating, using an established tool appropriate to the study designs, how well each study was conducted and how vulnerable its findings are to systematic error. A meta-analysis that pools strong and weak studies without regard to quality can be dominated by flawed work. Risk-of-bias assessment lets you weigh the evidence appropriately, explore whether conclusions depend on the weaker studies, and report honestly on the quality of the base you are summarising.

Pool the effects—and understand heterogeneity

When the studies are similar enough to combine, the meta-analysis calculates a weighted average of their effect sizes, giving more weight to more precise studies. The result is often displayed in a forest plot: each study as a point estimate with its confidence interval, and the pooled result as a diamond at the bottom. This visual makes it immediately clear how consistent the studies are and how much the pooled estimate rests on any single one.

The central question in pooling is heterogeneity: how much the true effect varies across studies. If studies are estimating essentially the same effect and differ only by chance, a fixed-effect model may be appropriate. If the true effect genuinely varies across contexts, populations, or designs—which is common in social-science and management research—a random-effects model, which allows for that variation, is usually more realistic. Heterogeneity is quantified and reported, and high heterogeneity is a signal to investigate why studies differ—through subgroup analysis or meta-regression—rather than to paper over the variation with a single average. A pooled estimate reported without attention to heterogeneity can obscure more than it reveals.

Report transparently

Throughout, the governing principle is transparency, and it has a widely adopted standard: the PRISMA reporting guidelines, which set out what a systematic review and meta-analysis should report so that readers can judge its rigour and others can reproduce it. Reporting to PRISMA is now an expectation in most fields that publish syntheses, and it structures everything from the search documentation to the flow diagram to the handling of bias. A review that cannot show its working—how studies were found, chosen, appraised, and combined—cannot expect to be trusted, however striking its headline number.

When not to pool

Finally, methodological honesty sometimes means not producing a single pooled estimate. If the included studies differ too much—measuring different outcomes, in incomparable populations, using incompatible designs—forcing them into one number produces a meaningless average. In such cases a well-structured narrative synthesis, or a synthesis that groups comparable studies without over-combining, is the more credible choice. Deciding whether the evidence can be pooled is itself part of the analysis, and resisting the temptation to compute a number just because the software will is a mark of a careful reviewer.

The bottom line

A systematic review and meta-analysis is only as trustworthy as the process behind it. Register a protocol before you begin; search comprehensively and document it; assess the risk of bias in what you find; pool only when the studies are similar enough, using a model that respects heterogeneity; and report everything to a recognised standard so others can check your work. Done this way, a synthesis can be the most authoritative statement a field has on its question. Done without that discipline, it is a confident-looking average that no careful reader should believe.