Instrumental Variables Explained

Suppose you want to estimate the effect of one variable on another—education on earnings, a policy on outcomes, a treatment on a result—but the predictor is endogenous: it is correlated with something in the error term, whether an omitted confounder, reverse causation, or measurement error. Ordinary regression will return a coefficient, but that coefficient is biased and cannot be read as a causal effect. This is one of the most common obstacles in empirical research, and the instrumental-variables approach is one of the most powerful ways around it—when the conditions are right.

This guide explains what an instrument is, the intuition for why it works, the two conditions a valid instrument must satisfy, and the practical difficulties that make instruments both prized and scrutinised. It builds on our guide to endogeneity and sits within our Causal Inference & Policy Evaluation practice.

The core idea

The problem with an endogenous predictor is that its variation is “contaminated”—part of why it moves is tangled up with the error term, so a regression cannot tell how much of the outcome’s change is really due to the predictor and how much to the confounding. The instrumental-variables idea is to find a third variable—the instrument, usually denoted Z—that influences the predictor but is otherwise disconnected from the outcome. The instrument isolates a portion of the predictor’s variation that is clean of the contamination, and uses only that clean portion to estimate the effect.

The intuition is that the instrument acts like a natural randomiser. If Z shifts the treatment X for reasons that have nothing to do with the confounders or the outcome, then the part of X driven by Z is as good as randomly assigned—and the relationship between that part and the outcome reflects a genuine causal effect. This logic is implemented through two-stage least squares: in effect, first predict X from the instrument, then use those predicted values—the clean variation—to estimate the effect on Y.

The two conditions for a valid instrument

An instrument is only valid if it satisfies two conditions, and both are essential.

The first is relevance: the instrument must genuinely affect the endogenous predictor. If Z is only weakly related to X, it captures too little clean variation to estimate the effect reliably—the problem of “weak instruments,” which can produce estimates that are more biased and more misleading than doing nothing at all. Relevance is the testable condition: you can examine the strength of the relationship between the instrument and the predictor directly, and a strong first-stage relationship is expected evidence.

The second is the exclusion restriction: the instrument must affect the outcome only through the predictor, and must be unrelated to the confounders in the error term. In other words, Z influences Y solely by way of X, with no direct path and no back door through omitted factors. This is the condition that makes the isolated variation clean—and it is the hard one, because it cannot be fully tested statistically. It rests on an argument about how the world works: you must make a convincing case, from theory and knowledge of the setting, that the instrument has no other route to the outcome.

Why good instruments are hard to find

The two conditions pull in opposite directions, which is what makes valid instruments genuinely scarce. A variable strongly connected to the treatment is often connected to the outcome through other routes as well, threatening the exclusion restriction; a variable safely disconnected from the outcome is often only weakly connected to the treatment, threatening relevance. Finding something that strongly moves the predictor and plausibly affects the outcome through no other channel is difficult, and much of the craft of IV analysis is in the search for, and defence of, such a variable.

The best instruments often come from sources of variation that are close to random for reasons outside the system being studied—an administrative rule, a lottery, a quirk of timing or geography, a natural experiment. When such a source exists and its exclusion can be credibly argued, IV can rescue a causal estimate that no amount of control variables could. But a poorly justified instrument is worse than none: it lends a false air of rigour to an estimate that may be badly biased. This is why a good IV analysis spends as much effort defending the instrument as estimating the effect.

What IV estimates—and what it doesn’t

A subtlety worth understanding is what effect IV recovers. When the effect of the treatment varies across individuals, an instrumental-variables estimate does not generally give the average effect across everyone. It gives the effect for those whose treatment is actually shifted by the instrument—the “compliers.” This is a real and useful quantity, but it is a local effect, specific to the group the instrument moves, and it may differ from the effect for the population as a whole. Interpreting an IV estimate as if it were the average effect for everyone is a common overreach; the honest interpretation is tied to whom the instrument actually influences.

The bottom line

Instrumental variables are among the most powerful tools in the causal-inference kit, because they can identify an effect even when the predictor is hopelessly entangled with the error term. But the power is conditional on a demanding pair of requirements: the instrument must be strongly relevant to the predictor, and—the harder, untestable part—it must affect the outcome only through the predictor. Good instruments are rare, weak or invalid ones are actively harmful, and the interpretation is local to those the instrument moves. Used with a well-argued instrument and honest interpretation, IV can turn an intractable endogeneity problem into a credible causal estimate; used carelessly, it dresses bias in the language of rigour. Knowing the difference is what separates a persuasive IV analysis from a fragile one.