High-leverage point

By Jude Wallis · Updated

A high-leverage point has an x-value far from the mean of x, which gives it the power to move the regression line, whether or not it actually does.

Leverage is a property of the explanatory variable alone. For point ii it is hi=1n+(xixˉ)2(xjxˉ)2h_i = \frac{1}{n} + \frac{(x_i - \bar{x})^2}{\sum (x_j - \bar{x})^2}, where xˉ\bar{x} is read x-bar, so you can work it out before looking at a single response value. It is at least 1n\frac{1}{n} and at most 1, and a point sitting at the mean of xx carries the least leverage available to it.

Five points, (1,3)(1, 3), (2,5)(2, 5), (3,4)(3, 4), (4,7)(4, 7) and (5,8)(5, 8), fit y^=1.8+1.2x\hat{y} = 1.8 + 1.2x. Now add a sixth at x=20x = 20. Whatever its response, its leverage is 0.967, against values of about 0.17 to 0.26 for the other five. Give it a value on the pattern, (20,26)(20, 26), and the line barely notices: y^=1.77+1.21x\hat{y} = 1.77 + 1.21x. Give it (20,10)(20, 10) instead and the slope collapses to y^=4.37+0.31x\hat{y} = 4.37 + 0.31x.

Now the trap. "If a point were distorting the fit, the residual plot would show it." In that second fit, the residual at x=20x = 20 is -0.52, the second smallest of the six in size, because the line swung far enough to almost meet the point. The damage shows up on the other five, whose residuals are now -1.68, 0.01, -1.30, 1.40 and 2.09, against 0, 0.80, -1.40, 0.40 and 0.20 under the original fit. A high-leverage point that has already bent the line hides itself behind a small residual.

Leverage is a warning, not a verdict. Both sixth points above have identical leverage, 0.967, and one of them changes almost nothing. Whether a point actually moves the fit is influence, and you settle that by refitting without it and comparing the slope, the intercept and rr. So keep the three labels apart: an outlier has a large residual, a high-leverage point has an extreme xx, and an influential point is one whose removal visibly changes the line.

Where this comes up

More regression and correlation terms, or browse the full statistics glossary.