High-Leverage Point vs Influential Point

Leverage is a property of the explanatory variable alone. For point $i$ it is $h_i = \frac{1}{n} + \frac{(x_i - \bar{x})^2}{\sum (x_j - \bar{x})^2}$, where $\bar{x}$ is read x-bar, so you can work it out before looking at a single response value. It is at least $\frac{1}{n}$ and at most 1, and a point sitting at the mean of $x$ carries the least leverage available to it.

Five points, $(1, 3)$, $(2, 5)$, $(3, 4)$, $(4, 7)$ and $(5, 8)$, fit $\hat{y} = 1.8 + 1.2x$. Now add a sixth at $x = 20$. Whatever its response, its leverage is 0.967, against values of about 0.17 to 0.26 for the other five. Give it a value on the pattern, $(20, 26)$, and the line barely notices: $\hat{y} = 1.77 + 1.21x$. Give it $(20, 10)$ instead and the slope collapses to $\hat{y} = 4.37 + 0.31x$.

Now the trap. "If a point were distorting the fit, the residual plot would show it." In that second fit, the residual at $x = 20$ is -0.52, the second smallest of the six in size, because the line swung far enough to almost meet the point. The damage shows up on the other five, whose residuals are now -1.68, 0.01, -1.30, 1.40 and 2.09, against 0, 0.80, -1.40, 0.40 and 0.20 under the original fit. A high-leverage point that has already bent the line hides itself behind a small residual.

Leverage is a warning, not a verdict. Both sixth points above have identical leverage, 0.967, and one of them changes almost nothing. Whether a point actually moves the fit is influence, and you settle that by refitting without it and comparing the slope, the intercept and $r$. So keep the three labels apart: an outlier has a large residual, a high-leverage point has an extreme $x$, and an influential point is one whose removal visibly changes the line.

Influence is defined by a comparison, not by appearance: fit the line with the point, fit it again without, and see how much the answer moves. That makes influence a verdict you reach by refitting, unlike the two labels it gets confused with. An outlier in regression has a large residual, a high-leverage point has an $x$ far from $\bar{x}$ (x-bar), and neither one is automatically influential.

Start with six students who study 1, 2, 3, 4, 5 and 6 hours and score 60, 72, 65, 78, 71 and 86, so $\hat{y} = 58 + 4x$ and $r = 0.8107$. Add a seventh who studied 10 hours and scored 75. That point is 23 below the old line, and refitting drops the slope from 4 to 1.6144 and the correlation to 0.5681. Now instead add a seventh who studied 3 hours and scored 100, a residual of 30, larger than the first. Refitting moves the slope only to 3.2742. The bigger miss shifted the line about a third as far, because it sat next to $\bar{x}$ where the line can pivot around it.

So the sentence to drop is "that point has a huge residual, so it is influential." Residual size is not the measure. Distance from the mean of $x$ is what converts a miss into leverage over the whole fit.

Leverage on its own is not the verdict either. Add a seventh student at 10 hours who scored 98, which sits exactly on the original line: that point has more leverage than any other in the set, and the fit stays at $\hat{y} = 58 + 4x$, unchanged to the last digit. It does lift $r$ from 0.8107 to 0.9245, so it is influential for the correlation while having no influence at all on the line. Always say what it is influential for.

Finding an influential point is not permission to delete it. Investigate it, and report the fit both ways if you cannot resolve it.