Extrapolation vs Influential Point

Both terms below come up in the same part of the course, and students mix them up. Here is each one defined on its own, side by side, so you can see where they part company.

Extrapolation

Regression and correlation

Extrapolation is using a regression line to predict outside the range of x-values the data covered, where the pattern may no longer hold.

Extrapolation is defined by where the explanatory value sits, not by how the answer looks. If the xx you substitute falls outside the interval of xx-values used to fit the line, the prediction is an extrapolation even when it comes out perfectly plausible, and an xx inside that interval is not an extrapolation even when the prediction turns out badly. The range of the response has nothing to do with it.

Six students study 1, 2, 3, 4, 5 and 6 hours and score 60, 72, 65, 78, 71 and 86, giving y^=58+4x\hat{y} = 58 + 4x. Predict for a student who studies 12 hours and the line returns 58+4(12)=10658 + 4(12) = 106 on a test scored out of 100. The arithmetic is correct. What is missing is any observation past 6 hours, so nothing in these data says the gain of 4 points per hour keeps going rather than flattening off, and the line has no way to know that scores stop at 100.

The defense to reject is "the fit is strong, so the prediction is fine." Strength is measured over the observed range only. Here r=0.8107r = 0.8107 and r2=0.657r^2 = 0.657 describe how the line does between 1 and 6 hours, and both are unchanged by which xx you feed the equation afterwards. Regression output prints no warning when you leave the data behind, so the check has to be yours: write down the smallest and largest xx before predicting.

Extrapolation is not banned. Forecasting is often the whole point of fitting a line, and a prediction just past the edge of the data is usually more defensible than one far beyond it. What you lose is the ability to check the assumption against evidence, so an extrapolated value gets stated with its assumption attached. The y-intercept is the case people miss: a=58a = 58 is the predicted score at x=0x = 0 hours, which is outside the range 1 to 6, so reading it as a claim about a student who does not study at all is itself an extrapolation.

Full entry for extrapolation

Influential point

Regression and correlation

An influential point is an observation that, if removed, would markedly change the regression line's slope, intercept, or correlation.

Influence is defined by a comparison, not by appearance: fit the line with the point, fit it again without, and see how much the answer moves. That makes influence a verdict you reach by refitting, unlike the two labels it gets confused with. An outlier in regression has a large residual, a high-leverage point has an xx far from xˉ\bar{x} (x-bar), and neither one is automatically influential.

Start with six students who study 1, 2, 3, 4, 5 and 6 hours and score 60, 72, 65, 78, 71 and 86, so y^=58+4x\hat{y} = 58 + 4x and r=0.8107r = 0.8107. Add a seventh who studied 10 hours and scored 75. That point is 23 below the old line, and refitting drops the slope from 4 to 1.6144 and the correlation to 0.5681. Now instead add a seventh who studied 3 hours and scored 100, a residual of 30, larger than the first. Refitting moves the slope only to 3.2742. The bigger miss shifted the line about a third as far, because it sat next to xˉ\bar{x} where the line can pivot around it.

So the sentence to drop is "that point has a huge residual, so it is influential." Residual size is not the measure. Distance from the mean of xx is what converts a miss into leverage over the whole fit.

Leverage on its own is not the verdict either. Add a seventh student at 10 hours who scored 98, which sits exactly on the original line: that point has more leverage than any other in the set, and the fit stays at y^=58+4x\hat{y} = 58 + 4x, unchanged to the last digit. It does lift rr from 0.8107 to 0.9245, so it is influential for the correlation while having no influence at all on the line. Always say what it is influential for.

Finding an influential point is not permission to delete it. Investigate it, and report the fit both ways if you cannot resolve it.

Full entry for influential point

Where each one fits in the course