Extrapolation vs Least-Squares Regression Line

Extrapolation is defined by where the explanatory value sits, not by how the answer looks. If the $x$ you substitute falls outside the interval of $x$-values used to fit the line, the prediction is an extrapolation even when it comes out perfectly plausible, and an $x$ inside that interval is not an extrapolation even when the prediction turns out badly. The range of the response has nothing to do with it.

Six students study 1, 2, 3, 4, 5 and 6 hours and score 60, 72, 65, 78, 71 and 86, giving $\hat{y} = 58 + 4x$. Predict for a student who studies 12 hours and the line returns $58 + 4(12) = 106$ on a test scored out of 100. The arithmetic is correct. What is missing is any observation past 6 hours, so nothing in these data says the gain of 4 points per hour keeps going rather than flattening off, and the line has no way to know that scores stop at 100.

The defense to reject is "the fit is strong, so the prediction is fine." Strength is measured over the observed range only. Here $r = 0.8107$ and $r^2 = 0.657$ describe how the line does between 1 and 6 hours, and both are unchanged by which $x$ you feed the equation afterwards. Regression output prints no warning when you leave the data behind, so the check has to be yours: write down the smallest and largest $x$ before predicting.

Extrapolation is not banned. Forecasting is often the whole point of fitting a line, and a prediction just past the edge of the data is usually more defensible than one far beyond it. What you lose is the ability to check the assumption against evidence, so an extrapolated value gets stated with its assumption attached. The y-intercept is the case people miss: $a = 58$ is the predicted score at $x = 0$ hours, which is outside the range 1 to 6, so reading it as a claim about a student who does not study at all is itself an extrapolation.

The least-squares regression line is the one line that the least-squares criterion selects out of all the straight lines you could draw: the line minimizing $\sum (y - \hat{y})^2$, the total of the squared vertical residuals. It is written $\hat{y} = a + bx$ (read y-hat for $\hat{y}$), and its two coefficients come from the summary statistics, with slope $b = r \cdot \frac{s_y}{s_x}$ and intercept $a = \bar{y} - b\bar{x}$, where $\bar{x}$ is x-bar, the mean of the explanatory values.

Six students study 1, 2, 3, 4, 5 and 6 hours and score 60, 72, 65, 78, 71 and 86. Here $\bar{x} = 3.5$, $\bar{y} = 72$, $s_x = 1.8708$, $s_y = 9.2304$ and $r = 0.8107$, so $b = 0.8107 \times \frac{9.2304}{1.8708} = 4.00$ points per hour and $a = 72 - 4(3.5) = 58$. The fitted line is $\hat{y} = 58 + 4x$, its residuals are -2, 6, -5, 4, -7 and 4, and their squares total 146. No other straight line drives that total below 146.

The picture to correct is the one where the best-fit line threads through as many points as it can. This line passes through none of the six, and its smallest miss is 2 points. The point it does pass through is the point of averages $(\bar{x}, \bar{y}) = (3.5, 72)$, since $58 + 4(3.5) = 72$, and that holds for every least-squares line because it is the intercept formula rearranged. Quality of fit is a statement about all the residuals at once, never about how many points were hit.

Two limits come with the line. The formulas return one as long as the $x$-values are not all identical, curved data included, so the existence of a line is no evidence that a line belongs on the data. And squaring the misses leaves the fit with no resistance, so a single observation far out in $x$ can move $b$ a long way.

Least-squares regression is topic 5.5 in Unit 5, Regression Analysis.

A physics lab runs this procedure almost every week. Data is rearranged so the relationship plots as a straight line, a least-squares line is fitted to it, and a physical constant is read off the slope: linearization.