Least-Squares Criterion vs Least-Squares Regression Line

Out of every line you could draw through a scatterplot, the least-squares criterion keeps the one that minimizes $\sum (y - \hat{y})^2$, where $\hat{y}$ (read y-hat) is the height of the line above each $x$. Two details in that sentence do the work. The distances are vertical, measured parallel to the $y$ axis, not perpendicular to the line, so the criterion is asking how badly the line predicts $y$ and nothing else. And they are squared, which stops positives and negatives cancelling and makes one big miss cost more than several small ones.

Take the four points $(1, 3)$, $(2, 4)$, $(3, 8)$ and $(4, 9)$. The obvious line through the first and last of them is $\hat{y} = 1 + 2x$, leaving residuals 0, -1, 1 and 0, which score $0 + 1 + 1 + 0 = 2$. Least squares instead returns $\hat{y} = 0.5 + 2.2x$, whose residuals are 0.3, -0.9, 0.9 and -0.3 and score $0.09 + 0.81 + 0.81 + 0.09 = 1.8$. No line scores below 1.8 on these four points.

What that comparison kills is the rule students reach for first: "the best line is the one whose residuals add to zero." Both lines above do that. So does every line through the point of averages $(\bar{x}, \bar{y}) = (2.5, 6)$, read x-bar and y-bar, and there are infinitely many of those. Residuals summing to zero is a property the least-squares line happens to have, not the test that identifies it.

Least squares is one criterion, not the only sensible one. Minimizing the total unsigned residual is a different rule, and on these same four points it prefers $\hat{y} = 1 + 2x$, whose absolute total is 2.0 against the least-squares line's 2.4. Squaring is also what lets a single far-out point swing the fit, since a miss of 4 costs sixteen times a miss of 1.

The criterion is topic 5.5, Least-Squares Regression, in Unit 5 of the Fall 2026 AP Statistics course.

The deeper reason for squaring is a calculus one. A sum of squares is smooth, so its minimum sits where the derivative is zero and can be found by solving one equation, while a sum of absolute values has corners where no derivative exists and the minimum has to be hunted for. That is the same machinery as any optimization problem, and the corners are why a critical point includes places the derivative fails to exist, not just places it is zero.

The least-squares regression line is the one line that the least-squares criterion selects out of all the straight lines you could draw: the line minimizing $\sum (y - \hat{y})^2$, the total of the squared vertical residuals. It is written $\hat{y} = a + bx$ (read y-hat for $\hat{y}$), and its two coefficients come from the summary statistics, with slope $b = r \cdot \frac{s_y}{s_x}$ and intercept $a = \bar{y} - b\bar{x}$, where $\bar{x}$ is x-bar, the mean of the explanatory values.

Six students study 1, 2, 3, 4, 5 and 6 hours and score 60, 72, 65, 78, 71 and 86. Here $\bar{x} = 3.5$, $\bar{y} = 72$, $s_x = 1.8708$, $s_y = 9.2304$ and $r = 0.8107$, so $b = 0.8107 \times \frac{9.2304}{1.8708} = 4.00$ points per hour and $a = 72 - 4(3.5) = 58$. The fitted line is $\hat{y} = 58 + 4x$, its residuals are -2, 6, -5, 4, -7 and 4, and their squares total 146. No other straight line drives that total below 146.

The picture to correct is the one where the best-fit line threads through as many points as it can. This line passes through none of the six, and its smallest miss is 2 points. The point it does pass through is the point of averages $(\bar{x}, \bar{y}) = (3.5, 72)$, since $58 + 4(3.5) = 72$, and that holds for every least-squares line because it is the intercept formula rearranged. Quality of fit is a statement about all the residuals at once, never about how many points were hit.

Two limits come with the line. The formulas return one as long as the $x$-values are not all identical, curved data included, so the existence of a line is no evidence that a line belongs on the data. And squaring the misses leaves the fit with no resistance, so a single observation far out in $x$ can move $b$ a long way.

Least-squares regression is topic 5.5 in Unit 5, Regression Analysis.

A physics lab runs this procedure almost every week. Data is rearranged so the relationship plots as a straight line, a least-squares line is fitted to it, and a physical constant is read off the slope: linearization.