Least-Squares Criterion vs Least-Squares Regression Line

Out of every line you could draw through a scatterplot, the least-squares criterion keeps the one that minimizes $\sum (y - \hat{y})^2$, the total squared vertical miss, where $\hat{y}$ is read y-hat. Squaring makes large misses count heavily and stops positive and negative residuals from cancelling, since the residuals of this line always add to 0. For example, if the least-squares line for three points leaves residuals 3, -1, and -2, it scores $3^2 + (-1)^2 + (-2)^2 = 9 + 1 + 4 = 14$, and no other line scores lower on those three points. One consequence worth remembering: the least-squares line always passes through $(\bar{x}, \bar{y})$, the point of averages, read x-bar and y-bar.

The least-squares regression line is the best-fit line for predicting the response from the explanatory variable, chosen to minimize the total squared vertical distance from the points. Its equation is $\hat{y} = a + bx$, where $\hat{y}$ (read y-hat) is the predicted response, $a$ is the y-intercept, and $b$ is the slope. For example, $\hat{y} = 50 + 5x$ predicts a score of 75 when $x = 5$ hours of study. The line always passes through the point of averages $(\bar{x}, \bar{y})$.