Least-squares regression line: find and interpret it

What the least-squares regression line is

The least-squares regression line is the straight line that best fits paired quantitative data by making the total squared prediction error as small as possible. The AP course writes it as $\hat{y} = a + bx$, where $\hat{y}$ (say "y-hat") is the predicted value of the response variable, $x$ is the explanatory variable, $a$ is the y-intercept, and $b$ is the slope.

The name comes from the rule that picks the line. For every data point, the vertical gap between the observed $y$ and the predicted $\hat{y}$ is a residual. The least-squares line is the single line that minimizes the sum of those residuals squared, which is why it is also called the LSRL, or least-squares regression line. Squaring the gaps makes larger misses count much more, and it makes the best-fit line unique.

One property is worth memorizing: the least-squares line always passes through the point $(\bar{x}, \bar{y})$, where $\bar{x}$ ("x-bar") is the mean of the x-values and $\bar{y}$ ("y-bar") is the mean of the y-values. That mean point acts like a pivot the line balances on.

This is Unit 5 material, topics 5.3 and 5.5, and Unit 5 makes up 10 to 20 percent of the multiple-choice section. Keep in mind that $\hat{y}$ estimates an average response, not a guaranteed outcome for any one individual.

How to find the least-squares regression line

On the AP exam a graphing calculator with statistical capabilities is expected, and it reports the slope, intercept, and correlation directly. The provided formula sheet gives the line's form, $\hat{y} = a + bx$. Seeing how the coefficients come from summary statistics still helps you understand and sanity-check that output.

Start with five summary numbers: the means $\bar{x}$ and $\bar{y}$, the sample standard deviations $s_x$ and $s_y$ (the typical spread of the x-values and of the y-values), and the correlation coefficient $r$ (a number from -1 to 1 measuring linear strength). The slope comes first:

The slope scales the correlation by the ratio of the two spreads, so it carries units of $y$ per one unit of $x$. Next, the intercept uses the fact that the line runs through $(\bar{x}, \bar{y})$:

Compute $b$ first, then substitute it into the intercept formula. The worked example below runs both steps on a small data set, digit by digit.

How to interpret slope and intercept in context

Interpreting the two coefficients is a common free-response task, and the AP course expects each to be stated in context, not as a bare number.

Slope. The slope $b$ is the predicted change in the response variable for a one-unit increase in the explanatory variable. Use this template: "For each additional [one unit of x], the model predicts [y] will [increase or decrease] by [b] [units of y]." Say predicted, not actual, and do not claim $x$ causes the change unless the data came from an experiment. The units matter: a slope of 7.5 points per hour is a complete answer, while "the slope is 7.5" is not.

Intercept. The intercept $a$ is the predicted response when the explanatory variable equals 0. Use this template: "When [x] is 0 [units], the model predicts [y] is [a] [units]."

The intercept does not always carry a sensible meaning. If $x = 0$ falls far outside the range of x-values used to fit the line, the intercept is an extrapolation. It can also be impossible in context, such as a negative value for something like height, in which case you say the intercept has no logical interpretation rather than force one.

For the line $\hat{y} = 23.5 + 7.5x$ in the worked examples below, where $x$ is hours spent studying and $y$ is a quiz score out of 100, the slope reads: for each additional hour of studying, the model predicts the quiz score increases by 7.5 points.

Extrapolation versus interpolation

Once you have the line, predicting looks easy: substitute an x-value and compute $\hat{y}$. The real question is which x-values you can trust.

Interpolation is predicting for an x-value inside the interval of x-values used to build the line. Those predictions are the reliable ones.

Extrapolation is predicting for an x-value beyond that interval. The AP course warns that a predicted value is less reliable the further you extrapolate, because no data out there confirms the trend keeps going. A line fit to study times from 1 to 5 hours says nothing trustworthy about 12 hours, and it can produce impossible values like a quiz score above 100.

In the worked examples, predicting a score for 3.5 hours of studying is interpolation because 3.5 sits between 1 and 5, while predicting for 12 hours is extrapolation and returns 113.5 points, off the 100-point scale.

Common mistakes with the regression line

A handful of slips show up over and over on this topic.

• Confusing the slope with the correlation. The correlation $r$ has no units and stays between -1 and 1, while the slope $b$ carries units and can be any size. They share a sign but answer different questions.
• Predicting x from y. The line $\hat{y} = a + bx$ predicts $y$ from $x$. You cannot rearrange it to read an x-value off a y-value and expect the least-squares fit to hold.
• Dropping the word predicted. The slope and intercept describe predicted values. Writing "the score goes up 7.5 points" as if it were guaranteed misses what regression actually claims.
• Rounding too early. Keep extra decimals in $s_x$, $s_y$, and $r$ through the slope and intercept steps, and round only the final coefficients.

Check your work and go further

To skip the arithmetic, paste your data into the regression calculator and read off $a$, $b$, $r$, and $r^2$. Working the line out by hand once builds the intuition; after that, let technology do the computing on the exam.

The square of the correlation, $r^2$, is the share of the variation in $y$ that the line explains; for this data $r^2 \approx 0.987$, about 98.7 percent. The guide on r versus r-squared explains the difference between the two. To watch a single unusual point drag the whole line, open the influential point interactive, and to check whether a line was even the right model, see how to interpret residual plots.