Linear model

A linear model claims that a response is well described by a straight line plus scatter around it. It is written $\hat{y} = a + bx$, where $\hat{y}$ (read y-hat) is the value the line returns, $a$ is the y-intercept, $b$ is the slope, and each observed $y$ differs from $\hat{y}$ by a residual. Any straight line through a scatterplot is a linear model; the least-squares regression line is the particular one that minimizes the sum of the squared residuals.

Six students study 1, 2, 3, 4, 5 and 6 hours and score 60, 72, 65, 78, 71 and 86, and the least-squares model is $\hat{y} = 58 + 4x$. Drop the hat and write $y = 58 + 4x$ and the equation becomes false for all six of them: it asserts scores of 62, 66, 70, 74, 78 and 82 when the students actually scored 60, 72, 65, 78, 71 and 86. The hat is what turns a false equation into an honest prediction.

So the misreading to name is "the data follow the equation." They do not, and no data ever do. A linear model is a summary of the pattern, and the leftover scatter is part of the description rather than a defect in it. A second version of the same slip is treating the model as a fact about the world instead of about these observations: refit on a different sample and $a$ and $b$ come out different.

Nothing in the arithmetic checks whether a line belongs on the data. Feed a clean parabola into a regression routine and it returns a slope, an intercept and a correlation without complaint. The form of the scatterplot and the shape of the residual plot are what justify the model, and the model speaks only for the range of $x$ the data covered.

Linear regression models are topic 5.3 in Unit 5, Regression Analysis.