Residual vs Outlier

A residual belongs to one data point and one fitted line, and the order of the subtraction is fixed: $\text{residual} = y - \hat{y}$, observed minus predicted, where $\hat{y}$ (read y-hat) is the height of the line above that point's $x$. The sign is where the meaning lives. A positive residual puts the observation above the line, so the model underpredicted it, and a negative residual puts it below, so the model overpredicted.

Six students study 1, 2, 3, 4, 5 and 6 hours and score 60, 72, 65, 78, 71 and 86. The least-squares line for them is $\hat{y} = 58 + 4x$. For the student who studied 5 hours the line gives $\hat{y} = 58 + 4(5) = 78$, and that student scored 71, so the residual is $71 - 78 = -7$. The line overpredicted that score by 7 points. Doing the same at every $x$ gives residuals of -2, 6, -5, 4, -7 and 4.

The error that costs points is the reversed subtraction. "The residual is 7, so the line missed by 7" keeps the size and throws the direction away, and computing predicted minus observed flips the sign of every residual in the table, turning each overprediction into an underprediction. Say observed minus predicted every time, and read a negative residual as the line sitting above the data point.

Two things bound what a single residual can tell you. It is measured against one particular line, so change the line and every residual changes with it. And for a least-squares line fitted with an intercept the residuals add to exactly zero: here $-2 + 6 - 5 + 4 - 7 + 4 = 0$. Their mean is zero for a good fit and for a hopeless one alike, which is why fit gets judged from the squared residuals and from the shape of the residual plot, never from their average.

Residuals are topic 5.4 in Unit 5, Regression Analysis, of the Fall 2026 AP Statistics course.

An outlier is a value that stands apart from the rest of the data, and the AP course gives two rules for deciding when apart counts. The first flags any value more than $1.5 \times \text{IQR}$ above the third quartile $Q_3$ or below the first quartile $Q_1$. The second flags any value more than 2 standard deviations from the mean. Neither rule is the definition of an outlier. They are criteria, and they do not always agree.

Take 12, 13, 19, 19, 20, 23, 25, 27, 35, 40. The quartiles are $Q_1 = 19$ and $Q_3 = 27$, so the IQR is 8 and the upper fence sits at $27 + 1.5(8) = 39$. The value 40 clears it, so the IQR rule calls 40 an outlier. Now the other rule: the mean is 23.3 and $s = 8.91$, so two standard deviations reach $23.3 + 2(8.91) = 41.12$, and 40 does not get there. Same data, two rules, opposite answers. Say which rule you used.

"There is an outlier, so it must be a mistake and I should drop it." An outlier is a statement about position inside this one data set and nothing more. A 40 from a broken sensor is an error; a 40 from an unusually productive week is the most informative point you have. Removing a flagged value is a decision about the study that needs a reason you can write down, not an arithmetic step.

The 2 standard deviation rule quietly fails on small samples. No observation among $n$ values can sit more than $\frac{n-1}{\sqrt{n}}$ sample standard deviations from the mean, because the value doing the stretching also inflates $s$. With $n = 5$ that ceiling is 1.79, so the rule flags nothing at all however extreme one of the five values is. The 1.5 IQR rule has no such blind spot.

Outliers sit in topic 1.7 of the Fall 2026 course, which is where both rules are listed, and topic 1.6 names outliers, gaps, and clusters as the unusual features every description of a distribution has to mention in context.