Standard deviation of the residuals (s)

By Jude Wallis · Updated

The standard deviation of the residuals, s, is roughly the typical size of a prediction error from the regression line, in the units of y.

The value ss answers the question a prediction raises: when I use this line, how far off am I usually? It is

s=(yy^)2n2s = \sqrt{\frac{\sum (y - \hat{y})^2}{n - 2}}

Square the residuals so they cannot cancel, divide by n2n - 2 rather than n1n - 1 because fitting a line spends two estimates, the slope and the intercept, then take the square root to land back in the units of the response. Software prints it as S.

Six students study 1, 2, 3, 4, 5 and 6 hours and score 60, 72, 65, 78, 71 and 86. Least squares gives y^=58+4x\hat{y} = 58 + 4x (read y-hat), leaving residuals of -2, 6, -5, 4, -7 and 4, whose squares sum to 146. With n=6n = 6, s=146/4=6.04s = \sqrt{146 / 4} = 6.04 points. Predicting a score from study time with this line misses by about 6 points in a typical case. Ignoring study time and predicting the mean of 72 for everyone would miss by more: the standard deviation of the six scores alone is 9.23.

The sentence to delete is "ss is the average residual." The residuals of a least-squares line add to zero, so their average is zero for every data set ever fitted, which is why ss works with their squares instead. A second slip treats 6.04 as a ceiling. It is a typical miss, not a maximum, and one residual in this very example is -7.

ss is also not a standard error, though it is the raw material for one. A standard error describes how much a statistic bounces from sample to sample; ss describes how far individual responses sit from the line. The standard error of the slope is built out of it, as s/(xxˉ)2s / \sqrt{\sum (x - \bar{x})^2}. And with n=2n = 2 the line passes through both points, so the formula reads 0/0\sqrt{0/0} and ss does not exist.

Where this comes up

More regression and correlation terms, or browse the full statistics glossary.