Y-Hat vs Residual

Both terms below come up in the same part of the course, and students mix them up. Here is each one defined on its own, side by side, so you can see where they part company.

Predicted value (y-hat)

Regression and correlation

The predicted value y-hat is the response a regression model gives for a chosen x, found by substituting that x into the equation of the line.

The predicted value is the height of the regression line above a chosen xx, not a value anyone observed. You get it by substituting: with y^=12+3x\hat{y} = 12 + 3x (read y-hat), an xx of 4 gives y^=12+3(4)=24\hat{y} = 12 + 3(4) = 24. If the individual with x=4x = 4 actually had y=26y = 26, the residual is 2624=226 - 24 = 2, so the model underpredicted by 2. Predicted values stay trustworthy only inside the range of xx used to fit the line; outside that range you are extrapolating.

Full entry for y-hat

Residual

Regression and correlation

A residual is the difference between an observed y-value and the value the regression line predicts for it: observed minus predicted.

A residual measures how far off the line is for a single point, so a positive residual sits above the line and a negative one below. It equals residual=yy^\text{residual} = y - \hat{y}, the observed response minus the predicted response y^\hat{y} (read y-hat). For example, if the line predicts 72 but a student actually scored 80, the residual is 8072=880 - 72 = 8. Least-squares regression chooses the line that makes the sum of the squared residuals as small as possible.

Full entry for residual

Where each one fits in the course