Y-Hat vs Residual
Both terms below come up in the same part of the course, and students mix them up. Here is each one defined on its own, side by side, so you can see where they part company.
Predicted value (y-hat)
Regression and correlation
The predicted value y-hat is the response a regression model gives for a chosen x, found by substituting that x into the equation of the line.
The predicted value is the height of the regression line above a chosen , not a value anyone observed. You get it by substituting: with (read y-hat), an of 4 gives . If the individual with actually had , the residual is , so the model underpredicted by 2. Predicted values stay trustworthy only inside the range of used to fit the line; outside that range you are extrapolating.
Residual
Regression and correlation
A residual is the difference between an observed y-value and the value the regression line predicts for it: observed minus predicted.
A residual measures how far off the line is for a single point, so a positive residual sits above the line and a negative one below. It equals , the observed response minus the predicted response (read y-hat). For example, if the line predicts 72 but a student actually scored 80, the residual is . Least-squares regression chooses the line that makes the sum of the squared residuals as small as possible.