Influential points and outliers: regression practice

Set the two tests. A point is an outlier in the y direction if it sits far from the pattern vertically, meaning a large residual $y - \hat{y}$ (observed minus predicted). A point has high leverage if its x-value sits far from $\bar{x}$, no matter how it lines up vertically.

Part (a), hive A. Its 9 frames equal $\bar{x} = 9$ exactly, so it sits dead center horizontally and has no leverage. The pattern predicts $\hat{y} = 2 + 3(9) = 2 + 27 = 29$ kg, and the hive gave 45 kg, so it lies $45 - 29 = 16$ kg above the pattern, far more than the other hives miss by.

Part (b), hive B. Its 24 frames sit well outside the 4 to 14 range of every other hive, so it has high leverage. The pattern predicts $\hat{y} = 2 + 3(24) = 2 + 72 = 74$ kg, and the hive gave 74 kg, so it lands right on the extension of the line with a residual of 0.

Part (c), hive C. Its 26 frames again sit far outside 4 to 14, so it has high leverage. The pattern predicts $\hat{y} = 2 + 3(26) = 2 + 78 = 80$ kg, and the hive gave 24 kg, a miss of $24 - 80 = -56$ kg, so it is also far off in the y direction.

Part (d), hive D. Its 6 frames sit inside the 4 to 14 range and only 3 frames from $\bar{x} = 9$, so no leverage. The pattern predicts $\hat{y} = 2 + 3(6) = 2 + 18 = 20$ kg against 22 kg observed, a residual of $+2$ kg, which is ordinary.

Sort out which one threatens the line. Hive C combines a far-out x-value with a huge vertical miss, so it is the one most likely to swing the slope. Hive A is far off vertically but sits at the center in x, where it has little grip on the tilt of the line, and hive B follows the pattern it extends.

Part (a), measure horizontal distance in standard deviations with $\frac{x - \bar{x}}{s_x}$, where $s_x$ is the standard deviation (typical spread) of the ages. Vine A: $\frac{13 - 12}{4} = \frac{1}{4} = 0.25$. Vine B: $\frac{26 - 12}{4} = \frac{14}{4} = 3.5$.

Finish part (a). Vine C: $\frac{25 - 12}{4} = \frac{13}{4} = 3.25$. Vine D: $\frac{11 - 12}{4} = -\frac{1}{4} = -0.25$.

Read the leverage verdict. Vines B and C sit more than 3 standard deviations from the mean age and outside the 6 to 18 year range of the rest, so both have high leverage. Vines A and D sit a quarter of a standard deviation from the mean, so neither has leverage.

Read the vertical verdict from the residuals. Vine A at 9.2 kg and vine C at 8.8 kg fall outside the -4 to +4 kg band that holds every vine not listed in the table, so both are outliers in the y direction. Vine B at -0.4 kg and vine D at -1.1 kg are ordinary.

Part (b), combine the two verdicts. Vine A is an outlier only, vine B is high leverage only, vine C is both, and vine D is neither.

Part (c), pick the likely troublemaker. Vine C pairs a far-out age with a large vertical miss, so it has both the grip and the pull to move the line, making it the best candidate for an influential point.

Explain why vine B stays unresolved. A high-leverage point can drag the fitted line toward itself, and once the line has moved, that point's own residual comes out small. So a residual of -0.4 kg is consistent with a vine that follows the pattern and also with a vine that bent the line to reach it. Refit the line without vine B and compare the slope, intercept, and $r$ to tell the two apart.

Part (a), find the means for all five weeks. $\bar{x} = \frac{2 + 4 + 6 + 8 + 20}{5} = \frac{40}{5} = 8$ cm and $\bar{y} = \frac{7 + 13 + 13 + 19 + 13}{5} = \frac{65}{5} = 13$ kg.

Find the deviations from each mean. For $x$: $-6, -4, -2, 0, 12$. For $y$: $7 - 13 = -6$, $13 - 13 = 0$, $13 - 13 = 0$, $19 - 13 = 6$, $13 - 13 = 0$.

Sum the products of the paired deviations. $\sum (x - \bar{x})(y - \bar{y}) = (-6)(-6) + (-4)(0) + (-2)(0) + (0)(6) + (12)(0) = 36 + 0 + 0 + 0 + 0 = 36$.

Sum the squared x-deviations, then divide. $\sum (x - \bar{x})^2 = 36 + 16 + 4 + 0 + 144 = 200$, so $b = \frac{36}{200} = 0.18$ kg per cm.

Part (b), drop the flood week and redo the means. $\bar{x} = \frac{2 + 4 + 6 + 8}{4} = \frac{20}{4} = 5$ cm and $\bar{y} = \frac{7 + 13 + 13 + 19}{4} = \frac{52}{4} = 13$ kg, unchanged.

Recompute the two sums with four weeks. The x-deviations are $-3, -1, 1, 3$ and the y-deviations are still $-6, 0, 0, 6$, so $\sum (x - \bar{x})(y - \bar{y}) = 18 + 0 + 0 + 18 = 36$ and $\sum (x - \bar{x})^2 = 9 + 1 + 1 + 9 = 20$. The slope is $b = \frac{36}{20} = 1.8$ kg per cm.

Part (c), compare. Dropping one of five weeks takes the slope from 0.18 to 1.8 kg per cm, ten times steeper, so the flood week is strongly influential. Its rainfall of 20 cm is well beyond the 2 to 8 cm of every other week, so it is high leverage as well.

Check the residual, which is the surprise. With all five weeks the intercept is $a = \bar{y} - b\bar{x} = 13 - 0.18(8) = 13 - 1.44 = 11.56$, so the flood week's predicted value is $11.56 + 0.18(20) = 11.56 + 3.6 = 15.16$ kg and its residual is $13 - 15.16 = -2.16$ kg. The 8 cm week has predicted value $11.56 + 0.18(8) = 13.00$ kg and residual $19 - 13 = +6.00$ kg, so the most influential point does not have the largest residual.

Part (a), find the means for all five sheds. $\bar{x} = \frac{1 + 2 + 3 + 4 + 10}{5} = \frac{20}{5} = 4$ panels and $\bar{y} = \frac{2 + 6 + 6 + 10 + 24}{5} = \frac{48}{5} = 9.6$ kWh.

Find the deviations. For $x$: $-3, -2, -1, 0, 6$. For $y$: $2 - 9.6 = -7.6$, $6 - 9.6 = -3.6$, $6 - 9.6 = -3.6$, $10 - 9.6 = 0.4$, $24 - 9.6 = 14.4$.

Compute the two sums and the line. $\sum (x - \bar{x})(y - \bar{y}) = 22.8 + 7.2 + 3.6 + 0 + 86.4 = 120$ and $\sum (x - \bar{x})^2 = 9 + 4 + 1 + 0 + 36 = 50$, so $b = \frac{120}{50} = 2.4$ kWh per panel and $a = 9.6 - 2.4(4) = 9.6 - 9.6 = 0$. The line is $\hat{y} = 2.4x$.

Part (b), drop the 10-panel shed and redo everything. $\bar{x} = \frac{10}{4} = 2.5$ panels and $\bar{y} = \frac{24}{4} = 6$ kWh, the x-deviations are $-1.5, -0.5, 0.5, 1.5$ and the y-deviations are $-4, 0, 0, 4$.

Finish part (b). $\sum (x - \bar{x})(y - \bar{y}) = 6 + 0 + 0 + 6 = 12$ and $\sum (x - \bar{x})^2 = 2.25 + 0.25 + 0.25 + 2.25 = 5$, so $b = \frac{12}{5} = 2.4$ and $a = 6 - 2.4(2.5) = 6 - 6 = 0$. The line is $\hat{y} = 2.4x$ again, identical to two decimal places and in fact exactly the same.

Part (c), use $r^2 = \frac{\left(\sum (x - \bar{x})(y - \bar{y})\right)^2}{\sum (x - \bar{x})^2 \cdot \sum (y - \bar{y})^2}$. With all five sheds, $\sum (y - \bar{y})^2 = 57.76 + 12.96 + 12.96 + 0.16 + 207.36 = 291.2$, so $r^2 = \frac{120^2}{50 \cdot 291.2} = \frac{14400}{14560} = 0.9890$ and $r = \sqrt{0.9890} = 0.994$.

Finish part (c) without the 10-panel shed. $\sum (y - \bar{y})^2 = 16 + 0 + 0 + 16 = 32$, so $r^2 = \frac{12^2}{5 \cdot 32} = \frac{144}{160} = 0.900$ and $r = \sqrt{0.900} = 0.949$.

Part (d), give the split verdict. With 10 panels against 1 to 4 for every other shed, the point is clearly high leverage. It leaves the slope and intercept untouched, so it is not influential on the line, but it lifts $r^2$ from 0.900 to 0.989 by stretching the x-range while staying on the pattern, so it is influential on the correlation. High leverage does not force influence, and the effect on the line and the effect on $r$ can point in different directions.

Part (a), run both tests. At 44 knots the crossing sits at double the largest of the other wind speeds, far from $\bar{x}$, so it has high leverage. It also sits far below the pattern of the other 29, so it has a large residual. It is both.

Part (b), picture where the point pulls. The crossing sits low and far to the right, so the least-squares line has to tip its right end down to reach it, flattening the fit. Take that anchor away and the right end springs back up, so the slope gets larger, meaning steeper in the positive direction.

Part (c), reason about $r$. The point contradicts the positive association shown by the other 29 crossings, so it weakens the linear pattern and holds $r$ closer to 0. Removing it leaves a cleaner positive pattern, so $r$ increases toward 1.

Part (d), square the result. Since $r$ is positive and rises, $r^2$ rises with it, so a larger share of the variation in fuel is explained by wind speed once the shortened-route crossing is out.

Name what parts (b) through (d) show. Removing one point out of 30 changes both the slope and $r$ noticeably, which is the definition of an influential point.

Part (e), switch to the separate scenario. Here the same 29 ordinary crossings are joined by a 30th that ran in 40 knots and burned almost exactly what the pattern predicts, replacing the shortened-route crossing of parts (a) through (d). At 40 knots it still sits far out in x, so it still has high leverage, but it lands on the pattern the other 29 set, so removing it barely moves the slope: the line was already pointing at it.

Finish part (e) with the effect on $r$. That crossing stretched the x-range while adding almost nothing to the scatter around the line, which usually makes the fit look tighter, so $r$ and $r^2$ typically go down when it is removed. It is high leverage, not influential on the slope, and still influential on $r$.

Part (a), square the correlation for all 18 islands. $r^2 = (0.48)^2 = 0.2304$, so about 23.0 percent of the variation in the number of nesting pairs is explained by the linear relationship with island area.

Square the correlation for the 17 islands. $r^2 = (0.86)^2 = 0.7396$, so about 74.0 percent of the variation in nesting pairs is explained by island area once the 300-hectare island is set aside.

Compare the two. The explained share goes from roughly 23 percent to roughly 74 percent, more than tripling, on the strength of one island out of 18.

Part (b), apply the definition. An influential point is one whose removal noticeably changes the slope, the intercept, or $r$. Removing this island moves $r$ from 0.48 to 0.86, so yes, it is influential. Its area of 300 hectares against a maximum of 40 for the others also makes it high leverage, which is why it has so much grip.

Part (c), find the direction. The island sits at a very large x-value and below the pattern, so it drags the right end of the fitted line downward. Removing it releases that end, so the slope gets larger.

Part (d), separate influence from error. Influence is a statement about arithmetic, not about correctness. A 300-hectare island with poor nesting habitat, heavy predation, or human traffic would produce exactly this point and be perfectly real, so a jump in $r$ is a reason to investigate the island, not a license to delete it.

Part (a), find the means for all five sessions. $\bar{x} = \frac{1 + 2 + 3 + 4 + 10}{5} = \frac{20}{5} = 4$ fixers and $\bar{y} = \frac{4 + 6 + 10 + 12 + 3}{5} = \frac{35}{5} = 7$ items.

Find the deviations. For $x$: $-3, -2, -1, 0, 6$. For $y$: $4 - 7 = -3$, $6 - 7 = -1$, $10 - 7 = 3$, $12 - 7 = 5$, $3 - 7 = -4$.

Compute the two sums and the line. $\sum (x - \bar{x})(y - \bar{y}) = 9 + 2 - 3 + 0 - 24 = -16$ and $\sum (x - \bar{x})^2 = 9 + 4 + 1 + 0 + 36 = 50$, so $b = \frac{-16}{50} = -0.32$ and $a = 7 - (-0.32)(4) = 7 + 1.28 = 8.28$. The line is $\hat{y} = 8.28 - 0.32x$.

Part (b), find the launch session's residual at $x = 10$. $\hat{y} = 8.28 - 0.32(10) = 8.28 - 3.2 = 5.08$ items, so the residual is $3 - 5.08 = -2.08$ items.

Find the 4-fixer session's residual at $x = 4$. $\hat{y} = 8.28 - 0.32(4) = 8.28 - 1.28 = 7.00$ items, so the residual is $12 - 7.00 = +5.00$ items, which is larger in size than the launch session's $-2.08$.

Part (c), drop the launch session and refit. $\bar{x} = \frac{1 + 2 + 3 + 4}{4} = 2.5$ fixers and $\bar{y} = \frac{4 + 6 + 10 + 12}{4} = \frac{32}{4} = 8$ items, with x-deviations $-1.5, -0.5, 0.5, 1.5$ and y-deviations $-4, -2, 2, 4$.

Finish part (c). $\sum (x - \bar{x})(y - \bar{y}) = 6 + 1 + 1 + 6 = 14$ and $\sum (x - \bar{x})^2 = 2.25 + 0.25 + 0.25 + 2.25 = 5$, so $b = \frac{14}{5} = 2.8$ and $a = 8 - 2.8(2.5) = 8 - 7 = 1$. The line is $\hat{y} = 1 + 2.8x$, and the slope has gone from $-0.32$ to $+2.8$, flipping the direction of the whole relationship.

Part (d), explain the small residual. Because the launch session sits far out in x, the least-squares line swings toward it, and most of its distance from the other points gets absorbed by tilting the line rather than left over as a residual. The tilt then pushes the leftover error onto the ordinary sessions, which is why the 4-fixer session ends up with the largest residual. A small residual at a high-leverage point is evidence that the point moved the line, not evidence that it is harmless.

Part (a), test the reasoning. Deleting whichever point fits worst always improves the reported $r^2$, so 'the fit improves' would justify deleting your way to any conclusion you like. The jump from 0.41 to 0.88 tells you the day is influential; it says nothing about whether the day is correct.

Part (b), handle a documented recording error. This is an error in transcription with a paper trail, so the fix is to correct the value from 104 to 74 degrees, refit with the corrected value, and state in the write-up what was changed and why. Correcting the number is better than deleting the row, because the sales figure for that day is still good data.

Part (c), handle a case that does not belong. The temperature is genuine, but a day when the shop was open half its usual hours is not measuring the same thing as the other 29 days. You may exclude it as long as you say so plainly: name the day, give the reason (the power outage), and report the fit both with and without it.

Part (d), handle the case with no reason at all. Keep the day. The honest report includes it, describes it as an influential high-leverage point, and shows the analysis both ways so a reader can see how much of the conclusion rests on that single day.

State the general rule. The justification for removing a point has to come from outside the arithmetic, in the form of a documented recording error, a measurement failure, or a case that does not belong to the population being studied. The size of the change in $r^2$ or in the slope is what makes a point worth investigating, never what makes removal legitimate.

Note what to write in all four cases. The write-up should name the point, say what was done to it, give the reason for that choice, and report the slope and $r^2$ for both versions of the analysis.