Selection Bias vs Nonresponse Bias

Selection bias is a property of the sampling method, not of any one sample. It is present when the chance of being included is related to whatever is being measured, so the statistic is centered somewhere other than the parameter no matter how many times the method is run. A description of it is only finished once it names a direction: too high or too low, and for what reason.

Put numbers on one. A school has 600 students, 200 seniors and 400 underclassmen, and 70 percent of seniors against 40 percent of the others want a later start time, so the true proportion is $\frac{140 + 160}{600} = 0.50$. Surveys handed out at the cafeteria door reach only 25 percent of seniors, who leave campus at lunch, and 90 percent of everyone else. That covers $50 + 360 = 410$ students, of whom $35 + 144 = 179$ want the later start, so the method is aimed at $179 / 410 = 0.437$, about 6.3 percentage points low. Every sample it produces is centered on 0.437.

"Our sample came out 60 percent female, so it is biased" is a different claim and usually a wrong one. One lopsided sample is evidence of nothing. In a simple random sample of 100 from a population split evenly, women reach 60 or more about 2.8 percent of the time, and one sex or the other does about 5.7 percent of the time. Bias is diagnosed by asking who the method could never reach, not by inspecting the sample it happened to give.

Selection bias is settled before anyone is contacted. If someone was chosen properly and then failed to answer, that is a later leak with its own name: selection bias vs nonresponse bias.

Topic 1.12 names four biases, voluntary response, undercoverage, nonresponse, and response bias. Selection bias is the umbrella over the ones that act while the sample is still being chosen.

Nonresponse bias needs two conditions at once, and students usually check only the first. Selected people fail to give data, and those who responded differ from those who did not in a way that matters for the question. A low response rate by itself is not the bias, only the opening that lets it in: if the silent group would have answered like the responders, the estimate is not pushed anywhere.

A district emails a survey to a simple random sample of 250 teachers asking whether they have enough classroom supplies. 100 reply and 65 say yes, so the reported statistic is $\hat{p} = \frac{65}{100} = 0.65$ (p-hat) on a response rate of $\frac{100}{250} = 0.40$. Suppose that among the 150 who never replied, only 45 would have said yes, a rate of 0.30. The truth across the 250 selected teachers is $\frac{65 + 45}{250} = 0.44$. The survey overstates supply adequacy by 0.21, and nothing in the returned data reveals it.

Here is the sentence that costs points: "only 100 responded, so treat it as a sample of 100." That describes the sample size correctly and the sample wrongly. The 100 are not a random subset of the 250; they are the subset that chose to reply, so a standard error built from $n = 100$ describes scatter around 0.65, a center in the wrong place that a wider interval does not move. Emailing 1,000 teachers changes nothing: at the same reply rate the lean returns, measured more precisely.

Nonresponse happens after selection, and that is what separates it from its neighbors. All 250 teachers were on the list and could have answered. Had the list covered only full-time staff, part-timers would have had no chance of selection, which is undercoverage. Had all 250 replied while shading answers toward what an administrator wanted to hear, that would be response bias.

The defense is procedural, not statistical: follow-up contacts, a shorter instrument, and reporting the response rate next to the estimate. Topic 1.12 also wants a direction: the teachers who answered are plausibly those with the strongest feelings about supplies.