Undercoverage

By Jude Wallis · Published

Undercoverage occurs when the list a sample is drawn from leaves out part of the population, so those individuals have no chance of being selected.

Undercoverage is a gap between two groups that are easy to run together: the population you want to describe, and the sampling frame, the list your method can actually draw from. Anyone in the first but not the second has probability zero of being selected, so those people are missing from every sample the method will ever produce, not just from this one.

A college with 4,000 students wants the proportion who eat at the dining hall at least twice a week, and draws a random sample from the residence hall roster. That roster holds the 2,500 students who live on campus and none of the 1,500 commuters. Suppose 0.80 of residents and 0.25 of commuters eat there that often. The parameter is 0.80(2500)+0.25(1500)4000=23754000=0.594\frac{0.80(2500) + 0.25(1500)}{4000} = \frac{2375}{4000} = 0.594, while the method is aimed at 0.80. Sample after sample scatters around 0.80, about 0.21 above the truth.

"Then survey more students" is the repair people reach for, and it is the wrong one. The frame does not change when the sample grows. Draw 2,000 of the 2,500 residents and the estimate is still centered on 0.80, now with a much smaller standard error, which only makes a wrong answer look authoritative.

The test that separates undercoverage from its neighbors is whether the person could have been selected at all. The commuters could not; they were never on the roster. A commuter who was on the list, was contacted, and never replied would be nonresponse bias instead, a failure one stage later.

One boundary keeps this honest. An incomplete frame biases an estimate only when the missing group differs on the variable being measured. If commuters ate at the dining hall at the same 0.80 rate as residents, this estimate would sit on the parameter even though the roster is just as incomplete, and that same roster would still wreck a survey about parking. Topic 1.12 wants both halves: name the excluded group, then say which way the estimate moves.

Where this comes up

More collecting data and study design terms, or browse the full statistics glossary.