AP Stats 1.12: Problems with Sampling

What bias is

Topic 1.12 is about the ways a sampling method can go wrong, chiefly bias, a systematic error in the sampling procedure that makes a statistic come out consistently larger or consistently smaller than the parameter it estimates. Bias is not bad luck on one sample; it is a lean built into the method, so taking a bigger sample does not fix it. The word systematic is the key: the error repeats in the same direction every time you use the method, which separates it from ordinary chance variation. The cure is a better method, usually one that uses random chance to select individuals.

The named types of bias

The CED names four kinds of bias plus a root cause, and each describes a different point where the method can lean.

• Voluntary response bias can occur when the sample is made up of volunteers, who often hold stronger opinions than the population.
• Undercoverage bias can occur when the method leaves out part of the population, or makes part of it less likely to be selected.
• Nonresponse bias can occur when some chosen individuals do not respond, and responders differ from non-responders in ways that matter for the study.
• Response bias can occur when answers tend to differ from the truth in one direction, from confusing or leading question wording or from sensitive self-reported answers.

Underlying several of these, nonrandom methods such as convenience or voluntary response introduce bias because they do not use random chance to select individuals.

Naming bias on the exam

On the exam, do more than label the bias. Say which direction it pushes the statistic and why, in the context of the study. A website poll that asks readers to opt in tends to attract stronger opinions, so its estimate is biased in that direction.

Random selection is the main defense, because it removes the human choices that create these leans; see simple random vs stratified sampling. Keep bias separate from a statistic simply varying from sample to sample, which is sampling variability, not bias, since a good method can still give a different value on each sample. That distinction between a statistic and a parameter is covered in parameter vs statistic.