Response bias vs nonresponse bias

By Jude Wallis · Published

Response bias happens when people answer a survey but their answers lean away from the truth, because of wording, social pressure, or faulty memory. Nonresponse bias happens when selected people give no answer and differ from those who do. One is about answers given, the other about answers missing.

AP Statistics: Unit 1 (topics 1.12 Potential Problems with Sampling). Response bias and nonresponse bias are both covered in topic 1.12, Potential Problems with Sampling, in Unit 1 of the Fall 2026 AP Statistics course. A typical task is to name the bias from a description of a survey and to explain the direction in which it is likely to move the estimate.

Response bias vs nonresponse bias: the short answer

The names are close and the problems are different. Response bias is about the answers you collected: people responded, and what they said tends to differ from the truth in one direction. Nonresponse bias is about the answers you did not collect: people chosen for the sample did not respond, and the ones who did respond are different from the ones who did not.

A quick test separates them. Ask whether the person gave an answer. If they answered and the answer is off, think response bias. If they were selected and gave no answer, think nonresponse bias. Both can push an estimate away from the population value, and both can occur in the same survey.

What response bias is

Response bias occurs when the responses themselves are systematically inaccurate. Common sources:

  • Question wording. A leading question pushes people toward one answer. A confusing question gets answers to something other than what was meant. See question wording bias.
  • Sensitive topics. People under-report behavior that is embarrassing or illegal and over-report behavior that looks good, such as voting or exercising.
  • The interviewer. Who asks, and whether the answer is anonymous, can change what people are willing to say.
  • Memory. People misremember how often or how recently they did something.

The sample can be chosen perfectly and every selected person can answer, and response bias can still be present. The fix is in the measurement: neutral wording, anonymity, or records in place of self-reports.

What nonresponse bias is

Nonresponse bias can occur when people selected for a sample cannot be reached or decline to answer. Two things must both be true. Some selected people gave no data, and those people differ from the responders in a way related to the question.

A mailed survey about job satisfaction illustrates it. If the employees who are busiest and most stressed are the least likely to send it back, the returned surveys over-represent the satisfied ones. Every answer on every returned form may be honest, and the result is still too high.

A low response rate alone does not prove bias. If the people who stayed silent would have answered the same way as the people who replied, the estimate is not pushed in either direction. The trouble is that the survey data cannot show whether that is so. The fix is in the follow-up: repeated contact attempts, reminders, and making the survey short and easy to complete.

The differences side by side

FeatureResponse biasNonresponse bias
Did the person answer?YesNo
Where the problem isIn the answers givenIn the answers missing
Typical causesLeading or confusing wording, sensitive questions, interviewer effects, faulty memoryUnreachable or unwilling people who differ from responders
Can occur with a 100% response rateYesNo
Warning sign in a problemA loaded question, a face-to-face interview on a private topic, self-reported behaviorA low response rate, mailed or emailed surveys, calls at one time of day
Fixed by a larger sampleNoNo
How to reduce itNeutral wording, anonymity, objective recordsFollow-up contacts, reminders, shorter surveys

Neither problem shrinks when the sample grows. A larger sample taken the same way repeats the same lean with less sampling variability around it, as does a bigger sample fix bias explains.

When each one applies

Read the description of the survey and find the step that went wrong.

If the description gives a response rate, or says that many people did not return the survey, did not pick up the phone, or refused, the issue is nonresponse. Go on to say how the nonresponders probably differ and which way that moves the estimate.

If the description quotes the question, names who asked it, or involves a topic people might not answer truthfully, the issue is response bias. Say which direction the answers lean and why.

A third problem sits nearby. People who were never on the list the sample was drawn from had no chance of selection, which is undercoverage. The undercoverage vs nonresponse bias page separates those two, and how to identify the type of bias covers the full set.

The classic mix-up and how to avoid it

The usual mistake is reading "response bias" as "a bias involving who responds" and using it for people who did not answer. The names point the other way. Response bias needs a response. Nonresponse bias needs a missing one.

A second mistake is stopping at the label. "There is nonresponse bias" earns little credit without the rest. A complete answer names the bias, explains how the people involved differ or why their answers lean, and states the direction: the sample proportion is likely higher, or likely lower, than the true population proportion.

Avoid describing every problem as "the sample is not representative." With response bias the sample may represent the population well and the answers are still wrong.

The same survey with each bias

A gym selects a simple random sample of 500 members and asks, "Did you work out at least three times last week?" Check-in records show that 255 of the 500 actually did. Scenario 1: the question is emailed, 200 members reply, and 150 of them say yes. Suppose all 200 replies are truthful. Scenario 2: a trainer asks all 500 members in person, everyone answers, and 380 say yes. For each scenario, find the survey's estimate, compare it with the truth for the 500, and name the bias.

  1. Find the true proportion for the 500 selected members from the records: 255500=0.51\frac{255}{500} = 0.51.

  2. Scenario 1 estimate: p^=150200=0.75\hat{p} = \frac{150}{200} = 0.75. The response rate is 200500=0.40\frac{200}{500} = 0.40.

  3. Find what the 300 nonresponders would have said. Of the 255 who worked out three times, 150 replied, so 255−150=105255 - 150 = 105 did not. Among nonresponders the proportion is 105300=0.35\frac{105}{300} = 0.35.

  4. Compare responders and nonresponders: 0.75 against 0.35. Frequent gym users were much more likely to reply, so the estimate is too high by 0.75−0.51=0.240.75 - 0.51 = 0.24. Every answer was truthful and 300 answers are missing, so this is nonresponse bias.

  5. Scenario 2 estimate: p^=380500=0.76\hat{p} = \frac{380}{500} = 0.76. The response rate is 100%, so nothing is missing.

  6. Compare with the records: 380 said yes and only 255 did it, so at least 380−255=125380 - 255 = 125 members said yes when the records say no. The estimate is too high by 0.76−0.51=0.250.76 - 0.51 = 0.25. Everyone answered and the answers lean toward the more flattering choice, so this is response bias.

Scenario 1 estimates 0.75 against a true 0.51, an overestimate of 0.24 caused by nonresponse bias: the 60% who did not reply worked out less often (0.35) than those who did (0.75). Scenario 2 estimates 0.76, an overestimate of 0.25 caused by response bias: all 500 answered, and at least 125 of them overstated their exercise when asked face to face by a trainer.

Naming the bias and its direction

For each survey, name the more likely bias and the direction of its effect. (a) A school emails a random sample of 400 parents a survey about satisfaction with the school, and 90 reply. (b) A principal asks a random sample of 60 students, one at a time in her office, whether they have ever cheated on a test. All 60 answer. (c) A random sample of adults is asked, "Do you agree that the city should stop wasting money on a new stadium?"

  1. Survey (a): find the response rate, 90400=0.225\frac{90}{400} = 0.225. Most selected parents gave no answer, so the concern is nonresponse.

  2. State a direction for (a) with a reason. Parents with strong feelings, often complaints, are more likely to take the time to reply, so the proportion who report being satisfied is likely lower than the true proportion among all parents. The opposite direction is possible, and the reasoning has to be stated either way.

  3. Survey (b): everyone answered, so nonresponse is ruled out. Students are being asked about misconduct by the principal, in person.

  4. State a direction for (b). Students are likely to deny cheating, so the sample proportion who admit it is likely lower than the true proportion. This is response bias.

  5. Survey (c): the question itself contains the phrase "wasting money", which pushes people to agree.

  6. State a direction for (c). The proportion who oppose the stadium is likely higher than it would be with neutral wording. This is response bias from question wording.

(a) Nonresponse bias, with a 22.5% response rate; the satisfaction estimate is likely too low if dissatisfied parents reply more readily. (b) Response bias; the proportion admitting to cheating is likely too low. (c) Response bias from leading wording; opposition to the stadium is likely overstated.

Frequently asked questions

Can a survey have both response bias and nonresponse bias?

Yes. They arise at different steps, so one survey can have both. A mailed survey with a leading question and a low return rate has missing answers from people who differ from the responders, and a lean in the answers that did come back.

Does a low response rate always mean nonresponse bias?

No. Nonresponse bias requires that the people who did not respond differ from those who did in a way related to the question. A low response rate makes that possible and gives no way to check it from the survey data, which is why it is treated as a serious warning.

Is voluntary response bias the same as nonresponse bias?

No. In a voluntary response sample nobody is selected, and people choose on their own to take part. Nonresponse bias arises after the researcher has selected specific people and some of them do not answer. Both leave the estimate depending on who chose to answer.