Scope of inference worksheet with answer key

8 problems on scope of inference, ready to print for class or homework. The answer key prints on its own page, so hand out everything before it.

Problem 1

For each study, say whether a cause-and-effect conclusion is supported and name the population the result covers.

(a) A cardiology group keeps a registry of 18,000 patients with one type of arrhythmia. It randomly selects 300 of them, then randomly assigns 150 to a new dosing schedule and 150 to the standard schedule.

(b) A polling firm randomly selects 1,000 people from a state's registered voter file and records how many hours each one volunteers per month and whether each one owns a dog.

(c) A gym owner posts a sign-up sheet, gets 40 volunteers, and randomly assigns 20 to each of two stretching routines before a timed sprint.

(d) A newsletter writer asks her subscribers to reply with their daily coffee intake and their sleep quality. 900 replies come back.

Problem 2

An auto insurer offers a phone app that tracks braking and speed, and customers choose whether to install it. Over one year, 1,820 of the 40,000 customers who installed the app filed a claim, compared with 3,900 of the 60,000 customers who did not.

(a) Find each claim rate and express the app group's rate as a percentage of the other group's. (b) Is this an experiment or an observational study? Say how you can tell. (c) The insurer's press release says the app makes drivers safer. Name a confounding variable and explain both links that make it confounding. (d) State the conclusion the insurer can defend, in the two-part form.

Problem 3

A sleep researcher recruits 90 volunteers from a university's psychology participant pool and randomly assigns 45 to a screen curfew, no phone or laptop after 9 p.m., and 45 to their usual evening habits, for two weeks. Mean time to fall asleep is 12.4 minutes in the curfew group and 21.8 minutes in the usual-habits group, a difference far larger than the study's chance variation.

(a) Find the difference in means and state which group fell asleep faster. (b) Write the causal half of the conclusion. (c) Write the generalization half of the conclusion. (d) The researcher wants to headline the result as "a screen curfew helps adults fall asleep faster." What single change to the study would justify the word adults, and why is that change hard to make? (e) Why does random assignment support a causal claim here even though nobody was randomly selected?

Problem 4

A state agriculture department draws a simple random sample of 900 of the state's 12,000 registered beekeepers and records, for each one, whether the hives sit within a mile of a sunflower field and whether the colony survived the winter. Of the 240 keepers with hives near sunflower fields, 204 colonies survived. Of the other 660, 429 survived.

(a) Find the survival rate in each group, the overall survival rate, and the gap. (b) Which cell of the selection-by-assignment grid is this study in? (c) State what may and may not be concluded, and about whom. (d) Name a confounding variable and explain both links. (e) Sketch a study that would support the causal claim, and say what population it would cover.

Problem 5

A state department of education randomly selects 60 of the state's 480 middle schools, then randomly assigns 30 of them to a 90-minute daily reading block and 30 to their current schedule, for one school year. Mean school reading score gain is 6.2 points under the new block and 2.9 points under the current schedule, a difference larger than chance variation would produce.

(a) What fraction of the state's middle schools were selected, and how many are in each group? (b) Identify the experimental units and explain why that matters for the wording of the conclusion. (c) Write the full two-part conclusion. (d) A principal in a neighboring state reads the study and adopts the block. Is that supported? Explain. (e) A reporter writes that the reading block raises any individual student's score by 3.3 points. Correct the sentence.

Problem 6

A researcher takes a simple random sample of 500 students from one large university's list of 21,000 enrolled students. Of the sampled students, 185 report studying in a group at least weekly and have a mean GPA of 3.28; the other 315 have a mean GPA of 3.09. The campus paper runs the headline "Group study raises grades for college students nationwide."

(a) Find the difference in mean GPA and verify that the overall sample mean GPA is 3.16. (b) The headline overreaches in two separate ways. Name each one and say which feature of the study it violates. (c) Rewrite the headline so that it claims exactly what the study supports. (d) Name a confounding variable and explain both links. (e) What one change would fix the first overreach, and what one change would fix the second?

Problem 7

A clinic recruits 200 patients with knee osteoarthritis from its own waiting list and randomly assigns 100 to a 12-week supervised exercise program and 100 to usual care. During the 12 weeks, 34 patients in the exercise group quit and 6 in the usual-care group quit. The clinic analyzes only the patients who finished: mean 6-minute walk distance is 462 meters for the exercise finishers and 418 meters for the usual-care finishers.

(a) How many patients are in each analyzed group, and what are the two dropout rates? (b) What would the study have supported if all 200 had been measured? (c) Explain why comparing the finishers breaks the causal claim. (d) Predict the direction of the resulting error in the 44 meter difference. (e) State what the clinic can defensibly report, and what it should do next time.

Problem 8

A city added protected bike lanes downtown in one year. In the year before, there were 84 cyclist collisions across 2.1 million bike trips. In the year after, there were 63 collisions across 2.8 million trips. The city also lowered the downtown speed limit from 30 to 25 miles per hour in the same year.

(a) Find the collision rate per 100,000 trips for each year and the percent decrease. (b) Why is comparing the raw counts 84 and 63 misleading here? (c) Classify the evidence and say what it supports. (d) Explain why the speed limit change is a confounding variable, using both links. (e) Describe a study that would support the causal claim, and name the population it would cover.

Scope of inference: answer key

  1. (a) Cause and effect supported; generalizes to the 18,000 patients in the registry. (b) Association only; generalizes to the registered voters on the state file. (c) Cause and effect supported; applies only to these 40 volunteers and people like them. (d) Association only; applies only to people similar to the subscribers who chose to reply.

  2. (a) 182040000=0.0455\frac{1820}{40000} = 0.0455 against 390060000=0.065\frac{3900}{60000} = 0.065; the app group's rate is 0.04550.065=0.70\frac{0.0455}{0.065} = 0.70, or 70% of the other rate, 1.95 percentage points lower. (b) Observational, because customers chose whether to install the app and no treatment was imposed. (c) Driving caution: cautious drivers are more willing to be monitored, and cautious drivers crash less anyway, so the app's effect and the drivers' pre-existing habits cannot be separated. (d) App use is associated with a lower claim rate among this insurer's customers; no causal claim is supported without random assignment, and no extension to drivers generally is supported without random selection.

  3. (a) 21.8−12.4=9.421.8 - 12.4 = 9.4 minutes faster for the curfew group. (b) Because the volunteers were randomly assigned, the 9.4 minute difference is evidence that the screen curfew caused faster sleep onset. (c) Because they were volunteers from one psychology participant pool rather than a random sample, the conclusion applies only to people similar to them. (d) Randomly selecting participants from the adult population, which is hard because selected adults can decline a two-week protocol and everyone who agrees is again a volunteer. (e) Random assignment balances the extraneous variables carried by these 90 people across the two groups, an argument that never refers to anyone outside the study.

  4. (a) 204240=0.85\frac{204}{240} = 0.85 and 429660=0.65\frac{429}{660} = 0.65, overall 633900≈0.703\frac{633}{900} \approx 0.703, a 20 percentage point gap. (b) Random selection, no random assignment: the top-right cell. (c) The association generalizes to the state's 12,000 registered beekeepers, but no causal claim is supported because placement was not assigned. (d) Keeper experience and management: experienced keepers both secure sunflower-adjacent sites and manage mites and feeding well, so their effect on survival cannot be separated from the sunflowers'. (e) Have each sampled keeper supply two comparable hives and randomly assign one of each pair to a sunflower-adjacent site; that adds random assignment to the existing random selection and would support causation for the state's registered beekeepers.

  5. (a) 60480=0.125\frac{60}{480} = 0.125, or 12.5%, with 30 schools per group. (b) The units are schools, since the schedule was assigned to a school, so every conclusion is a statement about schools and their mean gains. (c) Because the schools were randomly assigned, the 3.3 point difference in mean gain is evidence the reading block caused the higher gain; because they were randomly selected from the state's 480 middle schools, that conclusion extends to the state's middle schools. (d) No. The population sampled was one state's middle schools, so the result does not reach the neighboring state without replication there. (e) "Among the middle schools in the study, those randomly assigned the 90-minute reading block gained 3.3 more points on average than those keeping the current schedule."

  6. (a) 3.28−3.09=0.193.28 - 3.09 = 0.19 GPA points, and 185(3.28)+315(3.09)500=1580.15500=3.1603≈3.16\frac{185(3.28) + 315(3.09)}{500} = \frac{1580.15}{500} = 3.1603 \approx 3.16. (b) "Raises" claims cause without random assignment, since group study was self-selected; "nationwide" claims a population beyond the one university whose roster was sampled. (c) "Students at this university who study in groups have higher GPAs on average, by 0.19 points." (d) Major or prior preparation: both are tied to whether a student joins study groups and both affect GPA directly, so their effect cannot be separated from group study's. (e) Randomly assign group versus solo study to fix the causal overreach; randomly select from a national roster of enrolled students to fix the population overreach.

  7. (a) 66 and 94 analyzed; dropout rates 34100=0.34\frac{34}{100} = 0.34 and 6100=0.06\frac{6}{100} = 0.06, 28 percentage points apart. (b) That the program caused the longer walk distance, for patients similar to this clinic's waiting-list patients, since the 200 were not randomly selected. (c) The 66 and the 94 were not the groups chance created; patients filtered themselves out after assignment, in a way that depended on the treatment, so the comparison is now observational. (d) The 44 meters overstates the effect, because the patients doing worst on the program are the ones who left it while almost none left usual care. (e) Report an association among completers plus the 34% versus 6% dropout rates, and next time measure all 200 at week 12 and analyze by assigned group.

  8. (a) 842100000×100000=4.00\frac{84}{2100000} \times 100000 = 4.00 and 632800000×100000=2.25\frac{63}{2800000} \times 100000 = 2.25 collisions per 100,000 trips, a decrease of 1.75, or 1.754.00=43.75%\frac{1.75}{4.00} = 43.75\%. (b) Trips rose about 33%, so the raw counts compare different amounts of cycling and understate the improvement. (c) An observational before-and-after study with neither random assignment nor random selection, so it supports an association for this city's downtown only. (d) The speed limit dropped at the same time as the lanes went in, so it is tied to the before-versus-after variable, and lower speeds cut collisions on their own; the two effects cannot be separated. (e) Randomly select 40 of the city's 260 comparable corridors, randomly assign 20 to get lanes and 20 to keep the current layout with the speed limit held fixed, and compare rates after a year; that supports causation for this city's comparable downtown corridors.

The key gives final answers only. The full worked solution for each problem is on the matching practice set, and every other sheet is on the worksheets page.

More worksheets from this unit