Can you generalize these results? (scope of inference)

Two random acts, two different permissions

Scope of inference is two questions asked one after the other, and the answers come from two different design choices.

Random assignment decides which units receive which treatment. When treatments are randomly assigned to experimental units, the potential for confounding variables is reduced, so a difference in the response between the groups can be attributed to the treatments. That is what buys a cause-and-effect conclusion.

Random selection decides who is in the study at all. When the units in a sample are randomly selected from a population, it is appropriate to make generalizations about the entire population they were drawn from. When they are not, generalizations are appropriate only to a population of individuals similar to those used in the study.

These two choices are independent. You can randomize the assignment without randomizing the selection, and you can randomize the selection without assigning anything at all. The course warns specifically about mixing the two words up: write about random selection when you generalize to a population, and about random assignment when you explain a causal claim.

The 2x2 the exam tests

Read it as two independent dials. The columns set what kind of relationship you may claim. The rows set who the claim applies to.

Nothing in one dimension repairs a gap in the other. A convenience sample of 50,000 people still cannot be generalized, and a randomized experiment on 30 volunteers still supports causation for those kinds of volunteers. That is the point people lose most often, and it is why does a bigger sample fix bias answers no.

Both randomizations: the strongest conclusion, and the rarest

A logistics company employs 4,000 warehouse workers. It randomly selects 200 of them from the payroll list, then randomly assigns 100 to a new shift-swapping app and 100 to the existing paper sign-up sheet for one quarter, and records how many extra shifts each worker picks up. The app group picks up more.

Both dials are set. Random assignment means the two groups of 100 were built to be alike on everything except the tool, so the difference in shifts is attributable to the app. Random selection from the payroll list means the 200 stand in for all 4,000 workers, so the conclusion extends to the whole workforce.

This combination is uncommon in practice, and the course says why. It may be unethical or simply difficult to randomly select experimental units to take part in an experiment, so most experiments recruit volunteers instead. When a study does manage both, say both, and cite the matching randomization for each half of the claim.

Random assignment without random selection: the honest case

A researcher recruits 90 volunteers from a university psychology participant pool, randomly assigns 45 to a 9 p.m. screen curfew and 45 to no curfew for two weeks, and measures nightly sleep from a wearable. The curfew group sleeps longer.

The causal conclusion survives intact. Random assignment did its job: the two groups were built to be alike on everything except the curfew, so the difference in sleep is attributable to the curfew. What does not survive is the audience.

The 90 volunteers were not randomly selected from any defined population, so the finding applies to a population of individuals similar to those in the study, meaning undergraduates who signed up for a sleep experiment. It does not apply to adults in general, to shift workers, or to teenagers.

This is the most common shape in real research, and it is not a flaw to apologize for. Most experiments run on people who agreed to be in them, because drafting people into a treatment at random is usually impossible and often unethical. State the causal claim, state the narrow audience, and stop. Designing such a study is covered in how to design an experiment.

Random selection without random assignment: generalize, but not to a cause

A state agriculture office takes a simple random sample of 250 of the 4,000 licensed beekeepers in the state and asks about winter hive losses and hive insulation. Beekeepers who insulate report lower losses.

Here the audience is wide and the claim is weak. Because the 250 were randomly selected from the full list, generalizing to all 4,000 licensed beekeepers is appropriate. Because nobody assigned insulation, the study cannot say insulation caused the lower losses.

The reason is confounding. Beekeepers chose for themselves whether to insulate, so a variable associated with both insulating and hive survival, such as years of experience or how aggressively someone treats for mites, offers a rival explanation that the study cannot rule out. A confounding variable has to be associated with both the explanatory and the response variable, and in observational studies it usually is.

Every well-run survey lands in this cell, which is why survey results are reported as associations. The distinction between assigning conditions and merely recording them is drawn in experiments vs observational studies, and the reasoning trap it creates is in correlation vs causation.

Neither: describe the sample and stop

A gym posts a sign-up sheet for a new stretching class and later compares the flexibility of the 40 members who signed up with 40 regulars who did not. The stretchers are more flexible.

Both permissions are gone at once. There is no random selection, because participants put themselves on the list, and there is no random assignment, because they also chose their own group. You can describe what was observed in these 80 members, and that is the whole of what the study supports.

The rival explanations write themselves. People who join a stretching class may already be more flexible, younger, or more motivated than people who do not, so the observed gap has as many candidate causes as there are differences between the two groups. Self-selection into groups is the mechanism that produces confounding, which is why nonrandom sampling methods carry potential bias by construction. Picking a defensible method instead is the subject of how to choose a sampling method.

How to write the scope-of-inference answer

A scope-of-inference answer is two sentences, each with its reason attached. Both must name the actual study and the actual population, not the general rule.

1. The causation sentence. Either: because the treatments were randomly assigned, it is reasonable to conclude that the treatment caused the difference in the response. Or: because no treatments were assigned, the study shows an association but cannot establish that one variable caused the other.
2. The generalization sentence. Either: because the units were randomly selected from the population, the results can be generalized to that population. Or: because the units volunteered rather than being randomly selected, the results extend only to individuals similar to those in the study.

Two habits protect the points. Give a clear yes or no first and then the reason, because a hedge with no decision earns nothing. Then cite the right randomization for the right half: the causal sentence cites random assignment, and the generalization sentence cites random selection.

Naming a placebo, blinding, a control group, or a large sample as the reason for a causal claim does not work, because none of those balance the extraneous variables across groups. Only random assignment does that. Scope of inference sits in topic 1.10 and again in topic 1.13, and you can rehearse the wording in experimental design practice and sampling methods practice. The official framework is at AP Central.