Scope of inference

Scope of inference is two answers, not one: what kind of relationship the study supports, and about whom. The first is settled by whether treatments were randomly assigned to the units. The second is settled by whether the units were randomly selected from a stated population. Both are fixed by the design before any data exist, and nothing done during the analysis widens either one.

A state association has 1,200 licensed electricians. A researcher randomly selects 120 of them from the membership roll, randomly assigns 60 to a pre-inspection checklist and 60 to current practice, and records faults per 100 inspections: 4.1 for the checklist group against 6.3 for current practice, a difference of 2.2. Both randomizations are present, so the conclusion has both halves. Because the checklist was randomly assigned, it caused the lower fault rate. Because the 120 were randomly selected from the roll, that reaches the association's 1,200 electricians and stops there.

"The result was significant at $\alpha = 0.01$, so the effect holds for electricians generally" is the sentence to kill. Evidence is not scope. A p-value says how surprising the data would be if the null were true; it knows nothing about who was recruited or how treatments were handed out. A smaller p-value, a narrower interval, and a fancier model all leave the audience and the causal permission exactly where the design left them.

Missing a randomization is a smaller claim rather than a broken study, and the four combinations of the two are laid out under generalizability. The narrow cells are the ordinary ones: almost every experiment runs on volunteers, and almost every survey assigns nothing to anybody.

Scope of inference surfaces twice in the course, in topic 1.10 on data collection and again in topic 1.13 on experimental design. Write it as two sentences, each naming the randomization that licenses it.