Generalizability vs Scope of Inference

Both terms below come up in the same part of the course, and students mix them up. Here is each one defined on its own, side by side, so you can see where they part company.

Generalizability

Collecting data and study design

Generalizability is whether a study's results extend to a larger population, and it depends on how the sample was selected, not on random assignment.

Random selection from a population is what earns the right to generalize to that population. Without random selection, you can only generalize to a population of individuals similar to those actually studied. For example, a well-run randomized experiment on 80 student volunteers at one university supports a cause-and-effect claim for people like those volunteers, but not a claim about all college students. That is why the two randomizations get judged separately: assignment supports causation, and selection supports generalization.

Full entry for generalizability

Scope of inference

Collecting data and study design

Scope of inference is how far a study's conclusions reach: random assignment permits causal claims, and random selection permits claims about the population.

Handle the two randomizations separately, because they answer different questions. Random assignment of treatments makes the groups comparable, so a significant difference can be credited to the treatment; random selection makes the sample representative, so the result extends to the population sampled. For example, randomly assigning treatments to 60 volunteers supports cause and effect for people like those volunteers only, while a random survey of 60 town residents describes the town but cannot establish cause. A study with both random selection and random assignment is the only kind that supports a causal claim about the whole population.

Full entry for scope of inference

Where each one fits in the course