AP Statistics sampling and experiments cheat sheet
By Jude Wallis · Published
Random selection lets a result generalize to the population sampled. Random assignment lets a difference too large for chance be attributed to the treatment. This sheet lists the sampling methods, the named types of bias, the elements and designs of experiments, and what each study can conclude.
AP Statistics: Unit 1 (topics 1.10 The Investigative Question Revisited and Data Collection, 1.11 Random Sampling, 1.12 Potential Problems with Sampling, 1.13 Experimental Design). Data collection is Unit 1, topics 1.10 through 1.13, of the Fall 2026 AP Statistics course: study types, random sampling methods, bias, and experimental design.
Sampling methods
| Method | How it works | Useful when |
|---|---|---|
| Simple random sample (SRS) | Every possible sample of size has the same chance of being chosen | A list of the whole population exists |
| Stratified random sample | Divide the population into strata of similar individuals, take an SRS from each stratum, and combine | Groups differ on the response and each should be represented |
| Cluster sample | Divide the population into clusters, take an SRS of clusters, and include everyone in the chosen clusters | The population is spread across natural groups and a full list is hard to get |
| Systematic random sample | Choose a random starting point, then take every th individual | Individuals come in a list or a line |
Strata are built to be alike inside and different from each other. Clusters ideally each resemble the whole population. In a stratified sample some individuals from every group are chosen. In a cluster sample all individuals from some groups are chosen.
Two methods do not use chance and tend to produce bias: a convenience sample takes whoever is easy to reach, and a voluntary response sample takes whoever chooses to respond. See how to choose a sampling method and cluster vs stratified sampling.
Sources of bias
Bias is a systematic tendency for a method to give statistics that are too high or too low. A larger sample from the same method does not remove it.
| Bias | What goes wrong | Example |
|---|---|---|
| Voluntary response bias | The sample is made of volunteers, who often hold stronger opinions | An online poll that anyone can choose to answer |
| Undercoverage bias | Part of the population is left out or is less likely to be selected | A phone survey that cannot reach people without phones |
| Nonresponse bias | Selected individuals do not respond, and they differ from those who do | A mailed survey that few people return |
| Response bias | Answers differ from the truth in a consistent direction | A leading question, or a sensitive question answered face to face |
A full answer names the bias, explains how it arises in this study, and states the direction: whether the statistic is likely to be too high or too low. See how to identify the type of bias.
Types of studies
| Study | What happens | Can show cause and effect |
|---|---|---|
| Census | Data are collected from every individual in the population | No; a census imposes no treatments |
| Observational study | Variables are recorded without imposing treatments. A survey asks a standard set of questions, a prospective study follows individuals forward in time, and a retrospective study uses past data | No, because confounding variables may explain an association |
| Experiment | The researcher assigns treatments to experimental units and measures a response | Yes, when treatments are randomly assigned |
A confounding variable is associated with both the explanatory variable and the response variable, so it offers another explanation for the relationship between them. See experiments vs observational studies.
Elements of a well-designed experiment
| Element | Meaning |
|---|---|
| Comparison | At least two treatment groups, one of which may be a control group |
| Random assignment | Chance decides which experimental units get which treatment |
| Replication | More than one experimental unit receives each treatment |
| Control | Other variables that could affect the response are held the same for all units |
Random assignment spreads the other variables across the treatment groups by chance. A difference in the response then has two possible explanations: the treatments or the chance in the assignment. An inference procedure addresses the second.
| Term | Meaning |
|---|---|
| Experimental unit | The individual or object to which a treatment is assigned; people are called subjects or participants |
| Explanatory variable (factor) | The variable whose levels form the treatments |
| Response variable | The outcome measured after treatment |
| Control group | A group used for comparison, which may receive a placebo or no treatment |
| Placebo | An inactive treatment |
| Single-blind | Either the subjects or the researchers who interact with them do not know which treatment was given |
| Double-blind | Neither the subjects nor the researchers who interact with them know which treatment was given |
Three experimental designs
| Design | How treatments are assigned | Use when |
|---|---|---|
| Completely randomized design | Treatments are assigned to all experimental units completely at random | Units are similar, or no variable is known to matter |
| Randomized block design | Units are grouped into blocks that are alike on a blocking variable, and treatments are randomly assigned within each block | A known variable is expected to affect the response |
| Matched pairs design | Units are paired on a relevant variable and the two treatments are randomly assigned within each pair, or each unit receives both treatments in random order | There are two treatments and good matches exist |
Blocking accounts for the variation caused by the blocking variable, which makes the comparison of treatments more precise. A matched pairs design is a randomized block design with blocks of size two. Describing a design means saying how the random assignment is carried out, for example by numbering the units and using a random number generator. See how to design an experiment and blocking vs stratifying.
Scope of inference
Two separate questions decide what a study can conclude.
| How the study was done | Treatments randomly assigned | No random assignment |
|---|---|---|
| Individuals randomly selected from a population | Cause and effect, generalized to that population | Association only, generalized to that population |
| Individuals not randomly selected | Cause and effect, for individuals like those in the study | Association only, for individuals like those in the study |
Random selection is about who is in the study, and it supports generalizing. Random assignment is about who gets which treatment, and it supports a cause-and-effect conclusion. Neither one makes up for the absence of the other. See can you generalize results and random sampling vs random assignment.
Plan a stratified random sample
A school has 1,200 students: 420 freshmen, 330 sophomores, 270 juniors, and 180 seniors. The principal wants a stratified random sample of 80 students, with each grade's share of the sample equal to its share of the school. Find how many students to sample from each grade and describe how to select them.
Freshmen: .
Sophomores: .
Juniors: .
Seniors: .
Check: .
Selection: within each grade, number the students from 1 to the grade's size, use a random number generator to pick the required count of different numbers, and sample the students with those numbers.
Sample 28 freshmen, 22 sophomores, 18 juniors, and 12 seniors, each chosen as a simple random sample within its grade. The grades are the strata.
Decide what a study can conclude
A gym asks for volunteers and 60 members sign up. The gym randomly assigns 30 of them to morning workouts and 30 to evening workouts for eight weeks. At the end, the morning group has lost significantly more weight on average. Can the gym conclude that morning workouts cause more weight loss for all of its members?
Random assignment: yes. The 60 volunteers were randomly assigned to the two workout times.
Consequence: a cause-and-effect conclusion is supported for the people in the study.
Random selection: no. The 60 members volunteered, so they are not a random sample of the gym's members.
Consequence: the result generalizes only to members similar to the volunteers.
The gym can conclude that morning workouts caused more weight loss for members like those who volunteered. It cannot extend that conclusion to all members, because the participants were not randomly selected.
Frequently asked questions
What is the difference between a stratified sample and a cluster sample?
A stratified sample takes some individuals from every group. A cluster sample takes every individual from some groups. Strata work best when individuals within a stratum are alike. Clusters work best when each cluster resembles the population.
What is the difference between blocking and stratifying?
Both form groups of similar individuals before using chance. Stratifying is part of sampling: a random sample is taken from each stratum. Blocking is part of an experiment: treatments are randomly assigned within each block.
Does a bigger sample reduce bias?
No. A bigger sample reduces the variability of a statistic from sample to sample. Bias comes from the method, so a larger sample collected the same way has the same bias. See does a bigger sample fix bias.