AP Statistics sampling and experiments cheat sheet

By Jude Wallis · Published

Random selection lets a result generalize to the population sampled. Random assignment lets a difference too large for chance be attributed to the treatment. This sheet lists the sampling methods, the named types of bias, the elements and designs of experiments, and what each study can conclude.

AP Statistics: Unit 1 (topics 1.10 The Investigative Question Revisited and Data Collection, 1.11 Random Sampling, 1.12 Potential Problems with Sampling, 1.13 Experimental Design). Data collection is Unit 1, topics 1.10 through 1.13, of the Fall 2026 AP Statistics course: study types, random sampling methods, bias, and experimental design.

Sampling methods

MethodHow it worksUseful when
Simple random sample (SRS)Every possible sample of size nn has the same chance of being chosenA list of the whole population exists
Stratified random sampleDivide the population into strata of similar individuals, take an SRS from each stratum, and combineGroups differ on the response and each should be represented
Cluster sampleDivide the population into clusters, take an SRS of clusters, and include everyone in the chosen clustersThe population is spread across natural groups and a full list is hard to get
Systematic random sampleChoose a random starting point, then take every kkth individualIndividuals come in a list or a line

Strata are built to be alike inside and different from each other. Clusters ideally each resemble the whole population. In a stratified sample some individuals from every group are chosen. In a cluster sample all individuals from some groups are chosen.

Two methods do not use chance and tend to produce bias: a convenience sample takes whoever is easy to reach, and a voluntary response sample takes whoever chooses to respond. See how to choose a sampling method and cluster vs stratified sampling.

Sources of bias

Bias is a systematic tendency for a method to give statistics that are too high or too low. A larger sample from the same method does not remove it.

BiasWhat goes wrongExample
Voluntary response biasThe sample is made of volunteers, who often hold stronger opinionsAn online poll that anyone can choose to answer
Undercoverage biasPart of the population is left out or is less likely to be selectedA phone survey that cannot reach people without phones
Nonresponse biasSelected individuals do not respond, and they differ from those who doA mailed survey that few people return
Response biasAnswers differ from the truth in a consistent directionA leading question, or a sensitive question answered face to face

A full answer names the bias, explains how it arises in this study, and states the direction: whether the statistic is likely to be too high or too low. See how to identify the type of bias.

Types of studies

StudyWhat happensCan show cause and effect
CensusData are collected from every individual in the populationNo; a census imposes no treatments
Observational studyVariables are recorded without imposing treatments. A survey asks a standard set of questions, a prospective study follows individuals forward in time, and a retrospective study uses past dataNo, because confounding variables may explain an association
ExperimentThe researcher assigns treatments to experimental units and measures a responseYes, when treatments are randomly assigned

A confounding variable is associated with both the explanatory variable and the response variable, so it offers another explanation for the relationship between them. See experiments vs observational studies.

Elements of a well-designed experiment

ElementMeaning
ComparisonAt least two treatment groups, one of which may be a control group
Random assignmentChance decides which experimental units get which treatment
ReplicationMore than one experimental unit receives each treatment
ControlOther variables that could affect the response are held the same for all units

Random assignment spreads the other variables across the treatment groups by chance. A difference in the response then has two possible explanations: the treatments or the chance in the assignment. An inference procedure addresses the second.

TermMeaning
Experimental unitThe individual or object to which a treatment is assigned; people are called subjects or participants
Explanatory variable (factor)The variable whose levels form the treatments
Response variableThe outcome measured after treatment
Control groupA group used for comparison, which may receive a placebo or no treatment
PlaceboAn inactive treatment
Single-blindEither the subjects or the researchers who interact with them do not know which treatment was given
Double-blindNeither the subjects nor the researchers who interact with them know which treatment was given

Three experimental designs

DesignHow treatments are assignedUse when
Completely randomized designTreatments are assigned to all experimental units completely at randomUnits are similar, or no variable is known to matter
Randomized block designUnits are grouped into blocks that are alike on a blocking variable, and treatments are randomly assigned within each blockA known variable is expected to affect the response
Matched pairs designUnits are paired on a relevant variable and the two treatments are randomly assigned within each pair, or each unit receives both treatments in random orderThere are two treatments and good matches exist

Blocking accounts for the variation caused by the blocking variable, which makes the comparison of treatments more precise. A matched pairs design is a randomized block design with blocks of size two. Describing a design means saying how the random assignment is carried out, for example by numbering the units and using a random number generator. See how to design an experiment and blocking vs stratifying.

Scope of inference

Two separate questions decide what a study can conclude.

How the study was doneTreatments randomly assignedNo random assignment
Individuals randomly selected from a populationCause and effect, generalized to that populationAssociation only, generalized to that population
Individuals not randomly selectedCause and effect, for individuals like those in the studyAssociation only, for individuals like those in the study

Random selection is about who is in the study, and it supports generalizing. Random assignment is about who gets which treatment, and it supports a cause-and-effect conclusion. Neither one makes up for the absence of the other. See can you generalize results and random sampling vs random assignment.

Plan a stratified random sample

A school has 1,200 students: 420 freshmen, 330 sophomores, 270 juniors, and 180 seniors. The principal wants a stratified random sample of 80 students, with each grade's share of the sample equal to its share of the school. Find how many students to sample from each grade and describe how to select them.

  1. Freshmen: 4201200×80=28\frac{420}{1200} \times 80 = 28.

  2. Sophomores: 3301200×80=22\frac{330}{1200} \times 80 = 22.

  3. Juniors: 2701200×80=18\frac{270}{1200} \times 80 = 18.

  4. Seniors: 1801200×80=12\frac{180}{1200} \times 80 = 12.

  5. Check: 28+22+18+12=8028 + 22 + 18 + 12 = 80.

  6. Selection: within each grade, number the students from 1 to the grade's size, use a random number generator to pick the required count of different numbers, and sample the students with those numbers.

Sample 28 freshmen, 22 sophomores, 18 juniors, and 12 seniors, each chosen as a simple random sample within its grade. The grades are the strata.

Decide what a study can conclude

A gym asks for volunteers and 60 members sign up. The gym randomly assigns 30 of them to morning workouts and 30 to evening workouts for eight weeks. At the end, the morning group has lost significantly more weight on average. Can the gym conclude that morning workouts cause more weight loss for all of its members?

  1. Random assignment: yes. The 60 volunteers were randomly assigned to the two workout times.

  2. Consequence: a cause-and-effect conclusion is supported for the people in the study.

  3. Random selection: no. The 60 members volunteered, so they are not a random sample of the gym's members.

  4. Consequence: the result generalizes only to members similar to the volunteers.

The gym can conclude that morning workouts caused more weight loss for members like those who volunteered. It cannot extend that conclusion to all members, because the participants were not randomly selected.

Frequently asked questions

What is the difference between a stratified sample and a cluster sample?

A stratified sample takes some individuals from every group. A cluster sample takes every individual from some groups. Strata work best when individuals within a stratum are alike. Clusters work best when each cluster resembles the population.

What is the difference between blocking and stratifying?

Both form groups of similar individuals before using chance. Stratifying is part of sampling: a random sample is taken from each stratum. Blocking is part of an experiment: treatments are randomly assigned within each block.

Does a bigger sample reduce bias?

No. A bigger sample reduces the variability of a statistic from sample to sample. Bias comes from the method, so a larger sample collected the same way has the same bias. See does a bigger sample fix bias.