AP Stats 1.13: Experimental Design

Four elements of a well-designed experiment

Topic 1.13 closes Unit 1 with how to design an experiment. A well-designed experiment has four elements:

1. Comparison of at least two treatment groups, one of which may be a control group.
2. Random assignment of treatments to experimental units.
3. Replication, meaning more than one experimental unit gets each treatment.
4. Direct control of potential extraneous sources of variation, keeping their settings the same from unit to unit.

An extraneous variable is one known or believed to affect the response but not being studied, and controlling or balancing it keeps it from muddying the comparison.

Control, blinding, and random assignment

A control group exists for comparison and may receive a different treatment, such as a placebo (an inactive substance). The placebo effect is the difference between the average response to a placebo and the average response to no treatment. In a single-blind study the subjects do not know their treatment (or the interacting researchers do not); in a double-blind study neither the subjects nor the interacting researchers know.

Random assignment has one job: to make the treatment groups as similar as possible on extraneous variables, so that if it works, each extraneous variable is distributed about the same across groups. That balance is what reduces confounding, where an explanatory variable and another variable change together, and it is why a randomized experiment can support a cause-and-effect conclusion; see correlation vs causation.

Three experimental designs

The CED names three designs.

• In a completely randomized design, treatments are assigned to all experimental units completely at random. Group sizes are often equal but do not have to be.
• In a randomized block design, you first group units into blocks that are homogeneous on a blocking variable, then randomly assign treatments within each block so every treatment appears in every block. Blocking separates the variation caused by that variable, giving more precise comparisons.
• A matched pairs design is a randomized block design with two treatments: units are paired on an extraneous variable and each pair splits the two treatments at random, or one unit takes both treatments in random order.

Which design fits best depends on the goal, the population, and the variables involved; compare study types in experiments vs observational studies.