Experiment

By Jude Wallis · Published

An experiment imposes treatments on subjects and compares their responses, supporting a cause-and-effect conclusion when treatments are assigned at random.

An experiment imposes something. The researcher assigns conditions called treatments to experimental units and then measures a response, which is the line between an experiment and an observational study, where the groups assemble themselves. The course names four marks of a well-designed experiment: comparison of at least two treatment groups, random assignment of treatments to units, replication, meaning more than one unit per treatment, and direct control of extraneous variables.

A store tests two checkout layouts on 200 shoppers, using a random number generator to send 100 to each. Mean checkout time comes out at 84 seconds under layout A and 97 seconds under layout B, a difference of 13 seconds. Because chance decided who met which layout, no systematic difference in basket size or shopping habit is expected to pile up on one side, so a gap that size, if it is larger than chance variation would ordinarily produce, is fairly credited to the layout.

"It was an experiment, so it proves cause and effect" hides two separate conditions inside one word. Imposing a treatment is what makes a study an experiment. Random assignment is what makes its comparison causal, and the two do not always travel together. Give layout A to the morning shift and layout B to the afternoon shift and you have imposed treatments, so it is still an experiment, but time of day now moves in lockstep with layout and no causal reading is available.

The causal claim also has a limit on who it covers. Random assignment supports a conclusion about the units in the study, and extending that to a wider group takes random selection, a different act at a different stage; scope of inference works through the combinations.

Everything else in experimental design is machinery for those four marks: blocking to pull a known nuisance variable out of the comparison, a control group and a placebo to supply a baseline, and blinding to keep expectations from leaking into the response. Replication means many units per treatment, not repeating the whole study, which is the vocabulary slip the exam catches most often here.

Where this comes up

9 pages on the site use this term.

More collecting data and study design terms, or browse the full statistics glossary.