Replication

Replication is one of the four elements of a well-designed experiment, alongside comparison, random assignment and direct control, and it means one thing: more than one experimental unit receives each treatment. Count the units per treatment and you have counted the replication.

Why it is needed: every measured difference is a treatment effect plus the ordinary difference between units. Put the new fertilizer on one plot and the standard on one plot, and a 3 kilogram gap in yield could be the fertilizer or could be that the first plot drains better. Nothing in the data separates them. With many units per treatment, unit-to-unit variation averages out of each group mean. If the response has standard deviation $\sigma = 6$ points, a group of 4 units has a mean with standard error $6/\sqrt{4} = 3$ points, while a group of 36 has $6/\sqrt{36} = 1$. Nine times the units, one third the noise.

Two sentences to reject. "We replicated the experiment by running it again the next year" describes a replication study, which is a good thing to do and is not this design element; the count here happens inside one experiment. "Each subject was measured three times, so the treatment was replicated three times" is worse, because three readings on one person are one unit measured three times, and those readings are not independent of each other.

Watch what the unit actually is. If a treatment is applied to a whole classroom, the classroom is the experimental unit, so 4 classrooms of 25 students per treatment is a replication of 4, not 100. Counting the students makes the comparison look far more precise than it is.

Replication buys precision, not protection from bias. More units per treatment shrink the standard error and do nothing about a confounded design; that job belongs to random assignment. Both are listed in topic 1.13 Experimental Design.