Robustness

By Jude Wallis · Updated

A procedure is robust when it still gives roughly correct results even though one of its conditions is mildly violated.

Robustness is a property of a procedure, and always robustness against a named violation. Every interval and test comes with conditions, real data never satisfy them exactly, and robustness answers how much that costs. "The tt test is robust" is an incomplete sentence: robust against mild non-normality, yes; against a sample that was not random, not at all.

The word can be measured rather than asserted. A nominal 95% interval promises to capture the parameter in 95% of repeated samples, so break a condition, simulate, and count. Drawing from a strongly right-skewed population, one-sample t-intervals actually cover about 88% at n=5n = 5, about 93% at n=30n = 30, and about 94% at n=100n = 100 across 200,000 samples. Strong skew costs something at every size, and the cost shrinks as nn grows. For a test the same check reads the true false-alarm rate against α\alpha (alpha).

"My data are robust" is the sentence to catch. Data are never robust; procedures are. Watch the neighboring word too. In wider statistical usage, resistant describes a statistic whose value barely moves when a few observations sit far out, which is why the median is resistant and the mean is not, while robust describes a procedure that still performs as advertised when a condition holds only approximately. AP topic 1.7 does not draw that line: it treats the two as interchangeable labels for measures of center and variability, calling the median and IQR resistant or robust and the mean, range, and standard deviation nonresistant or non-robust. Read each word from what the sentence is about.

Robustness has a hard edge at bias. Keep a perfectly normal population and sample by a method that never reaches its bottom 20%: coverage of a nominal 95% interval falls from about 80% at n=10n = 10 to about 0% at n=100n = 100. It gets worse as the sample grows, because more data pin down the wrong number more tightly. Bias moves the center, and no sample size repairs that. Robustness is also never a licence to skip a condition check, which is scored separately.

Where this comes up

More hypothesis testing terms, or browse the full statistics glossary.