Robustness
By Jude Wallis · Updated
A procedure is robust when it still gives roughly correct results even though one of its conditions is mildly violated.
Robustness is a property of a procedure, and always robustness against a named violation. Every interval and test comes with conditions, real data never satisfy them exactly, and robustness answers how much that costs. "The test is robust" is an incomplete sentence: robust against mild non-normality, yes; against a sample that was not random, not at all.
The word can be measured rather than asserted. A nominal 95% interval promises to capture the parameter in 95% of repeated samples, so break a condition, simulate, and count. Drawing from a strongly right-skewed population, one-sample t-intervals actually cover about 88% at , about 93% at , and about 94% at across 200,000 samples. Strong skew costs something at every size, and the cost shrinks as grows. For a test the same check reads the true false-alarm rate against (alpha).
"My data are robust" is the sentence to catch. Data are never robust; procedures are. Watch the neighboring word too. In wider statistical usage, resistant describes a statistic whose value barely moves when a few observations sit far out, which is why the median is resistant and the mean is not, while robust describes a procedure that still performs as advertised when a condition holds only approximately. AP topic 1.7 does not draw that line: it treats the two as interchangeable labels for measures of center and variability, calling the median and IQR resistant or robust and the mean, range, and standard deviation nonresistant or non-robust. Read each word from what the sentence is about.
Robustness has a hard edge at bias. Keep a perfectly normal population and sample by a method that never reaches its bottom 20%: coverage of a nominal 95% interval falls from about 80% at to about 0% at . It gets worse as the sample grows, because more data pin down the wrong number more tightly. Bias moves the center, and no sample size repairs that. Robustness is also never a licence to skip a condition check, which is scored separately.
Where this comes up
More hypothesis testing terms, or browse the full statistics glossary.