Practical significance

A hypothesis test tells you whether there is convincing evidence of an effect; practical significance tells you whether anyone should change what they do about it. That is a judgment about size in context, and no part of the calculation produces it. The size of the effect and the setting decide it together, so the same difference can matter enormously in one place and not at all in another.

Put numbers on it. A supplement lowers mean systolic blood pressure by 0.4 mmHg in a trial of 100,000 people, with $s = 12$ mmHg. The standard error is $12/\sqrt{100{,}000} = 0.0379$, so $t = 0.4/0.0379 = 10.54$ and the p-value is far below 0.0001. The 95% confidence interval is $0.4 \pm 1.96(0.0379)$, running from 0.33 to 0.47 mmHg. The evidence that the true mean drop is not zero is overwhelming, and the interval says that drop is under half a millimetre of mercury, which is not a reason for anyone to take the supplement.

So the sentence to distrust is "the result was highly significant, so the supplement works." The word works is doing arithmetic the test never did. What was established is that the mean drop is very unlikely to be exactly zero; nothing was established about whether 0.4 mmHg buys anything. A narrow interval that excludes zero but sits close to it points to an effect that is small, however convincing the evidence for it is. Read the smallest value inside your interval and ask what it would be worth to the person making the decision.

The trap runs backwards too. A study too small to reject can produce "no significant difference," which then gets written up as no difference. Build the interval instead: if it stretches from a serious loss to a serious gain, the study settled nothing rather than settling on zero.

No topic in the Fall 2026 topic list is named practical significance. It is assessed through interpretation, in the justify-a-claim topics 3.4 for a proportion and 4.3 for a mean, and through the wording of a conclusion, which speaks of evidence rather than importance.