Decision Rule vs Significance Level

Both terms below come up in the same part of the course, and students mix them up. Here is each one defined on its own, side by side, so you can see where they part company.

Decision rule

Hypothesis testing

The decision rule is the standard you set before collecting data: reject the null hypothesis when the p-value is at or below alpha.

Fixing the rule in advance is what keeps a test honest, because α\alpha (alpha, the significance level) is the probability of rejecting a true null hypothesis and must not be adjusted once you see the result. The AP version is a direct comparison: if the p-value is at or below alpha you reject H0H_0 and conclude there is convincing evidence for the alternative, and otherwise you fail to reject. For example, at α=0.05\alpha = 0.05 a p-value of 0.032 leads you to reject, while a p-value of 0.080 does not. You never accept the null hypothesis, since failing to find evidence against a claim is not evidence that the claim is true.

Full entry for decision rule

Significance level

Hypothesis testing

The significance level, alpha, is the threshold a p-value is compared to, set before testing as the accepted probability of rejecting a true null hypothesis.

The significance level, written α\alpha (the Greek letter alpha), is the line you draw in advance for how much evidence counts as convincing. If the p-value is at or below α\alpha you reject the null hypothesis; if it is above, you fail to reject. For example, with α=0.05\alpha = 0.05 you reject when the p-value is 0.05 or less. Because α\alpha is also the long-run rate of falsely rejecting a true null hypothesis, a smaller α\alpha such as 0.01 demands stronger evidence.

Full entry for significance level

Where each one fits in the course