Significance Level vs P-Value

Both terms below come up in the same part of the course, and students mix them up. Here is each one defined on its own, side by side, so you can see where they part company.

Significance level

Hypothesis testing

The significance level, alpha, is the threshold a p-value is compared to, set before testing as the accepted probability of rejecting a true null hypothesis.

The significance level, written α\alpha (the Greek letter alpha), is the line you draw in advance for how much evidence counts as convincing. If the p-value is at or below α\alpha you reject the null hypothesis; if it is above, you fail to reject. For example, with α=0.05\alpha = 0.05 you reject when the p-value is 0.05 or less. Because α\alpha is also the long-run rate of falsely rejecting a true null hypothesis, a smaller α\alpha such as 0.01 demands stronger evidence.

Full entry for significance level

P-value

Hypothesis testing

A p-value is the probability, computed assuming the null hypothesis is true, of getting a result at least as extreme as the one you observed.

A p-value measures how well your data agree with the null hypothesis: a small p-value means the observed result would be surprising if the null were true, which counts as evidence against it. It is a probability about the data, not the probability that the null hypothesis is true. For example, a p-value of 0.02 says that if the null hypothesis held, data at least this extreme would occur about 2% of the time. You reject the null hypothesis when the p-value is at or below the significance level α\alpha (the Greek letter alpha).

Full entry for p-value

Where each one fits in the course