Confidence Level vs Significance Level

Both terms below come up in the same part of the course, and students mix them up. Here is each one defined on its own, side by side, so you can see where they part company.

Confidence level

Confidence intervals

The confidence level is the long-run percentage of confidence intervals, built the same way from repeated samples, that would capture the true parameter.

The confidence level, often 90%, 95%, or 99%, describes the method rather than one specific interval. It says that if you repeated the sampling and interval-building many times, that percentage of the intervals would contain the true parameter. For example, at a 95% confidence level about 95 of every 100 such intervals would trap the real value and about 5 would miss it. A higher confidence level uses a larger critical value, which makes each interval wider.

Full entry for confidence level

Significance level

Hypothesis testing

The significance level, alpha, is the threshold a p-value is compared to, set before testing as the accepted probability of rejecting a true null hypothesis.

The significance level, written α\alpha (the Greek letter alpha), is the line you draw in advance for how much evidence counts as convincing. If the p-value is at or below α\alpha you reject the null hypothesis; if it is above, you fail to reject. For example, with α=0.05\alpha = 0.05 you reject when the p-value is 0.05 or less. Because α\alpha is also the long-run rate of falsely rejecting a true null hypothesis, a smaller α\alpha such as 0.01 demands stronger evidence.

Full entry for significance level

Where each one fits in the course