Significance level vs confidence level
By Jude Wallis · Published
The significance level, alpha, belongs to a hypothesis test: it is the probability of rejecting a true null hypothesis. The confidence level belongs to a confidence interval: it is the share of intervals from repeated samples that capture the parameter. For a two-sided test the two add to 1.
This page spans Units 3 and 4 of the Fall 2026 AP Statistics course. The confidence level enters when an interval is constructed (topics 3.3 and 4.2), and the significance level is the Type I error probability from topic 3.8 that every test is carried out against (for example topic 4.5). The worked examples use means because the test and interval pairing is exact there.
Significance level vs confidence level: the short answer
Each number sets the error rate of a different procedure. The significance level, written (alpha), is the threshold a p-value must reach in a hypothesis test, and it equals the probability of a Type I error, rejecting when is true. The confidence level, often written , is the long-run success rate of a confidence interval method: the percentage of intervals, over many random samples, that contain the true parameter.
One is a rate of wrong rejections and the other is a rate of correct captures, which is why the usual pair, and 95% confidence, adds to 1. That link is exact for a two-sided test about a mean:
For a one-sided test the matching interval is , because all of sits in one tail and an interval leaves area in two.
What the significance level is
The significance level is chosen before the data are collected. Common choices are 0.10, 0.05, and 0.01. It does two jobs at once.
As a decision threshold, it is the line a p-value is held against: reject when the p-value is at or below .
As an error rate, it is the probability of a Type I error. If is true and the conditions hold, a test run at rejects in 5% of random samples.
The error rate is conditional on being true. The significance level says nothing about how often is true, and it is not the probability that a particular rejection was a mistake. Choosing a value is covered in how to choose a significance level.
What the confidence level is
The confidence level is also chosen in advance, with 90%, 95%, and 99% the common choices. It enters the interval through the critical value: a higher level needs a larger or , which widens the interval.
It describes the method across repeated sampling. If you drew many random samples and built a 95% interval from each, about 95% of those intervals would contain the true parameter and about 5% would miss it. For one computed interval the parameter is either inside or outside, so the 95% describes how reliable the procedure is.
No null hypothesis is involved. A confidence interval estimates a parameter and makes no claim about a specific value until you use it to check one. The interpretation is worked through in what 95 percent confidence means.
The differences side by side
| Feature | Significance level | Confidence level |
|---|---|---|
| Belongs to | A hypothesis test | A confidence interval |
| What it is a rate of | Rejecting a true (Type I error) | Intervals that capture the parameter |
| Typical values | 0.10, 0.05, 0.01 | 90%, 95%, 99% |
| Needs a null hypothesis | Yes | No |
| Used as | The threshold for the p-value | The source of the critical value |
| Raising it | Makes rejecting easier; more Type I errors | Makes the interval wider |
| Two-sided pairing | ||
| One-sided pairing |
The pairings in the last two rows are exact for procedures about means, where the test and the interval share one standard error. For proportions the test builds its standard error from and the interval builds it from , so the two can disagree in borderline cases, as confidence interval and p-value disagree shows.
When to use which
State a significance level when the question asks for a decision about a claimed value, such as whether there is convincing evidence that the mean differs from 50. You then compute a p-value and compare it with .
State a confidence level when the question asks for an estimate of the parameter with a margin of error. You then build an interval at that level.
When a problem gives you an interval and asks about a claim, use the pairing. A 95% interval for a mean that excludes the null value matches a two-sided test at that rejects, and an interval that contains the null value matches a test that fails to reject. Check that the levels are matched before leaning on this: a 99% interval answers a two-sided test at , and a 90% interval answers a one-sided test at , provided the interval misses the null value on the side the alternative names. The wider comparison of the two procedures is in confidence interval vs hypothesis test.
The classic mix-up and how to avoid it
The first error is treating the two as one number with two names. "The test was significant at 0.05, so we are 95% confident the alternative is true" is false. The 0.05 is a Type I error rate computed assuming is true. It does not give a probability that is true, and no interval was built.
The second error is misreading either one as a statement about a single result. does not mean there is a 5% chance that this rejection is wrong. 95% confidence does not mean there is a 95% probability that the parameter lies in this particular interval. Both numbers describe how a procedure behaves over many samples.
The third error is pairing the wrong levels: using a 95% interval to settle a one-sided test at . The second worked example shows a case where that shortcut gives the wrong decision.
To keep them apart, ask which procedure you are running. A test has hypotheses and an . An interval has a point estimate, a margin of error, and a confidence level.
One sample, a test and two intervals
A random sample of 36 batteries has mean life hours and hours. Test against at and at . Then build the 95% and 99% confidence intervals for and compare.
The sample is random and , so procedures are appropriate. Standard error: .
Test statistic: with degrees of freedom.
Two-sided p-value on 35 degrees of freedom: 0.0430.
Decisions: , so reject at . , so fail to reject at .
95% interval: at 35 degrees of freedom, so the margin is and the interval is , from 50.07 to 54.13.
99% interval: , so the interval is , from 49.38 to 54.82.
Compare. The 95% interval excludes 50, matching the rejection at . The 99% interval contains 50, matching the failure to reject at .
and the two-sided p-value is 0.0430. The test rejects at and fails to reject at . The 95% interval, 50.07 to 54.13, excludes 50, and the 99% interval, 49.38 to 54.82, contains it. Each interval agrees with the two-sided test whose is 1 minus its confidence level.
A one-sided test pairs with a 90% interval
A second random sample of 36 batteries has hours and hours. Test against at . Then check which confidence interval, 90% or 95%, agrees with that decision.
Standard error: . Test statistic: with 35 degrees of freedom.
One-sided p-value, the area to the right of 1.80 on 35 degrees of freedom: 0.0402.
Decision: , so reject .
95% interval: , from 49.77 to 53.83. This interval contains 50, which would suggest failing to reject.
90% interval: , so , from 50.11 to 53.49. This interval excludes 50, in agreement with the test.
Explain the match. A 90% interval leaves 5% in each tail, and the one-sided test puts its whole 5% in the upper tail, so both use the cutoff 1.690. This is .
, the one-sided p-value is 0.0402, and the test rejects at . The 90% interval, 50.11 to 53.49, lies entirely above 50 and agrees. The 95% interval, 49.77 to 53.83, contains 50 because it corresponds to a two-sided test at , whose p-value here would be 0.0805.
Frequently asked questions
Is the confidence level always 1 minus the significance level?
Only as a pairing between a two-sided test and an interval. A two-sided test at corresponds to a 95% interval. A one-sided test at corresponds to a 90% interval. A test and an interval are separate procedures, and you are free to run either one alone at any level.
Does a 95% confidence level mean a 5% chance the null hypothesis is true?
No. The confidence level describes how often the interval method captures the parameter across repeated samples. It involves no null hypothesis. Neither the confidence level nor the significance level gives the probability that a hypothesis is true.
What happens when I raise each one?
Raising the confidence level, say from 95% to 99%, widens the interval. Raising the significance level, say from 0.01 to 0.05, makes easier to reject and increases the Type I error rate. Because the pairing is , a higher confidence level corresponds to a lower significance level.