Significance level vs confidence level

By Jude Wallis · Published

The significance level, alpha, belongs to a hypothesis test: it is the probability of rejecting a true null hypothesis. The confidence level belongs to a confidence interval: it is the share of intervals from repeated samples that capture the parameter. For a two-sided test the two add to 1.

This page spans Units 3 and 4 of the Fall 2026 AP Statistics course. The confidence level enters when an interval is constructed (topics 3.3 and 4.2), and the significance level is the Type I error probability from topic 3.8 that every test is carried out against (for example topic 4.5). The worked examples use means because the test and interval pairing is exact there.

Significance level vs confidence level: the short answer

Each number sets the error rate of a different procedure. The significance level, written α\alpha (alpha), is the threshold a p-value must reach in a hypothesis test, and it equals the probability of a Type I error, rejecting H0H_0 when H0H_0 is true. The confidence level, often written CC, is the long-run success rate of a confidence interval method: the percentage of intervals, over many random samples, that contain the true parameter.

One is a rate of wrong rejections and the other is a rate of correct captures, which is why the usual pair, α=0.05\alpha = 0.05 and 95% confidence, adds to 1. That link is exact for a two-sided test about a mean:

C=1−αC = 1 - \alpha

For a one-sided test the matching interval is C=1−2αC = 1 - 2\alpha, because all of α\alpha sits in one tail and an interval leaves area in two.

What the significance level is

The significance level is chosen before the data are collected. Common choices are 0.10, 0.05, and 0.01. It does two jobs at once.

As a decision threshold, it is the line a p-value is held against: reject H0H_0 when the p-value is at or below α\alpha.

As an error rate, it is the probability of a Type I error. If H0H_0 is true and the conditions hold, a test run at α=0.05\alpha = 0.05 rejects in 5% of random samples.

The error rate is conditional on H0H_0 being true. The significance level says nothing about how often H0H_0 is true, and it is not the probability that a particular rejection was a mistake. Choosing a value is covered in how to choose a significance level.

What the confidence level is

The confidence level is also chosen in advance, with 90%, 95%, and 99% the common choices. It enters the interval through the critical value: a higher level needs a larger z∗z^* or t∗t^*, which widens the interval.

It describes the method across repeated sampling. If you drew many random samples and built a 95% interval from each, about 95% of those intervals would contain the true parameter and about 5% would miss it. For one computed interval the parameter is either inside or outside, so the 95% describes how reliable the procedure is.

No null hypothesis is involved. A confidence interval estimates a parameter and makes no claim about a specific value until you use it to check one. The interpretation is worked through in what 95 percent confidence means.

The differences side by side

FeatureSignificance level α\alphaConfidence level CC
Belongs toA hypothesis testA confidence interval
What it is a rate ofRejecting a true H0H_0 (Type I error)Intervals that capture the parameter
Typical values0.10, 0.05, 0.0190%, 95%, 99%
Needs a null hypothesisYesNo
Used asThe threshold for the p-valueThe source of the critical value
Raising itMakes rejecting easier; more Type I errorsMakes the interval wider
Two-sided pairingα=1−C\alpha = 1 - CC=1−αC = 1 - \alpha
One-sided pairingα=1−C2\alpha = \frac{1 - C}{2}C=1−2αC = 1 - 2\alpha

The pairings in the last two rows are exact for tt procedures about means, where the test and the interval share one standard error. For proportions the test builds its standard error from p0p_0 and the interval builds it from p^\hat{p}, so the two can disagree in borderline cases, as confidence interval and p-value disagree shows.

When to use which

State a significance level when the question asks for a decision about a claimed value, such as whether there is convincing evidence that the mean differs from 50. You then compute a p-value and compare it with α\alpha.

State a confidence level when the question asks for an estimate of the parameter with a margin of error. You then build an interval at that level.

When a problem gives you an interval and asks about a claim, use the pairing. A 95% interval for a mean that excludes the null value matches a two-sided test at α=0.05\alpha = 0.05 that rejects, and an interval that contains the null value matches a test that fails to reject. Check that the levels are matched before leaning on this: a 99% interval answers a two-sided test at α=0.01\alpha = 0.01, and a 90% interval answers a one-sided test at α=0.05\alpha = 0.05, provided the interval misses the null value on the side the alternative names. The wider comparison of the two procedures is in confidence interval vs hypothesis test.

The classic mix-up and how to avoid it

The first error is treating the two as one number with two names. "The test was significant at 0.05, so we are 95% confident the alternative is true" is false. The 0.05 is a Type I error rate computed assuming H0H_0 is true. It does not give a probability that HaH_a is true, and no interval was built.

The second error is misreading either one as a statement about a single result. α=0.05\alpha = 0.05 does not mean there is a 5% chance that this rejection is wrong. 95% confidence does not mean there is a 95% probability that the parameter lies in this particular interval. Both numbers describe how a procedure behaves over many samples.

The third error is pairing the wrong levels: using a 95% interval to settle a one-sided test at α=0.05\alpha = 0.05. The second worked example shows a case where that shortcut gives the wrong decision.

To keep them apart, ask which procedure you are running. A test has hypotheses and an α\alpha. An interval has a point estimate, a margin of error, and a confidence level.

One sample, a test and two intervals

A random sample of 36 batteries has mean life xˉ=52.1\bar{x} = 52.1 hours and s=6s = 6 hours. Test H0:μ=50H_0: \mu = 50 against Ha:μ≠50H_a: \mu \neq 50 at α=0.05\alpha = 0.05 and at α=0.01\alpha = 0.01. Then build the 95% and 99% confidence intervals for μ\mu and compare.

  1. The sample is random and n=36≥30n = 36 \ge 30, so tt procedures are appropriate. Standard error: 636=1\frac{6}{\sqrt{36}} = 1.

  2. Test statistic: t=52.1−501=2.10t = \frac{52.1 - 50}{1} = 2.10 with 36−1=3536 - 1 = 35 degrees of freedom.

  3. Two-sided p-value on 35 degrees of freedom: 0.0430.

  4. Decisions: 0.0430≤0.050.0430 \le 0.05, so reject H0H_0 at α=0.05\alpha = 0.05. 0.0430>0.010.0430 > 0.01, so fail to reject H0H_0 at α=0.01\alpha = 0.01.

  5. 95% interval: t∗=2.030t^* = 2.030 at 35 degrees of freedom, so the margin is 2.030×1=2.0302.030 \times 1 = 2.030 and the interval is 52.1±2.03052.1 \pm 2.030, from 50.07 to 54.13.

  6. 99% interval: t∗=2.724t^* = 2.724, so the interval is 52.1±2.72452.1 \pm 2.724, from 49.38 to 54.82.

  7. Compare. The 95% interval excludes 50, matching the rejection at α=0.05\alpha = 0.05. The 99% interval contains 50, matching the failure to reject at α=0.01\alpha = 0.01.

t=2.10t = 2.10 and the two-sided p-value is 0.0430. The test rejects H0H_0 at α=0.05\alpha = 0.05 and fails to reject at α=0.01\alpha = 0.01. The 95% interval, 50.07 to 54.13, excludes 50, and the 99% interval, 49.38 to 54.82, contains it. Each interval agrees with the two-sided test whose α\alpha is 1 minus its confidence level.

A one-sided test pairs with a 90% interval

A second random sample of 36 batteries has xˉ=51.8\bar{x} = 51.8 hours and s=6s = 6 hours. Test H0:μ=50H_0: \mu = 50 against Ha:μ>50H_a: \mu > 50 at α=0.05\alpha = 0.05. Then check which confidence interval, 90% or 95%, agrees with that decision.

  1. Standard error: 636=1\frac{6}{\sqrt{36}} = 1. Test statistic: t=51.8−501=1.80t = \frac{51.8 - 50}{1} = 1.80 with 35 degrees of freedom.

  2. One-sided p-value, the area to the right of 1.80 on 35 degrees of freedom: 0.0402.

  3. Decision: 0.0402≤0.050.0402 \le 0.05, so reject H0H_0.

  4. 95% interval: 51.8±2.03051.8 \pm 2.030, from 49.77 to 53.83. This interval contains 50, which would suggest failing to reject.

  5. 90% interval: t∗=1.690t^* = 1.690, so 51.8±1.69051.8 \pm 1.690, from 50.11 to 53.49. This interval excludes 50, in agreement with the test.

  6. Explain the match. A 90% interval leaves 5% in each tail, and the one-sided test puts its whole 5% in the upper tail, so both use the cutoff 1.690. This is C=1−2α=1−2(0.05)=0.90C = 1 - 2\alpha = 1 - 2(0.05) = 0.90.

t=1.80t = 1.80, the one-sided p-value is 0.0402, and the test rejects H0H_0 at α=0.05\alpha = 0.05. The 90% interval, 50.11 to 53.49, lies entirely above 50 and agrees. The 95% interval, 49.77 to 53.83, contains 50 because it corresponds to a two-sided test at α=0.05\alpha = 0.05, whose p-value here would be 0.0805.

Frequently asked questions

Is the confidence level always 1 minus the significance level?

Only as a pairing between a two-sided test and an interval. A two-sided test at α=0.05\alpha = 0.05 corresponds to a 95% interval. A one-sided test at α=0.05\alpha = 0.05 corresponds to a 90% interval. A test and an interval are separate procedures, and you are free to run either one alone at any level.

Does a 95% confidence level mean a 5% chance the null hypothesis is true?

No. The confidence level describes how often the interval method captures the parameter across repeated samples. It involves no null hypothesis. Neither the confidence level nor the significance level gives the probability that a hypothesis is true.

What happens when I raise each one?

Raising the confidence level, say from 95% to 99%, widens the interval. Raising the significance level, say from 0.01 to 0.05, makes H0H_0 easier to reject and increases the Type I error rate. Because the pairing is C=1−αC = 1 - \alpha, a higher confidence level corresponds to a lower significance level.