When a Confidence Interval and P-Value Disagree

Q: If my confidence interval excludes the null value, is the p-value automatically below 0.05?

For a mean, with a 95% interval and a two-sided test, yes. The t interval and the t test share the standard error s/sqrt(n), so they always agree. For a proportion it is a strong tendency, not a rule: the test builds its standard error from p₀ and the interval builds one from p-hat, so near the boundary they can split.

Q: Which result should I report when they disagree?

Report the one the question asked for, then say why the other differs. Neither is more correct. If the choice is yours, the interval carries more information, since it gives a range of plausible values instead of only labeling one value plausible or not.

Q: Does a 95% confidence interval always correspond to α = 0.05?

It corresponds to a two-sided test at α = 0.05. A one-sided test at α = 0.05 lines up with a 90% two-sided interval, or with a 95% one-sided bound. Check the tails before deciding the two procedures conflict.

Q: Can a confidence interval disagree with a chi-square test?

That comparison does not exist in AP Statistics. The chi-square tests for homogeneity and independence (topics 3.14 and 3.15) have no matching confidence interval in the course, so there is nothing to line up against the p-value. Follow-up there means examining the individual components of χ².

By Jude Wallis · Published July 28, 2026

Yes, they can disagree, and neither one is broken. Three things cause it: a one-sided test compared against a two-sided interval, a confidence level that does not match your significance level, and, for proportions, a test and an interval built on different standard errors.

AP Statistics: Unit 3 (topics 3.4 Justifying a Claim Based on a Confidence Interval for a Population Proportion, 3.6 p-Values, 3.7 Carrying Out a Test for a Population Proportion, 3.11 Justifying a Claim Based on a Confidence Interval for the Difference Between Two Population Proportions). This sits at the seam between the confidence interval topics (3.4 and 3.11) and the significance testing topics (3.6 and 3.7) of Unit 3 in the Fall 2026 AP Statistics course, where the same data can be judged either way.

Can a confidence interval and a hypothesis test disagree?

Yes. On the same data you can get a 95% confidence interval that excludes the null value and a p-value of 0.07, and both calculations can be correct.

That feels impossible because the two procedures get taught as two views of the same evidence. They almost are. A confidence interval and a two-sided test at the matching level answer the same question, so most of the time they hand you the same verdict.

Disagreements come from exactly three places:

Tails. A one-sided test is being read against a two-sided interval.
Level. The confidence level and the significance level $\alpha$ do not match.
Standard error. For proportions, the test divides by a standard error built from $p_0$ while the interval divides by one built from $\hat{p}$ .

The first two are setup mismatches, and you fix them by lining the procedures up. The third is real. For proportions the interval and the test are genuinely not the same test, so near the boundary they can split, and no amount of rechecking your arithmetic will make that go away.

Why a confidence interval and a test usually agree

A confidence interval is a list of plausible values for the parameter. A two-sided test asks whether one specific value, $p_0$ , is plausible. Those are the same question asked two ways, and that link is called the duality between intervals and tests.

Stated carefully: a confidence interval at level $C$ is very nearly the set of null values that a two-sided test at $\alpha = 1 - C$ would fail to reject. So:

If $p_0$ falls outside the interval, the two-sided test rejects $H_0$ .
If $p_0$ falls inside the interval, the two-sided test fails to reject $H_0$ .

For means, "very nearly" is exact. The one-sample t interval and the one-sample t test both use $\frac{s}{\sqrt{n}}$ as the standard error, and both use the same t distribution. Match the level, match the tails, and a mean will never hand you a contradiction. If it does, you made an arithmetic error.

For proportions the duality is only approximate. That is where most of the confusion on this page lives, and it is covered in reason 3 below.

Reason 1: a one-sided test read against a two-sided interval

A confidence interval reaches out in both directions from the estimate. A one-sided test spends all of $\alpha$ in a single tail. Those are different rules, so pairing them manufactures disagreements that are not really there.

Line them up like this:

Test you ran	Interval that matches it
Two-sided at $\alpha = 0.05$	95% two-sided interval
Two-sided at $\alpha = 0.10$	90% two-sided interval
One-sided at $\alpha = 0.05$	90% two-sided interval
One-sided at $\alpha = 0.025$	95% two-sided interval

The pattern is one line: a one-sided test at $\alpha$ pairs with a two-sided interval at level $1 - 2\alpha$ , not $1 - \alpha$ .

So a one-sided p-value of 0.04 rejects at $\alpha = 0.05$ while the 95% two-sided interval still holds the null value. Nothing is wrong. The interval is guarding both tails and the test is only watching one. The second worked example below runs those numbers. For picking the alternative in the first place, see one-tailed vs two-tailed tests.

Reason 2: the confidence level does not match alpha

A 95% interval matches $\alpha = 0.05$ . A 90% interval matches $\alpha = 0.10$ . A 99% interval matches $\alpha = 0.01$ . The link is $\alpha = 1 - C$ , and when the two numbers in front of you do not satisfy it, the procedures are not testing the same thing.

Free-response questions set this trap on purpose. Part (a) asks for a 95% confidence interval. Part (b) asks you to test at $\alpha = 0.10$ . A value can sit just inside the 95% interval and still get rejected at the 0.10 level, because that test corresponds to the narrower 90% interval.

Before you call anything a contradiction, write the confidence level and $\alpha$ next to each other. Most reported disagreements die right there. More on the decision rule in p-value vs alpha.

Reason 3: for proportions, the two standard errors are different

This is the real one, and it is specific to proportions.

The one-proportion z test standardizes using the standard deviation the sampling distribution would have **if $H_0$ were true**, so it uses $p_0$ :

z = \frac{\hat{p} - p_0}{\sqrt{\frac{p_0(1 - p_0)}{n}}}

The confidence interval has no null value to lean on, so it estimates the standard error from the data and uses $\hat{p}$ :

\hat{p} \pm z^{*}\sqrt{\frac{\hat{p}(1 - \hat{p})}{n}}

Same numerator distance, different denominators. The interval is really running its own z statistic,

z_{CI} = \frac{\hat{p} - p_0}{\sqrt{\frac{\hat{p}(1 - \hat{p})}{n}}}

and it excludes $p_0$ exactly when $|z_{CI}| > z^{*}$ . Two standardized distances from one data set, and they are not equal.

Which one comes out larger follows from the fact that $p(1 - p)$ peaks at $p = 0.5$ :

If $\hat{p}$ is farther from 0.5 than $p_0$ is, the interval's standard error is smaller, the interval is narrower, and the interval rejects more readily than the test does. That is the case that produces an interval excluding the null value alongside a p-value of 0.07.
If $\hat{p}$ is closer to 0.5 than $p_0$ is, the interval is wider, and the test rejects more readily than the interval does.

The ratio of the two standard errors is $\sqrt{p_0(1 - p_0)}$ over $\sqrt{\hat{p}(1 - \hat{p})}$ , so it depends only on where $\hat{p}$ and $p_0$ sit relative to 0.5, never on $n$ by itself. What shrinks with $n$ is the gap that matters here: a p-value sitting close to $\alpha$ forces $\hat{p}$ to lie within about two standard errors of $p_0$ , and that window closes as $n$ climbs. So the split lives in small and moderate samples, and only when the p-value is already sitting near $\alpha$ . It is a boundary effect, not a hidden flaw in either formula.

The same thing happens with two proportions (topics 3.10 through 3.13), and harder. The two-proportion z test (topics 3.12 and 3.13) pools both samples into a combined $\hat{p}_C$ for its standard error, while the two-proportion interval (topic 3.10) keeps the samples separate. Different denominators again, so a 95% interval for $p_1 - p_2$ that excludes 0 does not guarantee a p-value below 0.05. Compare the formulas side by side in one-proportion vs two-proportion z-test.

Which one to use on an AP question

Answer the question that was asked, using the procedure it named.

If the prompt says justify a claim based on the confidence interval (topics 3.4 and 3.11), use the interval. Write something like: the interval is $(0.096, 0.244)$ , and 0.25 is not in it, so 0.25 is not a plausible value for the true proportion and the claim is not supported.
If the prompt says carry out a test (topics 3.5 through 3.7), use the p-value against $\alpha$ . Write: the p-value is 0.065, which is greater than 0.05, so fail to reject $H_0$ ; there is not convincing evidence that the proportion differs from 0.25.

Do not split the difference, do not average them, and do not declare one of them wrong. If a question hands you both and they conflict, check the tails and the level first. If those already match and the parameter is a proportion, the two standard errors explain the rest, and naming that out loud is a strong answer.

One tiebreaker worth carrying: the interval holds more information. It names a range of plausible values, so it tells you how big the effect might be, while the p-value only rates the evidence against one specific value. That is why the course keeps asking you to interpret intervals in context. For the interpretation wording, see what 95% confidence means, and for the p-value wording, what a p-value means.

Mistakes to avoid

Do not compare a one-sided p-value against a two-sided interval. Halve $\alpha$ so a one-sided test at 0.025 matches the 95% interval, drop to a 90% interval, or use a one-sided bound instead.
Do not assume a 95% interval pairs with every $\alpha$ . It pairs with a two-sided test at $\alpha = 0.05$ and with a one-sided test at $\alpha = 0.025$ . It does not pair with a one-sided test at 0.05.
Do not put $\hat{p}$ in the test's standard error or $p_0$ in the interval's. That swap costs points on its own, and it can fake either an agreement or a conflict.
Do not reuse one condition check for both procedures. Large Counts for the test uses $np_0$ and $n(1 - p_0)$ ; for the interval it uses $n\hat{p}$ and $n(1 - \hat{p})$ . A problem can pass one and fail the other.
Do not hunt for an interval to match a chi-square test. Topics 3.14 and 3.15 have no matching confidence interval in this course, so the comparison does not exist.
Do not write "the interval and the test disagree, so something went wrong." Near the boundary, disagreement is the expected behavior of two procedures that are only approximately the same.

The interval excludes 0.25 but the p-value is 0.065

A school district claims that 25% of its students walk to school. In a random sample of 100 students from the district, 17 walk. Run a two-sided test of $H_0: p = 0.25$ at $\alpha = 0.05$ , build the 95% confidence interval, and explain why they disagree.

State the hypotheses. Let $p$ be the true proportion of district students who walk to school. $H_0: p = 0.25$ and $H_a: p \neq 0.25$ , with $\alpha = 0.05$ .
Check conditions, separately for each procedure. The sample is random and 100 is less than 10% of the district. Large Counts for the test uses $p_0$ : $np_0 = 100(0.25) = 25 \geq 10$ and $n(1 - p_0) = 100(0.75) = 75 \geq 10$ . Large Counts for the interval uses $\hat{p}$ : $n\hat{p} = 17 \geq 10$ and $n(1 - \hat{p}) = 83 \geq 10$ .
Sample proportion: $\hat{p} = \frac{17}{100} = 0.17$ .
Test standard error, built from $p_0$ . Multiply: $(0.25)(0.75) = 0.1875$ . Divide by 100: $0.001875$ . Square root: $\sqrt{0.001875} = 0.043301$ .
Test statistic: $z = \frac{0.17 - 0.25}{0.043301} = \frac{-0.08}{0.043301} = -1.8475$ .
P-value. The test is two-sided, so double one tail. Rounding to $z = -1.85$ , the z-table gives $P(Z < -1.85) = 0.0322$ , so the p-value is $2(0.0322) = 0.0644$ ; technology gives 0.0647. Either way it exceeds 0.05, so fail to reject $H_0$ .
Interval standard error, built from $\hat{p}$ . Multiply: $(0.17)(0.83) = 0.1411$ . Divide by 100: $0.001411$ . Square root: $\sqrt{0.001411} = 0.037563$ .
Margin of error at 95% confidence, where $z^{*} = 1.96$ : $1.96(0.037563) = 0.0736$ .
Interval: $0.17 \pm 0.0736$ , giving $(0.0964, 0.2436)$ . The claimed value 0.25 sits outside it, so the interval says 0.25 is not plausible.
Locate the conflict. The interval's own standardized distance is $z_{CI} = \frac{-0.08}{0.037563} = -2.130$ , which carries a two-sided p-value of about $2(0.0166) = 0.0332$ . The test standardized the identical gap of 0.08 to $-1.85$ and got 0.065. One numerator, two denominators: $0.037563$ against $0.043301$ .

The test fails to reject ( $z = -1.85$ , p-value about 0.065, greater than 0.05) while the 95% interval $(0.096, 0.244)$ excludes 0.25. Both are right. The test divides by $\sqrt{\frac{(0.25)(0.75)}{100}} = 0.0433$ and the interval divides by $\sqrt{\frac{(0.17)(0.83)}{100}} = 0.0376$ . Because $\hat{p} = 0.17$ lies farther from 0.5 than $p_0 = 0.25$ does, the interval is the narrower yardstick and rejects while the test does not.

A one-sided test rejects while the 95% interval does not

A coach claims that more than half of the students at a large school can run a mile in under 8 minutes. In a random sample of 50 students, 31 can. Test $H_0: p = 0.5$ against $H_a: p > 0.5$ at $\alpha = 0.05$ , then build a 95% confidence interval and reconcile the two results.

Hypotheses: $H_0: p = 0.5$ and $H_a: p > 0.5$ , with $\alpha = 0.05$ . This alternative points in one direction, so the test is one-sided.
Conditions. The sample is random and 50 is less than 10% of a large school. Test: $np_0 = 50(0.5) = 25 \geq 10$ and $n(1 - p_0) = 25 \geq 10$ . Interval: $n\hat{p} = 31 \geq 10$ and $n(1 - \hat{p}) = 19 \geq 10$ .
Sample proportion: $\hat{p} = \frac{31}{50} = 0.62$ .
Test standard error: $\sqrt{\frac{(0.5)(0.5)}{50}} = \sqrt{\frac{0.25}{50}} = \sqrt{0.005} = 0.070711$ .
Test statistic: $z = \frac{0.62 - 0.50}{0.070711} = \frac{0.12}{0.070711} = 1.697$ .
One-sided p-value. Rounding to $z = 1.70$ , the z-table gives $P(Z < 1.70) = 0.9554$ , so $P(Z \geq 1.70) = 1 - 0.9554 = 0.0446$ ; technology gives 0.0448. Since $0.0446 < 0.05$ , reject $H_0$ .
Interval standard error: $\sqrt{\frac{(0.62)(0.38)}{50}} = \sqrt{\frac{0.2356}{50}} = \sqrt{0.004712} = 0.068644$ .
95% interval: $0.62 \pm 1.96(0.068644) = 0.62 \pm 0.13454$ , giving $(0.4855, 0.7545)$ . That contains 0.5, so the interval keeps an even split on the table.
Match the procedures. A one-sided test at $\alpha = 0.05$ pairs with a two-sided interval at level $1 - 2(0.05) = 0.90$ . Rebuild at 90% confidence, where $z^{*} = 1.645$ : $0.62 \pm 1.645(0.068644) = 0.62 \pm 0.11292$ , giving $(0.5071, 0.7329)$ .
That interval excludes 0.5, so it agrees with the test. Equivalently, the 95% one-sided lower bound is $0.62 - 1.645(0.068644) = 0.507$ , and 0.5 falls below it.
Confirm the two-sided story. The interval's own standardized distance is $z_{CI} = \frac{0.12}{0.068644} = 1.748$ , short of 1.96, so the 95% interval keeps 0.5 inside. The test's two-sided p-value, $2(0.0446) = 0.0892$ , tells the same story here, though for a proportion the two are only approximately linked (reason 3). The matched pair agreed on this data set; only the one-sided test read against a two-sided interval looked like a contradiction.

The one-sided test rejects: $z = 1.70$ and the p-value is $0.0446 < 0.05$ . The 95% two-sided interval $(0.485, 0.755)$ contains 0.5, which looks contradictory but is not. The interval that matches a one-sided test at 0.05 is the 90% interval $(0.507, 0.733)$ , and it excludes 0.5, agreeing with the test.

Frequently asked questions

If my confidence interval excludes the null value, is the p-value automatically below 0.05?

For a mean, with a 95% interval and a two-sided test, yes. The t interval and the t test share the standard error $\frac{s}{\sqrt{n}}$ , so they always agree. For a proportion it is a strong tendency, not a rule: the test builds its standard error from $p_0$ and the interval builds one from $\hat{p}$ , so near the boundary they can split.

Which result should I report when they disagree?

Report the one the question asked for, then say why the other differs. Neither is more correct. If the choice is yours, the interval carries more information, since it gives a range of plausible values instead of only labeling one value plausible or not.

Does a 95% confidence interval always correspond to $\alpha = 0.05$?

It corresponds to a two-sided test at $\alpha = 0.05$ . A one-sided test at $\alpha = 0.05$ lines up with a 90% two-sided interval, or with a 95% one-sided bound. Check the tails before deciding the two procedures conflict.

Can a confidence interval disagree with a chi-square test?

That comparison does not exist in AP Statistics. The chi-square tests for homogeneity and independence (topics 3.14 and 3.15) have no matching confidence interval in the course, so there is nothing to line up against the p-value. Follow-up there means examining the individual components of $\chi^2$ .