P-value interpretation practice problems

Start with the assumption. A p-value is always computed by assuming the null hypothesis is true, so here you assume the true proportion of subscribers who check the dashboard is exactly $0.45$.

Name the event. The alternative is $H_a: p > 0.45$, so 'at least as extreme' means a sample proportion at least as far above $0.45$ as the one this sample produced.

Attach the number. The p-value $0.0271$ is that probability, and the only source of variation it describes is the random sampling.

Assemble the sentence. If the true proportion of subscribers who check the dashboard were $0.45$, then about 2.71% of random samples of this size would give a sample proportion at least as far above $0.45$ as the one observed.

Check what you did not say. You did not say the probability that $H_0$ is true, and you did not say the probability that the result is 'due to chance'. The probability is about the data, computed under a fixed assumption about $p$.

(a) The p-value is a probability about data, not about hypotheses. In this framework the true proportion $p$ is a fixed unknown number: it either equals $0.30$ or it does not, so it is never assigned a probability. Correct version: assuming $p = 0.30$, there is a $0.028$ probability of a sample proportion at least this far above $0.30$.

(b) Every result from a random sample 'happens by chance', so that phrase describes nothing. What the statement drops is the condition that makes the number meaningful, namely that $0.028$ was computed assuming $p = 0.30$. Correct version: if $p$ really were $0.30$, random sampling alone would produce a result this extreme or more extreme about 2.8% of the time.

(c) The chance that this particular rejection is a mistake is not $0.028$. A Type I error can only happen when $H_0$ is true, and its long-run rate is the significance level $\alpha$ that you chose in advance, not the p-value from one sample.

Spot the shared error. All three sentences delete the phrase 'assuming the null hypothesis is true', which is exactly the phrase that turns a tail area into a p-value.

Test yourself with a rewrite. Any correct interpretation should start with 'if the null hypothesis were true' or 'assuming $p = 0.30$', then describe a sample result, then give the probability.

Recall the rule. Reject $H_0$ when the p-value is less than or equal to $\alpha$; fail to reject when the p-value is greater than $\alpha$.

(a) Compare. $0.043 \le 0.05$, so reject $H_0$. There is convincing evidence that the true proportion of members who use the bulk bins differs from $0.25$.

(b) Compare again. $0.043 > 0.01$, so fail to reject $H_0$. At this stricter level there is not convincing evidence that the proportion differs from $0.25$.

(b) Say what that does not mean. Failing to reject is not evidence that the proportion equals $0.25$. It means this sample was not surprising enough under $0.25$ to clear a 1% bar.

(c) Apply the rule at the boundary. $0.02 \le 0.02$ is true, so the second co-op rejects $H_0$. The comparison uses 'less than or equal to', so an exact tie goes to rejection.

(d) Identify what moved. Nothing about the data changed between (a) and (b): the sample, the test statistic, and the p-value $0.043$ are identical. Only the cutoff agreed on in advance changed, and a smaller $\alpha$ demands stronger evidence before you will reject.

(a) Pick the tail from the alternative. The alternative is 'greater than', so the p-value is the area to the right of $z = 2.14$ under the standard normal curve.

(a) Read the table and subtract. $P(Z < 2.14) = 0.9838$, so the p-value is $1 - 0.9838 = 0.0162$.

(b) Interpret with the assumption in front. If the true proportion of weekday passengers bringing a bicycle were $0.12$, about 1.62% of random samples of this size would give a test statistic of $2.14$ or larger.

(c) Use symmetry. The standard normal curve is symmetric about $0$, so a two-sided p-value adds the matching left tail below $-2.14$: p-value $= 2(0.0162) = 0.0324$.

Compare the two. At $\alpha = 0.05$ both lead to rejecting $H_0$, The two-sided p-value is exactly twice the one-sided one only when the sample falls on the side the alternative points to, which is what happens here. Problem 5(c) is the other case: when the data go against the alternative, the one-sided p-value climbs above 0.5 and the two-sided p-value is the smaller of the two.

(a) Halve the two-sided value. A two-sided p-value is the sum of two equal tail areas, so one tail is $0.086 \div 2 = 0.043$. This works only because the sample fell on the side the alternative points to.

(b) Double the one-sided value. The matching opposite tail has the same area, so the two-sided p-value is $2(0.017) = 0.034$.

(c) Choose the tail from the alternative, not from the sign of $z$. The alternative is $H_a: p > 0.40$, so the p-value is the area to the right of $-0.90$.

(c) Compute it. $P(Z < -0.90) = 0.1841$, so $P(Z > -0.90) = 1 - 0.1841 = 0.8159$.

(c) Read what that means. The sample proportion landed below $0.40$, the opposite side from where the alternative predicted, so the data provide no support at all for $H_a$. You fail to reject $H_0$ at any usual $\alpha$.

Keep the warning. For a one-sided test, a p-value above $0.5$ is a signal that the sample went the wrong direction; taking the left tail instead would have given $0.1841$, which is the wrong answer to the question asked.

(a) Find the sample proportion. $\hat{p} = \frac{132}{400} = 0.33$.

(a) Find the standard error using the null value $p_0 = 0.35$. $SE = \sqrt{\frac{0.35(0.65)}{400}} = \sqrt{\frac{0.2275}{400}} = \sqrt{0.00056875} = 0.02385$.

(a) Standardize. $z = \frac{0.33 - 0.35}{0.02385} = \frac{-0.02}{0.02385} \approx -0.84$.

(a) Find the two-sided p-value. $P(Z < -0.84) = 0.2005$, so the p-value is $2(0.2005) = 0.4010$.

(b) Decide. $0.4010 > 0.05$, so fail to reject $H_0$. There is not convincing evidence that the true proportion of visitors from outside the county differs from $0.35$.

(c) Correct the claim. A large p-value says only that a sample proportion of $0.33$ would be unremarkable if $p$ were $0.35$. It gives no evidence that $p$ equals $0.35$, because values such as $0.32$ or $0.37$ would also survive this test.

(c) Restate it properly. 'There is not convincing evidence that the proportion of visitors from outside the county differs from 35%.' A hypothesis test can reject or fail to reject $H_0$, but it can never establish that $H_0$ is true.

(a) Find the sample proportion. $\hat{p} = \frac{20280}{40000} = 0.507$.

(a) Find the standard error using $p_0 = 0.500$. $SE = \sqrt{\frac{0.500(0.500)}{40000}} = \sqrt{\frac{0.25}{40000}} = \sqrt{0.00000625} = 0.0025$.

(a) Standardize. $z = \frac{0.507 - 0.500}{0.0025} = \frac{0.007}{0.0025} = 2.80$.

(a) Find the two-sided p-value. $P(Z < 2.80) = 0.9974$, so one tail is $1 - 0.9974 = 0.0026$ and the p-value is $2(0.0026) = 0.0052$. Technology reports $0.0051$.

(b) Decide. $0.0052 < 0.05$, so reject $H_0$. There is convincing evidence that the true completion rate differs from $0.500$.

(c) Size the effect. The estimate is $0.507$, and the interval says plausible values run from about $0.5021$ to $0.5119$. Even the top of that range sits only $0.5119 - 0.500 = 0.0119$ above one half, roughly one extra completed play per 100.

(c) Explain the small p-value. With $n = 40000$ the standard error is only $0.0025$, so a gap of $0.007$ is $2.8$ standard errors wide even though it is tiny in real terms. Large samples turn small differences into small p-values.

(c) Separate the two questions. The p-value answers 'could random variation alone plausibly produce a gap this big?' It does not answer 'is the gap large enough to act on?' For that you read the estimate and the interval.

(a) State. Let $p$ be the true proportion of trail users at this park who carry out all of their trash after the signs went up. $H_0: p = 0.80$ versus $H_a: p > 0.80$.

(a) Check conditions. Random: the 250 trail users are a random sample. 10%: the park logs roughly 12,000 trail users a season, and 250 is well under 10% of that. Large Counts under $H_0$: $np_0 = 250(0.80) = 200$ and $n(1 - p_0) = 250(0.20) = 50$, both at least 10.

(b) Find the sample proportion and standard error. $\hat{p} = \frac{212}{250} = 0.848$, and $SE = \sqrt{\frac{0.80(0.20)}{250}} = \sqrt{\frac{0.16}{250}} = \sqrt{0.00064} = 0.02530$.

(b) Standardize and find the tail. $z = \frac{0.848 - 0.800}{0.02530} = \frac{0.048}{0.02530} \approx 1.90$. The test is right-tailed, so the p-value is $1 - 0.9713 = 0.0287$.

(c) Interpret. If the true carry-out rate were still $0.80$, about 2.87% of random samples of 250 trail users would give a sample proportion at least as far above $0.80$ as $0.848$.

(d) Decide. $0.0287 \le 0.05$, so reject $H_0$. There is convincing evidence that the true carry-out rate is now greater than $0.80$.

(d) Correct the sentence. It is wrong twice. The p-value is not a probability about the signs or about $H_0$ being true, and it is computed by assuming the rate is still $0.80$. Correct version: assuming the rate is $0.80$, there is a $0.0287$ probability of a sample result at least this far above $0.80$.

(d) Add the design caution. Rejecting $H_0$ says the rate is above $0.80$; with no control group, it does not by itself show that the signs caused the increase, since weather or the season could differ from the earlier period.

(e) Decide at the stricter level. $0.0287 > 0.01$, so you fail to reject $H_0$. What changes is the decision. What does not change is the sample, the test statistic $1.90$, the p-value $0.0287$, or the strength of the evidence, and failing to reject still does not show the rate equals $0.80$.