Precision vs Confidence Level

Precision is how tightly an estimate pins a parameter down, and for a confidence interval you read it straight off the width: the narrower the interval, the more precise the estimate. It is a property of the procedure, fixed by the sample size, the confidence level, and how variable the data are. It says nothing about whether the estimate is centered on the right value.

Take $\hat{p} = 0.40$ (p-hat) from a random sample of 400. The standard error is $\sqrt{0.40(0.60)/400} = 0.024495$, so a 95% interval reaches $1.96(0.024495) = 0.0480$ on each side: 0.352 to 0.448, a width of 0.0960. Run the study again with 1600 people and the same $\hat{p}$ and the standard error halves to 0.012247, the interval becomes 0.376 to 0.424, and the width halves to 0.0480. Four times the data bought twice the precision, because the margin of error shrinks like $1/\sqrt{n}$.

The sentence to distrust is "the interval is narrow, so the estimate must be close to the truth." Precision is not accuracy. Suppose the survey reaches only the people who pick up unknown numbers, and that group runs 8 points high. The estimate from 1600 of them is still 0.048 wide, and it is still centered about 8 points off. Precision describes how tightly repeated estimates cluster around their own center. Accuracy is about where that center sits, and no sample size moves it, because bias appears nowhere in the arithmetic above.

There is also a cheap way to look precise. Keep the same 400 people and the same $\hat{p} = 0.40$, then drop the confidence level from 95% to 90%: the width falls from 0.0960 to 0.0806, about 16% narrower, bought entirely by accepting that 10 intervals in 100 miss instead of 5. Sample size buys precision, while a lower confidence level only relabels it. That is why a width means nothing unless the confidence level is quoted alongside it.

A confidence level is chosen before the data are collected and it does two jobs at once. It fixes the critical value the interval formula uses, and it names the rate at which that formula succeeds across repeated random samples of the same size. Write it $C$. The parameter never moves. The interval does, because a fresh sample produces fresh endpoints.

Suppose 240 of 400 randomly sampled adults say yes, so $\hat{p} = 0.60$ (p-hat) and the standard error is $\sqrt{\frac{0.60(0.40)}{400}} = 0.0245$. At 90% the critical value is $z^* = 1.645$ and the interval runs 0.560 to 0.640. At 95% it is $z^* = 1.960$ and the interval runs 0.552 to 0.648. At 99% it is $z^* = 2.576$ and the interval runs 0.537 to 0.663. One data set, three intervals. Nothing about the sample changed, only the level.

The sentence to stop writing is "there is a 95% probability that the true proportion is between 0.552 and 0.648." Once those two numbers exist they are fixed, the proportion was always fixed, and so that probability is 0 or 1 with no way to tell which. The randomness was spent when the sample was drawn. The 95% belongs to the procedure: it is a statement about the samples you did not take. The coverage simulator makes that visible, with the true $p$ drawn as a line and each new sample laying down one more interval to count.

A confidence level is what the method promises, not always what it delivers. For the proportion interval above the two come apart: at $n = 50$ with a true $p$ of 0.60, its exact long-run capture rate is 94.1% against a nominal 95%, because the standard error is built from $\hat{p}$ rather than from the unknown $p$. The rate a method actually achieves is its coverage probability.

Raising the level buys reliability with width and nothing else. At a fixed $n$ the 99% interval above is 0.126 wide against 0.096 at 95%, and a 100% interval would be every value from 0 to 1: certain to capture $p$ and worth nothing.