Coverage Probability vs Confidence Level

Both terms below come up in the same part of the course, and students mix them up. Here is each one defined on its own, side by side, so you can see where they part company.

Coverage probability

Confidence intervals

Coverage probability is the long-run proportion of intervals built by a given method that capture the true parameter.

Coverage probability describes the method, not any single interval you have already computed. A 95% method has coverage probability 0.95, so if you drew 1000 random samples and built an interval from each, about 950 of those 1000 intervals would contain the parameter. Once you compute a specific interval such as 0.48 to 0.56, that interval either contains the parameter or it does not, so saying it has a 95% probability of containing the parameter is wrong. The correct statement is about the procedure before you sample: P(the interval covers the parameter)=0.95P(\text{the interval covers the parameter}) = 0.95.

Full entry for coverage probability

Confidence level

Confidence intervals

The confidence level is the long-run percentage of confidence intervals, built the same way from repeated samples, that would capture the true parameter.

The confidence level, often 90%, 95%, or 99%, describes the method rather than one specific interval. It says that if you repeated the sampling and interval-building many times, that percentage of the intervals would contain the true parameter. For example, at a 95% confidence level about 95 of every 100 such intervals would trap the real value and about 5 would miss it. A higher confidence level uses a larger critical value, which makes each interval wider.

Full entry for confidence level

Where each one fits in the course