Unbiased Estimator vs Statistic

An estimator is unbiased when the mean of its sampling distribution is exactly the parameter, $E(\hat{\theta}) = \theta$, and when that holds for every possible value of the parameter rather than for one convenient case. The sample mean qualifies, $E(\bar{x}) = \mu$ (x-bar, mu), and so does the sample proportion, $E(\hat{p}) = p$ (p-hat). Unbiasedness pins down where the sampling distribution sits and says nothing about how wide it is.

Take the four-value population 1, 2, 3, 6, which has $\mu = 3$, $\sigma^2 = 3.5$ and $\sigma = 1.8708$. Write out all 16 samples of size 2 drawn with replacement and average each statistic across them. The sample mean averages to 3, exactly $\mu$. The sample variance with the $n-1$ divisor averages to 3.5, exactly $\sigma^2$. The sample standard deviation averages to $\sqrt{2} = 1.4142$, which is 75.6 percent of $\sigma$.

So this sentence is false: "the $n-1$ divisor makes $s$ an unbiased estimator of $\sigma$." What $n-1$ buys is $E(s^2) = \sigma^2$. Taking a square root is nonlinear, and the mean of a square root is not the square root of a mean, so the correction does not survive the step. $s$ comes out too small on average, at every sample size, for any population in which it varies at all.

The shortfall shrinks with $n$ without reaching zero: in the same population, all 1,024 samples of size 5 give an average $s$ of 1.7673, or 94.5 percent of $\sigma$. Inference for means uses $s$ in place of $\sigma$ regardless, because at ordinary sample sizes the gap is small and the exact correction depends on the population's shape.

Unbiasedness is also not the whole of a good estimator. Topic 3.1, Estimators, weighs it against variability: an estimator centered perfectly but scattered widely can be worse than one that leans slightly and lands close every time.

A statistic is a number computed from sample data alone. Because its value depends on which individuals were drawn, it changes from one sample to the next, and that movement is what separates it from a parameter, which stays put. Statistics take Roman letters or hats: $\bar{x}$ (x-bar) for the sample mean, $s$ for the sample standard deviation, $\hat{p}$ (p-hat) for the sample proportion, $r$ for the sample correlation. Each is paired with the parameter it estimates, $\mu$, $\sigma$, $p$, and $\rho$ in turn.

Suppose 60 percent of a large population would vote yes, so $p = 0.60$. Draw a random sample of 80 and find 52 yes votes, giving $\hat{p} = \frac{52}{80} = 0.65$. Draw a second sample of 80 and find 44, giving $\hat{p} = \frac{44}{80} = 0.55$. Two different statistics, one unchanged parameter, and no mistake in either sample. That spread is sampling variability, and the sampling distribution of $\hat{p}$ describes it.

"The sample proportion is 0.65, so the population proportion is 0.65" is the error to name out loud. A statistic is an estimate and it is almost never exactly right, so the honest version attaches an interval or a standard error to that 0.65. The written form of the same mistake is putting $p = 0.65$ where $\hat{p} = 0.65$ belongs. That collapse wrecks everything downstream, because a test compares an observed $\hat{p}$ against a hypothesized $p$, and no comparison is left once both symbols name one quantity.

A statistic has two lives. Before the sample is drawn it is a random variable with a distribution of its own, which is what makes phrases like the mean and standard deviation of $\hat{p}$ meaningful. Once the sample is in hand it is a single number.

Two boundaries. A value computed from an entire population is not a statistic, however it was calculated, and a statistic from a badly chosen sample is still a statistic, just a poor estimator. Topic 3.1 grades estimators on exactly those two axes: whether they are centered on the parameter, and how much they vary.