Variability of an estimator

Variability is the spread of an estimator's sampling distribution, measured by its standard deviation: $\frac{\sigma}{\sqrt{n}}$ for the sample mean $\bar{x}$ (x-bar) and $\sqrt{\frac{p(1-p)}{n}}$ for the sample proportion $\hat{p}$ (p-hat). Low variability means two different random samples would hand you nearly the same answer. It is the second axis of the target diagram: bias is where the shots are centered, variability is how tight the group is, and an estimator can sit anywhere on one axis regardless of the other.

Variability decides between estimators that bias cannot separate. For a normal population both the sample mean and the sample median are unbiased for $\mu$ (mu), so bias is 0 either way. Across two million simulated samples of $n = 25$ from a normal population with $\mu = 100$ and $\sigma = 15$, both estimators averaged 100.0, but $\bar{x}$ had a standard deviation of 3.00, matching $15/\sqrt{25}$, while the sample median had 3.73. Same aim, a group about 24 percent wider.

Here is the sentence to unlearn: "I took a bigger sample, so there is less bias." The sample size appears in the variability formula and nowhere in the bias. Raising $n$ narrows the sampling distribution around wherever it is already centered, so if that center is off, a larger sample only makes you more confident about the wrong number.

The narrowing is slower than it looks, because $n$ sits under a square root. With $p = 0.5$, moving from $n = 100$ to $n = 400$ takes the standard deviation of $\hat{p}$ from 0.05 to 0.025. Four times the data buys half the spread, and cutting the spread to a tenth costs a hundred times the data.

Topic 3.1, Estimators, is where the two axes are compared. Estimate this spread from sample data instead of from parameters and it is called the standard error.