Bias of an estimator

Bias is a number, not a verdict: subtract the parameter from the mean of the estimator's sampling distribution, $E(\hat{\theta}) - \theta$. Zero is what unbiased means. A negative value says the estimator lands below the parameter on average, a positive one says above. Because it is an average over all possible samples, bias is a property of the estimator and the sampling scheme, never of the single estimate in front of you.

Some estimators are biased for a reason you can see without arithmetic. A sample can never hold a value larger than the largest value in the population, so the sample maximum is either exactly right or too small and never too big; averaged over samples it has to fall short. Draw 10 values with replacement from the integers 1 through 100 and the sample maximum averages 91.40, a bias of $-8.60$ against a true maximum of 100.

The sentence to stop writing is this one: "my sample mean came out 3.2 when the true mean is 3.0, so my estimate is biased." A single estimate cannot be biased. Missing by 0.2 is sampling error, and unbiased estimators do it constantly, which is the whole point of having a sampling distribution. Bias only shows up across all possible samples, so you argue for it from the method rather than read it off your data.

Two different things get called bias in this course and they behave differently. The kind above is a mathematical property of a formula, and here it does shrink with $n$: at $n = 100$ the sample maximum averages 99.43. Bias built into how people were selected, such as undercoverage or nonresponse, does not shrink at all, and a larger sample only buys a more precise answer to the wrong question.

Topic 3.1, Estimators, pairs bias with variability. They are the two axes of the target diagram, and an estimator can be good on one and bad on the other.