Margin of error

The margin of error is a product of two pieces. The critical value comes from the confidence level; the standard error comes from the data and the sample size. Multiply them and you have the reach on each side of the estimate, so the interval is $\text{estimate} \pm \text{margin of error}$ and the margin is exactly half the interval's width. For a proportion it is $z^*\sqrt{\frac{\hat{p}(1-\hat{p})}{n}}$.

A poll of 1000 randomly selected adults finds 47% support. The standard error is $\sqrt{\frac{0.47(0.53)}{1000}} = 0.01578$ and the 95% critical value is $z^* = 1.960$, so the margin of error is $1.960 \times 0.01578 = 0.0309$, about 3.1 percentage points, and the reported interval runs from 43.9% to 50.1%. That is where a poll's plus or minus 3 points comes from.

"The margin of error tells you how far off the poll could be" is the reading to kill. It measures one source of error only: the variability from surveying a random sample rather than everybody. Nonresponse, a sampling frame that misses part of the population, leading question wording, and respondents who change their minds all sit outside it, and any of them can move a result by more than 3 points. A survey of self-selected volunteers has a perfectly computable margin of error and no useful accuracy, which is why undercoverage and nonresponse bias are named separately.

Two smaller slips are common. A margin of 3.1 percentage points is not 3.1 percent of 47%, which would be 1.5 points. And the margin is not a wall: a 95% method is built to miss about 1 time in 20, so a true value outside 43.9% to 50.1% is not evidence the poll was run badly.

Shrinking it is expensive. The margin falls with the square root of the sample size, so quadrupling a poll from 1000 respondents to 4000 takes 3.1 percentage points down to 1.5, not to 0.8. Raising the confidence level pushes it back up. To hit a target margin you solve the formula for $n$, which is a sample size calculation.

The most-watched monthly figure that carries one of these is the US unemployment rate, which is estimated from a household survey rather than counted from everyone: the unemployment rate.