Z-Score vs Standard Error

A z-score rescales one value into a count of standard deviations. For a population it is $z = \frac{x - \mu}{\sigma}$, where $\mu$ (mu) is the mean and $\sigma$ (sigma) is the standard deviation; for a sample it is $z = \frac{x - \bar{x}}{s}$, with $\bar{x}$ (x-bar) and $s$ in those roles. The units cancel in the division, so a z-score has no units of its own and values from different scales become comparable. The sign carries the direction and the size carries the distance.

A student scores 88 on a test with mean 80 and standard deviation 4, then 92 on a test with mean 86 and standard deviation 8. The raw scores say the 92 was the better performance. The z-scores say otherwise: $z = (88 - 80)/4 = 2.00$ against $z = (92 - 86)/8 = 0.75$. Relative to the class each score came from, the 88 is much the stronger result, and that comparison is the whole reason to standardize.

The wrong sentence to watch for is "her z-score of 1.5 means she beat 93 percent of the class." That 93.3 percent is the area to the left of 1.5 under the standard normal curve, so it is a fact about the normal model rather than about z-scores. A z-score is defined for any distribution that has a mean and a standard deviation, including badly skewed ones, and only when the distribution is roughly normal does the z-table turn it into a percentile.

Standardizing cannot repair shape. Subtracting and dividing by constants shifts and stretches the values without reordering them, so a right-skewed set of z-scores is still right-skewed and its outliers are still outliers.

Small samples put a hard ceiling on the answer. Using the sample standard deviation, no value among $n$ observations can have $|z|$ larger than $\frac{n-1}{\sqrt{n}}$. For $n = 5$ that ceiling is 1.79, so a z-score of 2 computed from five data points is an arithmetic error and not a rare event. The formula also breaks down when $s = 0$, which happens only when every value is the same.

The standard error is an estimate. It plays the same role for a statistic that the standard deviation plays for raw data, and it is called a standard error precisely because the quantity it reports is built from statistics rather than parameters. For a sample mean it is $SE_{\bar{x}} = \frac{s}{\sqrt{n}}$, using $s$, the sample standard deviation. For a sample proportion it is $SE_{\hat{p}} = \sqrt{\frac{\hat{p}(1-\hat{p})}{n}}$, using $\hat{p}$ (p-hat, the sample proportion). Put the parameters in instead, $\frac{\sigma}{\sqrt{n}}$ (sigma over the square root of n) and $\sqrt{p(1-p)/n}$, and you have the true standard deviation of the sampling distribution, which is the thing the standard error is estimating. That one swap, statistic in place of parameter, is the whole difference between the two names.

Suppose 25 measurements have a sample standard deviation of $s = 4.5$ grams. Then $SE_{\bar{x}} = \frac{4.5}{\sqrt{25}} = \frac{4.5}{5} = 0.9$ grams. Read the two numbers aloud: individual measurements sit about 4.5 grams from the sample mean, while the sample mean itself sits about 0.9 grams from the population mean across repeated samples of 25.

The misreading is "the standard error tells you how spread out the data are." It does not. That is $s$, and $s$ stays about the same as you collect more data, because a larger sample estimates the same population spread. The standard error shrinks toward zero with $\sqrt{n}$ because it describes a statistic, not an observation.

A second wrong reading is treating a small standard error as proof the estimate is close to the truth. It measures precision, not accuracy. A convenience sample of 4,000 produces a tiny standard error around whatever value its method is centered on, and no amount of precision repairs a center that is wrong.

One boundary worth memorizing: a one-sample z-test for a proportion puts the null value $p_0$ (p-naught) where $\hat{p}$ normally sits, giving $\sqrt{p_0(1-p_0)/n}$, because every quantity in a test is computed as though $H_0$ were true. Otherwise the standard error is the denominator of a standardized test statistic and the second factor in a margin of error.