Empirical Rule vs Z-Score

The empirical rule describes areas under a normal curve. For a distribution that is normal with mean $\mu$ (mu) and standard deviation $\sigma$ (sigma), about 68 percent of the area lies on $\mu \pm \sigma$, about 95 percent on $\mu \pm 2\sigma$, and about 99.7 percent on $\mu \pm 3\sigma$. Those three headline numbers are rounded. The exact areas are 0.6827, 0.9545, and 0.9973, so the rule is a fast sketch rather than a substitute for a table.

Take heights that are normal with $\mu = 68$ inches and $\sigma = 3$ inches. Two standard deviations reach from $68 - 2(3) = 62$ to $68 + 2(3) = 74$ inches, and the rule says about 95 percent land in there. The exact area between 62 and 74 is 0.9545, so the rule is low by roughly half a percentage point. One standard deviation, 65 to 71 inches, holds 0.6827 of the heights.

Here is the error that shows up most: "95 percent are within 2 standard deviations, so 5 percent are above 74 inches." The leftover is split between two tails, not piled into one. The exact area above 74 inches is 0.0228, about 2.3 percent, and the same amount sits below 62 inches. Every one-sided empirical-rule answer needs that halving step, and skipping it roughly doubles the reported probability.

The rule is a fact about the normal curve, not about data in general. Applied to a strongly skewed variable it can be badly off, and applied to data that are only roughly bell-shaped it is an approximation on top of an approximation. When a problem asks for exactly the middle 95 percent, the multiplier is 1.96 rather than 2: for these heights that runs 62.12 to 73.88 inches, an interval whose area is 0.9500.

The normal distribution the rule describes is topic 2.11 of Unit 2. Use the rule for a sketch and a sanity check, and the z-table when the answer has to be right.

A z-score rescales one value into a count of standard deviations. For a population it is $z = \frac{x - \mu}{\sigma}$, where $\mu$ (mu) is the mean and $\sigma$ (sigma) is the standard deviation; for a sample it is $z = \frac{x - \bar{x}}{s}$, with $\bar{x}$ (x-bar) and $s$ in those roles. The units cancel in the division, so a z-score has no units of its own and values from different scales become comparable. The sign carries the direction and the size carries the distance.

A student scores 88 on a test with mean 80 and standard deviation 4, then 92 on a test with mean 86 and standard deviation 8. The raw scores say the 92 was the better performance. The z-scores say otherwise: $z = (88 - 80)/4 = 2.00$ against $z = (92 - 86)/8 = 0.75$. Relative to the class each score came from, the 88 is much the stronger result, and that comparison is the whole reason to standardize.

The wrong sentence to watch for is "her z-score of 1.5 means she beat 93 percent of the class." That 93.3 percent is the area to the left of 1.5 under the standard normal curve, so it is a fact about the normal model rather than about z-scores. A z-score is defined for any distribution that has a mean and a standard deviation, including badly skewed ones, and only when the distribution is roughly normal does the z-table turn it into a percentile.

Standardizing cannot repair shape. Subtracting and dividing by constants shifts and stretches the values without reordering them, so a right-skewed set of z-scores is still right-skewed and its outliers are still outliers.

Small samples put a hard ceiling on the answer. Using the sample standard deviation, no value among $n$ observations can have $|z|$ larger than $\frac{n-1}{\sqrt{n}}$. For $n = 5$ that ceiling is 1.79, so a z-score of 2 computed from five data points is an arithmetic error and not a rare event. The formula also breaks down when $s = 0$, which happens only when every value is the same.