Z-Score vs T-Distribution

A z-score rescales one value into a count of standard deviations. For a population it is $z = \frac{x - \mu}{\sigma}$, where $\mu$ (mu) is the mean and $\sigma$ (sigma) is the standard deviation; for a sample it is $z = \frac{x - \bar{x}}{s}$, with $\bar{x}$ (x-bar) and $s$ in those roles. The units cancel in the division, so a z-score has no units of its own and values from different scales become comparable. The sign carries the direction and the size carries the distance.

A student scores 88 on a test with mean 80 and standard deviation 4, then 92 on a test with mean 86 and standard deviation 8. The raw scores say the 92 was the better performance. The z-scores say otherwise: $z = (88 - 80)/4 = 2.00$ against $z = (92 - 86)/8 = 0.75$. Relative to the class each score came from, the 88 is much the stronger result, and that comparison is the whole reason to standardize.

The wrong sentence to watch for is "her z-score of 1.5 means she beat 93 percent of the class." That 93.3 percent is the area to the left of 1.5 under the standard normal curve, so it is a fact about the normal model rather than about z-scores. A z-score is defined for any distribution that has a mean and a standard deviation, including badly skewed ones, and only when the distribution is roughly normal does the z-table turn it into a percentile.

Standardizing cannot repair shape. Subtracting and dividing by constants shifts and stretches the values without reordering them, so a right-skewed set of z-scores is still right-skewed and its outliers are still outliers.

Small samples put a hard ceiling on the answer. Using the sample standard deviation, no value among $n$ observations can have $|z|$ larger than $\frac{n-1}{\sqrt{n}}$. For $n = 5$ that ceiling is 1.79, so a z-score of 2 computed from five data points is an arithmetic error and not a rare event. The formula also breaks down when $s = 0$, which happens only when every value is the same.

The $t$-distribution is not one curve but a family, indexed by the degrees of freedom. It is the distribution of $t = \frac{\bar{x} - \mu}{s / \sqrt{n}}$ (x-bar minus mu, over s divided by root n) when the data come from a normal population. Swapping the fixed $\sigma$ (sigma) for the sample standard deviation $s$, which itself changes from sample to sample, is what puts the extra weight in the tails. Every member is symmetric about 0, and the family closes on the standard normal as the degrees of freedom grow.

The numbers make that convergence concrete. For a 95 percent interval the t-table gives $t^* = 2.145$ at 14 degrees of freedom, $2.042$ at 30, $1.984$ at 100 and $1.962$ at 1000, against $z^* = 1.960$ for the normal. The gap is 8.6 percent of the critical value at 14 degrees of freedom and about 0.1 percent at 1000.

"The sample is small so use $t$, and large so use $z$" is the wrong rule, and it is the one most students carry in. The trigger is whether $\sigma$ is known, not how big $n$ is. With 500 observations and a standard deviation estimated from them, the correct model is $t$ on 499 degrees of freedom, which happens to sit very close to the normal. Knowing $\sigma$ with $n = 8$ would put you back on $z$.

The heavier tails change verdicts, not just widths. A statistic of 2.00 read on $t$ with 14 degrees of freedom has a two-sided p-value of 0.0653, against the 0.0455 the normal returns for the same 2.00, so at $\alpha = 0.05$ one model rejects and the other does not.

Those tails cover the uncertainty in $s$ and nothing else. They do not repair a skewed population or a stray outlier, which is why a $t$ procedure still asks you to look at the shape of the sample first. The $t$-distribution enters the course at topic 4.2 of Unit 4, Inference for Quantitative Data: Means.