Is a higher standard deviation better? It depends

The honest answer: it depends on what you are measuring

The standard deviation ($s$ for a sample, $\sigma$ ("sigma") for a population) measures one thing: how far a typical value sits from the mean. It is a description of spread, not a score, and there is no threshold above which spread becomes bad or below which it becomes good.

The useful question is never "is this standard deviation high?" It is "what would go wrong if the values were more spread out, and what would go wrong if they were all identical?" Those two questions have different answers in different settings, and that is the entire reason this page cannot end with a rule.

Be suspicious of anyone offering one. A standard deviation of 5 is tiny for annual household incomes in dollars, enormous for the diameter of a machined bearing in millimeters, and meaningless without knowing which. Spread is judged against the unit, the mean, and the purpose.

When a lower standard deviation is better

Whenever the goal is to hit a target, consistency is the goal and spread is the enemy.

A filling machine. Two machines fill bottles labeled 500 mL. Both average exactly 500 mL, so the mean cannot tell them apart. Machine A has $\sigma = 2$ mL and Machine B has $\sigma = 8$ mL. If the acceptable range is 495 to 505 mL, then modeling each machine's output as approximately normal, Machine A puts about 98.76 percent of bottles in range and Machine B only about 46.80 percent. Same average, wildly different production line.

A measuring instrument. Weigh the same 100.0 g reference weight five times on two scales. A good scale returns 99.9, 100.0, 100.0, 100.1, 100.0, with a standard deviation of about 0.071 g. A worn scale returns 98.4, 101.3, 99.1, 101.7, 99.5, which also averages 100.0 g but has a standard deviation of about 1.432 g. Neither set of five readings gives evidence of bias, since both average 100.0 g, though five readings could not detect a small bias either way. Only one is precise enough to trust for a single reading.

Anything with a promise attached. Delivery times, dosages, the width of a runway, the time a surgery takes. In each case the average is a poor summary of the customer's experience, and reducing spread is the actual work.

When a higher standard deviation is better

The other direction is real, and it comes up more than students expect.

A test that has to separate people. Suppose eight students take Quiz A and score 80, 81, 81, 82, 82, 83, 83, 84. The mean is 82 and the standard deviation is about 1.31. The same eight take Quiz B and score 68, 72, 77, 81, 83, 87, 92, 96, again a mean of 82, with a standard deviation of about 9.59. Quiz B is the better instrument for ranking, because Quiz A compresses everyone into a five-point band from 80 to 84 where a single lucky guess reorders the class. Push it to the limit: if every student scored 82, the standard deviation would be 0 and the quiz would carry no information about who understood the material.

Choosing the conditions in a study. To learn how a fertilizer dose affects yield, you have to apply doses that differ. Testing 10 and 11 grams per plot gives you almost nothing to work with, while spanning 5 to 40 grams reveals the relationship. Deliberately building spread into the explanatory variable is standard design practice.

Anywhere the collection is supposed to span a range. A seed bank storing genetic diversity, a stock index tracking a whole market, a survey meant to capture the range of opinion in a town. When the collection is assembled deliberately, as with a seed bank, a small standard deviation is a sign the collector failed to cover the range. When the collection is meant to mirror something, as with an index or a survey, a small standard deviation is a finding about the thing being mirrored, not a fault in the collection.

The finance case, stated carefully

This example gets quoted often and usually gets mangled, so here is the accurate version.

In finance, the standard deviation of returns is the standard numerical measure of risk, often called volatility. Higher standard deviation means the returns bounce around more, which is not desirable in itself. Holding the average return fixed, a lower standard deviation is better. Combining investments that do not move in lockstep lowers the standard deviation of the combined return below the weighted average of the individual standard deviations, while the combined average return stays at the weighted average of the individual averages. That is the mathematical content behind the standard diversification argument.

The reason anyone tolerates a higher standard deviation is that historically, assets with higher average returns have also had higher volatility. So the choice is a tradeoff between two quantities, not a preference for spread. Saying "higher standard deviation is good for a portfolio" reverses the actual claim.

That is a description of how the statistic is used, not investment advice.

A standard deviation on its own tells you almost nothing

Before judging a standard deviation, you need three things it does not carry by itself.

• The unit. A standard deviation of 5 is 5 of something. It only means what that something means.
• The mean. For a variable measured on a scale with a true zero, comparing $s$ to $\bar{x}$ ("x-bar") is a quick first look: a standard deviation of 5 next to a mean of 8 is large relative to the values, while the same 5 next to a mean of 5000 is small relative to them. The comparison stops working when the mean is near zero or when the zero point of the scale is arbitrary, and it never settles whether the spread is acceptable for the purpose.
• A comparison. Spread becomes informative when it is compared to another group, to a specification, or to the same process last month. That is why AP questions ask you to compare distributions rather than to grade one.

One warning about a common mix-up. A larger sample does not shrink the standard deviation, because $s$ estimates how spread out the population is and that does not change when you look harder. What shrinks with a larger sample is the standard error, which measures how much a statistic like $\bar{x}$ varies from sample to sample. Standard error vs standard deviation separates the two.

How to answer this on the AP exam

Exam questions rarely ask whether a standard deviation is good. They ask you to compare spread between distributions and to justify a choice in context, which is the answerable version of the question.

Three habits earn the points.

• Say more or less variable, then say what that means here. "Machine B has a larger standard deviation, so its fill volumes are less consistent and more bottles fall outside the acceptable range" beats "Machine B has a higher standard deviation."
• Use the resistant summary when outliers are present. If a distribution is strongly skewed or has an extreme value, compare the interquartile ranges instead, since the standard deviation gets dragged around by a single far-out value the way the mean does. Standard deviation vs IQR covers the choice, and why the median is not affected by outliers covers the mechanism.
• Name the purpose. Whether more spread is a problem depends on what the process is for, so state that: filling bottles wants consistency, a placement test wants separation.

For the mechanics behind the number, see what standard deviation actually means, and for why the raw variance is not the thing you interpret, why variance is in squared units.