Type I error, Type II error, and power
Three probabilities live on the same picture: rejecting a null that is true, missing an effect that is real, and catching it. Move any control and one of them pays for another.
power against every true mean, holding alpha at 0.0500, n at 25, and the test one-sided
A test at alpha 0.0500 with n = 25 rejects the null once the sample mean passes 104.9346. If the truth really is 105.0, this test catches it with probability 0.5087 and misses it with probability 0.4913. Move any control and watch which error rate pays for the change.
Setup: a z test of H0 mu = 100 against Ha mu > 100, with the population standard deviation known to be 15. Sigma is held fixed here so the three factors you can move are visible one at a time; shrinking sigma is the fourth thing that raises power.
What to try
- Alpha is a price, not a gift. The tool opens on alpha 0.05, n = 25, and a true mean of 105, where power reads 0.5087. Click 0.10 and power jumps to 0.6499, but the hatched Type I region doubles. Nothing about the population changed; you agreed to reject a true null twice as often.
- A bigger effect is free power. Put alpha back to 0.05 and drag the true mean to 108. Power climbs to 0.8466 with the cutoff still sitting at 104.9346. Power is never a property of a test on its own, only of a test aimed at a specific alternative.
- Sample size beats both. Return the true mean to 105 and drag n from 25 to 100. Both curves narrow, the standard error falls from 3 to 1.5, and power reaches 0.9543 while alpha stays at 0.0500. This is the only control that improves both error rates at once.
- Two tails cost power. Switch to two-sided. Alpha 0.05 becomes 0.025 in each tail, the upper cutoff moves out from 104.9346 to 105.8799, and power against the same true mean of 105 falls from 0.5087 to 0.3848.
- The self-check. Drag the true mean to exactly 100. Power now reads the same as alpha, to four decimals, at every sample size you try. With no effect to detect, the only rejections available are Type I errors, which is why the two-sided power curve in the lower panel bottoms out at alpha instead of at zero.
Why the arithmetic is checkable here
The setup is a z test for a population mean with the population standard deviation known, which is the one inference setting where nothing is estimated on the way to the answer. The critical value is the null mean plus invNorm(1 minus alpha) times sigma over the square root of n, and beta is the area of the true curve on the fail-to-reject side of that cutoff. Power is 1 minus beta by definition, so the two readouts can never disagree.
With sigma = 15 and n = 25 the standard error is exactly 3, so the one-sided cutoff at alpha 0.05 is 100 + 1.6449 times 3, or 104.935 to three decimals. A true mean of 105 puts that cutoff 0.0218 standard errors below the centre of the true curve, and the area of the true curve above the cutoff is 0.5087. Both lines are a z table away from being checked by hand.
What the AP exam actually asks
Be honest with yourself about the scope here. AP Statistics topic 3.8, Potential Errors When Performing Tests, asks you to identify which error is which in context, to describe what each one would cost, and to explain the factors that raise or lower power. It does not ask you to calculate power, and there is no power formula on the formula sheet.
The four factors are sample size, the significance level, the size of the true effect, and the variability of the data. Three of them are the controls above; the fourth is sigma, held fixed at 15 so the others read cleanly one at a time. Use the four-decimal readouts to see which direction each factor pushes, then answer exam questions in sentences.
The reasoning in full is in Type I vs Type II error and power, the case for picking one alpha over another is in how to choose a significance level, and the quantity these errors are decided from is in what a p-value means. When you can predict the picture before you move the slider, go and practice Type I and Type II error questions.
Frequently asked questions
Can power ever equal alpha?
Yes, and it always does at one specific point. When the true mean equals the null value there is no effect to detect, so every rejection the test makes is a Type I error and power equals alpha exactly. On a two-sided power curve that point is the minimum, which is why two-sided power never falls below alpha no matter how small the sample is.
Why does a two-sided test have less power?
A two-sided test splits alpha between two tails, so each tail gets alpha/2 and the critical value sits further from the null value. Against an alternative in one specific direction, that larger cutoff is harder to clear, so power drops. What you buy for it is the ability to detect an effect in the other direction, which a one-sided test cannot see at all.
What counts as good power?
Researchers usually plan for at least 0.80, meaning a study that would catch a real effect about four times in five. That is a convention rather than a rule, and it is reached before any data is collected by choosing a large enough sample. AP Statistics asks you to explain what moves power, not to calculate it or hit a target.
Does a larger sample raise the Type I error rate?
No. Alpha is chosen by you, not produced by the data, so it stays exactly where you set it no matter how large the sample gets. What a larger sample does is shrink the standard error, which narrows both sampling distributions and pushes beta down. That is why sample size is the one factor that improves both error rates at the same time.
Embed this interactive on your class site (free)
Paste this into any webpage, LMS page, or class site. Free for educational use; the small credit link stays on.
Every interactive on the site is embeddable the same way. See the rest, with notes on where each one fits a lesson, on the resources for teachers page.