Type I vs Type II Error and Statistical Power

The two ways a hypothesis test can go wrong

Every hypothesis test compares two claims: a null hypothesis, written $H_0$ (the "no effect" or "no difference" claim), and an alternative hypothesis, written $H_a$ (the claim you are testing for). You collect data and decide whether to reject $H_0$. Because you never see the whole population, that decision can be wrong in two different ways.

A Type I error is rejecting $H_0$ when $H_0$ is actually true. A Type II error is failing to reject $H_0$ when $H_0$ is actually false. These are the only two mistakes a test can make, and only one of them is even possible on a given test, because $H_0$ is either true or false (you just do not know which).

The probability of a Type I error is $\alpha$ (alpha), the significance level you choose before running the test, often 0.05. The probability of a Type II error is $\beta$ (beta), which you do not set directly. For a refresher on setting up the two hypotheses, see null vs alternative hypothesis.

The decision table

Line up your decision against the hidden truth and there are four boxes. Two are correct and two are errors.

Read it by column. When $H_0$ is true, the only way to be wrong is to reject it, which is a Type I error. When $H_0$ is false, the only way to be wrong is to fail to reject it, which is a Type II error. The main diagonal (top-left and bottom-right cells) holds the two errors, and the other diagonal holds the two correct decisions.

Type I error: rejecting a true null hypothesis

A Type I error is a false alarm. You reject the null hypothesis and announce an effect when there is no real effect at all. Its probability is exactly $\alpha$, the significance level, so you control it directly. Testing at $\alpha = 0.05$ means that if $H_0$ is true, there is a 5% chance your sample leads you to reject it by random chance alone.

You lower the Type I error rate by choosing a smaller $\alpha$, such as 0.01. In a test you reject $H_0$ when the p-value is less than $\alpha$, so a smaller $\alpha$ demands stronger evidence before you call an effect real. For how the p-value drives that decision, see what does a p-value mean.

Type II error: failing to reject a false null hypothesis

A Type II error is a missed detection. There really is an effect, but your data did not give you enough evidence to reject $H_0$, so you come away saying there is not enough proof. Its probability is $\beta$.

You do not choose $\beta$ the way you choose $\alpha$. It depends on how big the true effect is, how much data you have, and the significance level. A tiny real effect measured with a small sample is easy to miss, which makes $\beta$ large. A big real effect measured with a large sample is hard to miss, which makes $\beta$ small.

Statistical power explained

Statistical power is the probability of correctly rejecting a false null hypothesis. In symbols, $\text{power} = 1 - \beta$. If a test has a power of 0.80, then whenever a real effect exists the test detects it 80% of the time and misses it (a Type II error) the other 20% of the time.

Power and $\beta$ are two sides of the same coin: anything that lowers the Type II error rate raises power, and anything that raises $\beta$ lowers power. A high-power test is good at catching real effects, which is why researchers plan sample sizes to reach a target power (0.80 is a common goal) before they collect any data.

Which error is worse depends on the stakes

Neither error is automatically worse. Which one hurts more depends on the consequences in context, so write both consequences out and then judge the stakes. State what a Type I error would cause (acting on an effect that is not real) and what a Type II error would cause (missing an effect that is real), then decide which is more costly for that specific situation.

When a Type II error is worse. A cancer screening test uses $H_0$: the patient does not have cancer. A Type I error flags a healthy patient for follow-up tests they did not need, causing worry and expense. A Type II error tells a patient who does have cancer that they are fine, delaying treatment. Missing the disease is usually judged the worse outcome, so screening tests are built for high power even though that allows more false positives.

When a Type I error is worse. A new drug must prove it beats a placebo before approval, with $H_0$: the drug is no better than a placebo. A Type I error approves a drug that does not actually work, so patients take a useless treatment and may skip one that helps. A Type II error keeps a genuinely helpful drug off the market until further testing. Regulators often treat approving an ineffective drug as the worse outcome, so they require a small $\alpha$.

What increases statistical power

You can raise power (equivalently, lower $\beta$) in several ways. Each one works only if the others stay fixed.

• **A larger sample size $n$.** More data tightens the sampling distribution, so a real effect is easier to detect. This is the safest lever because it raises power without raising the Type I error rate.
• **A larger significance level $\alpha$.** Moving from $\alpha = 0.01$ to $\alpha = 0.05$ makes you quicker to reject $H_0$, which raises power, but it also raises the Type I error rate. That is a genuine tradeoff.
• A true effect farther from the null. The further the truth sits from $H_0$, the easier it is to detect. You usually cannot control the real effect size, but a bigger true difference gives more power for free.
• A smaller standard error. Less variable data, or a more careful and consistent measurement process, shrinks the standard error and sharpens the signal, which raises power.

Notice that raising $\alpha$ trades one error for another: it lowers $\beta$ but raises the chance of a Type I error. Increasing $n$ is the one lever that improves both at once.

How this appears on the AP exam

This topic is 3.8, "Potential Errors When Performing Tests," in Unit 3 of the Fall 2026 AP Statistics course, Inference for Categorical Data: Proportions. The exam expects you to identify which mistake is a Type I error and which is a Type II error in a given scenario, describe each error's consequence in context, and explain how factors like sample size and significance level affect power. You are expected to find error probabilities from given values or a simulation, as in the worked examples above, but there is no power formula to memorize.

These ideas show up on the free-response section too. Section II Question 3 is a full inference question (a hypothesis test or a confidence interval), and it can ask you to interpret an error or a power statement as part of your conclusion. For the wider list of inference tools, see which statistical test to use. The official course page is AP Statistics on AP Central.