Type I Error vs Significance Level

A Type I error requires two conditions at once: $H_0$ is true, and the test rejects it anyway. Its long-run rate under a true null is exactly the significance level $\alpha$ (alpha) you chose, which makes it the one error probability you set directly rather than inherit.

Take $H_0: \mu = 100$ against $H_a: \mu > 100$, with $\sigma$ (sigma) known to be 15 and $n = 25$, so the standard error is 3 and an $\alpha = 0.05$ test rejects once $\bar{x}$ (x-bar) clears 104.93. If the true mean really is 100, that happens in 5 percent of samples. Those 5 percent are Type I errors, and there is nothing wrong with the data or the arithmetic in any of them: the sample was unlucky, not mistaken. Drop $\alpha$ to 0.01 and the cutoff moves to 106.98, so the rate falls to 1 percent.

The sentence to retire: "my p-value was 0.03, so there is a 3 percent chance this rejection is a Type I error." Once the decision is made, either $H_0$ is true and you erred or it is false and you did not; the coin has already landed. $\alpha$ describes the procedure across many repetitions, not the single conclusion in front of you, and 0.03 is the p-value, computed under a null that may well be false.

The error is also unavailable when $H_0$ is false, no matter how badly the test performs. Rejecting a false null is the correct decision, and failing to reject it is a Type II error. Which of the two mistakes you could even make depends on a truth you never observe.

Lowering $\alpha$ cuts this error and, with $n$ and the true effect held fixed, raises the other: at $\alpha = 0.01$ the test above has $\beta = 0.7453$ against a true mean of 105, where $\alpha = 0.05$ gave 0.4913. A larger sample cuts both at once. Which error costs more is a judgment about consequences, and AP Statistics topic 3.8, Potential Errors When Performing Tests, asks for that judgment in context.

$\alpha$ (alpha) does two jobs with one number. It is the cutoff the p-value is compared against, and it is the probability the test rejects $H_0$ when $H_0$ is true. Those coincide because the rejection region is built to hold exactly that much area under the null distribution, which is also why $\alpha$ has to be chosen before the data arrive.

Take $H_0: \mu = 100$ against $H_a: \mu > 100$, with $\sigma$ (sigma) known to be 15 and $n = 25$, so the standard error is 3. Setting $\alpha = 0.05$ puts the cutoff at $100 + 1.645(3) = 104.93$: if the true mean really is 100, 5 percent of samples of 25 land above 104.93 through nothing but sampling variability. Set $\alpha = 0.01$ instead and the cutoff moves out to 106.98.

The misreading is subtle and common: "I test at $\alpha = 0.05$, so 5 percent of the results I call significant are wrong." It is not 5 percent of your rejections. It is 5 percent of the tests you run on null hypotheses that happen to be true. What fraction of your significant findings are false alarms depends on how often the nulls you test are true in the first place, and $\alpha$ carries no information about that.

A fixed threshold also does not make 0.049 and 0.051 different evidence. They sit either side of an agreed line, nothing more, which is why the p-value gets reported alongside the decision rather than swallowed by it.

Say which lever you mean. Holding $n$, the true effect, and the spread fixed, lowering $\alpha$ lowers power: in the test above, $\alpha$ of 0.10, 0.05, and 0.01 gives power 0.6499, 0.5087, and 0.2547 against a true mean of 105. That is a trade between the two error rates, not a law binding power to $\alpha$, because raising $n$ from 25 to 100 lifts power to 0.9543 with $\alpha$ still 0.05. AP Statistics covers the errors this threshold governs in topic 3.8, Potential Errors When Performing Tests.