Decision Rule vs Significance Level

A decision rule is the verdict written down before the data exist: fix $\alpha$ (alpha, the significance level), then reject $H_0$ when the p-value is at or below $\alpha$, and fail to reject otherwise. Fixing it first is what gives $\alpha$ any meaning. $\alpha$ is the long-run probability of rejecting a null hypothesis that is true, and that guarantee attaches to a rule chosen without reference to the result.

At $\alpha = 0.05$, a p-value of 0.032 rejects and a p-value of 0.080 does not. The comparison is at or below, so a p-value of exactly 0.050 rejects. Notice how little the data have to do with it: that same 0.080 rejects at $\alpha = 0.10$, and that same 0.032 fails to reject at $\alpha = 0.01$. The rule, not the evidence, is what changed.

"We accept the null hypothesis" is not one of the two outputs. Failing to reject means the data would not be surprising if $H_0$ were true, and that is equally true of a whole range of parameter values near it, which is what a confidence interval displays. The quieter version of the same offence is "$p = 0.06$ was almost significant, so we reject at $\alpha = 0.10$." Moving the threshold after seeing the p-value converts a 5 percent rule into a 10 percent one and throws away the guarantee $\alpha$ was carrying.

The rule returns a verdict, not a strength, and it says nothing about whether this particular call was correct. $\alpha$ controls how often true nulls get rejected across many tests; the single test in front of you either got it right or committed a Type I error, and no rule can tell you which. That is why choosing $\alpha$ is really a judgment about which mistake costs more, since lowering it trades Type I errors for Type II errors.

Comparing the test statistic against a critical value instead is the same rule in different clothing. The statistic falls in the rejection region exactly when the p-value is at or below $\alpha$, so the two routes cannot disagree.

$\alpha$ (alpha) does two jobs with one number. It is the cutoff the p-value is compared against, and it is the probability the test rejects $H_0$ when $H_0$ is true. Those coincide because the rejection region is built to hold exactly that much area under the null distribution, which is also why $\alpha$ has to be chosen before the data arrive.

Take $H_0: \mu = 100$ against $H_a: \mu > 100$, with $\sigma$ (sigma) known to be 15 and $n = 25$, so the standard error is 3. Setting $\alpha = 0.05$ puts the cutoff at $100 + 1.645(3) = 104.93$: if the true mean really is 100, 5 percent of samples of 25 land above 104.93 through nothing but sampling variability. Set $\alpha = 0.01$ instead and the cutoff moves out to 106.98.

The misreading is subtle and common: "I test at $\alpha = 0.05$, so 5 percent of the results I call significant are wrong." It is not 5 percent of your rejections. It is 5 percent of the tests you run on null hypotheses that happen to be true. What fraction of your significant findings are false alarms depends on how often the nulls you test are true in the first place, and $\alpha$ carries no information about that.

A fixed threshold also does not make 0.049 and 0.051 different evidence. They sit either side of an agreed line, nothing more, which is why the p-value gets reported alongside the decision rather than swallowed by it.

Say which lever you mean. Holding $n$, the true effect, and the spread fixed, lowering $\alpha$ lowers power: in the test above, $\alpha$ of 0.10, 0.05, and 0.01 gives power 0.6499, 0.5087, and 0.2547 against a true mean of 105. That is a trade between the two error rates, not a law binding power to $\alpha$, because raising $n$ from 25 to 100 lifts power to 0.9543 with $\alpha$ still 0.05. AP Statistics covers the errors this threshold governs in topic 3.8, Potential Errors When Performing Tests.