Significance Level vs Power

The significance level, written $\alpha$ (the Greek letter alpha), is the line you draw in advance for how much evidence counts as convincing. If the p-value is at or below $\alpha$ you reject the null hypothesis; if it is above, you fail to reject. For example, with $\alpha = 0.05$ you reject when the p-value is 0.05 or less. Because $\alpha$ is also the long-run rate of falsely rejecting a true null hypothesis, a smaller $\alpha$ such as 0.01 demands stronger evidence.

Power is a test's ability to detect a real effect when one truly exists. It equals $1 - \beta$, where $\beta$ (the Greek letter beta) is the probability of a Type II error. For example, a test with power 0.80 has an 80% chance of rejecting the null hypothesis when the alternative is genuinely true. Power rises with a larger sample size, a bigger true effect, less variability, or a larger significance level $\alpha$.