alpha
critical value 104.9346in z units 1.6449standard error 3.0000effect +5.0 (1.67 SE)
null curve, centered at 100true curve, centered at 105.0alpha 0.0500 Type I: reject a true nullbeta 0.4913 Type II: miss a real effectpower 0.5087 1 minus beta

power against every true mean, holding alpha at 0.0500, n at 25, and the test one-sided

A test at alpha 0.0500 with n = 25 rejects the null once the sample mean passes 104.9346. If the truth really is 105.0, this test catches it with probability 0.5087 and misses it with probability 0.4913. Move any control and watch which error rate pays for the change.

Try to:○ get power above 0.80✓ make beta bigger than alpha○ reach power 0.80 with alpha still 0.05

Setup: a z test of H0 mu = 100 against Ha mu > 100, with the population standard deviation known to be 15. Sigma is held fixed here so the three factors you can move are visible one at a time; shrinking sigma is the fourth thing that raises power.

Type I error, Type II error, and power visualizer · free from StatsLearn