Randomization test

If the null hypothesis of no treatment effect holds, each response would have been the same no matter which group its subject landed in, so you can reassign the labels at random and recompute the statistic thousands of times. The resulting randomization distribution approximates the sampling distribution, and the p-value is the fraction of reshuffles at least as extreme as what you actually observed. For example, if 38 of 1000 reshuffles produce a difference in means of 4.2 or more in absolute value, the two-sided p-value is $38/1000 = 0.038$. The Fall 2026 AP course defines the randomization distribution in Unit 2 and expects you to build distributions by simulation, but the tests you carry out in Units 3 and 4 are the formula-based z, t, and chi-square procedures.