Randomization Test vs Hypothesis Test
Both terms below come up in the same part of the course, and students mix them up. Here is each one defined on its own, side by side, so you can see where they part company.
Randomization test
Hypothesis testing
A randomization test builds the null distribution by reshuffling the observed data through simulation instead of relying on a formula.
If the null hypothesis of no treatment effect holds, each response would have been the same no matter which group its subject landed in, so you can reassign the labels at random and recompute the statistic thousands of times. The resulting randomization distribution approximates the sampling distribution, and the p-value is the fraction of reshuffles at least as extreme as what you actually observed. For example, if 38 of 1000 reshuffles produce a difference in means of 4.2 or more in absolute value, the two-sided p-value is . The Fall 2026 AP course defines the randomization distribution in Unit 2 and expects you to build distributions by simulation, but the tests you carry out in Units 3 and 4 are the formula-based z, t, and chi-square procedures.
Hypothesis test
Hypothesis testing
A hypothesis test uses sample data to weigh a null claim against an alternative, gauging how surprising the data would be if the null claim were true.
A significance test decides whether the evidence in a sample is strong enough to reject a default claim about a parameter. You state a null and an alternative hypothesis, compute a test statistic, and find its p-value under the null. For example, to test whether a coin is fair you compare the observed number of heads to what a fair coin predicts. If the p-value is at or below the significance level, you reject the null hypothesis; otherwise you fail to reject it.