Hypothesis test classroom activity: is the coin fair
By Jude Wallis · Published
Students judge a claim that a coin favors heads after it lands heads 18 times in 25 flips. Each student flips fair coins in sets of 25, the class plots the counts, and the share of sets with 18 or more heads estimates the p-value. The exact value is 0.022.
AP Statistics: Unit 3 (topics 2.3 Estimating Probabilities Using Simulation, 3.5 Setting Up a Test for a Population Proportion, 3.6 p-Values). Students state hypotheses about a proportion (topic 3.5), estimate a p-value by simulating the null hypothesis (topics 2.3 and 3.6), and interpret it in context before meeting the one-proportion z test.
What the activity shows
A p-value answers one question: if nothing unusual is going on, how often would chance alone produce a result at least this extreme? Students can answer that with coins before they have seen a test statistic or a formula.
The period has two parts. A short demonstration with a stacked deck lets students feel the reasoning: they assume the deck is ordinary, watch the evidence pile up, and abandon the assumption. Then the class builds a null distribution by hand for a coin question and reads p-values off its own dotplot.
Run it as the introduction to significance tests, before the one-proportion z test.
Materials and timing
- One standard 52-card deck, arranged before class so that the top 8 cards are red. Do not shuffle it in front of students.
- Five coins and a cup per student or pair.
- A number line on the board marked from 0 to 25, labeled number of heads in 25 flips.
- Sticky notes or markers for the dots.
Plan on 45 minutes: 8 for the deck, 5 to set up the coin question and the hypotheses, 10 to flip, 7 to plot, and 15 to read p-values and discuss.
Steps
- Hold up the deck and say it is an ordinary deck. Announce a game: you will turn over cards from the top, and the class scores a point for every black card.
- Turn over the first card, then the next, one at a time. After each red card, ask who still believes the deck is an ordinary shuffled deck. Stop when most of the class objects, and write down how many red cards it took.
- Ask what made them object. Steer toward this sentence: if the deck were shuffled, this many reds in a row would be very unlikely. Show the probabilities from the first worked example below.
- Pose the coin question. A coin is flipped 25 times and lands heads 18 times. The owner of the coin says it is fair. Is 18 heads convincing evidence that the coin favors heads?
- Write the hypotheses with as the coin's true probability of heads: and .
- Each student shakes five coins in the cup and tips them out 5 times, for 25 flips, and counts the heads. Then each student does a second set of 25. These coins stand for a coin that is known to be fair.
- Each student places two dots on the number line, one for each set.
- Count the dots at 18 or more and divide by the total number of dots. That fraction is the class's estimated p-value for 18 heads.
- Repeat the count for a milder result: suppose the coin had landed heads 16 times. Count the dots at 16 or more.
- For each result, have students write a conclusion in context using a significance level of 0.05.
What students should see
The deck. For a well-shuffled deck, the chance that the first 5 cards are all red is . For the first 4 it is 0.055, and for the first 6 it is 0.011.
The dotplot. The number of heads in 25 flips of a fair coin is binomial with and . Its mean is 12.5 and its standard deviation is . The dotplot should be mound-shaped, centered between 12 and 13, with about 77% of the dots from 10 to 15.
The exact p-values.
| Heads in 25 flips | Probability of that many or more, fair coin |
|---|---|
| 15 | 0.212 |
| 16 | 0.115 |
| 17 | 0.054 |
| 18 | 0.022 |
| 19 | 0.007 |
What the class count should look like. With 30 students and 60 dots, expect about 7 dots at 16 or more and only 1 or 2 at 18 or more. There is a 27% chance that no dot at all reaches 18. A class in that position has an estimated p-value of 0 out of 60, which tells them the p-value is small. It does not tell them it is zero.
The two conclusions. For 18 heads, the p-value of 0.022 is below 0.05: a fair coin rarely does this, so the class has convincing evidence that the coin favors heads. For 16 heads, the p-value of 0.115 is above 0.05: a fair coin does this more than one time in ten, so the class does not have convincing evidence. In neither case has anyone shown that the coin is fair.
Discussion questions
- What does one dot on the board represent? One set of 25 flips of a fair coin, summarized by its number of heads.
- The question was whether the coin is fair. Why did we simulate with coins we assumed were fair? A p-value is computed as if the null hypothesis were true. We need to know what a fair coin does before we can say 18 heads is unusual for one.
- The p-value for 18 heads is about 0.02. Does that mean there is a 2% chance the coin is fair? No. It means a fair coin produces 18 or more heads in about 2% of sets of 25 flips.
- With 16 heads we did not reject the claim of fairness. Did we show the coin is fair? No. Sixteen heads is consistent with a fair coin and also with a coin that lands heads 60% of the time.
- Where on the number line would a result have to fall before you doubted the coin? Compare the class's sense of this with the table: 17 heads is just above 0.05 and 18 is below it.
- In the card demonstration, at which card did the class object, and what was the probability at that point? Compare it with 0.05.
- Suppose the suspicion had been that the coin is unfair in either direction. How would the count on the dotplot change? Dots at 7 or fewer would count as well, since 7 is as far below 12.5 as 18 is above it. The exact p-value doubles to 0.043.
Extension
Compare with the z test. Once students know the one-proportion z test, have them run it on 18 heads in 25 flips. The sample proportion is , the standard deviation under the null is , and , for a one-sided p-value of 0.014. The exact binomial value is 0.022. The large counts condition is met, since on each side, but the sample is small enough that the normal approximation and the exact value visibly differ. Both lead to the same decision at the 0.05 level.
Find the rejection cutoff. Using the table, a test at the 0.05 level rejects when the count is 18 or more. The chance that a fair coin lands there is 0.022, so that is the test's actual probability of a Type I error, lower than the 0.05 it was allowed.
Tools. The binomial probability calculator gives the exact tail: set n to 25, p to 0.5, and ask for at least 18. The proportion z test calculator gives the z version. For the interpretation students most often get wrong, see what a p-value means.
How unlikely is a run of red cards?
A standard deck has 26 red and 26 black cards. If the deck is well shuffled, find the probability that the first 4, 5 and 6 cards turned over are all red.
The first card is red with probability .
Cards are not replaced, so each later probability uses what is left. The second is red with probability , the third , the fourth .
Four reds: .
Five reds: multiply by to get about 0.0253.
Six reds: multiply by to get about 0.0113.
The probabilities are about 0.055, 0.025 and 0.011. A class that objects at the fifth red card has rejected the claim of an ordinary shuffled deck at a point where the evidence had a probability of about 0.025 under that claim.
Reading a p-value from the class dotplot
Example data, not from a real class: 30 students each plot two sets of 25 fair-coin flips, for 60 dots. Eight dots are at 16 or more heads and two dots are at 18 or more. Estimate the p-values for 16 heads and for 18 heads, compare each with the exact value, and state a conclusion at the 0.05 level.
Estimated p-value for 16 heads: . The exact value is 0.115.
Estimated p-value for 18 heads: . The exact value is 0.022.
Both estimates are close to the exact values and fall on the same side of 0.05 as the exact values do.
For 16 heads: 0.133 is greater than 0.05, so do not reject the null hypothesis.
For 18 heads: 0.033 is less than 0.05, so reject the null hypothesis.
With 18 heads in 25 flips there is convincing evidence that the coin's probability of heads is greater than 0.5. With 16 heads there is not convincing evidence. Sixty dots give a rough estimate: one dot more or fewer at 18 changes the estimate by about 0.017.
Frequently asked questions
What if no dot lands at 18 or more?
With 60 dots that happens about 27% of the time. Report the estimate as 0 out of 60 and ask what it means: a result that a fair coin produced zero times in 60 tries is rare, though not impossible. Then give the exact value, 0.022, or add a second round of flips.
Why use a made-up result of 18 heads and not a real suspicious coin?
A fixed result gives every class the same question and lets you check the dotplot against known values. If you want real data, have a student produce 25 outcomes some other way, such as spinning a coin on a desk, and test that count against the same dotplot. This guide makes no claim about what spinning will show.
Can I skip the card demonstration?
Yes, the coin simulation stands on its own. The deck takes eight minutes and gives students the reasoning in a setting where they supply the suspicion themselves, which makes the later vocabulary easier to attach.
Why is the alternative one-sided?
The suspicion stated before any data was that the coin favors heads, so only large counts of heads count as evidence. The last discussion question covers the two-sided version, where the exact p-value for 18 heads doubles to 0.043.
Are classroom coins fair enough to stand for the null hypothesis?
The activity assumes that ordinary coins shaken in a cup and tipped out land heads half the time. If the class dotplot is centered near 12.5, the assumption is holding up. A calculator command such as sum(randInt(0,1,25)) returns the number of heads in 25 simulated flips and can stand in for the coins.