Investigative question practice (8 problems)

(a) Hold the sentence against the three components. The variable is missing. "Enough sleep" is a judgment, not something a data collector can record: two students could disagree about whether 7 hours is enough and both be answering honestly, so the sentence does not say what to write down. What is already fine is the population. "Our students" is a defined group of 1,480 people with an enrollment list behind it, which is more than most wonderings supply.

(b) The collection component. Replace the judgment with a variable someone can measure: "On a typical school night, how many hours of sleep do students at this school get?" Hours of sleep on a school night is quantitative, every student has a value, and a data collector now knows exactly what to ask. Any wording that names hours of sleep on a school night and the students at this school does the job.

(c) The analysis component, interval version. A confidence interval question names the parameter and says the goal is to estimate it within a range: "What is the mean number of hours of sleep on a school night for the 1,480 students at this school, and what range of values is plausible for it?" The parameter is $\mu$ ("mu"), the mean for all 1,480. It is not the mean of the 120 who get asked, which is $\bar{x}$ ("x-bar"), a statistic.

(d) The analysis component, test version. A test question names the parameter and the direction of the alternative. The 8 hour target comes from the stem, so a direction is available: "Is the mean number of hours of sleep on a school night for students at this school less than 8 hours?" Parameter $\mu$ again, direction less than. The interval version and the test version are different questions about the same parameter, and the counselor has to choose one before collecting rather than after seeing the numbers.

(e) The conclusion component and the fraction. The conclusion will describe the mean sleep of the 1,480 students at this school, and it will be a description rather than a cause, because nobody is assigning anyone a bedtime. It reaches all 1,480 because the 120 are randomly selected from the enrollment list. The fraction sampled is $\frac{120}{1480} \approx 0.081$, about 8.1% of the school.

(a) Census. The population is 62 buses sitting in the district's own yard, and the district needs the exact number rather than an estimate. A census records information from every item in the population, so nothing is inferred and there is no sampling variability to account for. It is not a survey, because buses do not answer questions.

(b) Experiment. The manager can impose the condition: she decides which days the second line opens. Imposing a treatment is the whole definition of an experiment, so an experiment is available to her. Deciding the days by hand is not enough to earn the causal claim, though. If she opens the second line on the days she expects to be busy, the treatment moves with the crowd, and the explanation that busy days are simply the days she opens it survives untouched. What removes that explanation is random assignment: draw which days get the second line with a random mechanism and every other variable, the day of the week, the menu, the weather, the size of the lunch period, is spread across the two groups by chance rather than by her expectations. A shorter mean wait is then left with only two explanations, the second line or the luck of the draw, and that is what a causal claim rests on. Note where the treatment lands: the waits are timed on students, but the condition is assigned to a day, and the experimental unit is whatever the treatment is assigned to, so the units here are days.

(c) Survey. The data are what people report about what they would do, which no record contains, and the council can reach only $\frac{200}{900} \approx 0.222$, about 22.2%, of the school, so it is a sample rather than a census. Whether that 22.2% supports a statement about all 900 depends on how the 200 are chosen, which the stem does not say. Students who happen to sit in one homeroom are a convenience sample, and the estimate would then describe those 200 only.

(d) Observational study. The explanatory variable is years of night-shift work, and the stem says outright that it cannot be assigned. With no assignment there is no experiment, so the researcher compares people who already differ, and the comparison can report an association only.

(e) Observational study. The drivers chose the course for themselves; the team registered them and watched, but assigned nothing. Following units forward in time does not make a study an experiment. The only thing that would is imposing the course on a randomly chosen half of the 900.

(f) Direction in time. Situation (d) reads work history that already happened, so it is retrospective. Situation (e) starts at registration and follows the same 900 drivers forward for three years, so it is prospective. Both are observational, and neither earns a causal claim, which is the point: the direction in time changes the logistics, not the scope of the conclusion.

Read the set as a decision tree. First ask whether anyone imposes a condition, which picks out (b) alone. Then ask whether everyone can be measured, which picks out (a). Then ask whether the data are people's answers to a standard set of questions, which picks out (c). What is left is observational, and the only remaining question is which way it faces in time.

(a) Two proportions. Of those who answered, $\frac{3051}{4812} \approx 0.634$, so 63.4% clicked Yes, which is where the headline's 63% comes from. The people who answered are $\frac{4812}{210000} \approx 0.023$ of the daily readership, about 2.3%, so roughly $210000 - 4812 \approx 205{,}000$ readers who could have clicked said nothing.

(b) Sort the three questions. (i) Yes, and exactly. The 4,812 who answered are not a sample of themselves, they are all of themselves, so 63.4% answers (i) with no inference and no margin of error attached. (ii) No. The respondents were not randomly selected from the city's residents; they selected themselves by clicking, and they had to be reading this site's transit article to be in the pool at all. Nothing in the data connects them to the city. (iii) No. Nobody was assigned to read or not read the article, and no support was measured beforehand, so there is neither a comparison group nor a before value.

(c) Name the sample. It is a voluntary response sample: every respondent decided for themselves to be counted. People who feel strongly about a levy click far more often than people who feel mildly, so 63.4% is pushed off the readership's true share in a consistent direction. That is bias, a systematic error, not the random wobble a random sample would have.

(d) The editor's fix. It does not work. A larger voluntary response poll measures the same wrong quantity more precisely: the tilt lives in who chooses to click, and that is unchanged at 50,000. Sample size shrinks variability and does nothing to bias, which is the argument in does a bigger sample fix bias. What the site would have on 50,000 responses is a confident wrong number.

(e) A question the site could answer. "What proportion of the city's registered voters support the transit levy, and what range of values is plausible for that proportion?" To answer it, draw a random sample from the city's voter registration list, contact those people directly, read every one of them the same wording, and record how many could not be reached. The parameter is $p$, the proportion of registered voters who support the levy. Note where the conclusion stops: registered voters, because that is the list the sample came from, and not all residents, since anyone not on the list had no chance of selection.

(a) Interval, no direction. The manager asked for a range of plausible values, which is the confidence interval form of the analysis component: name the parameter, and say the goal is to estimate it within a range. The parameter is $\mu$ ("mu"), the mean weight in grams of all the bags the machine fills. Nothing is being decided against a stated value, so there is no direction to give.

(b) Test, direction greater than. The contract supplies the value to test against, 0.05, and the words above that supply the direction. The parameter is $p$, the proportion of all this operation's packages that arrive late, and the alternative is $p > 0.05$. Keep the parameter a proportion of packages: a count of late packages and a percentage of one particular sample are both something else.

(c) Test, direction not equal. Differs is two-sided. The researcher has not said which county she expects to be higher, so the alternative points both ways. The parameter is $p_1 - p_2$, the difference between the two counties' population proportions, and the alternative is that this difference is not 0. Two populations are in play, so the question is not answerable until it names both counties.

(d) Test, direction association or not independent. That is the fourth kind of direction the analysis component allows, and it is the one that does not come with a single number. The claim is that favorite sport and class year are related among this school's students, so the alternative is that the two variables are associated, or equivalently not independent.

(e) The one with no interval. It is (d). An interval estimate needs a parameter to place an interval around, and with 3 sports crossed against 4 class years there is no single number that captures the relationship. Contrast (c): a difference between two proportions is a number, so it could be estimated with an interval instead of tested, and there the choice comes from the stated goal rather than from the data. That is worth noticing, because it means the analysis component is a decision you make, not one the data makes for you.

(a) Two responses, and why you have to choose. Works could mean a faster 20-meter sprint time in seconds, or fewer hamstring injuries over a season, or a longer sit-and-reach distance in centimeters. Any two of those count. They are different studies: a sprint time is one afternoon's measurement taken on every player, while injuries are rare enough that a season and far more than 48 players would be needed before a difference meant anything. The reason both cannot stay open is that the investigative question has to be fixed before the data arrive. A coach who measures five things and reports the one that came out well has not answered a question, he has chosen one, and problem 6 puts a number on how often that produces a result from nothing.

(b) The explanatory variable. It is the warm-up routine, a categorical variable with two levels: the new routine and last season's routine. Those two levels are the treatments. Sprint time is the response, measured after the treatment, and no second factor is in play, so this is one explanatory variable with two levels rather than a crossed design.

(c) The collection method. An experiment. The feature that makes it available is stated in the stem: he can decide which players do which warm-up. Imposing the condition is what separates an experiment from everything else, so the causal question is open to him. Earning the answer takes the random assignment set up in part (d): letting players pick a routine, or handing the new one to the players he thinks need it, would leave the difference in sprint times explained just as well by which players ended up in which group.

(d) Units and sizes. A warm-up is assigned to a player, so the experimental units are the 48 players. An even split is $\frac{48}{2} = 24$ players per routine. Do the assignment with a mechanism rather than by preference: number the players 1 to 48, generate random integers from 1 to 48 ignoring repeats until 24 distinct numbers appear, give those 24 the new routine, and give the remaining $48 - 24 = 24$ last season's.

(e) The director's question. This study covers 48 players on one squad, and they were not randomly selected from anything. Random assignment earns the causal claim for those 48 and players like them; it says nothing about the district's other 13 schools, whose players differ in age, sport, and training history. The change that would answer him is random selection: draw players, or whole squads, at random from across the 14 schools, then randomly assign the two warm-ups within that sample. Random assignment and random selection do two different jobs, and he is asking for the one this study does not have.

(a) Count the false alarms you should expect. If no club is really related to GPA, each comparison still has a 5% chance of coming out significant on its own, so across 30 of them you expect $30 \times 0.05 = 1.5$ significant results from chance alone. That figure needs nothing beyond the 5% per comparison; it does not require the 30 comparisons to be unrelated to each other, which they are not, since a student can belong to several clubs. He found 2. That is about what running 30 comparisons produces when there is nothing there.

(b) The requirement he broke. An investigative question has to be posed before the data are collected and analyzed, and it must not be changed once the results are visible. He had no question at all, so the data chose one for him. The written-up question exists because of its result, which is exactly what stops that result from being evidence for it. The first component of an investigative question names the variable of interest in advance, and here the variable of interest was picked last.

(c) Answer the defense on its own terms. Every one of the 30 comparisons uses real data and correct arithmetic, and none of that is in dispute. The problem is which comparison got reported. He ran 30 and published the ones that looked good, and part (a) says roughly 1.5 would look good even if every club were unrelated to GPA. Correct arithmetic on a comparison selected because of how it turned out tells you the arithmetic was right, not that the comparison was worth making.

(d) The second unsupported claim. The population. His sentence says "students," which reads as students in general, while the file covers the 1,200 students at one high school and nobody was randomly selected from anywhere. Even a solid association here reaches those 1,200 and students like them, and the sentence has to say so.

(e) What to do with it. Treat it as a question rather than as an answer. Write the investigative question down first, before any new data: "Is mean GPA higher for debate club members than for non-members at this school?" That fixes the variable, the parameter, and the direction in advance. Then collect a fresh set of data and answer that one question. A pattern spotted in a file is a reason to run a study, not the result of one. Note that the fresh study is still observational, since nobody assigns students to debate club, so even a confirmed difference stays an association.

(a) Answerable, and exactly. Days absent is recorded for every one of the 1,140 students, so the records are a census of the school on that variable. Compute the mean and you have the parameter itself. There is no sample, no interval, and no margin of error, because nobody was left out and nothing is being inferred.

(b) Answerable as an association. Both variables sit in the records for all 1,140 students, so the relationship between them can be described for this school with no new collection at all. What the records cannot supply is a direction of influence, because neither variable was assigned to anyone.

(c) Not answerable. Nobody assigned bus riding: students ride or do not ride depending on where they live, and where a family lives is tied to plenty of things that affect attendance on their own. Earning the word cause would take an experiment that randomly assigns students to ride or not, which a school cannot run. The version of (c) these records can answer is (b)'s shape: are absences associated with riding the bus at this school.

(d) Not answerable. Sleep is not in the file, and no analysis creates a variable nobody recorded. To answer it, collect the variable: ask a random sample of the school's students how many hours they slept on a stated school night, which supports a conclusion about all 1,140, or ask every student for a census. Either way this is new collection, and since the school holds no record of anyone's sleep it has to come from the students themselves, which in practice means a survey.

(e) Not answerable. The records are a census of this school and contain no student from any other school, so there is nothing to compare against. You would need absence data for a random sample of the state's students, or a published statewide figure, and the comparison would then be between this school and that population rather than between this school and itself.

(f) The proportion and its label. $\frac{486}{1140} \approx 0.426$, about 42.6% of the school rides a bus, and the other $1140 - 486 = 654$ do not. Because the records cover every student at the school, that number describes the whole population of interest, so it is a parameter for this school, not a statistic. The same arithmetic would produce a statistic if the 1,140 were a sample of something larger. Parameter or statistic depends on what the number describes, never on how it was computed.

(a) The descriptive question. Collection component: in the past 30 days, how many bike share trips did a registered rider take for which a car was available to them at the time and they chose the bike instead? That is quantitative, and it is something only the rider can report, which is why the office's own trip records cannot supply it. Note what the wording avoids: asking how many trips a rider "would otherwise have driven" asks for a guess about a trip that never happened, and two riders making the same trip could answer it differently and both be honest, which is the objection problem 1 used to reject "enough sleep." Whether a car was available is something the rider knows. Analysis component: what is the mean number of such car-available bike trips for the 12,000 registered riders, and what range of values is plausible for it? The parameter is $\mu$, that population mean. Conclusion component: the conclusion describes the 12,000 registered riders and claims no cause, and it reaches all 12,000 only if the 400 are randomly selected from the office's list. The method is a survey of $\frac{400}{12000} \approx 0.033$, about 3.3% of the riders.

(b) The causal question. Collection component: how many trips start at a docking station over an 8-week period, and does that station have a covered shelter? Analysis component: is the mean number of trips started per station over 8 weeks greater for stations with a covered shelter than for stations without one? The parameter is the difference between the two treatment means, and the direction is greater than. Conclusion component: because shelters will be randomly assigned, a difference can be attributed to the shelter, and because the stations in the study will be randomly selected from the 240, that conclusion reaches the city's 240 comparable stations. The method is an experiment, available because the office can decide which stations get shelters.

(c) The randomization, in two stages. First select. Number the 240 comparable stations 1 to 240, use a random number generator to draw integers from 1 to 240 ignoring repeats until 90 distinct numbers appear, and those 90 stations are the study's units. That is $\frac{90}{240} = 0.375$, 37.5% of the comparable stations. Then assign. Number those 90 from 1 to 90, draw distinct integers from 1 to 90 ignoring repeats until 45 appear, build covered shelters at those 45, and leave the remaining $90 - 45 = 45$ as they are. Count trips started at each of the 90 over the same 8 weeks and compare the two mean trip counts. The experimental units are stations, because a shelter is built at a station, so every conclusion is a statement about stations.

(d) What each licenses. The survey: because the 400 riders were randomly selected from the list of 12,000, the estimate of the mean number of car-available bike trips generalizes to all 12,000 registered riders, and because nothing was assigned, it describes them rather than explaining anything. The experiment: because shelters were randomly assigned, a higher mean trip count at sheltered stations is evidence the shelter caused it, and because the 90 stations were randomly selected from the 240, that causal conclusion reaches the city's 240 comparable stations. The experiment has both randomizations, which is the strongest position a study can be in and the rarest.

(e) Why working stays unanswered. Working is not a variable. Each study answers one narrow question about one measurable thing, and the office still has to decide in advance what it will count as working: car-available bike trips, trips per station, rider retention, cost per trip. That decision belongs to the first component of the investigative question, and it belongs before any data arrive, because a measure chosen after the results are in is chosen for its result. What the two studies do show is that the choice of measure decides the collection method: a condition the office can impose points at an experiment, and something only a rider can report points at a survey.