Part 4 – Multiverse and preregistration
Part 4 opens on the chart that the class has just built and closes on the single analysis you would defend if you had to pick one. The reading on the specification curve is its companion.
| Work through | What you do | Timing |
|---|---|---|
| Multiverse and slides | Read the exploratory dots from the class against the full curve and the five published analysts, and find your own among them. | 10 minutes |
| Slides | Bet on which analytical choice moves estimates most before the decomposition is shown. The decomposition then attributes about half of the variation across specifications to the choice of outcome. Then where the five analysts are placed on the claim-aligned grid, and where the wrong-sign results concentrate. | 16 minutes |
try_spec() on the Browser lab page |
Rerun the specification you submitted, then read off your own percentile, which is the share of the 840 specifications more negative than the estimate you submitted, and send it when the facilitator says go. Without a submission of your own, use the reproduced baseline instead. | 7 minutes |
| Slides, the multiverse tutorial and what the curve leaves out | Optional depth on where the idea came from and what a multiverse is for, followed by a slide that is always shown, on the four things that a curve leaves out. | 10 minutes |
| Reading a specification curve and slides | Why hundreds of estimates on the same data are not hundreds of independent tests, what a curve-level permutation test would have to do, and how to read one if it were run. Then the discussion, argued in chat or at the table. | 12 minutes |
| Slides, with Reproducibility repositories as the follow-up | The three critical positions on multiverse analysis, the argument that a fully specified question has one correct analysis, what preregistration settles and what it leaves open, and the OSF pages that hold the materials of the case. | 10 minutes |
| The report skeleton in the workspace, or Browser lab Task 5 | Complete the preregistration block before you run anything, translate the specification in the block into model syntax yourself, run it, and submit with report_result(). Your declared result is drawn on the chart as a filled circle beside the open circles of the exploratory round. |
20 minutes |
| Multiverse | Wrap up with the chart on screen: two points from the literature, and the reminder that the path you declared was one choice among the alternatives you had just seen. | 5 minutes |
By the end of the part you have your own percentile in the 840-specification grid, one specification fixed in the preregistration block of your report before it was run, and a report of your own that contains the result of that specification.