The Multiverse
Every line is a justifiable analysis
This page shows the specification curve for the workshop exercise. Every justifiable analysis of one claim sits on one chart. The chart is claim-aligned: the claim predicts opposite signs for the positive framings (mcosmo, mutil, mpos) and the negative framings (mcomm, mlib, mneg), so the negative framings are reverse-coded (1 − y) before plotting. That is the same convention that is recorded in your preregistration (claim_align), applied here automatically, and it puts every result on one readable scale. A specification below zero supports the claim, whichever outcome it uses.
The main chart renders at build time and is interactive. Hover over any point to read the full recipe behind it. The chart at the foot of the page is live: it fetches the room’s submissions in your browser and redraws when you press Run Code. Before the session, or when the data files are absent, the page renders with placeholder messages.
The logic of the visualisation follows the specification-curve idea (Simonsohn, Simmons and Nelson): one point per justifiable analysis, ordered by size. Because the menu mixes outcome scales and predictor forms, raw coefficients are not comparable across specifications. The chart therefore plots the partial correlation (r = t/sqrt(t^2 + df)), the same metric that Multi100 used to compare analysts, on the claim-aligned scale. The horizontal line at zero is the boundary between support for the claim and evidence against it, and your own submission will land on one side of it.
The claim-aligned specification curve
The chart holds all 2,520 pre-computed specifications. The canonical 840 are the gaussian family, drawn in blue as open triangles with unemployment as the predictor. The beta-family twin of each one is in green, and the two halves together make the family universe of 1,680. The remaining 840 carry the growth predictor – GDP growth as the claim carrier, alone or beside unemployment, in both families – and are drawn as open squares. Colour therefore carries the outcome family and the marker carries the claim-carrying predictor. Faded points are not significant at 5%.
Both the outcome family and the choice of predictor are workshop additions beyond the Multi100 menu, which fixed unemployment as the predictor. Task 1 of Multi100 shows three of the five analysts freely choosing GDP measures before that instruction existed. Each family curve is ranked within itself, and the claim-alignment convention extends once more: the claim-expected sign for growth is positive on the positive framings, so growth-predictor results are sign-flipped on top of the outcome recoding and below zero still means support for the claim, whichever variable and whichever framing a specification uses.
The five Multi100 analysts sit on the gaussian curve styled as what they ran – blue unemployment triangles, slightly enlarged – each found by its name tag. C6HJR’s gold tag marks the constrained baseline you reproduce in Task 3. The original author (OA) joins them under a red tag, their own published anchor result converted to the common axis exactly as the analysts’ results were. Every mark answers on hover, and hovering a name tag or its marker highlights the pair while the rest of the chart dims.
That the paper’s own anchor lands beside PRL47’s near-zero triangle rather than out among the strong results is the degrees-of-freedom lesson of the statistical methods module in one picture: a person-level t of −4.03 with no recorded residual df is read against its 335,566 respondents, and almost nothing survives the conversion.
On the claim-aligned scale every supportive specification sits below zero: 69.8% of the family universe of 1,680 (68.7% of the canonical gaussian half, 71.0% of the beta half), with 63.5% significant at the 5% level overall. The two family curves all but coincide, and in the cosmopolitan universe (outcome mcosmo) the wrong-sign pattern is identical in each family. The same 33 pooled and 8 two-way specifications run against the claim whichever likelihood is used. The beta family sits on the menu beyond the constraints set by Multi100. The specification menu describes both the extra axis and the claim-alignment convention, and the module on statistical methods explains the beta model itself. Across the 840 growth-predictor specifications the direction agrees with the claim just as often (70.5%), but only 46.9% are significant – the two halves of OA’s two-indicator construct are not equally strong evidence, exactly as analyst KQXUE’s free-phase growth null hinted.
Where the room lands
This chart is live, and it uses the same visual conventions as the chart above. The full universe of 2,520 pre-computed specifications forms the muted backdrop: the gaussian curve in grey, its beta twin in green-grey, triangles carrying unemployment and squares GDP growth, each family ranked within itself. The analysts and OA sit on the gaussian curve as name-tagged triangles, and the room’s own submissions overlay them as black circles, numbered in arrival order and placed where the claim-aligned partial r of each result falls on the curve.
The room submits twice, and the two waves are drawn differently. An open circle is an explored specification – tried while working through the menu, chosen after its result was visible. A filled circle is a declared one, written down before it ran. Hover any circle for the specification behind it. Press Run Code to pull the latest submissions.
The R engine downloads into your browser the first time (10–30 seconds); the button changes from a spinner to Run Code when it is ready.