Specification Menu
Choosing your specification
The specification menu defines the space of justifiable analyses you choose from. Every option is defensible, but you have to write the defence yourself.
The menu mirrors the structure of Gelman’s ‘garden of forking paths’: at each decision point, there is more than one reasonable choice, and choosing without reasons is what turns a multiverse into noise.
One note on ordering. The day runs the Multi100 sequence in reverse: its analysts chose freely first and were standardised months later, while you start from the standardised baseline model and recover the freedom one declared axis at a time. The menu therefore expands a single constrained option rather than restricting a blank page.
Every result is claim-aligned
One convention applies across the room. The claim predicts a negative unemployment coefficient for the positive framings (mcosmo, mutil, mpos) and a positive one for the negative framings (mcomm, mlib, mneg). Plotted raw, the class chart would therefore straddle zero by construction and read as noise. The day’s convention is claim-alignment: a negative framing is reverse-coded before fitting,
d$mcomm_ca <- 1 - d$mcomm # claim-aligned communitarian framingso that ‘supports the claim’ reads as a negative sign for every outcome, and the class chart is drawn on that scale. Reverse-coding a bounded scale flips the sign of the coefficient and leaves its magnitude, standard error, |t| and p-value untouched (exactly so for the beta family too, since logit(1 − μ) = −logit(μ)), so nothing about the evidence from your model changes, only the direction convention it is stated in. It is nonetheless a real analytic decision. That is why it rides on the claim_align switch in your preregistration, set to TRUE as the day’s convention and applied for you rather than buried silently in the fitting code. If your outcome is a positive framing, there is nothing to reverse-code.
One further flip applies when your claim variable is GDP growth. The claim then predicts a positive coefficient on the positive framings, so claim-aligned reporting multiplies your b, t and r by −1 (the lab applies this for you). After both conventions, ‘supports the claim’ reads as a negative sign for every specification on the menu, whatever it measures.
Worked example
Choice: outcome = mpos, outcome family = gaussian, predictor = unemployment, predictor form = raw unemp, copredictor = none, estimator = two-way FE via feols, sample = all years, weights = none.
Justification (2 sentences): The “positive dimensions” wording in the claim maps more directly onto the composite mpos scale than onto mcosmo alone, so using mpos is the more literal operationalisation of the stated hypothesis. Grand-mean centring the unemployment rate changes the intercept but not the slope when fixed effects are present, so using raw unemp produces an identical coefficient; the choice is made explicit here to avoid an undocumented transformation. (mpos is a positive framing, so the claim-alignment step requires nothing here.)
This is the kind of justification that makes a specification defensible: it connects the analytical choice to the wording of the claim or to a causal argument, rather than to convenience or preference.
Commit before you run
Commit to your specification before you run it. That is what stops you picking the path that happens to give the answer you want. You record the commitment by completing the preregistration block in your report before you add the model-fitting code, then running. It is a real template – the OSF “Preregistration Template from AsPredicted.org”, shortened to five fields – so the exercise mirrors what a registered secondary-data analysis actually does.
- Positron workspace (Route 1): the report skeleton (
index.qmd) carries the preregistration block in Task 5. Fill in your one specification and its DAG-grounded justification, and render the report before you run. The rendered block, timestamped in your report, is your record. - Browser lab (Route 2): there is no render step, so complete the commented preregistration block in the Browser lab and copy it into the plain-text file where you are saving your work. That is your commitment.
An optional mirror form may be used to log preregistrations for the room, and the link is shared in the session. It is not required. The completed block in your report is your official preregistration.
Where your numbers come from
Every specification on the menu already sits in the pre-computed universe, so the browser lab never fits a model live. run_spec() looks your declared choice up in the grid and prints its b, se, t (or z for a beta family), df and claim-aligned partial r, and report_result() carries the same numbers to the class chart. The values arrive claim-aligned: for a negative framing the coefficient, statistic and r flip sign, and for a GDP-growth predictor they flip as well. Nothing else changes, so they match what your own claim-aligned fit would produce. If a machine will not run the cells at all, the facilitator can look the same eight choices up in data/spec_grid_full.csv by hand and enter the numbers directly.
Send your result to the room
After running your model, submit your result to the class-results Google Form. You do not fill it in by hand and you do not transcribe anything. report_result() takes no arguments: it reads your preregistered choices, builds the spec label itself, and prints a one-click link with all your numbers already in it, on the claim-aligned scale. Open it and press Send, and your result appears on the live Multiverse chart.
The helper standardises every submission to a partial correlation, r = t / sqrt(t^2 + df) – the same conversion that Multi100 used – so that models on different outcome scales and estimators all sit on one comparable axis.
Route 2: the helper is already on the Browser lab page. Route 1: it lives in your Positron workspace at R/report_result.R:
source("R/report_result.R")
report_result()This prints your submission link, ready to click.
What each choice commits you to
Outcome variables. The four EU-frames dimensions (mcosmo, mutil, mcomm, mlib) are 0–1 relative-frequency scales. mpos and mneg are composites (means of the positive and negative sub-dimensions respectively). All are in EUframes_cy.csv, and the codebook (data/EUframes_cy_codebook.md in the template repo) carries the exact definitions. The negative framings (mcomm, mlib, mneg) take the claim-alignment step above; name the recoded variable so the transformation is visible (the _ca suffix is the room’s habit).
Predictor form. Log-transforming unemployment makes sense if you believe the effect is proportional rather than additive; centring or standardising makes coefficients more comparable across specifications but does not change the t-statistic. Document your choice.
GDP growth as a copredictor. From the DAG perspective: is growth a confounder (a common cause of unemployment and EU framing), a second predictor of interest (another pathway you want to estimate separately), or a mediator (part of the path from unemployment to framing)? Your answer determines whether including it is theoretically defensible for your estimand.
Estimator choice. Two-way FE controls for time-constant country traits and year-specific EU-wide shocks. Country FE only removes country traits but leaves year shocks in the error. Random effects assumes the country-level heterogeneity is uncorrelated with the covariates – testable via a Hausman test. Pooled OLS with cluster-robust SE ignores the panel structure entirely; it requires the strongest assumption.
Sample restrictions. Dropping Greece and Spain removes the two countries with unemployment rates far above the rest during the crisis period. If the effect is driven mainly by these outliers, the coefficient will shrink substantially. That is a legitimate sensitivity check.
Outcome family. Beta regression is a defensible fork here because the outcomes are bounded 0–1 proportions and the country-year cell means sit strictly inside the interval, so a logit-link beta model respects that support where a linear model does not. Beta fits report a z statistic rather than a t; report_result() handles both, and the partial-correlation conversion is unchanged (r = z / sqrt(z^2 + df)), exactly how Multi100 treated analyst PRL47’s z. Two comparisons show how much the fork does. The beta twin of the workshop anchor gives z = −4.075 and r = −0.258 against the Gaussian r = −0.242: same direction, still significant, slightly stronger. And the wrong signs in the cosmopolitan universe break down identically across families – 33 pooled + 8 two-way FE in each. So the family choice moves the numbers a little and leaves the conclusions where they were. The theory behind the family choice is in the statistical methods module.