Part 3 – The claim and its reproduction
Part 3 reads one published paper in full, asks what quantity its claim is an estimate of, draws the graph that shows what has to be held constant, and then has you reproduce, to the third decimal, the t-statistic that Multi100 recorded for analyst C6HJR, one of the five analysts who reanalysed this paper. Without the estimand and the graph the choices you make in the afternoon would be arbitrary, which is why a day on tools keeps them.
| Work through | What you do | Timing |
|---|---|---|
| Slides, the paper in full | The four framing scales, the model that the original author (OA) fitted, Model 1 and Model 2, which put the robustness question inside the paper, the failed attempts to reproduce the paper and the replication on new data whose estimate had the opposite sign, and the five questions that the Multi100 analysts asked in the free phase, when each chose their own analysis. | 20 minutes |
| Slides, theoretical and empirical estimands | Lundberg’s three steps from a theoretical quantity to an empirical one. | 8 minutes |
| Slides, the causal-graph crash course | Confounders, mediators and colliders; open and closed paths; total against direct effects. Then our own claim: everyone writes a one-sentence estimand, holds it, and sends it together. | 25 minutes |
R/dag_starter.R in the workspace, or Browser lab Task 2 |
Look at the panel; write the estimand in words; take the # off the arrows you are willing to defend, re-run adjustmentSets(), and write two sentences on what two-way fixed effects adjust for. |
12 minutes |
| Slides, the constrained task | The three Multi100 instructions read verbatim, what each assumes, the columns of the panel, and the aim of the reproduction. | 8 minutes |
| Your working script, or Browser lab Task 3 | Fit mcosmo ~ unemp_c with country and year fixed effects, which should give t = −3.804, the value recorded by Multi100 for analyst C6HJR. Type your t without sending it, and send it when the facilitator says go. Then the question of why everyone in the class got the same t while five analysts, each writing their own code, did not. |
17 minutes |
By the end of the part you have the estimand written in one sentence, a graph that includes only the arrows you would defend, and a reproduction of t = −3.804 in your own report.