Part 1 – Credibility and software
Part 1 asks what a claim in social science is for and why people describe the field as being in crisis, and then gives figures for it from the three SCORE studies. It ends inside the one of those studies in which the facilitator took part, on five analysts of the same claim who disagreed but whose work could be checked afterwards, because every one of them shared a readable, runnable document. The next part goes through the toolchain that makes that kind of checking possible.
| Work through | What you do | Timing |
|---|---|---|
| Slides, the credibility talk | What social science is for; false positives and forking paths; incentives and questionable research practices, and why the worry is structural rather than a matter of fraud; the red-card study and its twenty-nine teams; why ‘replication’ names three different operations; what open research proposes. | 26 minutes |
| Slides, SCORE and its three studies | The programme; reproducibility, robustness and replicability each put to a hundred or more papers; why the three verdicts do not go together. | 20 minutes |
| Slides, Multi100 from the inside | What a Multi100 analyst received, the paper in brief, the two Multi100 tasks and how their results are put on a common scale; then a forest plot of the five analysts’ results, one row at a time, with a column for the software that each of them used. Then open-source software. | 18 minutes |
| Slides, the workspace and the browser lab | The workspace, the report and the browser lab in one slide, and what the tools cannot do. | 8 minutes |
| Your workspace, or Browser lab Task 1 | Get to the point where a line of R runs and the panel of 270 country-years is in front of you. | 8 minutes, and the 30-minute break after it if you need more |
By the end of the part you have R running with the panel loaded, you can distinguish the three operations that go under the word ‘replication’, and you can say why five analysts of one claim, all of them working carefully, returned five different numbers.