The case analysed where its author analysed it, on data you can run
Every other page on this site works at the country-year level: 270 rows, one per country and year, which is where the workshop’s own exercise lives. The study underneath it did not. Its central result comes from a model fitted to individual survey respondents, and some things visible there are invisible from above.
This page fits that model. It is optional, it is not part of the workshop day, and it exists so that the individual-level version of the argument is available rather than merely described. The original author is referred to throughout as OA, following the convention of the rest of the site.
1. Why the level matters at all
The claim is about countries: when a national economy performs badly, do the people in it frame the EU less positively? That is a statement about country-years, and the workshop’s panel answers it directly.
But the same words license a second reading. Perhaps unemployed people frame the EU differently, so a country with more of them shows a lower average without anything having changed for anyone else. The first reading is contextual – living through high unemployment changes how everyone frames the EU. The second is compositional – the mix of people changed, not the people.
A country-year panel cannot separate them, because it contains no people. Individual data can, and this page does it.
2. The dataset, and why you can trust it
The Eurobarometer microdata behind the case are GESIS-licensed and cannot be redistributed, so this page cannot ship the real respondents. What it ships instead is a synthetic twin: EUframes_person_full.csv, 41,660 simulated respondents across the real 270 country-years, carrying every variable that OA’s published regressions use. No real respondent appears in it. The simulation module explains how it is built; the short version is that a respondent’s characteristics come from a correlation structure estimated on the real file, and their answers from a model of the survey instrument conditional on those characteristics.
What licenses a number from it is the validation rather than the care in construction: the same model was fitted to the real respondents and to the twin, and the two coefficient vectors were compared.
Table 1: Every coefficient of the anchor model, fitted on 416,698 real respondents and on the twin. The last column is the difference over the two-sample standard error.
valid |>summarise(`coefficients compared`=n(),`within 1 standard error`=sum(abs(z) <1),`within 2 standard errors`=sum(abs(z) <2),`largest discrepancy`=round(max(abs(z)), 2))
# A tibble: 1 × 4
`coefficients compared` `within 1 standard error` `within 2 standard errors`
<int> <int> <int>
1 64 46 60
# ℹ 1 more variable: `largest discrepancy` <dbl>
60 of 64 agree within two standard errors, where roughly three would be expected to fall outside by chance alone. The comparison lives in _model-outputs/twin_validation.csv and can be read line by line.
Two cautions belong here rather than in a footnote. The yardstick above is the two-sample standard error, which asks whether a difference exceeds the sampling noise of the twin itself – the question that a reader quoting one of its numbers has. Judged instead against the precision that 416,698 respondents afford, only 39 agree; the twin holds a tenth as many people and cannot stand in at full precision. And the model validated is OA’s specification with its cross-level interactions removed. The twin reproduces that. It is not validated for the interacted form, and should not be quoted for it.
3. The anchor, at the level it was estimated
fit <-function(y) { f <-reformulate(c("female", "mage", "edu3", "ses", "urban","mgrowth", "munemp", "(1 | cntry)", "(1 | cy)"),response = y)lmer(f, data = twin, REML =FALSE)}m_cosmo <-fit("cosmo")parameters::model_parameters(m_cosmo, effects ="fixed",ci_method ="satterthwaite")
Uncertainty intervals (equal-tailed) and p-values (two-tailed) computed
using a Wald t-distribution with Satterthwaite approximation.
Read the macro terms first. A one-point rise in a country’s unemployment rate goes with a fall of about 0.0054 in an individual’s cosmopolitan framing score, net of who they are.
Now read sesunemployed, which is the same respondent’s own employment status. Being unemployed rather than a manual worker moves the score by -0.028 – roughly 5 times as much per unit. Individual circumstance matters far more to an individual than the national rate does.
That comparison is the one that people get wrong, so it is worth doing the arithmetic rather than eyeballing it.
4. A large individual effect that explains almost nothing
The two coefficients above are not competing explanations of the same thing, and the units make the difference. The individual effect applies to whoever is unemployed. The contextual effect applies to everyone. So how much of the country-level association could the individual effect account for?
That depends on a quantity that neither coefficient contains: how much a country’s unemployment rate changes who is in the sample. Measured on the real respondents, a one-point rise in the published rate adds 0.0074 to the share of respondents who are themselves unemployed – well under one percentage point.
b_ind <-fixef(m_cosmo)[["sesunemployed"]]b_ctx <-fixef(m_cosmo)[["munemp"]]grad_w <- grad$gradient[2]tibble(quantity =c("individual effect of being unemployed","extra unemployed share per point of the national rate","compositional contribution to the country-level slope","the country-level slope itself","compositional share of it"),value =c(b_ind, grad_w, b_ind * grad_w, b_ctx, b_ind * grad_w / b_ctx)) |>mutate(value =signif(value, 3))
# A tibble: 5 × 2
quantity value
<chr> <dbl>
1 individual effect of being unemployed -0.0278
2 extra unemployed share per point of the national rate 0.00743
3 compositional contribution to the country-level slope -0.000207
4 the country-level slope itself -0.00544
5 compositional share of it 0.038
So the compositional path accounts for roughly 4% of the country-level association. An effect several times the size, spread over a group that barely grows, moves the average hardly at all.
One caution about that figure. It is computed from the coefficients of the twin itself, and the unemployed contrast is where the twin does worst: it comes back at -0.0278 against -0.0350 on the real respondents, the largest single discrepancy in the validation table. Using the real value the compositional share is 4% rather than 4%. Both are small; the conclusion does not depend on which you use, and it is worth seeing a case where the twin is visibly imperfect and the argument survives anyway.
This is the calculation to demand whenever an aggregate association is explained by ‘it must be the people who changed’. It is rarely done, and it usually does not survive being done. Note also what it takes: an individual-level effect, and a prevalence gradient. The second is the part that gets assumed, and it is measurable.
5. Two designs, one claim, different answers
The workshop’s panel and OA’s own model are both defensible readings of the same claim, and they do not agree.
Table 2: The unemployment coefficient on cosmopolitan framing under two designs. The aggregate model is the workshop’s anchor; the individual model is the published one, fitted on the twin.
agg <-lm(mcosmo ~ unemp + growth +factor(year) +factor(cntry), data = panel)tibble(design =c("country-year panel, two-way fixed effects","individual, country and country-year random intercepts"),n =c(nrow(panel), nrow(twin)),estimate =c(coef(agg)[["unemp"]], fixef(m_cosmo)[["munemp"]]),removes_common_year_shocks =c("yes", "no")) |>mutate(estimate =signif(estimate, 3))
# A tibble: 2 × 4
design n estimate removes_common_year_…¹
<chr> <int> <dbl> <chr>
1 country-year panel, two-way fixed effec… 270 -0.00356 yes
2 individual, country and country-year ra… 41660 -0.00544 no
# ℹ abbreviated name: ¹removes_common_year_shocks
The individual-level estimate is about half again as large, and the reason is in the last column. OA’s design carries country and country-wave random intercepts but no year effects. Nothing in it removes a shock that hit every country at once, and 2009 was exactly that: unemployment rose everywhere and EU framing moved everywhere, for reasons that include but are not limited to unemployment. The two-way fixed-effects design removes it; OA’s does not, so its estimate carries some of it.
Neither is wrong. They are answers to different questions, which is the workshop’s own argument arriving one level down – and it is worth noticing that the difference here is larger than most of the forks on the specification menu.
On the panel it is inert. At individual level it is clearly moving. An effect that exists or does not depending on which defensible design you chose is a strong argument for declaring the design before looking.
6. What the levels do to the evidence
The multilevel structure matters here for the standard error, and for what the model is willing to count as information.
Table 3: One predictor, three random structures, fitted on the twin. The real-data values are read from a committed table produced by the same models on the 416,698 licensed respondents.
ml <-list("country only"= cosmo ~ unemp + (1| cntry),"country and country-year"= cosmo ~ unemp + (1| cntry) + (1| cy),"with a random slope"= cosmo ~ unemp + (1+ unemp | cntry) + (1| cy))keys <-c("two_level", "three_level", "slopes")imap(ml, \(f, nm) { m <-lmer(f, data = twin) p <-coef(summary(m))["unemp", ] k <- keys[match(nm, names(ml))]tibble(structure = nm,twin_estimate = p[["Estimate"]], twin_t = p[["t value"]],twin_df = p[["df"]],real_estimate =rv(k, "unemp"), real_df =rv(k, "unemp", "df"))}) |>list_rbind() |>mutate(across(where(is.numeric), \(x) signif(x, 4)))
# A tibble: 3 × 6
structure twin_estimate twin_t twin_df real_estimate real_df
<chr> <dbl> <dbl> <dbl> <dbl> <dbl>
1 country only -0.00534 -8.99 28910 -0.00479 391200
2 country and country-year -0.00564 -5.24 251. -0.00524 253.
3 with a random slope -0.00601 -2.95 15.4 -0.00571 18.9
In the first row, declaring only the country level lets the model treat every respondent as fresh information about unemployment, and the degrees of freedom run into the tens of thousands – on the twin, and into the hundreds of thousands on the real file. But the predictor takes only 270 distinct values in the whole dataset. A model reporting six-figure degrees of freedom for a variable that varies 270 times is counting people as though they were countries.
Declare the country-year level and the same data answer differently: the estimate barely moves, and the degrees of freedom collapse to the order of the number of cells. That is the correct count, and the t falls accordingly.
The variance components tell the same story from the other side.
Table 4: Where the variation sits, on the twin and on the real respondents. Standard deviations, from the country and country-year model.
# A tibble: 3 × 3
component twin real
<chr> <dbl> <dbl>
1 between countries 0.0646 0.0613
2 between country-years 0.0446 0.0385
3 between people 0.356 0.368
Person-to-person variation dwarfs everything else – around 95% of the total. That is the same fact as the shape of the outcome at this level, where a respondent who mentions two things and finds one of them cosmopolitan scores exactly one half: a lumpy, bounded, near-categorical variable whose scatter within any cell is enormous compared with the differences between cells. It is also why so many respondents buy so little precision about a country-level predictor.
The country-year component in the twin runs about a sixth above the real one, and the reason is worth knowing rather than smoothing over: its cells hold a tenth as many respondents, so their means are noisier, and some of that sampling noise is attributed to the cell level rather than to people. The country and residual components sit within a few per cent.
Person-level sample sizes advertise precision that the design does not possess whenever the predictor lives at a higher level. Two of the five Multi100 analysts fitted person-level models, and this is a large part of why their strengths of evidence differ so much from the other three.
7. What this dataset will and will not support
Stated plainly, because a synthetic dataset that is trusted for the wrong things is worse than none.
It will support the specification validated above and things close to it: the four framing dimensions, the published covariate set, country and country-year random structures, and the composition-against-context arithmetic. Coefficients from those may be quoted with the validation table beside them.
It will not support three things. Anything at the survey-wave level, because the twin is built on 270 country-years and its finest cluster is the country-year, while the real file has sixteen waves across ten years. The interacted form of the published model, which was excluded from the validation deliberately: a synthesis built to reproduce a covariance structure does not reproduce a cross-level interaction unless it is put in. And analyses of individual framing items, since the twin reproduces the average salience of each item but not its country-by-country variation – no consequence for any model using the four dimension scores, which is every model here.
Its weight is not a survey weight.w1 reverses a deliberate over-sampling of younger respondents that the generator introduces so that weighting has something to correct. It is a teaching device with a known answer, not the Eurobarometer design weight, and it should not be read as one.
TipTry it
Fit the model of section 3 again with ses removed, and watch what happens to munemp. It should barely move, and the reason is section 4: the compositional path is a few per cent of the total, so deleting the variable that carries it changes little.
Then do the same with mgrowth removed, and with the country-year random intercept removed. One of those three matters far more than the others; find out which, on data whose answer you can check.
Sources
The specification is OA’s published Table 3 Model 1 with its cross-level interactions dropped, taken from the author-provided Stata code in the SCORE dossier (Teney 2016). The twin is generated by companion/_model-outputs/simulate_person_joint.R from the public country-year panel and committed calibration files; the calibration was estimated once, locally, from a person-level rebuild of the sixteen GESIS waves that is never committed and never redistributed – the repositories module explains that regime. The validation table comes from validate_twin.R, the real-data multilevel reference from fit_person_multilevel.R, and the prevalence gradient from fit_person_anchor.R, all in the same folder. Every model on this page is fitted when it is built, from files that ship with the site.
References
Teney, Céline. 2016. “Does the EU Economic Crisis Undermine Subjective Europeanization? Assessing the Dynamics of Citizens’ EU Framing Between 2004 and 2013.”European Sociological Review 32 (5): 619–33. https://doi.org/10.1093/esr/jcw008.