2.8 Comparing Effects Across IVs
Research question: what social factors most strongly pattern political views?
- Sometimes, want to compare effect sizes across predictors (IVs) within a model
- Complicated by variables on different metric (e.g., age vs income), by variables of different types (e.g., binary vs continuous), and by nonlinearities of continuous IVs
- For binary IVs, the 0 vs 1 effect already provides the full summary
- For nominal IVs, can use an ME inequality
- For continuous IVs, calculate marginal effects, even with nonlinearities
- Can still provide an answer for the average effect
- For comparing effect sizes across binary and continuous predictors, a +2 SD change is sometimes recommended (Gelman 2008) as comparable
- More robust alternative is ME across trimmed range (5th to 95th percentile)
- Logic: binary 0 vs 1 or ME inequality is entire range of variable’s effect; trimmed range similarly expresses entire range of continuous IV’s effect while protecting against outliers (Mize and Han 2025)
Comparing marginal effects within a model
- In Stata, mecompare can simultaneously calculate multiple MEs, and then use metest for additional tests of equality
- amount(trimrange) for 5th to 95th percentile change for continuous IVs
- meinequality option for nominal IVs
- In R, use avg_comparisons and specify each predictor of interest in a list( )
- For a significance test of the difference in marginal effects, use the hypothesis option*
- hypothesis = ~ pairwise will calculate all contrasts, but be careful with multiple exploratory hypothesis tests
- For a significance test of the difference in marginal effects, use the hypothesis option*
- To compare pure effect sizes and ignore direction, take the absolute value of the marginal effects before testing their equality
The example is adapted from Mize and Han (2025):
use "https://tdmize.github.io/data/data/cda_gss", clear(cda_gss.dta | GSS 1972-2021 CDA - Categorical Data Analysis | date created 202
> 3)keep if year == 2021(64,814 observations deleted)drop if missing(polviews, race4, woman, class, age)(424 observations deleted)clonevar conpolviews = polviews
regress conpolviews c.age##c.age i.woman i.race4 i.class Source | SS df MS Number of obs = 3,608
-------------+---------------------------------- F(9, 3598) = 16.61
Model | 339.734948 9 37.7483275 Prob > F = 0.0000
Residual | 8175.67969 3,598 2.27228452 R-squared = 0.0399
-------------+---------------------------------- Adj R-squared = 0.0375
Total | 8515.41463 3,607 2.3608025 Root MSE = 1.5074
-------------------------------------------------------------------------------
conpolviews | Coefficient Std. err. t P>|t| [95% conf. interval]
--------------+----------------------------------------------------------------
age | 0.037 0.009 4.021 0.000 0.019 0.055
|
c.age#c.age | -0.000 0.000 -2.487 0.013 -0.000 -0.000
|
woman |
Women | -0.175 0.051 -3.438 0.001 -0.275 -0.075
|
race4 |
Black | -0.331 0.084 -3.945 0.000 -0.496 -0.167
Other | -0.129 0.111 -1.161 0.246 -0.348 0.089
Hispanic | 0.011 0.083 0.134 0.893 -0.152 0.174
|
class |
working cl.. | -0.021 0.096 -0.223 0.823 -0.209 0.166
middle class | -0.279 0.094 -2.952 0.003 -0.464 -0.094
upper class | -0.536 0.151 -3.556 0.000 -0.831 -0.240
|
_cons | 2.986 0.243 12.273 0.000 2.509 3.463
-------------------------------------------------------------------------------est store mod42b
mecompare age woman race4 class, amount(trimrange) meinequality model(mod42b)Predicting: Linear prediction
Marginal effects (N_mod42b=3608)
| ME # Estimate SE P>|z|
---------------------------------+---------------------------------------
age(5-95%) |
mod42b | 1 0.784 0.082 0.000
---------------------------------+---------------------------------------
woman |
Women - Men |
mod42b | 2 -0.175 0.051 0.001
---------------------------------+---------------------------------------
race4 |
ME Inequality | 3 0.172 0.053 0.001
Black - White |
mod42b | 4 -0.331 0.084 0.000
Other - White |
mod42b | 5 -0.129 0.111 0.246
Hispanic - White |
mod42b | 6 0.011 0.083 0.893
---------------------------------+---------------------------------------
class |
ME Inequality | 7 0.273 0.059 0.000
working cla - lower clas |
mod42b | 8 -0.021 0.096 0.823
middle clas - lower clas |
mod42b | 9 -0.279 0.094 0.003
upper class - lower clas |
mod42b | 10 -0.536 0.151 0.000metest, clear
metest 1 - abs(2), add rowname("age - woman") | estimate se pvalue
---------------------------------+--------------------------------
age - woman | 0.609 0.097 0.000 metest 1 - 3, add rowname("age - race4") | estimate se pvalue
---------------------------------+--------------------------------
age - woman | 0.609 0.097 0.000
age - race4 | 0.612 0.095 0.000 metest 1 - 7, add rowname("age - class") | estimate se pvalue
---------------------------------+--------------------------------
age - woman | 0.609 0.097 0.000
age - race4 | 0.612 0.095 0.000
age - class | 0.511 0.097 0.000 metest 3 - abs(2), add rowname("race4 - woman") | estimate se pvalue
---------------------------------+--------------------------------
age - woman | 0.609 0.097 0.000
age - race4 | 0.612 0.095 0.000
age - class | 0.511 0.097 0.000
race4 - woman | -0.003 0.074 0.972 metest 3 - 7, add rowname("race4 - class") | estimate se pvalue
---------------------------------+--------------------------------
age - woman | 0.609 0.097 0.000
age - race4 | 0.612 0.095 0.000
age - class | 0.511 0.097 0.000
race4 - woman | -0.003 0.074 0.972
race4 - class | -0.101 0.079 0.202 metest 7 - abs(2), add rowname("class - woman") | estimate se pvalue
---------------------------------+--------------------------------
age - woman | 0.609 0.097 0.000
age - race4 | 0.612 0.095 0.000
age - class | 0.511 0.097 0.000
race4 - woman | -0.003 0.074 0.972
race4 - class | -0.101 0.079 0.202
class - woman | 0.098 0.075 0.194 metest, title("Effect size comparison of predictors of political views")Effect size comparison of predictors of political views
| estimate se pvalue
---------------------------------+--------------------------------
age - woman | 0.609 0.097 0.000
age - race4 | 0.612 0.095 0.000
age - class | 0.511 0.097 0.000
race4 - woman | -0.003 0.074 0.972
race4 - class | -0.101 0.079 0.202
class - woman | 0.098 0.075 0.194 Age has a larger effect on political views than does gender, race-ethnicity, or social class
The effects of gender, race-ethnicity, and class are similar in size, with no significant differences in effect size