Multiverse Analysis: On the Robustness of Functional Form and Data Pre-Processing Decisions

Functional form assumptions are central ingredients of a model specification.

Just as there are many possible control variables, there is also an abundance of estimation commands and strategies one could invoke, including ordinary least squares (OLS), logit, matching, and many more. How much do empirical results depend on the choice of functional form? In this talk we demonstrate the functional form multiverse with two empirical applications: how job loss affects wellbeing in panel data and the effect of education on voting for Trump. We find in our cases that OLS and logit produce very similar results, but that matching estimators can be surprisingly unstable. We also reconsider a key many-analysts study and find that human researchers produce a much wider range of results than does the multiverse algorithm.