Citation

Young C. Am. Sociol. Rev. 2009; 74(3): 380-397.

Copyright

(Copyright © 2009, American Sociological Association)

DOI

10.1177/000312240907400303

PMID

unavailable

Abstract

Model uncertainty is pervasive in quantitative research. Classical statistical theory assumes that only one (true) model is applied to a sample of data. In practice, however, researchers do not know which exact model specification is best. Modern computing power allows researchers to estimate a huge number of plausible models, yet only a few of these estimates are published. The result is a severe asymmetry of information between analyst and reader. The applied modeling process produces a much wider range of estimates than is suggested by the usual standard errors or confidence intervals. I demonstrate this using the work of Barro and McCleary on religion and economic growth. Small, sensible changes in their model specification produce large changes in the results: the results are inconsistent across time, and the instrumental variables strategy suffers from a weak instrument set. Also, the observed relationship between religiosity and economic growth does not hold in the West; it is largely a feature of Asian and African countries and of countries whose data is poor quality. In short, empirical findings should be evaluated not just by their significance but also by their robustness to model specification. I conclude with suggestions for incorporating model uncertainty into practice and improving the transparency of social science research.