That said, while we agree that the primary story regarding the insertion of the sociology of religion in the larger discipline, and the empirical topics addressed, is primarily the lack of change (and we will address this in a future post), we think there are some clear and important changes with respect to the emergence of a strong program, the increase in socio-evaluative findings, as well as the relationship of these latter with funding.

Evans is right to point out the difference between statistical significance and substantive importance in quantitative research. With a large sample size, a minuscule difference can achieve high levels of statistical significance. However, this is less of a problem in our study because our sample is relatively small (around 600 cases). Furthermore, we use the relatively high threshold of two-tailed tests. This means that when our findings reach statistical significance the reader can be relatively sure that there is something important going on.

But while statistical significance has clear measures, substantive significance has no measure other than whether you think there is something important going on. To get a feel for it, one needs to put differences into everyday narratives. The following hypothetical situations would be consistent with the findings in our working paper.

• If in the late 1970s a graduate student were to examine the causal models used in sociology of religion articles she would find almost 40% having religion as a dependent variable and only 25% having religion as an independent variable. If she were to do the same review thirty years later she would find only 20% of articles having religion as a dependent variable and over half with religion as an independent variable. Put differently, thirty years ago it was approximately fifty percent more common for religion to be explained than explanatory while in the most recent measurement it was more than twice as likely to be explanatory than explained.

• If our graduate student were charged with reading 100 journal articles on religion in 1978, 12 would reflect negatively on religion while 15 would reflect positively. If she did the same thing in 1993, 6 articles would reflect negatively on religion while 28 would be positive. If she carried out the same task in 2003 21 would negative and 34 positive.

We think our hypothetical graduate student would have quite different experiences in each of these cases and these experiences would likely affect her own developing perspective.

Indeed, other than in certain key moments such as social revolutions or natural disasters, social structure tends to work precisely through small inputs iterated over time. Race, gender, class, residential location, pollution, networks, quality of education, public services, and social programs each have very small effects on individuals when they are compared to each other. But together and iterated over time these and other variables combine to create the radical, structured inequality that characterizes our contemporary world.

Consider funding. The data show that both private and public funding are positively and significantly correlated with positive socio-evaluative findings. If articles with funding are between 12% and 20% more likely to report positive socio-evaluative findings than articles without funding, over time this can have a huge impact (as anyone with credit card debt could readily testify). Thinking about it in substantive terms, for every research project that gets funded there are several more that don’t. And these unfunded researchers tend to observe those who did get funding, think about what went wrong with their own proposal and how they can be more successful the next time. When this experience is iterated over and again, the results can be considerable.

Think about it in another way. Imagine that, for better or for worse, many public and private institutions with power and resources want US citizens to maintain a positive image of religion, and they promote this goal not only through funding for scholarship, but through funding for faith-based social programs, tax breaks for religious institutions, or providing political access to religious leaders. Each of these initiatives likely has a small impact; but taken together and over time, they undoubtedly would affect people’s perceptions of religion.

What has not changed is that religious activists and foundations have continued to control the discourse in the sociology of religion. The same parochial orientations toward data, theory, and methodology that made the sociology of religion a backwater for most of the 20th century continue to hold sway, despite a surge of more general interest in the last 15 years.

Hopefully, a strong program in the sociology of religion will include a critical examination of the factors influencing religious organizations, individual religiosity, and the dynamics of religion over time and across nations. Ignoring religion as a dependent variable is not a strong move.