Amateur Post-hoc Research in public policy discourse

In the wake of the high profile mass-shooting incidents, I’ve been watching with interest the debates from policy makers through news media and even social media. It’s got to be disheartening for academics. Answering questions like the ones being posed right now is a scholar’s reason for living. Unfortunately, a multitude of knee jerk reactions have been proposed by various interest groups and factions. Today, we look at the Salem witch trials of the late 1600s as foolish- but these witch hunts are still going on today in how we react to public policy data. 

“If you torture the data long enough, it will confess.”  Ronald Coase, (as cited by Gordon Tullock, 2001)

ImageConsider how my dog does science. I sometimes buy sunflower seeds that come in a particular type of resealable plastic bag. I pick up the bag and its sound resonates with her as what she knows to be the sound of the “Pup-Peroni” dog treat bag. She approaches, ears perked attentively, tail wagging. I want to please her, so I pull out the dog treat bag and give her one. She’s a good dog. She doesn’t know the difference between the sunflower seed bag and the dog treat bag. She has evidence to show that the majority of times she hears that noise, she gets the expected result, too. So from her perspective, it’s good science. 

I work on a number of journal articles and my dissertation, typically, with CNN in the background. It keeps my mind’s back-channels occupied enough that I can focus on what I’m doing, if that makes any sense. I hear the story about the shooting in Newtown. I glance up at the TV and the same plastic bag noises come through for me: white male, troubled, semi-automatic weapons, defenseless victims with no apparent connection, no apparent motive. This shooter ended his own life along with the others’. I recall some children were involved in the Colorado shooting, but this one took place in an Elementary school. I pause. I look up and take in more of the news and have very little time for emotional response before my eye-rolling reflex kicks in toward the media coverage, the attention that this shooter gets, and the post-hoc research being kicked off by the news network. I realize this coverage will get worse and I feel fatigue in advance for the sensationalism and armchair analysis that is about to take place. I know the amateur researchers are gearing up for a season of witch hunting. 

Post-hoc means “after this” or “after the fact”- and it basically seeks to make sense of what’s observed, rather than choosing what to observe beforehand. Some of the advantages to post-hoc research include the ability to take into account more variables; as a pre-design scheme would dictate more exact research questions and conditions. Unfortunately, amateur post-hoc research often involves a dog’s eye view of the treat bag. Post-hoc analysis often concludes that because two things co-occur, one must have caused the other.


For instance, I know that cavities are caused by visiting the Dentist, because I don’t seem to have them until my Dentist points them out to me. I’d be much healthier if I just avoided going to the Dentist.

“Three out of four mass shooting perpetrators had the following traits,” says the post-hoc analysis. The NRA proposes more guns; other policymakers propose less guns; conspiracy theorists conspire; lobbyists perspire. Media coverage has been overbearing, and some of the loudest voices on both ends provide us all with a spectacle. 

Unfortunately, the politicians I’ve seen on both ends of the argument resemble my dog responding to the sunflower seed bag. They propose legislation on semi-automatic weapons, video games, mental health, and whatever other traits they’ve seen. 

Scholars sit on the sidelines. They would propose a multivariate model of understanding the interconnectedness of the variables whose interaction precipitated such behavior, but nobody asked. They would weigh the factors’ comparative influence on violent outcomes, but nobody wants to wait for that. They would propose that an outcome like what happened in Newtown is just one of the many unacceptable outcomes that result from a system of variables, and that we need to look at the health and environment of all our people. They would want to consider all the factors- in advance. They would want to do professional research. 

If we made policy decisions based on the same science with which we build skyscrapers, satellites, airplanes, and other things whose failure have dire costs, imagine the success we could enjoy as a nation. But to achieve this, we need to elect researchers, thinkers, and scholars to lead us. My dog can do amateur post-hoc research. We need to choose leaders who want to use professional research to make decisions. 


One thought on “Amateur Post-hoc Research in public policy discourse

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s