An interesting news item in what is becoming an increasingly common genre of news: scientific non-reproducibility.
I’m not surprised at such stories’ growing prevalence, having become increasingly suspicious of the durability of findings, even in large and prestigious studies, in my own field of medicine during my professional career. But over 50% non-reproducibility in psychological research begins to change the rules by which one can claim to “know” anything in that field at all. After all, there is no culture of routinely replicating research, any more than there is in other sciences. It would seem that independent replication of research ought to be mandatory – but it’s hardly likely to happen.
Admittedly, one might expect things to be less secure in a “soft” science like psychology, as opposed to those disciplines not involving those awkward human beings as a subject (though it ought to be of concern that science is all done by awkward human beings!), but that doesn’t seem to stop more items being churned out in the popular press on psychology than on anything else – perhaps because it is a lot more important to us than the other sciences, and ought to be dealing in real knowledge.
Yet the fact remains that the current state of psychological truth remains pretty insecure, unless a lot more confirmatory research happens. Although such research is based on experimentation and data, it is more likely to be untrue than not, according to this survey… assuming that this survey itself is reliable. How ironic if it reflected the trend it describes!
And that raises a further thought: how likely to be true, do you think, is research that is not based on controlled experimental trial results, and is intrinsically not capable of being repeated – like evolutionary psychology, for example?
This reminds me of what Dennis Prager – a conservative talk radio host – says:
“Studies either confirm common sense, or their wrong”
That could be true …
I suppose another way of putting it is that if your theoretical presuppositions are counter-intuitive, your results will be too, until they’re repeated by someone less batty.