The Problem of How We Practice “Science”

[T]he reason I feel compelled to dabble in [the replication crisis in science] . . . is that I don’t feel like anyone else is conveying the level of absolute terror we should be feeling right now. As far as I can tell, this is the most troubling outbreak of the replication crisis so far. And it didn’t happen in a field like social psychology which everyone already knows is kind of iffy. It happened in neuroscience, with dramatic knock-on effects on medicine, psychology, and psychiatry.

The above quote comes from psychiatrist Scott Alexander discussing a recent study that found that neurogenesis – the creation of new brain cells in humans – does not happen in any appreciable number in adult humans.  Adult neurogenesis was supposedly proven in a 2002 study and thereafter became a foundation for a significant number of derivative theories, some with real world consequences in medicine.  Thousands of scholarly articles cited to either the original study or to follow on studies.

To that, Mr. Alexander comments:

. . .  We know many scientific studies are false. But we usually find this out one-at-a-time. This – again, assuming the new study is true, which it might not be – is a massacre. It offers an unusually good chance for reflection.

And looking over the brutal aftermath, I’m struck by how prosocial a lot of the felled studies are. Neurogenesis shows you should exercise more! Neurogenesis shows antidepressants work! Neurogenesis shows we need more enriched environments! Neurogenesis proves growth mindset! I’m in favor of exercise and antidepressants and enriched environments, but this emphasizes how if we want to believe something, it will accrete a protective layer of positive studies whether it’s true or not.

I’m also struck by how many of the offending studies begin by repeating how dogmatic past neuroscientists were for not recognizing the existence of adult neurogenesis sooner. . . .

Science is not a label one simply attaches to a finding.  Science is defined by the four step process of the scientific method:  hypothesis, experiment, findings, replication.  Skip any step — as so often happens, particularly as regards global warming and our temperature records, and what you are left with is not science, it is a polemic to the faithful.  Thus, this post is a reminder of three things.

One, anyone who tells you the “science is settled” in any complex area is a con man or a complete idiot unworthy of being allowed in the public square.  Two, even well meaning scientists can make honest errors, ergo the absolute necessity of replication — and a true pox on all who would ever have the temerity to argue for the use of “secret science.”  Three, we rely far too much on science in the modern world to leave its policing to modern scientists, many of whom seem to be budding Lysenkos and many others with their own agenda.  As I argued here, we need to reform how science is practiced, published and financed in this country, including applying the same standards of fraud that we apply to securities, if science is to ever regain a modicum any trustworthiness.

My only qualification of the above is in regards to the marvelous health benefits of chocolate found in the ground breaking studies funded not long ago by Hershey, Inc.  I have been assured by Ms. Bookwormroom that those studies carry their own indicia of trustworthiness and thus have no need of replication.