This op-ed by Lawrence Krauss explains the amazing picture of the universe assembled by the Planck satellite. It’s a good read. As some may know, Krauss is no stranger to controversy. In the past year, he has been at the epicenter of the feud between scientists and philosophers about the existence of nothing and the value (or lack thereof) of philosophy. (See post Science Vs. Philosophy). In Krauss’ current piece, he makes a couple of ironic comments and takes the obligatory jab at philosophy. Speaking of some of the anomalies in the data, he writes:
If history is any guide, most of these anomalies will disappear. It is an unfortunate facet of science reporting that it isn’t often made clear that most anomalies in experiments tend to go away, just as most theoretical ideas turn out to be wrong.
Indeed. It would be refreshing if physicists would finally come to terms with the fact that String Theory is one such theory. In fact, as far as String Theory is concerned, it is even a worse state of affairs. As Peter Woit of Columbia University points out, respectable theories can turn out to be wrong (because they are testable). Alas, String Sheory is not even wrong (see Woit’s Book). It is not a testable theory now…and there is good reason to believe it will never be tested. So, for all the print spilled on this theory…not to mention the research dollars, someone of repute should step forward and say this theory is dead. (By someone of repute I mean one of the flag wavers who advanced this theory and popularized it for the public). The irony here is that Krauss and his cohorts are leading the attacks on philosophy and religion. And yet, String Theory was worked on for years more as a matter of faith than good science. So, it would be helpful to remember that scientists are susceptible to collective cognitive episodes of irrationality. And Krauss goes on…
Instead of attributing significance to potentially strange results, it is the business of science to try and prove them wrong before we blindly move forward. Skepticism is the business of the day, and it is wise to remember this next time you read an astounding discovery claimed in the press.
Yes. And not only should we cast a skeptical eye on discoveries, we should also be aware that the medical research community has failed to police itself when it comes to publicizing the results of their studies. Each week we read about some study that claims some benefit from consuming this or that…or abstaining from one thing or another. Other studies focus on the benefits of a certain drug or intervention for a variety ailments. Despite the attention afforded these studies in the media, most are either proven wrong or demonstrated to have exaggerated the effects of the independent variable. Want some numbers? A 2005 study on this topic was conducted by John P. A. Ioannidis, MD (Harvard) examining how often highly cited clinical research studies were either contradicted, proven less than reported, confirmed with other studies, or unchallenged. Here are the results from the study:
Of 49 highly cited original clinical research studies, 45 claimed that the intervention was effective. Of these, 7 (16%) were contradicted by subsequent studies, 7 others (16%) had found effects that were stronger than those of subsequent studies, 20 (44%) were replicated, and 11 (24%) remained largely unchallenged. Five of 6 highly-cited nonrandomized studies had been contradicted or had found stronger effects vs 9 of 39 randomized controlled trials (P = .008). Among randomized trials, studies with contradicted or stronger effects were smaller (P = .009) than replicated or unchallenged studies although there was no statistically significant difference in their early or overall citation impact. Matched control studies did not have a significantly different share of refuted results than highly cited studies, but they included more studies with “negative” results.
It’s 2013 and I guarantee nothing in the medical research community has been done to change this sad state of affairs. Let’s put these numbers in perspective. Highly cited studies get reported in the media and their conclusions adopted by many doctors and practitioners. Of the 49 studies Ioannidis examined, less than half were replicated! Replication is the controlling mechanism in all of science! And aside from the fact that 16% were contradicted, 24% of these studies had never been challenged. I suspect the rate of contradiction would go up if the remaining 24% went under some scrutiny. Here is the Ionnidis study: JOC50060
So, I agree with Krauss, skepticism is the business of the day. And it applies to all fields. Read about how Vitamin E increases your ability to solve differential equations? Don’t run out and buy a barrel of sunflower seeds. Read about how pomegranates gives you everlasting health? Don’t run out to Whole Foods just yet. The bottom line here is that most of what is reported by medical research needs to be taken with a grain of salt. But just a grain..because too much salt will kill you.
High-res image of the Planck picture of the universe.