Nature reports how more rigorous documentation requirements are demonstrating the intrinsic unreliability of scientistry (the profession of science) and showing how the substitution of scientistry for scientody (the actual scientific process) makes what subsequently passes for science unreliable.
The launch of the clinicaltrials.gov registry in 2000 seems to have had a striking impact on reported trial results, according to a PLoS ONE study1 that many researchers have been talking about online in the past week.
A 1997 US law mandated the registry’s creation, requiring researchers from 2000 to record their trial methods and outcome measures before collecting data. The study found that in a sample of 55 large trials testing heart-disease treatments, 57% of those published before 2000 reported positive effects from the treatments. But that figure plunged to just 8% in studies that were conducted after 2000. Study author Veronica Irvin, a health scientist at Oregon State University in Corvallis, says this suggests that registering clinical studies is leading to more rigorous research. Writing on his NeuroLogica Blog, neurologist Steven Novella of Yale University in New Haven, Connecticut, called the study “encouraging” but also “a bit frightening” because it casts doubt on previous positive results.
Irvin and her co-author Robert Kaplan, chief science officer at the Agency for Healthcare Research and Quality in Rockville, Maryland, focused on human randomized controlled trials that were funded by the US National Heart, Lung, and Blood Institute (NHLBI). The authors conclude that registration of trials seemed to be the dominant driver of the drastic change in study results. They found no evidence that the trend could be explained by shifting levels of industry sponsorship or by changes in trial methodologies.
Translation: there is good reason to be dubious about more than 7 in every 8 historical corporate sponsored medical trials. Keep this in mind when you are basing an argument in support of the safety and efficacy of vaccines on research published by the pharmaceutical industry.
This higher standard of documentation is very welcome, but it underlines the way in which that the human factor is the weak link in the scientific process. No amount of “training” can substitute for forcing scientists to be completely transparent about their work.