Reproducibility, the ability to replicate or reproduce experimental results, is one of the major tenets of the scientific method.
However, in the case of academic preclinical research, reproducibility (or more accurately the lack of reproducibility) has become a significant problem. An increasing number of reports have found discrepancies in published preclinical studies across scientific disciplines. For instance:
- Amgen found that 47 of 53 “landmark” oncology publications could not be reproduced.
- Bayer found that 43 of 67 oncology & cardiovascular projects were based on contradictory results from academic publications.
- Dr. John Ioannidis and his colleagues found that of 432 publications purporting sex differences in hypertension, multiple sclerosis, or lung cancer, only one data set was reproducible.
These studies, and the many others that report similar results, highlight a significant problem in the development of new therapies to treat disease. The identification of potential drug candidates typically happens in academic research labs. Pharmaceutical companies then use these new drug candidates as the basis for their drug development efforts. With increasing reports of discrepancies in preclinical publications, pharmaceutical companies are being forced to re-evaluate their reliance on academic research (see Bayer’s decision to halt nearly two-thirds of target-validation projects).
So why do so many preclinical publications contain research that can’t be reproduced?
The blame may be attributable to the lack of double-blind experiments which, while required in clinical trials, are rarely enforced in preclinical settings. In the absence of double-blind conditions, there is a risk that researchers—driven by a desire to get high impact publications to support their tenure and grant funding aspirations—selectively publish positive results. Further, with an increasing number of researchers around the globe competing for grant funding and tenure positions, there may be decreasing incentives for collaboration in reproducing preclinical results.
I’m not alone in believing that we need to address the lack of reproducibility in preclinical academic research. There is widespread agreement that infrastructural changes are needed to create incentives that promote collaboration and reproducibility of preclinical academic research (see the 2010 Yale Law School Round Table report or C. Glenn Begley and Lee M. Ellis’ recent Nature paper). Any new set of incentives should focus, in part, on the funding agencies and journal publications that support academic research. For instance:
- Funding agencies should allocate some proportion of a grant to allow research groups to include independent replication in their project work flow.
- Journal editors should encourage independent replication of key results prior to publication.
I believe that Science Exchange can be used to facilitate the reproducibility of experiments and improve the reliability of preclinical academic research. As an online marketplace for outsourcing experiments, we make it easy to verify experiments and key findings at independent research sites. Our platform is well positioned to help funding agencies and journal editors ensure greater reproducibility in the research they fund and publish by making it easy for researchers to implement reproducibility into their project work flow.
At Science Exchange, we’re actively developing ways to help scientists independently replicate their research. I’m sure many scientists will feel that their research is now being unfairly judged by association. One option we are considering is to provide scientists with an ability to obtain a reproducibility ‘stamp of approval’ for their work through Science Exchange. If you’d like to be part of this initiative or learn more about the programs we’re working on please feel free to email me at [email protected].
[about_box image=”http://thebenchapp.s3.amazonaws.com/wp-content/uploads/2012/03/Elizabeth-80.png”]Elizabeth Iorns is Co-Founder & CEO of Science Exchange. Elizabeth conceived the idea for Science Exchange while an Assistant Professor at the University of Miami and as CEO she drives the company’s vision, strategy and growth. She is passionate about creating a new way to foster scientific collaboration that will break down existing silos, democratize access to scientific expertise and accelerate the speed of scientific discovery. Elizabeth has a B.S. in Biomedical Science from the University of Auckland, a Ph.D. in Cancer Biology from the Institute of Cancer Research in London, and conducted postdoctoral research in Cancer Biology from the University of Miami’s Miller School of Medicine where her research focused on identifying mechanisms of breast cancer development and progression.[/about_box]