The Need for Reproducibility in Academic Research

Reproducibility, the ability to replicate or reproduce experimental results, is one of the major tenets of the scientific method.

However, in the case of academic preclinical research, reproducibility (or more accurately the lack of reproducibility) has become a significant problem. An increasing number of reports have found discrepancies in published preclinical studies across scientific disciplines. For instance:

• Amgen found that 47 of 53 “landmark” oncology publications could not be reproduced.
• Bayer found that 43 of 67 oncology & cardiovascular projects were based on contradictory results from academic publications.
• Dr. John Ioannidis and his colleagues found that of 432 publications purporting sex differences in hypertension, multiple sclerosis, or lung cancer, only one data set was reproducible.

These studies, and the many others that report similar results, highlight a significant problem in the development of new therapies to treat disease. The identification of potential drug candidates typically happens in academic research labs. Pharmaceutical companies then use these new drug candidates as the basis for their drug development efforts. With increasing reports of discrepancies in preclinical publications, pharmaceutical companies are being forced to re-evaluate their reliance on academic research (see Bayer’s decision to halt nearly two-thirds of target-validation projects).

So why do so many preclinical publications contain research that can’t be reproduced?

Read the rest of this entry »

The future of universities

Paul Graham just published a new essay on Frighteningly Ambitious Startup Ideas in which he discusses a number of area that appear ripe for disruption by startups. One of the areas he mentioned was the university system:

Replace Universities. People are all over this idea lately, and I think they’re onto something. I’m reluctant to suggest that an institution that’s been around for a millennium is finished just because of some mistakes they made in the last few decades, but certainly in the last few decades US universities seem to have been headed down the wrong path. One could do a lot better for a lot less money.

The essay has sparked a lot of commentary and debate over on Hacker News, including the following comment by me (view on Hacker News):

Read the rest of this entry »