This week we are featuring a guest post on how peer review can improve reproducibility. Check out Aimee Whitcroft from Publons’ thoughts below.
There has been much talk over the last few years about the fact that most research, particularly in the medical fields, may not be reproducible – a stunning waste of time and resources.
At Publons, we’ve been following the crisis closely, and we at think improved peer review is a vital first step towards reproducibility in academic and scientific literature.
As a piece in PLOS recently put it:
“The importance of replication and reproducibility has recently been exemplified through studies showing that scientific papers commonly leave out experimental details essential for reproduction, studies showing difficulties with replicating published experimental results, an increase in retracted papers, and through a high number of failing clinical trials.”
There are several ways we can improve reproducibility through peer review.
Firstly, making the possibility of reproduction a necessary precondition for the publishing of a paper. By this, we mean ensuring a manuscript gives full details of methodology and results – give pre-publication peer review the ability to ensure only reproducible papers are published.
Post-publication peer review should identify and mark published papers whose work is not reproducible. Indeed, we think Publons is the perfect forum for this, because it allows papers to remain ‘alive’ after publication and gather valuable context.
Further, reviewers should mention whether or not there were sufficient details published to allow other researchers to attempt replication. Reviewers wanting to contribute even more could attempt to reproduce the studies themselves or organize their validation.
Reviewers could also identify those studies most in need of reproduction – studies which could be the foundations for major change in a discipline.
From there, initiatives such as the Reproducibility Initiative could organize to have those studies verified. The positive effects are huge, and include:
1) Preventing shoddy papers from being perpetuated and engrained in scientific research.
2) Identifying papers of high significance as targets for reproduction.
3) Saving huge amounts of time and money on poor methodologies.
Of course, our platform allows reviewers – and it’s our mission to make all reviewers look awesome and, in the process, to speed up and improve science – to generate both pre- and post-publication reviews, and we’d love to encourage all our reviewers to comment on the reproducibility of papers they’re reviewing.
We’re even considering making it a necessary feature of any review – we’d welcome your thoughts and feedback in the comments below and, of course, you can always come and check out our site at publons.com!