“Reproducibility is actually the heart of science. The fact that not everything is reproducible is not a surprise.” – Eric Lander, head of the Broad Institute at MIT in a recent Washington Post article.
“We’re always in a gray area between perfect truth and complete falsehood,” The best researchers can do is edge closer to truth. – Giovanni Parmigiani, Dana-Farber Cancer Institute in a recent ScienceNews article
The Reproducibility Project, a collaboration between Science Exchange and the Center for Open Science, is independently replicating some of the most impactful studies in cancer biology. Along the way, not only will the collaboration shepherd 50 studies through the process of replication and meta-analysis, but it will also help to mature the discussion around reproducibility more generally. Where do the biggest challenges lie? What are some of the key predictors of whether experiments are reproducible? The answer to these questions will be critical as the reproducibility initiative gains traction.
Since December, experimental work has begun on four more replication studies, and three more Registered Reports have been published in eLife (with a fourth* accepted and on the way):
In total, eleven replications have begun or are poised to begin in the coming weeks. Read the rest of this entry »
We’re excited to announce that our introductory article and the first three of our Registered Reports have been published by our partner eLife.
In “An open investigation of the reproducibility of cancer biology research”, the Reproducibility Project: Cancer Biology core team details the impetus for and the specific goals of the project:
“The resulting open methodology and dataset will provide evidence about the reproducibility of high-impact results, and an opportunity to identify predictors of reproducibility” (1).
The first three Registered Reports are:
In addition, Sean Morrison, director of the Children’s Medical Institute at the University of Texas–Southwestern and a senior editor at eLife, has written an editorial introducing the Reproducibility Project: Cancer Biology, highlighting the role this project could play in beginning to reform scientific discovery methods to maximize reproducibility. He notes that:
“to be responsible stewards of the public’s investment in this work we have to maximize the pace of discovery and the efficiency with which discoveries get translated to the benefit of patients. By gauging the fraction of high-impact results that are not reproducible, we can consider what further steps should be taken to promote good science….[M]easuring the magnitude of the problem with efforts like the Reproducibility Project: Cancer Biology is an important step in the right direction” (2).
You can find all five articles on eLife’s website: http://elifesciences.org/collections/reproducibility-project-cancer-biology Read the rest of this entry »
Recently I spoke with Ries Robinson from our lab Medici Technologies. Everything about Medici Technologies is captivating, from the story behind their unique name to their interesting approach to data analysis. Check out more on their specialized approach below!
Q: What is Medici Technologies’ specialty?
Ries: We analyze data for groups or companies that have data that is so complex that it exceeds their resources. We are a consulting firm that provides expertise in data analysis.
Q: Why did you choose Medici as your name?
Ries: The Medici Effect is the idea that significant breakthroughs in innovation and technology often occur when you cross-pollinate fields. It stems from the Renaissance. For example, a Renaissance family would make the plumber work with the weaver, or someone with a different skill set, and that’s part of what initiated the Renaissance movement.
A lot of what we do is pull different ideas or algorithms from different places. Historically, we’ve worked on complex data analysis of optical signals for measuring chemicals or analytes in the body, but some of our greatest breakthroughs have been by taking algorithms from non-traditional sectors. For example, we can utilize song recognition and gesture recognition tools to classify tissue types. Utilizing methods developed in other applications has been extremely beneficial. Read the rest of this entry »
Today, we’re excited to introduce a new, centralized order management experience on Science Exchange, focused on improving collaboration and messaging.
We take feedback from our researchers and labs very seriously and have designed the new order management experience with your input in mind. The new design, which went live earlier today, moves messaging front and center and introduces a timeline of all events.
A collaborative workspace
The new workspace allows researchers and lab members to intuitively collaborate on orders. Any member of the lab can see all messages on an order, enabling multiple members of the same lab to coordinate easily and efficiently under this team view. Everyone who has access to this order is shown on the righthand side.
Communication is key
Since open communication between researcher and lab is of utmost importance, the new order page focuses heavily on messaging. Messages and files can be sent during any stage of an order, from the top of any page. Users can start sending messages as soon as a request is posted, and continue sending them even after an order is complete. As always, there is no limit on the size or quantity of uploaded files. All messages and files appear in the main Timeline tab, along with all other events.
Keep track of your order
It can be difficult to keep track of research projects, so we’ve made it easy for you. The main timeline shows everything that has happened in reverse chronological order. You can always reference the order status indicator in the upper right hand corner. All available actions, like shipping samples or accepting a quote, are shown below the indicator for easy access.
Easy access to the latest details
Because research projects are updated regularly, we’ve added the Order Details tab. Here, you can always see the latest version of the order, no matter what state it is in. You can also download quote summaries and billing documents from the Files tab.
We think you will find that the rest of your overall workflow is largely the same, just more beautiful and intuitive. As of this morning, all orders will reflect the changes. If you have any questions, please don’t hesitate to reach out to us at firstname.lastname@example.org or share feedback.
About the author
Becca joined as Science Exchange’s first Product Manager, excited to help bring efficiency to scientific research. She previously launched SurveyMonkey Enterprise and WePay Canada. She has a degree from Stanford in Mathematical & Computational Science and enjoys cooking and home improvement in her spare time.
We are proud to announce today that we have partnered with the Prostate Cancer Foundation (PCF), with funding from the Movember Foundation, to reproduce findings that have implications for prostate cancer patients. We will be collaborating with PCF to identify faster, high-impact biomedical findings that that can improve early detection and new cures.
PCF’s Chief Science Officer, Dr. Soule stated “This first-in-field foundation initiative is all about getting the smart stuff to patients quicker. We will see an acceleration of progress due to the mobilization of resources against the robust findings.”
Our Software Engineer Michael Kompanets with last year’s Movember mustache.
Science Exchange has a been long-time fan and supporter of PCF and the Movember Foundation (see picture to the right), so we are thrilled to be working with them to incorporate replication studies into their funding strategy. We will be utilizing the best practices that we’ve established for our Reproducibility Project: Cancer Biology to enable confirmation of high potential exploratory research results. Our hope is that by identifying robust reproducible results, we can accelerate prostate cancer research.
“The Movember Foundation is committed to accelerating the translation of promising discoveries into new tests and treatments,” said Paul Villanti, Executive Director of Programs, Movember Foundation. “Through quicker validation of the science, and if the science is true, we can help find new cures and prevent prostate cancer in more men at a faster rate. The Movember Foundation is confident that this initiative will play an important role in supporting this goal.” Read the rest of this entry »
There has been growing concern in the scientific community over the last several years about a lack of reproducible results in the biomedical research community. Recently, two large pharmaceutical companies (Amgen and Bayer) announced that they could only reproduce a small fraction of published preclinical cancer biology studies. These results have shocked the scientific community, and have lead to calls mandating an overhaul of both funding and publishing practices to address the crisis. The NIH, as well as the journals Nature and Science, are all proposing strategies to help improve the research process.
However, a major question remains: Why weren’t these experiments reproducible? Valid arguments exist suggesting scientists are falling prey to poor experimental design, flawed statistical analysis, and/or biased data interpretation, all of which can prevent their results from being replicable. However, there are many innocuous reasons why a particular experiment might fail to replicate the original results, from errors or changes in the protocol, to a lack of expertise in performing a particular technique, to unknown factors that produce variability in results. Unfortunately, it’s hard to draw conclusions from the Amgen and Bayer studies because these companies made none of their data or methods public.
The birth of the Reproducibility Project: Cancer Biology
We believe that in order to really understand the crisis in reproducibility, including its prevalence, scope and underlying causes, we need a large dataset of actual replication experiments. These replications must be conducted in a rigorously empirical fashion, using detailed protocols as close to the original study as possible, and conducted by expert scientists trained in the original techniques. Most importantly, the details of these replication datasets must be freely available to everyone.
These criteria led us to create the Reproducibility Project: Cancer Biology (RP:CB), a large-scale initiative to systematically replicate key findings from 50 highly impactful recent papers in the field of cancer biology. The project is a partnership between Science Exchange (and our network of expert service providers) and the Center for Open Science, and is funded through a grant from the Arnold Foundation, as well as through donations from many generous vendors. The goal of the project is to clarify the variety of challenges that exist for reproducibility, and encourage discussion of data-driven solutions from researchers themselves, as well as for policy makers at funding, publishing, and government institutions. To that end, all our findings will be published by the open-access journal eLife. Additionally, all of the methods, data, and results of the replication studies to be available for anyone to review on the Open Science Framework. Read the rest of this entry »
When I was in grad school, I was expected to track and know every new piece of research that related to Indian Monsoons. However, no one told me how to do that. I didn’t have the years of experience and I definitely didn’t have the time to sort through the endless new articles coming out on a regular basis. There is now a website called Sparrho that simplifies the overwhelming process of tracking scientific articles.
Sparrho compiles scientific sources to one place and brings the latest and most relevant scientific news (including papers, grants, and patents) to users. Most importantly, Sparrho learns. As you continue to use Sparrho, it will learn your preferences and needs so that you can spend more time reading relevant articles, rather than digging for them. Read the rest of this entry »
I recently spoke with our user Ethan Perlstein, whose one-of-a-kind independent lab is flipping traditional drug discovery on its head. Check out how he is changing the paradigm of traditional research, pharmacology, and more below.
Q: What is the focus of the Perlstein Lab?
Ethan: The Perlstein Lab is focused on personalized orphan drug discovery. We take a two-pronged approach. We first create a primordial disease model for a given patients’ mutation; that involves taking a change in the DNA that you see in the disease and putting it into the model organisms.
We use yeast, worms, flies, and fish that have ancestral versions of that gene. We can use those models to do drug discovery, and we can validate the hits that we get in patient derived cells of the same genotype. So it’s a closed system where everything is personalized from the outset.
Q: How did it come into existence? What was the progression from your very first crowdfunding experience to starting your own lab?
Ethan: The science behind it has been incubating a long time, since I was in grad school, so it’s been a ten-year process. Screening using a model organism is something I did in grad school, so it’s existed for awhile. As a post-doc, I took some of those scientific concepts and drilled down deeper, so that put me in a good position to have a scientific foundation.
I spent the next 18 months leaving academia and navigating the business side. Last fall, I put together a business plan, had it reviewed by business people, improved my plan, and by the end of 2014 I began fundraising.
The team started to come together in early April. The lab started to come together in terms of equipment and structure in mid-April. And now we have a fully functional lab that has yeast, worms, and flies, and it’s off to the races. Read the rest of this entry »
We are excited to announce that eLife has joined our partnership with the Center for Open Science to work on the Reproducibility Project: Cancer Biology!
eLife is an open access journal co-founded by the Howard Hughes Medical Institute, the Wellcome Trust and the Max Plank Institute. We are proud to have the work of the RP:CB published through them.
Each study in the RP:CB will undergo two rounds of review and publication. The first round will present the proposed replication plan to the public in the form of a Registered Report. This Registered Report will ensure that the proposed protocols have been reviewed by scientific and statistical experts prior to experimental work commencing. The completed work and all data will then be published as a Replication study. All data generated will be freely available to the public through eLife’s open access platform. Registered Reports are now under review by the eLife Board of reviewing editors and will be published in the eLife journal as available.
The Reproducibility Project: Cancer Biology aims to replicate key findings from 50 high profile papers from the field of cancer biology.
“We need an objective way to evaluate reproducibility,” said Randy Scheckman, who is the Editor-in-Chief of eLife and a Nobel prize winning cell biologist at the University of California- Berkeley. “This project is a valuable opportunity to generate a high-quality dataset to address questions about reproducibility constructively and rigorously.”
For more information, please see eLife (http://elifesciences.org/eLife-the-Center-for-Open-Science-and-Science-Exchange-partner-to-assess-the-reproducibility-of-cancer-biology-research) and the Reproducibility Project: Cancer Biology (http://validation.scienceexchange.com/#/cancer-biology). Read the rest of this entry »
Can We Defeat EBOLA with an Experimental CANCER Drug? from Experiment on Vimeo.
We are proud to share that Science Exchange user, OncoSynergy, is crowdfunding to test their experimental cancer drug, OS 2966, against ebola infection.
OS2966 is a monoclonal antibody that inhibits CD29 , a main cellular adhesion receptor that is key to cancer progression. Interestingly, CD29 is also thought to be hijacked by the ebola virus during infection. With the current ebola outbreak’s death toll exceeding 1,000 victims, it is more important than ever that promising drugs are investigated as soon as possible. As a result, OncoSynergy is using the Science Exchange network to test whether OS2966 can block ebola infection in cultured human cells.
“We have a unique opportunity to potentially effect a major impact on the current global ebola crisis,” said
Dr. W. Shawn Carbonell, MD, PhD, Founder and CEO of OncoSynergy. “However, as a seed stage
biotech startup with 6 employees, we don’t have the bandwidth to take on projects beyond our central
mission focused on cancer. We are teaming up with Science Exchange and Experiment to accomplish
the initial experiments which are an important first step towards possible clinical testing of OS2966. We
now need the public’s help to fund the work so we can start as soon as possible.”
“I am pleased to be working with Dr. Carbonell on this timely project,” said our CEO, Dr. Elizabeth Iorns. “This is a great example of how the Science Exchange platform can help facilitate science and accelerate scientific discoveries.”
If you’re interested in helping this research progress, please donate to OncoSynergy’s campaign here: https://experiment.com/projects/can-we-defeat-ebola-with-an-experimental-cancer-drug