Reproducibility Initiative to Increase the Value of Biomedical Research

By Bio-IT World Staff 

August 17, 2012 | Science Exchange, the open-access publisher Public Library of Science (PLOS), and open data repository figshare, have announced the launch of the Reproducibility Initiative – a new program to help scientists, institutions and funding agencies validate their critical research findings.

Science Exchange CEO and co-founder, Elizabeth Iorns, said: “In the last year, problems in reproducing academic research have drawn a lot of public attention, particularly in the context of translating research into medical advances. Recent studies indicate that up to 70% of research from academic labs cannot be reproduced, representing an enormous waste of money and effort.”

Iorns syas that in her research experience, the problem lays primarily in the lack of incentives and opportunities for validation. “The Reproducibility Initiative directly tackles these missing pieces,” she said.

The Reproducibility Initiative provides both a mechanism for scientists to independently replicate their findings and a reward for doing so. Scientists who apply to have their studies replicated will be matched with experimental service providers based on the necessary expertise.

The Initiative leverages Science Exchange’s existing marketplace for scientific services, which contains a network of more than 1,000 expert providers at core facilities and contract research organizations (CROs).

“Core facilities and commercial scientific service providers are the solution to this problem,” said Iorns. “They are experts at specific experimental techniques, and operate outside the current academic incentive structure.”

Scientists will receive the results of their validation studies and have the opportunity to publish them in the open-access journal PLOS ONE as part of a special collection highlighting the importance of reproducibility in scientific research. They can also upload their primary data to the open-access repository, figshare. Replications published in PLOS ONE will link back to the original publications upon which they are based. The Nature Publishing Group and Rockefeller University Press are among the publishers that have expressed their support for this acknowledgement of reproducibility.

The Reproducibility Initiative will potentially provide a mechanism for industry to identify robust drug targets for developing effective new therapies. “Improving the robustness of published data on early targets would have a significant impact on the efficiency of the drug development process,” said Christopher Haskell, head of the US Science Hub at Bayer HealthCare. “The Reproducibility Initiative seeks to address this challenge.”

“It is critically important to independently validate preclinical data before moving to the clinic,” said Lee Ellis (MD Anderson), who co-authored a widely read article in Nature earlier this year on the need to improve the reliability of preclinical cancer studies, and is an advisor to the Reproducibility Initiative.

Stanford University’s John Ioannidis, who published a study in PLOS Medicine titled “Why Most Published Research Findings Are False,” also offered his support for the initiative, calling it “a very important pilot effort, offering valuable insights on how reproducibility checks can work in real life.”

The Reproducibility Initiative is initially accepting 40-50 studies for validation. Studies will be selected on the basis of potential clinical impact and the scope of the experiments required and, in aggregate, may eventually serve as a proof-of-concept for this mechanism of validation to funding agencies and patient groups.

Josh Sommer, Executive Director of the Chordoma Foundation, agreed with the potential promise of the Initiative, noting: “As a patient group ourselves, we can't afford to fund anything but research that is reproducible and robust enough to be developed into new therapies.”

Reproducible science incentives could build a foundation for robust translational research and improved therapies. "As awareness of irreproducibility grows, we wanted to provide a way for top quality researchers to distinguish themselves. This is truly a great opportunity for scientists with potentially groundbreaking results to garner direct validation of their work, and to lead the charge for reproducible science," said Iorns.

Scientists can submit their study for validation at: www.reproducibilityinitiative.org 

Core facilities or CROs can join Science Exchange at: www.scienceexchange.com/provider