Biomedical Research Experts Reach Consensus on Open Science Practices to Monitor Institutions Worldwide

By Brittany Wade 

February 9, 2023 | A group of 80 research stakeholders—administrators, open science specialists, performance managers, and research scientists—convened to determine the best open science practices in biomedicine. The team’s efforts will set the groundwork to establish future open science policies and develop educational guidelines to improve data sharing in the biomedical space. 

Open science provides the community access to studies, data, published documents, and software, making research transparent and accessible. It exists to improve research quality, foster innovation and discovery, and encourage research sustainability. 

In November 2021, the United Nations Educational, Scientific, and Cultural Organization released a Recommendation on Open Science, urging the scientific community to develop policies and practices to establish a standardized open science regimen. In response, a global team hailing from 20 major biomedical research institutions—led by Dr. Kelly Cobey, Ph.D., MRes, HBSc, scientist and director of the Open Science and Meta Research Program at the University of Ottawa Heart Institute, Canada—set out to formulate a framework for their industry.  

To date, no other global framework exists in the biomedical space. “Having an agreed set of open science practices to monitor is an important milestone for the community. Through taking this community-centered approach, we hope to develop and make available a tool for biomedical institutions to monitor open science practices,” said Cobey in a press release.  

Published in PLoS Biology (DOI: 10.1371/journal.pbio.3001949), the study’s goal was to create a fully automated dashboard or digital tool that monitors each institution’s progress in adopting the team’s finalized set of core principles. “Ultimately, we need to track open science practices in order to ensure that we are taking timely steps to open up research and to ensure we comply with existing open science mandates,” said Cobey. 

To weigh each stakeholder’s perspective equally, the team conducted a Delphi Study, a well-established method to glean the anonymized opinion from multiple individuals and obtain a clear consensus on the topics at hand (Evidence-Based Nursing, DOI: 10.1136/ebnurs-2020-103303). The Delphi method consists of a series of rounds where each expert is asked a group of questions or provided with statements and instructed to express their thoughts.  

The responses from each round are then shared with the collective, where ideas and feedback govern the content of subsequent rounds. Initially, the stakeholders were presented with 17 open science principles or practices and told to form a consensus on whether they should be included within the final core list.  

The stakeholders participated in three rounds. In the first two rounds, entries were submitted through an online survey portal. Round three consisted of two half-day virtual meetings to discuss the principles that had yet to reach a consensus. As a result, some items were heavily modified, split into two or more, newly generated through the study, or removed altogether. Ultimately, the stakeholders agreed on 19 principles: 12 open science practices and seven transparency practices. 

The traditional open science practices include standardized trial registration reporting, adding identifying statements to track the proportion of openly available data, systemic review registration reporting, and marking whether shared data has received citations in other works to measure study impact. 

The broader transparency practices include adding author statements clarifying their role in a publication, standardized reporting of author conflicts of interest, and reporting funding statements to determine clear links between financial resources and research outputs. 

The team will use the core principles to pilot an open science dashboard at three institutions and provide individualized feedback through comparative meta-research. Then, with a better understanding of how the practices are adopted within each institution, the team will tailor the platform’s settings for optimal use and improved monitoring. 

“Policy in the absence of monitoring is not effective. We have reached agreement on how to design a digital dashboard to track open science practices to determine if we are doing a good job implementing them or not,” said Cobey. Moving forward, the team hopes their work will inform the future development and adoption of digital dashboards in various other biomedical institutions worldwide.