New Issue Brief: Federal Health Journals and the Reproducibility Crisis

New York, NY – Throughout the pandemic, the U.S. Department of Health and Human Services (HHS) played a role as central as it was controversial. Its scientific journals helped shape policies guiding American life for the better part of two years, but lack of clarity about their data undermined public trust in these policies. In a new Manhattan Institute issue brief, senior fellow Randall Lutter highlights the challenge facing federal health journals and offers ways to bolster their scientific and public legitimacy.

Reproducibility is a cornerstone of scientific research, and Lutter demonstrates how lack of transparency about research data and computer code hinders independent verification of research results. Elite private journals have all taken steps to increase transparency, but federal health journals are woefully behind, making it difficult – if not impossible – for other experts to check the work shaping public health policy. Research journals, including the HHS’s, are also subject to publication bias and cherry-picking of results, posing further challenges to scientific robustness and public trust. Lutter’s issue brief proposes solutions for the HHS, including:  

• Directing each of its journals to adopt public-access policies for data, materials, and code no weaker than those of journals like Proceedings of the National Academy of Sciences or Science.

• Sponsor/encourage studies of the independent reproducibility of its journals’ published research.

• Encourage preregistration of research plans – for studies of animals and human subjects alike – as well as the registration of randomized trials generally.

The problems besetting the HHS’s journals are not new, but the speed of reform has been glacial. Should this continue, Lutter suggests that Congressional action or privatization of the journals may be necessary for change.

Read the full issue brief here.