Press "Enter" to skip to content

Publishing Scientific Studies During a Pandemic: Can We Trust the Process?

Ever since COVID-19’s emergence as a potential threat to global health, there has been unrelenting pressure on researchers to discover the best ways to treat it and prevent its spread – and to do it very quickly. Back in March, the urgent scientific questions ran the gamut: How many different ways can the virus be transmitted? What preventive measures are most effective in reducing the risk of contagion? Which of the drugs in our pharmaceutical arsenal might be effective in combatting the disease? How long will it take to develop, test, manufacture, and distribute an effective vaccine?

Following a similar trajectory, a wide range of entities have sought immediate answers to those questions – healthcare providers treating patients with the disease, public health organizations, government agencies, and an understandably anxious public.

The urgency to implement research and rapidly report findings has led to an unprecedented explosion of communications. As of July 6, 2020, a PubMed search yielded nearly 30,000 COVID-related reports. These were closely followed by a staggering number of public-facing communications. Unfortunately, journalists, talk shows, government entities, and press releases from industry and academia often fail to communicate results well.

In their recent article for JAMA, authors Richard Saitz, MD, MPH, and Gary Schwitzer describe communication flaws and failures that have occurred during the pandemic and their consequences. They observed three main causes of faulty communications:

  • Focusing on a single study without putting in context of other studies, or even acknowledging that a solitary study is rarely definitive.
  • Overemphasis on results, especially emphasizing observed effects without recognition of limitations.
  • Basing communications on incomplete reports of studies that have not been adequately reviewed.

A classic example of communication flaws is the now-familiar saga of hydroxychloroquine. It all began with the encouraging results reported from a small trial (20 study patients/16 control patients). Media reports suggested that the drug cut death rates significantly and helped improve patient survival. In quick succession: the U.S. president announced he was taking the drug and promoted its use; the FDA approved an Emergency Use Authorization; a dramatic spike in the drug’s use rapidly led to shortages; and the U.S. stockpiled 63 million doses. Subsequent studies in the weeks and months that followed conclusively demonstrated the drug’s lack of efficacy.

The public is eager for, and entitled to, scientific explanations and information regarding solutions to health issues. What is the role of industry public relations in clearly communicating scientific information without bias? Are journalists held accountable for the clarity and accuracy of what they write? Can information shared via news conferences and talk shows be trusted?

Perhaps the most important question is one that can only be answered by scientific publishers. How do journals maintain their stringent standards during a pandemic given the rapid pace and enormous volume of research being conducted in a compressed time period? The response from one of the U.S.’s premiere medical journals is exemplary. From January 1 to June 1, 2020, JAMA published 11,000 manuscripts, compared to a more typical 4,000 during the same period. Howard Bauchner, MD, and colleagues recently published an article on the journal’s approach to maintaining editorial and peer review standards despite the deluge.

Evaluation and opinion from peer reviewers with subject matter knowledge and methodological and statistical expertise is essential in assessing the scientific rigor of studies and the plausibility of results. That being the case, a typical peer review process is deliberately thorough and very time-consuming. Rushing leads to mistakes that inevitably undermine public trust. Rapid publication is possible only if authors, scientific editors, and manuscript editors are available to review and revise manuscripts.

The sheer volume of manuscripts submitted to JAMA during the pandemic necessitated changes in their usual editorial process; chiefly, evaluation by external peer reviewers is limited to manuscripts in which results are likely to influence clinical practice or public health policy (or are likely to be of interest or possible concern to the public). A more expeditious review by internal peer reviewers is conducted for other manuscripts. In all cases, the publication is delayed if additional analyses are considered to be important.

Good communication is always vital, but especially so during a national pandemic. Ultimately, those of us who serve as editors of scientific journals are responsible for protecting the integrity of peer review and ensuring total transparency regarding what is published. As editors, we understand this. Our answer to the question, “Can we trust the process?” should always be a resounding “Yes!”

David Nash, MD, MBA, is founding dean emeritus and the Dr. Raymond C. and Doris N. Grandon Professor of Health Policy at the Jefferson College of Population Health. He serves as special assistant to Bruce Meyer, MD, MBA, president of Jefferson Health. He is also editor-in-chief of the American Journal of Medical Quality and of Population Health Management.

Last Updated October 30, 2020