Long before COVID-19, academics squabbled about the methodology, clinical sample sizes, and data manipulation of published scientific papers—especially if they did not agree with conclusions or were educational rivals.
Problems With Peer Review
What is peer review? Independent reviewers who are knowledgeable in a related field, sometimes called referees, are chosen by a journal’s editor, to assess a submitted manuscript for originality, validity, and significance.These reviewers are typically employed at universities and institutions that task professors and researchers with doing peer reviews as part of their responsibilities. In other words, reviewers are not typically paid by the journal. Being a reviewer is a boost to a researcher or expert’s professional credibility—and by extension that of their employer.
Peer review is the “gold standard” for scientific papers and its absence is a key feature of arguments against preprint publications. Yet the potential flaws in peer review are also well recognized. According to the Lancet, one of the top medical journals, the peer review system is costly, time-consuming, and favors reviewers from “high-income countries” rather than low-income countries because the latter may not be able to afford to do the uncompensated review work.
More significant than that issue is likely the influence of status-quo reviewers who have careers built on specific theories or paradigms. Evidence that challenges or overturns established ideas can directly undermine a reviewer’s professional credibility.
Peer Review Errors Can Happen
The Cambridge piece included another caveat—“peer review also frequently misses major errors in submitted papers.” Such errors go back at least to the revered JAMA when, in 2006, an article defending antidepressants during pregnancy was found to have ties to antidepressant manufacturers and authors of an article linking migraines with auras to cardiovascular disease was found to be financially linked to migraine and heart medication manufacturers.A New Focus on Preprints
Preprints allow authors to reach an audience more quickly, without peer review, and thus can be more “democratic.” However, during COVID-19, those very attributes have sometimes made preprints the “bad guys.”“[M]anuscripts are examined by volunteer academics or subject specialists who scan for non-scientific content and health or biosecurity risks,” says the article. Richard Sever, a cofounder of both servers told Nature that the vetting process first seeks out articles that “might cause harm” rather than evaluating quality.
The vetting includes “flagging papers that might contradict widely accepted public-health advice or inappropriately use causal language in reporting on a medical treatment,” says the Nature article, and rejecting “papers that might fuel conspiracy theories.”
Of course, sometimes radically divergent science can be mislabelled because it is particularly groundbreaking. Many will remember that the “germ theory,” proven by French chemist Louis Pasteur in the 1860s, also contradicted widely accepted public-health advice.
Is the Public Served By Such Tightened Publication Criteria?
The lack of experts reviewing preprint papers can raise questions about the quality of their papers, but those attacking this publication process may incidentally support scientific elitism and even censorship.Albert-László Barabási, a computational scientist at Northeastern University in Boston, Massachusetts, had another observation when his paper was rejected by the preprint site bioRxiv because the site was no longer accepting manuscripts forecasting COVID-19 predictions about treatments on the basis of computational work.
Barabási said he understood the concerns about protecting patient safety but added, “It’s precisely the coronavirus that creates an environment where you need to share,” such breaking information, he said. The purpose of a preprint server, “is that we decide what is interesting, not the referees,” he added.
Many readers would agree with Barabási, especially during a worldwide viral pandemic. A diversity of medical viewpoints should be desirable especially when a closed medical establishment seems heavily tilted toward vested interests.
It is also true that junk science can further confuse public opinion. But that point seems less pertinent when public distrust appears heightened, rather than reduced, by efforts to enforce scientific hegemony.