Emergency use authorisation, pharmaceutical companies announcing the results of clinical trials first through press releases, free access to all COVID-19 scientific papers published in journals and a sharp increase in the number of preprints posted became more common during the pandemic. Except free access to all COVID-19 papers in journals, which otherwise would have been behind paywalls, all the others came under severe criticism at some point.
As per a comment in The Lancet, an average of 39.5 COVID-19 preprints were posted each day during the pandemic compared with just 10.5 per day during the Ebola virus epidemic of 2014. Also, the first preprint was posted just 22 days after health authority notification of the initial cluster of cases in Wuhan, China. In contrast, no preprints were posted for about six months after first cluster notifications in the case of Ebola virus and Zika virus.
Early in the pandemic, bioRxiv added a banner on its home paper and on top of every preprint dealing with COVID-19. The banner read, “bioRxiv posts many COVID-related papers. A reminder: they have not been formally peer-reviewed and should not guide health-related behaviour or be reported in the press as conclusive”. The decision to publish the disclaimer was prompted by a highly questionable study by Indian researchers posted on bioRxiv preprint server, which was soon withdrawn.
Scientists submit their early drafts to the preprint server, where an in-house team weeds out “obvious spam and clear garbage,” as well as submissions that are not scientific in nature, Richard Sever, co-founder of bioRxiv preprint server, was quoted as saying following the Indian researchers’ study.
‘Readers cautioned’
Because preprints are yet to be peer-reviewed, most newspapers while detailing the results posted in preprint servers did caution their readers that the study in question is yet to be peer-reviewed. The notion is that since preprints are yet to be peer-reviewed by domain experts prior to posting, readers should exercise caution and not take the study results on face value. In contrast, results and findings of all papers published in journal papers after the scientific vetting process are deemed to be reliable and correct.
A small study of retracted preprints and journal papers on and about COVID-19 has turned these notions on its head. A team led by Dr. Jitendra Kumar Meena from AIIMS, Delhi found that there were more papers published in journals post peer-review that were retracted than preprints — 143 retracted peer-reviewed journal papers compared with just 40 retracted preprints.
The results of this study were posted as a preprint in medRxiv server; preprints are yet to be peer-reviewed. The researchers looked at the following preprint servers that published COVID-19 studies — medrRxiv, bioRxiv, arXiv, Social Science Research Network (SSRN), OSF Preprints, and Research Square.
Reasons for retraction
The retracted COVID-19 preprints and journal papers published between January 1, 2020 and March 10, 2022 were identified based on alerts posted by the Retraction Watch database. According to the authors, concern about data and results was the most common reason for retraction of preprints, while fake peer review and duplication of articles were the reasons for retraction of peer-reviewed articles. As per the authors, six journals accounted for nearly 35% of all the retractions among peer-reviewed articles. In the case of preprints, most of the retracted studies were posted on medRxiv (23 studies, 57.5%) followed by the SSRN (eight articles, 20%) and bioRxiv (seven preprints, 17.5%).
That peer-reviewing process has broken down is best exemplified by innumerable manipulated images that accompany many papers. The PubPeer website, a post-publication peer-reviewing by independent scientists, provides daily proof of how broken the journal peer-reviewing process is.
While both preprints and journal papers were freely accessible from the day the results were posted/published, it took far fewer days for a preprint to be retracted compared with journal papers. As per the study, the median retraction time for preprint was 29 days, while it was 139 days for peer-reviewed articles. Faster retraction by preprint servers meant that readers became aware that the results of a particular preprint were not to be trusted and hence, saved time. Unlike the traditional journal papers, any reader is free to critique the study immediately after the manuscript is posted on the preprint server. The comment section, following the manuscript, thus served to warn other scientists, and particularly journalists, about the problems with the study. The authors thus conclude,“The increased adoption of pre-prints results in faster identification of erroneous articles compared to the traditional peer-review process.”
Faulty studies exposed
Preprints, particularly during the pandemic, facilitated rapid dissemination of results. But by virtue of not being peer-reviewed, they had potential of disseminating faulty research findings to a large audience. But the quick, open and transparent peer-reviewing by hundreds, if not thousands, of scientists from across the globe meant that faulty preprints studies were exposed within a very short time. For instance, a virology “study” posted on bioRxiv preprint server by IIT Delhi and University of Delhi researchers erroneously claimed that SARS-CoV-2 virus contained HIV “insertions”. The “study” was widely criticised on the preprint server, Twitter, Facebook and blogs. The bioRxiv preprint server alone carried 135 comments by scientists critiquing the study. The preprint was quickly withdrawn.
In early March 2020, The New England Journal of Medicine published a correspondence that claimed that a Chinese lady had spread the virus to her German hosts during the incubation period even while not exhibiting any symptoms. While asymptomatic people can spread the virus became clear later during the pandemic, the authors had wrongly concluded the asymptomatic spread of the virus without verifying if the Chinese lady had symptoms during her visit to Germany. The journal provided a supplementary appendix with details of symptoms and timeline that clearly showed that she had symptoms while in Germany and thus it was not a case of asymptomatic spread of the virus to her German hosts.
Also, a paper published in a journal by a Chinese team based on codons favoured by SARS-CoV-2 virus and other potential intermediate hosts found that two snakes — Bungarus multicinctus (the many-banded krait) and Naja atra (the Chinese cobra) — sold at the Wuhan seafood and animal market were likely to be wildlife animal reservouirs for SARS-CoV-2 virus. The paper was published on January 22, 2020 in the Journal of Medical Virology. However, it was a non-peer-reviewed, early access article published in the journal.
‘Wider, faster reach’
“The present study shows that there is another benefit to encouraging people to put up their work on preprint servers in the form of earlier recognition of errors and open review leading to prompt retractions. As we go forward, the scientific community should promote preprints as they provide wider and faster reach, more attention, and rapid peer feedback,” the authors conclude.
