Shepherding preprints through a pandemic

Coping with volume

In March and April, as our teams in the northeastern US and the UK entered lockdown, we all worked long hours (from home), dealing with medRxiv submissions that increased 10-fold in three months. Our “day jobs” were also expanding—the Yale clinicians were seeing patients, while at BMJ and Cold Spring Harbor we learnt to produce journals with entirely remote teams. In April and May it became clear that, no matter how hard everyone worked, we had to scale up to deal with the volume of submissions: first, training more screeners who establish that a submission is within scope and meets reporting requirements; next, recruiting more affiliates—clinical researchers who confirm that a submission describes clinical research and would not be dangerous if posted; and, finally, securing help for the single clinical adviser who looked at every paper before posting.

Studies with human participants are posted on medRxiv only if they’ve had ethical oversight. We thought this a straightforward requirement, but we heard from many clinicians who were certain that they didn’t need oversight because they had “the answer” to treating the most seriously ill people, sometimes with treatments that were banned or reported to be dangerous. When we saw President Trump promoting unproven treatments and branding a (preprinted) study that countered his view an “enemy statement,”5 this upped the ante on how we handled treatment claims, and we ensured that all had clinician input.

Preprint v press release

Another area of debate was studies of small numbers of people. Early on we saw reports of loss of taste and smell in handfuls of patients in Italy: was this important or just an interesting curiosity? Ensuring that patients aren’t identifiable became a focus as we received increasing numbers of submissions from clinicians that read like a referral letter to a close colleague—describing patients’ home life, work, family relationships, dates of illness, and comorbidities. Some countries’ governments also disregarded patient confidentiality in the drive to control viral spread.6

And now, with a light appearing at the end of the pandemic tunnel7 and our workloads and workforces stabilised, discussions focus on claims about treatments and vaccines. Should we post studies of ill defined herbal remedies, for example, even within a registered clinical trial? Are there particular requirements for vaccine studies? Discussions continue.

One example of when we had no doubt about posting a preprint as quickly as possible was the Recovery trial’s preliminary result on dexamethasone.8 This was submitted a few days after the UK chief medical officer recommended the treatment and a few weeks before the peer reviewed article appeared in the New England Journal of Medicine.9 How many lives were saved by improved treatment in those weeks mid-year? And how much better was it that physicians could read the full preprint before prescribing, rather than relying on a press release?

Such examples drive us to carry on the work of preprint shepherding—despite occasional accusations of censorship and bias from authors whose preprints we decline to post, and despite the view of some eminent journal editors that, when we focus on speed, we devalue accuracy or reliability. Of course, sharing research openly and at pace has risks, the worst of which medRxiv aims to mitigate, but aren’t the risks of opacity and delay even greater? If posting preprints weren’t an option, how much of this research would remain behind closed doors today, and how much less informed would be the global discussions of policy and treatment options?

We launched medRxiv as a service to the clinical research community, and we look forward to hearing your ideas and contributions as its development continues.