Objectives In patient-facing healthcare workers delivering secondary care, what is the evidence behind UK Government personal protective equipment (PPE) guidance on surgical masks versus respirators for SARS-CoV-2 protection?
Design Two independent reviewers performed a rapid review. Appraisal was performed using Critical Appraisal Skills Programme checklists and Grading of Recommendations, Assessment, Development and Evaluations methodology. Results were synthesised by comparison of findings and appraisals.
Data sources MEDLINE, Google Scholar, UK Government COVID-19 website and grey literature.
Eligibility criteria Studies published on any date containing primary data comparing surgical facemasks and respirators specific to SARS-CoV-2, and studies underpinning UK Government PPE guidance, were included.
Results Of 30 identified, only 3 laboratory studies of 14 different respirators and 12 surgical facemasks were found. In all three, respirators were significantly more effective than facemasks when comparing protection factors, reduction factors, filter penetrations, total inspiratory leakages at differing particle sizes, mean inspiratory flows and breathing rates. Tests included live viruses and inert particles on dummies and humans. In the six clinical studies (6502 participants) included the only statistically significant result found continuous use of respirators more effective in clinical respiratory illness compared with targeted use or surgical facemasks. There was no consistent definition of ‘exposure’ to determine the efficacy of respiratory protective equipment (RPE). It is difficult to define ‘safe’.
Conclusions There is a paucity of evidence on the comparison of facemasks and respirators specific to SARS-CoV-2, and poor-quality evidence in other contexts. The use of surrogates results in extrapolation of non-SARS-CoV-2 specific data to guide UK Government PPE guidance. The appropriateness of this is unknown given the uncertainty over the transmission of SARS-CoV-2.
This means that the evidence base for UK Government PPE guidelines is not based on SARS-CoV-2 and requires generalisation from low-quality evidence of other pathogens/particles. There is a paucity of high-quality evidence regarding the efficacy of RPE specific to SARS-CoV-2. UK Government PPE guidelines are underpinned by the assumption of droplet transmission of SARS-CoV-2.
These factors suggest that the triaging of filtering face piece class 3 respirators might increase the risk of COVID-19 faced by some.
- infection control
- occupational & industrial medicine
- public health
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
The results of this study will allow for future study with a real and tangible effect towards the well-being of healthcare workers nationwide, and perhaps internationally.
This article has an exceptionally broad range—from infection control, to public health, to biomechanical engineering, to industry; the hope is to increase multidisciplinary discussion.
This study reviews evidence specific to a novel virus and inevitably there is a paucity of specific evidence.
Eight hundred seven healthcare workers have died of COVID-19 worldwide as of 30 April 2020.1 One hundred six of these tragedies have occurred in the UK.2 On 11 April 2020, WHO COVID-19 SitRep3 was focused solely on the need for robust reporting of SARS-CoV-2 in healthcare workers (HCWs) in order to better guide infection prevention and control measures.
To have confidence in the indications for use of respiratory protective equipment (RPE), the fluid repellent surgical mask (FRSM) and the filtering face piece class 3 (FFP3) respirator, UK HCWs must have confidence in the evidence base behind UK Government (HMG) PPE guidance (see online supplemental appendix 1 for HMG PPE guidance).4 5
It is widely accepted that filtering face piece respirators (that meet UK/EU standards of FFP2/3 and US standards of N95/100) (see online supplemental appendix 2 for a comparison of the various international standards of testing of respirators and surgical facemasks) are more effective in the protection of the wearer from aerosolised pathogens than FRSMs, which are not designed to protect the wearer from aerosols.6 This is reflected in global RPE guidelines,7–11 which demonstrate the triaging of respirators to those more likely to encounter aerosolised SARS-CoV-2, and the recommendation of FRSMs to those deemed less likely.
The need for triaging of RPE includes several considerations other than the protective ability of these respirators. These include the shortage of global stock and supply,7–11 the need to ensure that low-income to middle-income countries are also able to access RPE,12 and the relative risk of SARS-CoV-2 exposure by the current understanding of the transmission of virus.
The latter consideration causes concern. HMG PPE guidance4 on the indications for use of FFP3 respirator relies on two assumptions. First, that its list of aerosol generating procedures (AGPs)5 and high-risk areas are exhaustive. Second, that the droplet theory of SARS-CoV-2 transmission13 14 is correct. If either of these two postulates are incorrect and the role of aerosolisation transmission in SARS-CoV-2 is greater than currently thought, the current triaging system of respirators may result in HMG PPE guidance indicating a less effective form of RPE in a higher-than-expected risk setting.
This rapid review aims to determine the evidence base to the protective ability of respirators versus FRSMs to aerosolised SARS-CoV-2.
This is a rapid systematic review of heterogenous studies with no summary estimate due to vastly different study protocols.
Following the widely used PICO structure,15 the research question was framed as:
In patient-facing healthcare workers delivering secondary care, what is the evidence behind UK Government PPE Guidance on surgical masks versus respirators for SARS-CoV-2 protection?
Full PICO strategy and search strands are available in online supplemental appendix 3.
Preliminary search for similar reviews
Two similar systematic reviews were found.16 17 The focus of the UK-based review by Greenhalgh et al 16 was on the efficacy of FRSMs and respirators in primary care; while in 2016, Smith et al 17 did not focus on SARS-CoV-2 prevention, rather respiratory disease in general in Canada. These reviews employed a similar methodology but varied in their population16 and their outcomes.17 Greenhalgh et al found no difference between FRSMs and FFP3 respirators in primary care. Smith et al found insufficient data to draw conclusions of the protective ability of N95 respirators compared with FRSMs.
Following keywords were searched: ‘respirator’, ‘surgical mask’, ‘mask’, ‘FFP’, ‘FFP3’, ‘PPE’, ‘personal protective equipment’ AND ‘viral’, ‘infection’, ‘respiratory’, ‘covid’, ‘COVID-19’, ‘coronavirus’, ‘SARS-CoV-2’.
Full PICO strategy and search strands are available in online supplemental appendix 3.
Authors PR and JS conducted the database search and eligibility check independently.
Grey literature search—by searching for references behind the RPE guidelines of the UK, the USA and EU/EEA(bodies outside of the UK were searched since it was felt that these populations have similar demographics and pandemic response measures).
Snowball search—by reviewing the references of included and excluded articles, and the references of these references, for eligibility and appraisal.
Table 1 shows the eligibility criteria for search results.
The authors independently used the relevant Critical Appraisal Skills Programme (CASP) checklists.18 All studies were included for qualitative analysis since it is noted that during a time of global crisis, the need for rapid evidence based on a novel virus may reduce the viability of gold-standard randomised controlled trials and shorten timelines for follow-up. The need to appraise studies thoroughly for ‘bad science’ is vital during such a time, and therefore comments arising from critical appraisal of all articles included are attached to their results to allow for informed decision making.
Disagreement resulted in full-text review for eligibility and, if accepted, individual appraisals conducted independently. A third author was tasked to review for eligibility had there been any further disagreements on full-text review.
The Grading of Recommendations, Assessment, Development and Evaluations (GRADE) tool was used to systematically and reproducibly assess the quality of the included studies. Authors PR and JS undertook quality assessment with GRADE independently. On completion, the authors allocated the mean of their independent individual GRADE subscores to each study.
Data from the nine included articles were extracted independently onto independent electronic spreadsheets.
Databases were re-searched in the timeframe 11–30 April 2020 to identify new literature. An additional similar systematic review was discovered.20 This did not contain primary data so was not included, however is discussed as a similar study. During the peer-review process of this manuscript, another similar review21 was published, the results of which are discussed in this review.
Due to the heterogeneity of study designs and the parameters of results, data extracted from accepted articles were compared directly.
For laboratory studies, these data included study design, research question, masks/respirators tests, testing particle/pathogen, findings and appraisal comments.
For clinical studies, these data included setting, participants, interventions, outcomes and limitations raised in appraisal.
Patient and public involvement
This study did not include patient or public involvement due to the rapid nature of the review.
Review of laboratory studies comparing respirators with FRSMs
Three key laboratory studies were identified that met the inclusion criteria. All studies used a methodology that evaluated the effectiveness of masks by measuring the concentration of particles (either NaCl aerosol or live Influenza) between the mask/wearer and externally, hence determining a property known as protection factor (PF) (PF is a ratio of the test particle/pathogen per unit volume on the outside of the test mask/respirator compared with that on the inside, over a standardised time frame with standardised temperature, humidity and windspeed).21
Lee et al compared the PF of FRSMs with FFP2 and FFP3 respirators in filtering NaCl aerosol when donned on human subjects.22 It was found that respirators offered an average PF of 24.5 vs 1.7 for FRSMs. Despite this relative improvement, in several cases the measured PFs for the masks did not meet the standards they were approved to. Overall, the PF afforded by the mask was affected by the quality of fit test. A key limitation of the study was that NaCl surrogate particles used had a wide range of particle sizes, and therefore the ability of the masks to filter in the specific particle size range of those in the size range of respiratory viruses (including SARS-CoV-2) was not measured.
In order to more accurately reproduce the particle size of viral and bacterial aerosols, the Health and Safety Laboratory used live influenza and NaCl aerosol with controlled particle size.6 The study reported a similar PF to Lee et al 22 of 2 for FRSMs, and a similar increase in effectiveness for respirators (17 times increase). Live, infectious particles were detected behind all FRSMs after testing. This indicates that the influenza sized particles could either penetrate or circumnavigate the mask and remain infectious to the user. However, a key limitation of this study was that the same effect was not reported for respirators due to testing issues—preventing comparison between the two types of mask.
He et al 23 identified that the PF difference between FRSMs and respirators also varies according to breathing rate, or mean inspiratory flow (MIF). At the lowest MIF, FRSMs were 108 times more effective, while just 13.4 at the highest MIF. Akin to the methodology by Lee et al, 22 NaCl aerosol was used as a substitute for virus. During the study, the same respirator was used up to 20 times. Respirators are known to clog with increased use.22 As this effect was not quantified or otherwise accounted for, its effects on overall filter performance cannot be assessed.
Review of clinical trials comparing respirator with FRSMs
Six clinical studies were identified that met the inclusion criteria. The number of subjects varied between 41 and 2862. In contrast to laboratory studies, where PF was used as a metric of effectiveness, the identified clinical studies used symptomatic viral infection of HCWs in healthcare environments as a measure of mask effectiveness.
Radonovich et al 24 and Loeb et al 25 both found no significant differences in infection rates between HCWs wearing N95 (FFP2 equivalent) respirators and FRSMs. Both studies used self-reported data to track respiratory symptoms. If detected, the HCW would provide a laboratory sample to confirm a positive case of influenza. During both studies, the use of infection control and PPE use in the respective institutions was observed and evaluated. A limitation of the study by Radonovich et al 24 was that only symptomatic participants were PCR-tested. Therefore, asymptomatic but infected individuals were not included. Additionally, the results were not stratified for influenza vaccination. A limitation of the study by Loeb et al 25 was that data on other infection control measures (hand hygiene, or use of gloves and gowns) were not collected, nor whether participants had received influenza vaccination.
In contrast, MacIntyre et al reported a statistically significant reduction in self-reported respiratory symptoms between groups of HCWs employing either targeted FRSM or N95 use (11.8% vs 17%).26 Further reduction in infection rate was measured if a respirator was worn continuously and not just during AGPs. However, when symptomatic cases were laboratory tested, no significant differences in confirmed cases between the two groups were found. A later study from the same group27 did find a reduction of laboratory confirmed cases in N95 vs FRSM, but no difference between those who wore FRSMs and those who did not wear masks. A limitation of this study was that only symptomatic participants were tested.
Ng et al retrospectively assessed the symptoms of 41 HCWs (divided into FRSM and respirator cohorts) after exposure to a single patient with SARS-CoV-2, lab testing all workers every 1–5 days.28 During the study timeframe, 85% of workers were exposed to an AGP from the patient. During the study, none of the HCWs tested positive for the virus, and therefore no difference between the mask wearing groups was detected. Loeb et al conducted a retrospective study based on interviews with HCWs exposed to SARS with a similar sized cohort.29 In contrast to the study by Ng et al,28 this study showed an almost 80% reduction in risk of infection for nurses who consistently wore masks (either FRSM or N95). On comparison of use of N95 respirators wih FRSMs, the relative SARS risk associated with the N95 mask was half that for the surgical mask. However, due to the small sample size, the result was not statistically significant. A limitation of this study was its small sample size (n=43) likely underpowering its results. The retrospective methodology used self-reported retrospective data, and is therefore subject to recall bias.
There were common methodological issues to all of the clinical studies that may affect the reported results. First, no study accurately accounts for vaccination history—either a percentage of the cohort are assumed to be vaccinated, or simply omitted as a limitation. Second, the activities of each HCW outside of the working environment (eg, unprotected interaction with other individuals at risk of infection) were not tracked or considered.
A summary of the results of this review is mentioned in table 2. Table 3 tabulates the GRADE quality assessment conducted on included studies in this review. More details of each of the evaluated studies can be found in online supplemental appendix 4.
Statement of principal findings
All three laboratory studies suggest respirators are significantly more effective than facemasks in PFs. However, an important limitation is the need for generalisation from the laboratory setting to the clinical setting. It was difficult to compare clinical studies as different methodologies and parameters were used to define the protective ability of each form of RPE. Studies were often underpowered and might suffer from confounding variables.
Both the quality and quantity of evidence regarding the use of RPE against SARS-CoV-2 is low. This is understandable given the novel nature of the COVID-19 pandemic. It was found that there is a paucity of evidence specific to SARS-CoV-2, with the current evidence being very low quality. On review of the references within HMG PPE guidance,4 indirectness and imprecision were detected in included studies. Indirectness was due to the need to use low-quality evidence from other pathogens as a surrogate due to the paucity of SARS-CoV-2-specific evidence. Imprecision was usually due to either small population sizes or the inability to stratify for confounders.
Strengths and weaknesses of the study
The CASP checklist for the appropriate study designs was used to critically appraise studies while GRADE methodology was used to assess the quality of evidence in included studies. It was noted that there is a paucity of evidence regarding RPE specific to SARS-CoV-2. HMG’s PPE guidance4 was found to reference non-SARS-CoV-2 and non-FFP3-specific studies, therefore these were included for review. Any study design containing primary data was included. Non-English language studies were not included, although translated studies were screened. Other databases such as the Cochrane Library, could have been searched, although unpublished literature and public health reports were discovered through snowball searching. Due to the heterogeneity of study designs, and statistically insignificant results, it was not possible to perform a quantitative analysis. The potential harm of RPE use, such as pressure sores and stress, is poorly documented and requires further study for mitigation and improvement.
This review aimed to ascertain and evaluate the evidence base behind RPE policy specific to SARS-CoV-2 in order to most effectively protect HCWs from SARS-CoV-2 infection. While the subject is novel, this review adds to the rapidly expanding discussion on the use of RPE in the inpatient setting.
Strengths and weaknesses in relation to other studies, discussing important differences in results
This review found just one study that directly compares FRSMs and respirators. Ng et al 28 conclude that FRSMs and N95s are equally effective. Limitations of this study include retrospective design, small sample size and a wide range of scenarios defined as ‘exposure’. There is no stratification of confounding variables such as age, sex, health, community exposure to SARS-CoV-2, or exposure to other patients with COVID-19. In the participants tested, none tested positive for SARS-CoV-2 in either intervention. It is unclear how quantitative analysis was performed. No retrospective significant difference was found in SARS-CoV-2 test results of these HCWs. While this does not prove either more effective, it also does not support the study conclusion that FRSMs and N95s are equally effective.
On re-searching of the literature, two similar systematic reviews20 21 were published during the study period of this review. One, by Bartoszko et al 20 was a systematic review and meta-analysis of four clinical randomised controlled trials (RCTs) comparing FRSM and N95 use. Three24 25 27 of the four studies included in the review by Bartoszko et al.20 were included in our review. Bartoszko et al 20 used search terms specific to RCTs, not specific to SARS-CoV-2/COVID-19, and excluded laboratory studies or tests on manikins. These authors also highlight the paucity and low quality of evidence comparing FRSMs and respirators. Their review adjusted for the collation of results from cluster RCTs with individual RCTs. However, the review was not specific to SARS-CoV-2, nor was the meta-analysis of aggregate data specific to any coronavirus. Similar to our review, that team20 draws outcomes from laboratory-confirmed illnesses of other viruses to postulate conclusions. The review by Bartoszko et al 20 might be limited by the exclusion of other study designs. It is unclear why three studies were included for analysis externally to their search strategy at a late stage, nor why an RCT27 included in our review, providing statistically significant findings, was excluded by that team.
The second, a rapid review of facemasks and respirators by MacIntyre et al,21 was similar to the review by Bartoszko et al 20 in that it included only RCTs. Similar to this review, MacIntyre et al 21 focused on SARS-CoV-2 and included data on other respiratory transmissible viruses and studies from community, healthcare and source control settings. MacIntyre et al 21 also find that comparison between different RCTs is difficult due to the varying parameters used to define ‘safe’. Akin to our review, MacIntyre et al 21 also find that continuous respirator use is effective in the protection of HCWs when compared with intermittent use. They also comment on the lack of current understanding of the degree of aerosolisation of SARS-CoV-2 outside recognised AGPs, and the role this plays in transmission.
Meaning of the study: possible explanations and implications for clinicians and policymakers
There is no high-quality evidence regarding the efficacy of RPE in protecting HCWs against SARS-CoV-2 transmission. The evidence base suggests FFP3s may be a more effective form of RPE than FRSMs. There is uncertainty on the transmission mechanism of SARS-CoV-2.30 If SARS-CoV-2 aerosolisation is more common than thought, FFP3 respirators would be indicated in a greater variety of settings as first line. There are challenges to the droplet model of respiratory illness transmission.31 Procedures classified as AGPs vary in international guidance.7–11 If the droplet model is inaccurate, aerosolisation might occur to a greater degree than currently thought, portending increased use of FFP3 respirators. Given this uncertainty, HMG PPE guidance4 should take a cautious approach rather than risk underprotecting staff. RPE guidance is increasingly stock driven.8–11 If RPE must be triaged due to unavailability of stock, FRSM wearing HCWs may be exposed to aerosolised SARS-CoV-2.
Unanswered questions and future research
Further rigorous study is required into the transmission of SARS-CoV-2, as recent studies liken it more to SARS-CoV-1 than to influenza.30 HMG PPE guidance4 is based on preparedness for an influenza pandemic.
The validity of the droplet versus aerosol dichotomy of respiratory illness transmission is uncertain.31 It must be substantiated since it underpins HMG PPE guidance on RPE. This may have wider implications as nations consider guidance on large-scale mask-wearing interventions as a public health measure.
Expedited research is required to further understand aerosol-generating procedures, including an effort to standardise the classification of AGPs by different organisations as AGPs are the key indication for RPE triaging in HMG PPE guidance.4 This is even more pertinent if COVID-19 becomes an established disease.
HMG PPE guidance on the indications for use of FFP3 and FRSM is underpinned by the droplet theory of transmission of SARS-CoV-2, based on the flow chart suggested by Coia et al 32 as shown in figure 2.
Hence, an immediate consideration for the choice of RPE in patient-facing HCW is that an FRSM may not be ‘safe’, as it is difficult to define ‘safe’. Due to lack of stock and supply, FFP3 respirators may have to be prioritised for those exposed to areas of greatest aerosolisation of SARS-CoV-2. This is further compounded by our lack of understanding of the aerosolisation of SARS-CoV-2. An FFP3 respirator may provide greater protection than an FRSM given the uncertainty on the aerosolisation of SARS-CoV-2. It is suggested that if there were an unlimited stock of FFP3 respirators, the guidance would be that all inpatient facing HCWs would wear them continuously.
HCWs away from work, self-isolating or on sick leave due to COVID-19 reduce the health system’s capacity to deal with the ongoing pandemic.3 In order to reduce sickness burden on health systems, local policy makers must be able to make informed, evidence-based decisions on their choice of RPE.
This review concludes that the evidence base for HMG’s PPE guidelines4 is not based on SARS-CoV-2 and requires generalisation from low-quality evidence in which other pathogens/particles were tested. There is a paucity of high-quality evidence regarding the efficacy of RPE specific to SARS-CoV-2. HMG’s PPE guidelines are underpinned by the assumption of droplet transmission of SARS-CoV-2.
It is evident from the WHO,33 the European Centre for Disease Prevention and Control9–11 and the Centers for Disease Control and Prevention8 guidance that the indications for the use of RPE are not based solely on the protective abilities of respirators and FRSMs. Instead, a triaging system based on an expected shortage of global stock and supply, combined with current understanding of likelihood of exposure to aerosolised SARS-CoV-2 is used.
There is active discussion regarding the droplet transmission of SARS-CoV-2 with an accepted uncertainty in understanding.30 31 Given this uncertainty, a cautious approach should be taken in the protection of HCWs. This review found that in all laboratory studies respirators were more protective to the wearer than FRSMs in all parameters tested. In the clinical studies reviewed, the only statistically significant finding was that respirators provided significant protection against bacterial-viral coinfection compared with FRSMs. No statistically significant evidence was found to support the conjecture that an FRSM might provide the same level of protection as a respirator against SARS-CoV-2, or indeed any tested live virus or inert submicron particle. Therefore, use of a respirator would be the more cautious option.
While the triaging of RPE due to a lack of global stock is understandable and appropriate during the strains of a pandemic, it must be noted that by increasing the protection of some through the provision of respirators, HMG PPE guidance might be increasing the risk of COVID-19 faced by others.
The authors would like to thank the following for their peer review of the completed manuscript prior to submission: Mr Isaac Florence MSc; Dr Rory Wilson MBChB BSc (Hons), University of Glasgow School of Medicine, Glasgow; Miss Roseanna Jenks RM MA (Cantab) BSc (Hons), West Middlesex University Hospital, London; Dr Sivakami Ramaraj MOrth (RCS Eng) MSc (Lond).
Contributors All authors meet ICMJE’s four criteria of authorship. PR conceived the idea, contributed to the methodology, conducted the search, extracted and appraised data and wrote the article. JTS contributed to the methodology, conducted the search, extracted and appraised data and reviewed the article. RD contributed to the writing of the article, and provided review and guidance throughout. CA contributed to the writing of the article and provided review and guidance throughout. SH contributed to the development of concept, contributed to the writing of the article and provided review and guidance throughout. All authors approved the final manuscript and article submission. Guarantorship: the corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.
Funding The Wellcome Trust kindly covered the Article Processing Charges of this manuscript. Access to the Wellcome Trust Article Processing Charges block grant was sought and secured after the manuscript was accepted for publication by BMJ Open.
Competing interests All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: no support from any organisation for the submitted work; no research grants and honorariums. RD has recently begun to design not-for-profit, small scale items of PPE for the amelioration of the widely documented PPE stock crisis, aside from RD’s core business; no other relationships or activities that could appear to have influenced the submitted work.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data are available on reasonable request. The authors will support data sharing on request by emailing the corresponding author, PR.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.