Article Text
PDF
Abstract
Background/objectives Urinary antigen tests have been used for the rapid identification of Streptococcus pneumoniae infection in patients with pneumonia, thereby leading to earlier targeted therapy than when using conventional diagnostic culture methods. This study aimed to update the knowledge on the diagnostic accuracy of urinary antigen tests for S. pneumoniae among patients with acute respiratory failure suspected of pneumonia based on a systematic review and meta-analysis.
Methods A systematic search was performed using MEDLINE and the Cochrane Central Register of Controlled Trials for studies published up to 3 June 2020. Prospective and retrospective cohort studies (in English) that reported on the diagnostic performance of urinary antigen tests versus culture or smear diagnostic methods in adult patients with clinically diagnosed pneumonia were selected and analysed. The QUADAS-2 tool was used to assess the risk of bias, and a bivariate random effects model was applied to perform a meta-analysis of the selected studies.
Results A total of 2179 studies were screened, of which 30 met the eligibility criteria for quality assessment and meta-analysis. Overall, data from 12 366 patients, including 1548 patients (12.5%) with the target condition and suspected pneumococcal pneumonia, were included in the analysis. The overall quality of the included studies was determined to be serious. The calculated pooled sensitivity and specificity were of 0.66 (95% CI 0.62 to 0.69) and 0.90 (95% CI 0.85 to 0.93), respectively.
Conclusions The urinary antigen test is useful for achieving a definitive diagnosis of S. pneumoniae infection in patients with pneumonia.
- INFECTIOUS DISEASES
- Diagnostic microbiology
- INTERNAL MEDICINE
- MICROBIOLOGY
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information. Not applicable.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
The protocol was registered in advance and submitted according to Preferred Reporting Items for Systematic Reviews and Meta-analyses.
Statistical analysis methods and bias risk assessment tools recommended in the Cochrane Handbook were used.
Papers reported in languages other than English were not included, and the Embase database and Grey literature were not used as data sources, which may have resulted in missing relevant papers or publication bias.
Introduction
Streptococcus pneumoniae is the most common cause of community-acquired pneumonia in adults and is also the main cause of pneumonia-causing acute respiratory failure.1 The mortality rate of pneumococcal pneumonia varies depending on the severity of the disease at onset and host factors, but the 90-day mortality rate is very high, at approximately 25%–30%, when accompanied by bacteraemia.2
Pneumococcal pneumonia diagnosis is usually confirmed by the identification of S. pneumoniae in sputum samples by Gram staining, or its detection in cultures of blood samples or respiratory specimens, such as sputum or pleural fluid. Gram staining is rapid but not highly sensitive/specific.3 Culture tests take several days to provide results. Therefore, many cases of pneumonia are treated empirically. Empiric therapy is generally useful because it takes into account knowledge of the pathogen, including resistance/susceptibility, based on local high-quality surveillance system, the patient’s risk factors and comorbidities, and the severity of symptoms. However, not every country, region or hospital has high-quality surveillance system, and even if they do, if the urine antigen test can rapidly diagnose S. pneumoniae infection, a more specific antimicrobial agent may be selected. As a result, we believe that it may be helpful to reduce the individual risk of infections caused by resistant bacteria and antibiotic-associated Clostridium difficile infections.4 5
BinaxNOW Streptococcus pneumoniae Antigen Card (BinaxNOW-SP; Abbott, Abbott Park, Illinois, USA) is an immunochromatographic test that detects the C-polysaccharide coat protein of S. pneumoniae in urine. It is used worldwide as a rapid diagnostic method for S. pneumoniae infection in patients with pneumonia, allowing to obtain clinically valuable results within 15 min of urine collection. In 2013, two systematic reviews and meta-analyses were conducted using BinaxNOW-SP as the index test, along with established culture or smear methods as reference standard, in patients with community-acquired pneumonia.6 7 However, several studies were subsequently reported that consisted of very large sample sizes, which tended to be slightly less sensitive than the previous systematic reviews, therefore, we considered that it was necessary to update the evidence for the diagnostic accuracy.6–10 Therefore, to close this knowledge gap, we conducted a systematic review and meta-analysis to compare the sensitivity and specificity of urinary antigen tests (UATs) with established culture or smear methods in the diagnosis of patients with acute respiratory failure suspected of pneumonia.
Methods
We performed a systematic review and meta-analysis of previously published studies on diagnostic test accuracy (DTA). We adhered to the methodological standards outlined in the handbook for DTA reviews of Cochrane11 and used the Preferred Reporting Items for Systematic Reviews and Meta-analyses of DTA Studies12 to report our findings. The review protocol was prospectively registered in the University Hospital Medical Information Network Clinical Trials Registry (UMIN 000041081) and it is also available in online supplemental material. The need for ethical approval and consent was waived for this systematic review.
Supplemental material
Patient and public involvement
There was no patient and public involvement in the whole process of conducting this research.
Population, index test and target condition
The target participants were adult patients with acute respiratory failure suspected of having pneumonia. If a study involved both children and adults, we extracted only the data related to the adult patients and excluded studies in which data related to adults could not be distinguished from those of children. The index test of interest was UATs, such as BinaxNow-SP. The target condition was pneumococcal pneumonia diagnosed using reference standard methods. For the reference standard, data were considered positive if at least one positive result was obtained by Gram stain of sputum, or culture of blood, pleural fluid or respiratory samples (ie, sputum, bronchoalveolar lavage fluid or other).
Study eligibility and selection
All studies written in English, including prospective, retrospective and observational (cohort or cross-sectional) studies, and secondary analyses of randomised controlled trials, were selected. We excluded diagnostic case-control studies (two-gate study) and case studies that lacked DTA data, namely true positive (TP), false positive (FP), true negative (TN) and false negative (FN) values. Two authors independently screened each study for eligibility and extracted the data. Disagreements among reviewers were resolved by discussion or a third reviewer.
Electronic searches
To identify all eligible studies, we searched the Medical Literature Analysis and Retrieval System Online (MEDLINE) via PubMed (accessed on 3 June 2020), and the Cochrane Central Register of Controlled Trials (CENTRAL) (accessed on 3 June 2020). The details of the search strategy are provided in online supplemental file 1. The developing search strategy was supported by medical librarians at the Kyoto Prefectural University of Medicine (Kyoto, Japan).
Supplemental material
Data extraction and quality assessment
The following data were extracted using a predefined data extraction form that included the study characteristics (author, year of publication, country, design, sample size, clinical settings, conflict of interest and funding source), patient characteristics (inclusion/exclusion criteria, and clinical and demographic characteristics), index test (UATs), reference standard (Gram stain of sputum, blood culture, sputum culture, pleural fluid culture, or culture of any other respiratory sample) and diagnostic accuracy parameters (TP, FP, FN and TN). Two investigators evaluated the risk of bias using the QUADAS-2 tool,13 which includes four risk of bias domains and three domains of applicability. Any disagreements were resolved by discussion or a third reviewer. Given the absence of evidence for publication bias in DTA studies and the lack of reliable methods for its assessment, no statistical evaluation of publication bias was performed.11
Statistical analysis and data synthesis
The ‘Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy’ was used.11 Diagnostic sensitivity and specificity estimates with 95% CIs were captured in paired forest plots to evaluate study heterogeneity. We used a bivariate random effects model for the meta-analysis. Random-effects model attempted to generalise findings beyond the included studies by assuming that the selected studies are random samples from a larger population.14 All analyses were performed using Review Manager V.5.3 (Cochrane Collaboration, London, UK) and STATA/SE V.16.1 (StataCorp). All statistical analyses were conducted using a two-sided alpha error of 5%.
Heterogeneity assessment
A subgroup analysis, prespecified in the protocol, was performed to investigate whether sensitivity and specificity were different among the following subgroups: (1) differences in reference standard, (2) presence of immunocompromised patients, (3) antibiotics use prior to UAT was performed, (4) inpatients or outpatients and (5) types of UAT used.
Sensitivity analysis
We performed a sensitivity analysis restricted to studies with a low risk of bias in all domains of QUADAS-2.
Results
A total of 2179 studies were screened, among which 30 met the eligibility criteria and were included in the quality assessment and meta-analysis. The detailed selection protocol and reasons for exclusion are shown in figure 1.
Study selection protocol (PRISMA flow chart). PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-analyses.
Study characteristics
Data from 12 366 patients were retrieved from the 30 studies, among who were 1548 patients (12.5%) with pneumococcal pneumonia. The median and mean age of the patients included ranged from 45 to 74 years, and most were male (51%–79%). The median prevalence of pneumococcal pneumonia was 16.7% (IQR: 9.5–22.9). Twenty-one studies were prospective, five studies were retrospective and four were of unknown nature. Most studies were conducted in inpatient settings, but three studies included outpatients. The patient characteristics and reference standards used in each study are summarised in table 1. The index test was BinaxNow-SP in all the studies. Reference standards were defined by either positive cultures of blood, respiratory samples (or smear) or pleural fluid in 18 studies, by either positive cultures of blood or respiratory samples (or smear) in 9 studies, by blood culture alone in 2 studies and by respiratory samples culture (or smear) alone in 1 study.
Summary of the characteristics of the studies included in the analysis
Risk of bias assessment
For patient selection, we evaluated 10 studies as having a high risk of bias or high concern for applicability (figure 2) as the exclusion of immunocompromised patients was inappropriate and that the sample was not consecutive when the time frame for inclusion of patients was limited. For the index test, one study was rated as high risk of concern in applicability. The study used early morning urine for the diagnosis of pneumococcal pneumonia; however, it was not considered as common and thus the applicability of the results to other settings and actual clinical practice was considered as high concern in the applicability. For the reference standard, we evaluated 11 studies as having a high risk of bias or high concern for applicability because the reference standard was not a composite of blood culture, respiratory sample culture (or smear) and pleural fluid culture. In the patient flow assessment, we assessed 16 studies to have a high risk of bias because blood cultures as a reference standard were not performed for all patients. Taken together, the overall quality of the included studies was serious.
Summary of bias risk assessment. Bias risk was evaluated using the QUADAS-2 tool. The QUADAS-2 tool is designed to assess the quality of primary diagnostic accuracy studies. Green: low risk of bias or low concern in applicability. Red: high risk of bias or high concern in applicability.
Meta-analysis results
A summary of the diagnostic accuracy of UTAs for pneumococcal pneumonia in each study is presented in a forest plot in figure 3. The calculated pooled sensitivity and specificity were of 0.66 (95% CI 0.62 to 0.69) and 0.90 (95% CI 0.85 to 0.93), respectively. In a population of 1000 patients with a given target condition prevalence of 10%, the following was detected: 66 patients (95% CI 62 to 69) with TP, 34 patients (95% CI 31 to 38) with FN, 810 patients (95% CI 765 to 837) with TN and 90 patients (95% CI 63 to 135) with FP. The findings for the different prevalence estimates (5%–15%) are presented in online supplemental file 2.
Supplemental material
Paired forest plot of the studies included in the meta-analysis. FN, false negative; FP, false positive; TN, true negative; TP, true positive.
Subgroup and sensitivity analyses
We conducted subgroup analysis among two reference standards: (1) either positive culture of blood, respiratory samples (or smear) or pleural fluid, (2) either positive culture of blood or respiratory samples (or smear), (3) positive blood culture only and (4) positive respiratory samples (or smear) (figure 4A). Based on the 17 studies using reference standard A (see table 1), the sensitivity and specificity were of 0.69 (95% CI 0.64 to 0.73) and 0.88 (95% CI 0.80 to 0.93), respectively. Based on the nine studies using the reference standard B, the sensitivity and specificity were of 0.59 (95% CI 0.52 to 0.65) and 0.93 (95% CI 0.87 to 0.96), respectively. We could not conduct a subgroup analysis of reference standards C and D as each of these diagnostic approaches was only used in a few studies.
Paired forest plot of the studies included in the subgroup analysis based on (A) difference in reference standard, (B) presence of immunocompromised patients, and inclusion of (C) only inpatients, or both inpatients and outpatients. FN, false negative; FP, false positive; TN, true negative; TP, true positive.
Regarding the presence of immunocompromised patients, since no study assessed exclusively this population, we conducted subgroup analysis between excluded immunocompromised participants and mixed, 10 and 8 studies, respectively (figure 4B). The sensitivity and the specificity of the included group of studies were of 0.64 (95% CI 0.57 to 0.70) and 0.93 (95% CI 0.78 to 0.97) and in the mixed group were of 0.65 (95% CI 0.60 to 0.70) and 0.83 (95% CI 0.79 to 0.87), respectively.
Regarding inpatients or outpatients, since there was no study comprising only outpatients, we conducted a subgroup analysis between studies with only inpatients and with both inpatients and outpatients (23 and 5 studies, respectively) (figure 4C). The sensitivity and specificity of the studies with only inpatients were of 0.66 (95% CI 0.62 to 0.71) and 0.89 (95% CI 0.82 to 0.93), and in studies with mixed populations were of 0.67 (95% CI 0.56 to 0.76) and 0.90 (95% CI 0.79 to 0.96), respectively.
We could not conduct subgroup analysis regarding presence of prior antibiotics at UAT nor on different types of UATs due to lack of data. Sensitivity analysis was also not performed because of the low number of studies with a low risk of bias in all the domains of QUADAS-2.
Discussion
This systematic review and meta-analysis included 30 studies, in which the pooled sensitivity and specificity were of 0.66 (95% CI 0.62 to 0.69) and 0.90 (95% CI 0.85 to 0.93). These results indicate moderate sensitivity and high specificity, which were in accordance with those of previous meta-analyses.5 6 However, a previous analysis comprising 10 studies reported a higher sensitivity (0.75, 95% CI 0.71 to 0.79) and specificity (0.95, 95% CI 0.92 to 0.98).5 This difference may be due to the exclusion of studies in which more than one-third of the target population was administered prior antibiotics and by excluding pneumonia of unknown causative organisms.
Here, the subgroup analysis revealed no significant variations in sensitivity and specificity owing to differences in the reference standards. Moreover, the specificity tended to be lower in studies including immunocompromised patients, suggesting that caution may be necessary in clinical practice when interpreting UAT results from immunocompromised patients. In addition, studies including outpatients showed similar results to those including only inpatients, suggesting that the UATs may be as useful in outpatients as in inpatients. However, no studies have included outpatients; thus, further research is required.
The strengths of our study are as follows. It has been more than 7 years since the last existing systematic review was published, and the results of two investigations showed some variability.5 6 In the present systematic review, we were able to report updated results by including eight studies published since 2013, including studies with large sample sizes. Second, we were able to evaluate the sensitivity and specificity by including immunocompromised patients, which may be helpful in clinical practice. Third, the results of this study suggest that the collected information may be applicable to both outpatients and inpatients.
Nonetheless, this study also has some limitations. First, we did not include the Embase database or the grey literature as data sources and only included articles described in English for inclusion. Therefore, there is a possibility of missing relevant studies or publication bias. However, we attempted to extract as many related studies as possible by manual and citation search. Second, acute respiratory failure was diagnosed based on the clinical symptoms used to diagnose pneumonia and the degree of severity that required hospitalisation. Therefore, some patients with pneumonia without acute respiratory failure might be included in this study population. Third, paediatric patients and patients with nosocomial pneumonia were not included in this systematic review. The diagnostic accuracy for these patients needs further study. Fourth, the overall quality of the study was judged to be serious, mainly because blood cultures were not collected from all participants, but it was unknown which participants in each study did not have blood cultures collected, which may have affected the results of this study. Fifth, although we set the reference standard as gram stain of sputum, or culture of blood, pleural fluid, or respiratory samples and it may be accepted in the previous literature, there is some opinion that it is difficult to define the gold standard for the diagnosis of pneumococcal pneumonia as well as other infectious disease. Recently, in a research question of diagnostic accuracy for infectious disease without widely accepted gold standard, the latent class analysis is suggested as an alternative method. We did not perform it because it was not planned in the analysis protocol; however, in further research, it may be valuable to consider the latent class analysis. Sixth, we were unable to evaluate the effect of antibiotic use, although some studies reported that 21%–70% of the participants received prior antibiotics. Fourth, all the included studies used BinaxNow-SP; thus, we could not evaluate the effect of different types of UATs. Seventh, we were not able to evaluate the effect of differences in disease severity. Nonetheless, we believe that this effect is small because the standard mechanism of the antigen test regardless of the symptoms. Eighth, previous reports have indicated that UATs inspection performance may vary from period to period, although the cause is not known.15 16 This study did not examine this point in the subgroup analysis, however, future analyses might need to consider heterogeneity due to differences in inspection performance by period. Ninth, UAT does not detect antimicrobial susceptibility/resistance and may have no direct impact on prescriptions of antimicrobials. Even if UAT is used, antimicrobials should be selected based on knowledge of the pathogen based on local surveillance system, patient risk factors, underlying disease and severity. Tenth, patients who have recently developed pneumococcal pneumonia may provide FP results for several weeks after onset.17 Therefore, when using UAT, the patient’s recent history of pneumonia should be ascertained and interpreted with this in mind. Finally, the diagnostic accuracy of UAT for pneumococcal pneumonia might have changed during the COVID-19 epidemic. At the time of the search for inclusion studies, there were no studies available after the COVID-19 outbreak, so the impact of the COVID-19 outbreak could not be considered; the diagnostic accuracy of UAT for pneumococcal pneumonia after the COVID-19 outbreak will be considered in future studies.
Taken together, the reported high pooled specificity and moderate pooled sensitivity indicate that UATs are useful to rule in pneumococcus pneumonia rather than to rule it out. Despite of the above-mentioned limitations, our study indicates that UATs may be useful for the diagnosis of patients suspected of having pneumonia.
Conclusion
We conducted a systematic review and meta-analysis, which included recent studies with a large sample size, to compare the sensitivity and specificity of UATs as diagnostic tools for the assessment of patients with acute respiratory failure suspected of pneumonia. Our results were similar to those of previous studies, demonstrating that UATs have high specificity and moderate sensitivity. Hence, UATs can be useful for rapidly achieving a definitive diagnosis in patients suspected of having pneumonia.
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information. Not applicable.
Ethics statements
Patient consent for publication
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @YasuoShun
Collaborators Shunsuke Yasuo, Maki Murata, Natsuki Nakagawa, Takeshi Kawasaki, Takuo Yoshida, Koichi Ando, Satoshi Okamori, and Yohei Okada
Contributors All authors were involved in the study design. SY, MM, NN, TK, TY, KA, SO and YO identified the studies included in the meta-analysis and analysed the data. SY and MM drafted the manuscript, and YO supervised the drafting of the manuscript. All authors were involved in the data interpretation and discussion. All authors read and approved the final manuscript. SY is responsible for the overall content of the study as a guarantor.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.