Introduction

Pneumonia is one of the worldwide leading causes of mortality in children below 5 years of age [1] and one of the leading causes of hospitalization in children [2]. In uncomplicated community-acquired pneumonia (CAP), diagnosis is based mainly on clinical parameters [3]. In most cases of pneumonia, physicians are limited to empiric therapy due to a lack of suitable microbiological testing that is quick, non-invasive, and reliable. Empiric therapy might be adequate for non-complicated pneumonia [3, 4], but patients with severe pneumonia, with or without prior treatment, usually require further investigations. Children that are at increased risk for severe pneumonia are those with chronic diseases or those hospitalized and/or undergoing mechanical ventilation. Hospital-acquired pneumonia (HAP) and ventilator-associated pneumonia (VAP) are among the most common nosocomial infections in intensive care patients [5,6,7,8]. Most commonly encountered pathogens in VAP are Pseudomonas aeruginosa, Staphylococcus aureus, Streptococcus spp., Klebsiella pneumoniae, Escherichia coli, Enterobacter spp., Acinetobacter baumannii, and coagulase-negative staphylococci, while some cases have a polymicrobial etiology [9]. The most important issue is appropriate sampling [10]. Bronchoalveolar lavage (BAL) and protected specimen brush are well-established in adults, but are regarded as either too invasive or as technically inapplicable for most children. Tracheal aspirates (TA) have been described as an alternative to BAL, especially in the neonatal intensive care unit (NICU). Intubated newborns undergo routine endotracheal suctions for mucus relief, during which it is feasible to collect TA for diagnostic purposes. The main disadvantage is the potential contamination of TA and, to a lesser extent, of BAL by microorganisms colonizing the upper airways.

Rapid pathogen detection is crucial for early and appropriate treatment of HAP or VAP, but also of severe CAP, yet routine culturing, pathogen identification, and antimicrobial susceptibility testing (AST) take 48–72 h. Prior antibacterial therapy might contribute to false-negative culture findings and thus hampers the identification of pathogens [11, 12]. On the other hand, inappropriate antimicrobial therapy contributes to longer hospitalization and even higher mortality [13, 14]. Therefore, rapid molecular diagnostics, e.g., based on genetic testing, that yield quick, but also reliable results, are increasing in importance, both for viral [15,16,17,18] and for bacterial etiology [18,19,20]. The Unyvero P50 Pneumonia application, developed by Curetis AG (Holzgerlingen, Germany), is a multiplex PCR-based assay that has been CE marked and approved for use in adults in Europe for rapid pathogen detection. The previous studies on adults had varying results [21,22,23,24]. We sought to evaluate Unyvero in children and neonates and compare the findings with routine culturing.

Materials and methods

In this prospective study, we compared findings of routine microbiology with findings of Unyvero in hospitalized children and neonates in whom microbiological testing was ordered due to suspected pneumonia. BAL fluids were obtained from in-patients undergoing bronchoscopy, TA from intubated children and newborns, and pleural fluids from patients who underwent pleural puncture. The specimens underwent routine culturing according to the standard microbiology laboratory procedures for diagnostic purposes [25], including inoculation on columbia agar with 5% sheep blood, chocolate, and MacConkey agar plates (Becton–Dickinson, New Jersey, USA), identification by plate reading and confirmation by VITEK® 2 (bioMérieux, Marcy-l’Étoile, France). AST comprised disk-diffusion method, VITEK® 2, and minimum inhibitory concentration determination using Etest (bioMérieux, Marcy-l’Étoile, France). Remaining sample volumes (only samples containing more than 180 µL) were irreversibly anonymized and used according to manufacturer instructions as explained elsewhere [22, 23] in Unyvero P50. This application is intended to detect gene products of the following 17 pathogens: S. aureus, Streptococcus pneumoniae, E. coli, Enterobacter spp., Proteus spp., K. pneumoniae, Klebsiella oxytoca, Serratia marcescens, Morganella morganii, Moraxella catarrhalis, Pseudomonas aeruginosa, A. baumannii, Stenotrophomonas maltophilia, Legionella pneumophila, Chlamydophila pneumoniae, Haemophilus influenza, and Pneumocystis jirovecii. Besides, 18 resistance genes are covered: ermA, ermC, ermB, msrA, mefA/E, mecA, bla TEM, bla SHV, bla CTX-M, bla EBC, bla DHA, bla KPC, bla OXA-51-like, int1, sul1, gyrA83, gyrA87, and parC.

Data were collected in Microsoft Excel (Microsoft Corporation, Richmond, USA). Results from the conventional culturing and Unyvero were compared in terms of detected pathogens and resistance patterns. Concordance was analyzed for each pathogen specifically, and was defined as identical results for a single pathogen from both techniques was calculated. For each pathogen, results were regarded as concordantly positive (CP) when both Unyvero and culture were positive; concordantly negative (CN) when both Unyvero and culture were negative; Unyvero positive (UP) when Unyvero was positive and culture was negative; and Unyvero negative (UN) when Unyvero was negative and culture was positive. The performance of Unyvero with regard to every single pathogen covered by the application was evaluated by determining prevalence, sensitivity, specificity, positive-predictive value, and negative-predictive value. 95% confidence intervals (CI) were calculated according to the efficient score method [26]. The parameters were calculated, as specified in Supplemental Table 1.

Apart from age and sex, no clinical data were available, including gestational age in neonates, so chronological age was used. Correlation between the specified signal strength for each detected pathogen in Unyvero and the corresponding bacterial count was assessed for specimens with concordant findings. Bacterial count was designated either as colony forming units per milliliter from BAL or as categorical variables for tracheal aspirates (many, moderate, or little). Data were excluded when all PCR chambers were invalid or if the failure concerned chambers containing organisms that yielded positive results in the corresponding culture. Data were included if the partial invalidity affected other chambers than the culturally detected pathogen.

The specimens were obtained between March 2014 and November 2015. This observational study was reviewed and approved by the ethic committee of the Ludwig-Maximilians-Universität München (UE Nr. 012-14). The committee declared the study unobjectionable and waived the need for informed consent, since no additional patient samples were obtained.

Results

From 80 samples available, one had to be excluded due to total runtime error of the Unyvero. The remaining 79 samples (from 79 patients) were analyzed, of which 43 specimens were BAL fluids, 30 specimens were TA, and six specimens were pleural fluids (Table 1). We observed partially invalid runs due to errors in some of the PCR chambers in 21/79 cases (26.6%). Of these, three occurred in specimens, where cultures were positive for a pathogen which could not be detected by Unyvero; however, the chamber failure did not affect the specific pathogen.

Table 1 Descriptive data
Full size table

Concordance rates

Both traditional culture and Unyvero each yielded positive results in 32/80 (40%) specimens, while only 13 of those were concordantly positive. In ten cases, the Unyvero failed to conform with the positive culture result, while Unyvero was positive in three cases of completely sterile culture. Both culture and Unyvero were concordantly negative in 35/80 (43.8%) specimens. Unyvero yielded no pathogens in 43 specimens, one pathogen in 17 specimens, two pathogens in eight specimens, and three or more pathogens in nine specimens, while in routine culturing, no pathogens were detected in 36 specimens, one pathogen in 27 specimens, two pathogens in eight specimens, and three or more pathogens were detected in six specimens.

Pathogen identification

The most frequently detected pathogens in BAL fluids were M. catarrhalis in 4/13 cases (30.8%), Haemophilus influenzae in 3/13 cases (23.1%), and Pseudomonas aeruginosa in 2/13 cases (15.4%) of concordantly positive findings. For concordantly positive findings and those where only Unyvero detected pathogens, the most frequently encountered pathogens were M. catarrhalis in 7/27 cases (25.9%) and H. influenzae in 5/27 of cases (18.5%). In TA, the most frequent concordantly positive findings were P. aeruginosa in 9/28 cases (32.1%), S. aureus in 5/28 (17.9%), and K. pneumoniae 4/28 (14.3%). When taking Unyvero-positive findings also into account, P. aeruginosa with 9/43 (20.9%), S. aureus with 8/43 (18.6%), and K. pneumoniae with 5/43 cases (11.6%) remained the most prevalent, while E. coli with 5/43 (11.6%) was more often detected than in CP findings only.

The Unyvero showed best results for Gram-negative bacteria, especially for non-fermenting bacteria such as P. aeruginosa, A. baumannii, M. catarrhalis, and S. maltophilia, who in all amounted to 47.4% of detected bacteria (Table 2). The sensitivity for this group, i.e., the rate of culture-positive specimens that were detected by Unyvero was 90% (95% CI 66.9–98.2%). The specificity, i.e., the rate of negative results in Unyvero conforming with culture, was 97.3% (95% CI 94.5–98.7%), due to eight Unyvero-positive specimens that had no cultural detection of these bacteria. These values lead to a positive-predictive value of 69.2% (95% CI 48.1–84.9%) and to a negative-predictive value of 99.3% (95% CI 97.3–99.9%).

Table 2 Pathogen and group specific results; concordantly positive (CP), Unyvero negative and culture positive (UN), Unyvero positive and culture negative (UP), and concordantly negative (CN), given as absolutes; prevalence (Prev), sensitivity, specificity, positive-predictive value (PPV) and negative-predictive value (NPV) given as accuracy measures, in percent; 95% confidence intervals in brackets
Full size table

For Enterobacteriaceae, the sensitivity was slightly lower with 84.6% (95% CI 53.7–97.3%), the specificity was 98.8% (95% CI 97.2–99.5%), calculated PPV and NPV were 64.7% (95% CI 38.6–84.7%) and 99.6% (95% CI 98.3–99.9%), respectively.

In contrast, the performance for Gram-positive bacteria was poor, yielding a sensitivity of only 46.2% (95% CI 20.4–73.9%), while specificity was also lower with 93.9% (95% CI 84.4–98%), influenced by a substantial amount of Unyvero-positive findings that were culture negative. Thus, PPV was low with 60% (95% CI 27.4–86.3%), and NPV yielded 89.9% (95% CI 79.6–95.5%).

There were no cases of M. morganii, Legionella pneumophila, or C. pneumoniae, while in one TA, Unyvero detected P. jirovecii (Supplemental Table 2). In one case, culture yielded Acinetobacter junii, while Unyvero detected A. baumanii. Due to sequence homology, we rated this finding as concordant. Furthermore, we did not calculate accuracies for commensal bacteria such as Streptococcus viridans group, coagulase-negative staphylococci, and other important pathogens that are not covered by Unyvero, such as Burkholderia cepacia. Pathogens in brackets were below the signal cutoff of 250 or 200 (for first and second generation cartridges, respectively). No pathogens were detected by Unyvero in any of the pleural fluids (Supplemental Table 2).

Resistance markers

In total, 39 resistance genes were detected by Unyvero in 23 specimens (six BAL, 17 TA). The most frequently encountered resistance gene was ermB in seven cases; followed by mecA and tem in six specimens each (Table 3). In nine specimens, Unyvero detected more than one resistance gene. In addition, Unyvero yielded positive findings for several specimens in which no pathogen could be detected, such as mecA, a resistance gene of Staphylococcus spp., detected in three specimens, in which the corresponding culture yielded commensal flora such as coagulase-negative Staphylococci.

Table 3 Resistance genes and correlating findings
Full size table

With the exception of mecA, ermB, and mefA/E, there was a perfect concordance of detected resistance genes and perfect correlation with the culture result, while there was weaker correlation for the corresponding result of AST. Overall, the detected resistance genes correlated with a plausible pathogen finding in Unyvero in 71.8% of cases, while 85.7% were concordant with the culturally proven pathogen. In those, the result of the AST was in accordance with the proven resistance gene in 75%. Four cases of medically important drug resistances were missed: one specimen with A. baumannii and proven multi-drug resistance to four out of four classes of antibiotics (acylaminopenicillins, third and fourth generation cephalosporins, carbapenems, and fluoroquinolones; 4-MRGN); one specimen with a methicillin-resistant S. aureus (MRSA; genetically confirmed at a reference laboratory); one specimen with P. aeruginosa and proven multi-drug resistance to three out of four classes (sensitive to fluoroquinolones; 3-MRGN); and one specimen with an ESBL-producing E. coli. In seven specimens, Unyvero yielded results concordant with the microbial sensitivity tests. In 17 cases, Unyvero detected resistance genes, while the microbial testing yielded no resistance, and in seven specimens, Unyvero failed to confirm the microbiologically proven antibiotic resistance (Supplemental Table 3).

Correlation between signal strength and microbial count

To evaluate the congruency between the PCR and culture, we compared Unyvero signal strengths and bacterial counts. For signal strength values of above 1500 in BAL fluids and above 1800 in TA, there was a correlation between the signal strength (Fig. 1a) and the detected bacterial count (Fig. 1b), while for lower signal strengths, this correlation was weaker. We specifically regarded signal counts in specimens in which culture had yielded no pathogen (Fig. 1c, d); here, we observed lower means of signal strength for BAL fluids (355.7 vs. 722.4) and TA (773.5 vs. 1169.7) with sterile cultures but findings in Unyvero.

Fig. 1
figure 1

Correlation between signal strength measured by Unyvero and bacterial count assessed by culturing, bacterial count indicated as colony forming units per ml (a) or bacterial count (1: little; 2: moderate; and 3: many) (b); signal strength: no specified unit. A: BAL fluids with concordant findings only; b TA with concordant findings only; c BAL with culture-negative findings only; d TA with culture-negative findings only

Full size image

Discussion

In this prospective study, we assessed the performance of a point-of-care multiplex PCR, the Unyvero, an automated and fast diagnostic tool for respiratory pathogens and resistance genes, in comparison with the standard diagnostic procedures in children and neonates. Unyvero succeeded in detecting all culture-positive specimens with P. aeruginosa, A. baumannii, Klebsiella oxytoca, and Enterobacter spp. These Gram-negative bacteria become clinically more relevant, especially with increasing rates of antimicrobial resistance [27,28,29,30]. A fast and reliable identification would be helpful, especially in patients with risk factors such as mechanical ventilation [31], cystic fibrosis [32] or other chronic lung diseases, and primary [33] or secondary immunodeficiency diseases [34]. Furthermore, there were 28 specimens in which Unyvero detected a pathogen, while the corresponding culture remained negative for the respective pathogen. This issue is one drawback inherent to the study design of comparing two different diagnostic methods. It remains debatable if these “false positive” findings represent a higher sensitivity inherent to PCR to detect the smallest amounts of genetic material of bacteria that cannot (any longer) be cultured, which has also been shown for other PCR assays [35]. In addition, PCR could have detected DNA from bacterial decay, without correlation in culture, notwithstanding clinical relevance. Since the objective of the study was to evaluate the Unyvero system, we compared it to the current gold standard, i.e., the cultural method, keeping the inherent problem in mind. The possible overdetection of pathogens due to a higher sensitivity was further intensified by the possible contamination of the main specimens, TA and BAL, which is underscored by the negative findings in pleural fluids.

Overall, Unyvero showed a sensitivity of 71.4%, failing to identify viable pathogens in as many as 16 specimens. Sensitivity was especially low for Gram-positive bacteria such as S. aureus, a major pathogen causing severe hospital-acquired pneumonia. This finding might represent biologic properties inherent to Gram-positive bacteria, i.e., their cell wall, which in contrast to Gram-negative bacteria display a thicker peptidoglycan layer and thus might render them more stable to the process of lysis.

Previous studies had shown divergent results, while not all analyses had indicated sensitivity and specificity. Our results are similar to what Schulte at al. [22] had shown in their study, where the test yielded a sensitivity of 70.6% and a specificity of 95.2%, but differ substantially from what Personne and colleagues [24] calculated (95.7% sensitivity and 32.6% specificity). These differences might be caused using different versions of cartridges. Recently, Ozongwu et al. reported a sensitivity and specificity of 88.8 and 94.9%, respectively, yet their analyses were done with a newer generation of cartridges [36]. Apart from the Pneumonia assay, there are other assays offered by Curetis, e.g., the implant and tissue infection assay. Studies on this assay had replicated non-superiority of the multiplex PCR over conventional methods except for the time factor [37, 38] or the benefit of detecting pathogens in patients under antimicrobial therapy [39].

In 17 cases, Unyvero detected resistance markers, while the particular pathogen was fully sensitive in culture. This might be explained by the presence of commensal flora such as Staphylococcus epidermidis in several specimens with proof of mecA without the detection of a true pathogen. In addition, the presence of resistance markers does not necessarily imply a de facto resistance of the bacterium. Nevertheless, Unyvero also failed to detect important gene products conferring resistance to broad-spectrum antibiotics, as in the two cases of 3MRGN and one case of 4MRGN and MRSA each.

The analysis of S. pneumoniae findings was hampered by the fact that the Unyvero P50 application was unable to differentiate it from other members of the Streptococcus mitis group, which was resolved in later generations of the cartridge. Therefore, we decided not to analyze the findings with S. pneumoniae.

Another drawback of our study was the lack of clinical data, especially the information on antimicrobial therapy prior to the diagnostic procedure, which has been shown to influence discrepancies between culture and PCR findings in other studies [11], and details concerning severity of disease.

At the time of study initiation, not all bacteria relevant to pediatric pneumonia were covered by the Unyvero panel. During the study and after it was completed, the manufacturer further developed the cartridges, re-labeled as P55 and, later, as HPN, and by the end of December 2016, the panel detected additionally Mycoplasma pneumoniae, Citrobacter freundii, and Klebsiella variicola; Enterobacter spp. have been subdivided into Enterobacter cloacae complex and Enterobacter aerogenes; in addition, other important resistance genes have been incorporated [40]. It would be advantageous to include other organisms, e.g., B. cepacia, a pathogen frequently encountered in patients with cystic fibrosis, which was also detected in one of our specimens by culture, or Mycobacterium tuberculosis, a major global killer which has been increasingly found also in German children in the last years, not only due to the influx of refugees [41].

Only one Unyvero test had to be excluded from our analysis due to complete failure of all PCR chambers. Although we did not have to exclude any of the partially invalid results, since no chamber failure affected the pathogens proven by culture, we still regard the partial invalidity rate of 26.6% as more than worrisome. In one of the early studies with the prototype system, Schulte et al. [22] were confronted with as many as 34.3% invalid results. Another German study group [23] had a rate of complete failure of 10% and partial failure of 30%, while Jamal et al. [21] had only 6.1% invalid runs. Lately, Personne and colleagues from the UK [24] reported an invalidity rate of 12.6%, which bespeaks an improvement, but is still quite high.

Conclusions

In conclusion, our data support that Unyvero might represent a feasible additional tool for rapid pathogen detection in suspected cases of pneumonia, especially in hospitalized children and neonates in whom tracheal aspirates or BAL fluids can be obtained. Feasibility must be reassessed in patients undergoing antimicrobial therapy prior to analysis. The sensitivities for Gram-positive bacteria and resistance markers need to be improved substantially, and the concern of invalid results has to be addressed as well. Further studies are needed to confirm our results and correlate with clinical parameters, but also investigate the application’s potential impact on decision-making policies and economic aspects.