Article Text
Abstract
Objective To identify the best clinical decision rules (CDRs) for diagnosing group A streptococcal (GAS) pharyngitis in children. A combination of symptoms could help clinicians exclude GAS infection in children with pharyngitis.
Design Systematic review and meta-analysis of original articles involving CDRs in children. The Pubmed, OVID, Institute for Scientific and Technical Information and Cochrane databases from 1975 to 2010 were screened for articles that derived or validated a CDR on a paediatric population: 171 references were identified.
Setting Any reference including primary care for children with pharyngitis.
Data extraction The methodological quality of the articles selected was analysed according to published quality standards. A meta-analysis was performed to assess the statistical performance of the CDRs and their variables for the diagnosis of GAS pharyngitis.
Primary outcome measure The main criterion was a false-negative rate in the whole population not any worse than that of a rapid diagnostic test strategy for all patients (high sensitivity and low negative likelihood ratio).
Results 4 derived and 12 validated CDRs for this diagnosis in children. These articles involved 10 523 children (mean age, 7 years; mean prevalence of GAS pharyngitis, 34%). No single variable was sufficient for diagnosis. Among the CDRs, that of Joachim et al had a negative likelihood ratio of 0.3 (95% CI 0.2 to 0.5), resulting in a post-test probability of 13%, which leads to 3.6% false-negative rate among low-risk patients and 10.8% overall, equivalent to rapid diagnostic tests in some studies.
Conclusions The rule of Joachim et al could be useful for clinicians who do not use rapid diagnostic tests and should allow avoiding antibiotic treatment for the 35% of children identified by the rule as not having GAS pharyngitis. Owing to its poor specificity, such CDR should be used to focus rapid diagnostic tests to children with high risk of GAS pharyngitis to reduce the antibiotic consumption.
this is an open-access article distributed under the terms of the creative commons attribution non-commercial license, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. see: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.
Statistics from Altmetric.com
Article summary
Article focus
-
Pharyngitis is a frequent diagnosis in children leading to excessive antibiotic treatments by the inability to distinguish a bacterial from a viral cause of the disease despite the availability of rapid diagnostic test, too rarely done in children with pharyngitis.
-
Several studies have analysed the performance of clinical signs and developed clinical decision rules for the diagnosis of group A streptococcal (GAS) pharyngitis in children.
-
Through a meta-analysis, this article focus on the methodological quality of these studies and on the value of clinical signs and decision rules that could be helpful to focus rapid diagnostic test to the children with higher risk of GAS pharyngitis.
Key messages
-
No symptom considered alone is predictive enough of GAS pharyngitis.
-
Some clinical decision rules are performing as well as some rapid diagnostic test to exclude GAS pharyngitis in children, but are not performing enough for the positive diagnosis of GAS pharyngitis, which can lead to a still important antibiotic prescription level.
-
Therefore, clinical decision rules could be used to focus rapid diagnostic tests to children with high risk of GAS pharyngitis to reduce the use of antibiotic.
Strength and limitations of this study
-
Meta-analysis of all relevant articles, from 1975 to 2010 that analysed the performance of clinical signs and derived or validated clinical decision rules for the diagnosis of GAS pharyngitis in children.
-
A decision rule that performed as well as the most rapid diagnostic tests to rule out GAS pharyngitis in children was identified, but has not been validated until now.
-
Comparison between studies was limited by methodological weaknesses and heterogeneity between patients.
Introduction
Acute pharyngitis is one of the most common diseases in the world. It is diagnosed annually in 11 million patients in US emergency departments (ED) and other outpatient settings1 and is responsible for more than 140 physician office visits and 96 antibiotic prescriptions per 1000 US children under 15 years of age.2 The group A streptococcal (GAS) form is identified in 20–37% of children with pharyngitis.3 ,4
The priority over the past 50 years has been the prevention of GAS complications such as local suppurations, invasive infections and acute rheumatic fever (ARF). These complications are rare in industrialised countries; however, among children treated with antibiotics in GAS pharyngitis, less than 1% have suppurative complications,5 3/100 000 have invasive infections6 and 0.08–0.15/100 000 have ARF.7 ,8 ARF rates were declining even before the use of antibiotics because non-rheumatogenic types of streptococci were replacing the rheumatogenic types.9 The prevention of these complications, however, has induced large-scale prescription of antibiotics, which in turn might induce drug side effects and the emergence of multidrug-resistant organisms owing to pressure on the ecosystem.10
National guidelines are different from one country to another.11 To optimise the use of antibiotics, in 2012 the Infectious Diseases Society of America (IDSA) recommended the use of pharyngeal swabs to take samples for bacterial cultures or rapid diagnostic tests (RDTs) because the clinical features alone do not reliably discriminate between GAS and viral pharyngitis.12 These recommendations have changed some medical practices, but adhesion remains partial.13 Although diagnostic performances of RDT are good (sensitivity (Se), 85–90%, specificity (Sp), 90–100%),14 ,15 their use is still not widespread,16 they are offered to less than 50% of patients with pharyngitis17 and antibiotic prescriptions for children with pharyngitis remain excessive in industrialised countries.2 Moreover, RDTs are not recommended in practice in all settings internationally.18 Clinical decision rules (CDRs) have been proposed to help physicians decide whether or not the patient needs further tests (RDTs or culture) or direct antibiotic treatment without further testing. The IDSA recommends the use of such CDRs.12 Although several authors have suggested CDRs for children,19–24 most of these have been validated only partially.25–36
The aim of this study is to conduct the first systematic review, including a meta-analysis, of these CDRs and their variables for the diagnosis of GAS pharyngitis in children, to identify CDRs not any worse than that of an RDT strategy.
Methods
Search strategy and study selection criteria
This systematic search and quality assessment of studies were performed independently by FLM and FD in August 2010. To identify eligible original articles, we searched four electronic databases: Medline via PubMed, Institute for Scientific and Technical Information (INIST) at article@inist, database now accessible at http://www.Refdoc.fr, the OVID library at http://ovidsp.ovid.com/ and the Cochrane library. In the Medline search, we used the medical subject heading terms ‘pharyngitis’ (MeSH, restricted to major topic) and ‘predictive value of tests’ (MeSH), separated by the Boolean operator AND. Limits were set to specify ‘human’ as the species, ‘all child’ as the age and year of publication from 1975 to 2010, without limits on language of publication. In the other databases, only the MeSH term ‘pharyngitis’ was used and less limits to broaden the research: in INIST via Refdoc, we used the terms ‘pharyngitis’ and ‘children’ from 1975 to 2010; in OVID, we used the terms ‘pharyngitis’, ‘children’ and ‘sensitivity’ with limits set to specify ‘clinical medicine’ as journal subset, and year of publication from 1975 to 2010; in the Cochrane library, we used the term ‘pharyngitis’ alone without limits of dates.
The study selection criterion was the presence of original data used to derive or validate a CDR for predicting GAS pharyngitis in a paediatric population. We reviewed the titles of all articles identified by electronic searches and then the abstracts of those that appeared eligible. Related articles and references in the articles that met the selection criterion were examined to identify references that our electronic research might have missed. Eligible articles were fully reviewed.
Quality criteria for the CDR derivation and validation studies
The quality of the selected articles was determined by applying the methodological standards of Wasson et al37 and Laupacis et al38 Two of the authors (FLM and FD) separately screened each article for the 10 criteria enlisted below. Each criterion applied to GAS pharyngitis was split into 1–4 items (one point per item), as detailed below. Derivation studies could have up to 24 items and validation studies 21. The criteria were:
(1) The outcome for the selected articles was GAS pharyngitis. It should have been defined and diagnosed with the gold-standard, a throat culture. The culture technique should have been specified. The test used as the gold-standard should have been assessed blinded, without the knowledge of the value of the predictive variables. (2) The predictive values used in the studies that derived each CDR should have been exhaustively identified and well defined, to facilitate its reproducible use. The choice of the variables should have been explained, and the potentially important variables not included should have been mentioned. The studies that validated CDRs should have used the predictive variables as listed and defined in the derivation. Analyses should have been performed blinded to the outcome. (3) Important patient characteristics should have been described, for example, age, sex ratio and any characteristics that might cause the predictive value to differ within the cohort of patients, such as the prevalence of GAS pharyngitis. (4) The study site should have been specified, including the medical setting and the country. (5) The statistics used to derive the CDRs should have been described and justified. The authors should have assessed the possibility that the logistic regression model overfitted the data.38 (6) The statistical performance of the CDRs should have been described. (7) The reproducibility of the predictive variables and of the CDR should have been assessed. (8) The study should have been prospective, and the CDR should have been fully validated, in accordance with recommendations39: derivation study, internal validation, external validation and prospective study of the rule’s impact on clinical behaviour. (9) The CDR should be clinically sensible, easy to use (simple and quick) and should suggest a course of action rather than a probability of disease. (10) The effects of clinical use should have been prospectively measured. This last criterion (impact of the CDR) was evaluated at point 8.
Main criteria of CDR performance
The aim of a CDR strategy is to identify a group of children at low risk of GAS pharyngitis to allow them to avoid antibiotic treatment for these patients and to propose an action (eg, RDT) for patients classified in the high-risk group. A strategy including a CDR was considered useful if it did not increase the false-negative rate in the overall population (high-risk and low-risk patients), compared to an RDT strategy for all patients (figure 1). The RDT strategy (median Se 89%, median Sp 96%) has a median false-negative rate of 11%.14 Therefore, our criteria for evaluating the performance of each CDR were an Se as good as that of RDTs and a probability of GAS pharyngitis in the low-risk group of patients <11%. This corresponds to a negative likelihood ratio (LR−) of 0.2 or less when the prevalence of GAS pharyngitis is 30%.3 ,4 In the literature, an LR− under 0.2 is considered useful38 and the median LR− for RDTs is 0.15.14
Statistical analysis
After the identification of the CDRs, the entire population was described, in percentages and 95% CI for dichotomous variables and means and ranges for continuous variables. The absence of the raw data prevented us from calculating the SD. The statistical performance of the variables and the CDRs was analysed for paediatric studies only and not in studies that included both children and adults. When possible, we focused on children older than 3 years,27 because younger children rarely have GAS pharyngitis.12
The meta-analysis of the variables included in the CDRs and their validations used the DerSimonian and Laird40 method. For the Se, Sp, positive and negative predictive values (PPV and NPV), we tested the heterogeneity between studies, applying the LR test. For the OR, positive LR (LR+) and LR−, we used Cochran's Q test. In analyses with significant heterogeneity or with four or more studies, a random effect model was used to assign the weight of each study. Pooled Se, Sp, PPV, NPV, LR+, LR− and OR with their 95% CIs were calculated for CDRs and their variables.
CDRs in the literature propose different courses of action according to the individual’s clinical risk of GAS pharyngitis. In the selected studies, four CDRs proposed a course of action based on three levels of probability of GAS pharyngitis: high risk (antibiotics), intermediate risk (culture and antibiotics if positive) and low risk (no culture and no antibiotics). One CDR proposed a course of action based on two risk groups,20 and two CDRs offered four or five risk groups without any courses of action.19 ,25 We chose to identify the CDRs with a useful LR– that would allow us to rule out GAS pharyngitis, as most second-generation RDTs do. Therefore we dichotomised each population into two groups: the low-risk group on one side and the intermediate and high-risk group on the other side (see online supplementary material).
Results
Search strategy results
After excluding duplicates, our search strategy identified 65 references from PubMed, another 89 from INIST, 8 from OVID and 9 from the Cochrane database (see flowchart, figure 2). Reading the titles and abstracts of these 171 references led us to exclude 150 articles that did not meet the inclusion criterion. Complete reading of the remaining 21 articles and reviewing their references, related articles and authors’ publications identified 15 additional relevant references. Of these 36 references, 18 were excluded because they did not report the derivation or validation of a CDR on a paediatric population. In the 18 articles that fulfilled the inclusion criterion, six studies derived CDRs19–24 and 12 validated them in children.25–36 Of these 18 studies, the article cited as the source from which the WHO CDR20 was derived did not provide details about it, and the CDR by Centor et al19 used for validation in children was derived on adult patients. These two derivation studies were thus screened for methodological quality, but were excluded from the meta-analysis. However, the studies that validated these two CDRs among children28–31 were included in the meta-analysis.
Patient characteristics
The 16 studies with data for children included 10 523 children. Eleven studies were conducted in industrialised countries and five in emerging countries. Nine studies were conducted in hospitals or clinics, six in paediatricians’ or general practitioners’ (GPs) offices, and one in GPs’ offices and an ED. Overall, the derivation studies that could be reviewed (n=4) included 963 children (mean number per study 241, range 90–356).21–24 All the validation studies (n=12) together included 9560 children (mean number per study 797, range 79–1848).25–36 The mean prevalence of GAS pharyngitis was 34% (median 34%, range 24–58%) and did not differ between the derivation and validation studies (33% vs 34%; p=0.54) or between industrialised and emerging countries (34% vs 33%; p=0.30). The children’s mean age was 7 years, 5.9 in the derivation and 7.2 in the validation sets, 5.7 in the emerging and 8.4 in the industrialised countries. The studies used different inclusion criteria: ‘pharyngitis’ (n=5),23 ,24 ,27 ,35 ,36 ‘suspected GAS pharyngitis’ (n=4),22 ,26 ,29 ,34 ‘sore throat’ (n=3),28 ,31 ,33 ‘new upper respiratory tract infection’ (n=2)21 ,25 and both ‘new upper respiratory tract infection’ and ‘sore throat’ (n=2).30 ,32
Methodological quality for derivation and validation studies
Overall, the derivation studies correctly followed a median of 65% of the quality criteria (range 13–83%). The derivation of WHO's CDR was not found. The validation studies correctly followed a mean of 69% of these quality criteria (range 43–86%; table 1).
One study used an RDT as the gold-standard,24 and two others used RDTs or throat culture.29 ,34 No derivation studies defined a predictive variable; three validation studies did so for at least one variable (ie, cervical lymph node,25 ,27 ,30 abnormal pharynx25 and exudate30), but 7/12 validation studies changed a variable (eg, tender node for node, fever ≥38°C for fever >38°C). All studies described the CDRs, although one modified it.36 No study specifically described whether assessments were blinded, but for validation studies, we considered that a prospective study based on the culture result and without any RDT validated this item. One derivation study simplified a rule without reconducting the statistical analyses.24 Only one study was retrospective.34
Performance of the variables
The CDRs considered 17 variables in all, most frequently lymph nodes, exudate, age, fever and cough. The online supplementary materials include a table describing the types of variables by CDR and the details of the CDRs. OTable 2 presents the meta-analysis of the statistical performance of these variables. ‘Node >1.5 cm’, ‘sore throat’ and ‘no diarrhoea’ each had an LR− under 0.5. The Se of these three variables exceeded 0.81, and their NPV exceeded 0.72. The statistical performance of ‘Node >1.5 cm’ was not reproducible with the other ‘node’ variables. ‘Scarlatiniform rash’ had the highest LR+ (4.7) and OR (4.8). All other LR+ were less than 2.
Performance of the CDRs
After meta-analysis of the validation studies, three rules had high Se (99%, 95% and 88%) and NPVs (87–88%). However, the rules of McIsaac et al and Attia et al were not discriminative (Sp, 14% and 4%), and were negative for only 10% and 3% of the population, respectively (Otable 3). These rules were therefore not useful in clinical practice. The rule tested by Joachim et al had one of the best LR− (table 3), with a value of 0.3 (95% CI 0.2 to 0.5), which should help clinicians to rule out the diagnosis of GAS pharyngitis. Application of this CDR brought the probability of GAS pharyngitis down from 34% to 13% when the score of the CDR is negative (figure 3). The rule of Joachim et al also had the best performance, with an Se of 88%, a post-test probability of 13% in 28% of the low-risk patients, and an Sp of 35% (95% CI 30 to 40). This rule leads to a 3.6% false-negative rate in this low-risk population and 11.5% overall with an RDT strategy for the intermediate and high-risk patient groups and on the assumption that the RDT Se was 89%.
Discussion
The large number of studies and CDRs proposed for the diagnosis of GAS pharyngitis is the evidence of physicians’ desire to improve their management of this common disease and to limit antibiotic prescriptions, bacterial resistance and costs. Our study shows how difficult it is to develop and validate an effective and useful CDR. We identified 16 articles that described the derivation or validation of seven CDRs for the diagnosis of GAS pharyngitis in children. The meta-analysis confirmed, as others recently,41 that symptoms alone were not sufficient to rule out this diagnosis. Examination of the statistical performance of the variables included in the CDRs showed that none had a significant positive (>5) or negative (<0.2) LR.42 Two CDRs brought the post-test probability of GAS pharyngitis to around 10%.22 ,24 Only the CDR of Joachim et al was considered useful for clinical use to exclude GAS pharyngitis.
The poor performance of each of these variables requires comment. It might be owing to the low Sp of some signs (such as rhinorrhoea and cervical nodes), their subjectivity in children (sore throat) or a lack of definition. For these reasons, several variables might have been recorded differently from study to study and possibly within some studies. Because the individual variables predict GAS pharyngitis so poorly, researchers have suggested combining potential predictive variables within a CDR. Our systematic review of standards for the derivation of CDRs,37 ,38 however, shows that none of the studies followed all of the methodological quality criteria, in particular, the studies that derived CDRs did not. The construction of two CDRs was not available for methodological analysis.20 ,25 ,43 A rule that proposed an empirical simplification of the Breese score, without following any methodological standards was not included.44 Two other rules, not specifically derived for children,19 ,21 have nonetheless been used for validation in a paediatric population,28 ,29 ,32–35 despite the methodological requirement that rules be applied only in populations with the same characteristics as those used in the derivation sets.39 We also identified statistical biases. When a CDR is derived on a population, the validation set should not include members of the derivation set.21 Moreover, the logistic regression model of multivariate analyses in some studies might have been overfitted.19 ,23 Finally, the validation of a CDR may entail its refinement,24 ,36 which in turn requires a new validation. The CDRs with the lowest LR− in our meta-analysis were those of Attia et al22 and Joachim et al,24 which brought the post-test probability of GAS pharyngitis down to 9% and 13%, respectively. Nonetheless, the CDR by Attia et al was validated only once36 and was not discriminative for clinical practice. The rule developed by Joachim et al performed the best but has not been externally validated yet and requires the collection of nine variables for its application, which may limit its use in practice. Since CDRs were more useful than individual symptoms, they might help the clinicians who do not use RDTs in ruling out the diagnosis of GAS pharyngitis. The IQR of LR− for second-generation RDTs varies from 0.07 to 0.19.14 Thus the probability of GAS pharyngitis would have been reduced from 34% to a post-test probability of less than 10% for most RDTs.14 ,15 Compared to this full RDT strategy, the CDR of Joachim et al leads to a maximum 11.5% false-negative rate globally: 3.6% in the low-risk group of patients (28%) and 7.9% in the intermediate-risk and high-risk group (72%), if we assume an RDT strategy with 89% Se (probably an underestimate in this group). Nevertheless, none of the CDRs included in this study reached the level of performance required to bring both the probability of GAS pharyngitis and the risk of a false-negative test to less than 10%, as most RDTs do.
Our study has some limitations. Because of the lack of access to individual data, except from two authors who provided the complete set of data from their derivation study,23 ,24 we could only perform a meta-analysis of the pooled data. Moreover, the populations involved in our analysis were heterogeneous and difficult to compare. These differences concerned (1) the objective of the study, since some studies sought to validate a CDR while others tested RDTs35 or serological titres28; (2) the inclusion criteria, which differed between CDRs and even within the same CDR and (3) the mean age of the patients, which might influence the prevalence of the disease and the type of symptoms.27 The prevalence of the disease varied and could double between studies, as a result of differences in patients’ ages31 or study sites or because of a short study period when GAS might be more or less prevalent.19 ,22 ,25 Although prevalence did not influence Se, Sp or the LRs of the variables, it might influence the choice of variables for building the CDR, especially when methodological standards are not adhered to closely. A spectrum bias is possible if reference standards were not performed in all patients of the included studies. Variables might be defined differently between studies of the same CDR, and one CDR might suggest a different course of action in different studies.36 Another important variation may come from the definition of the disease (pharyngitis), which was not provided in the studies and which varies between countries.12 ,45 We had to create artificial risk groups and courses of action for three CDRs, although they had not been derived for that purpose; our results thus cannot reflect exactly the performance of each risk group. Finally, a review was recently published on this subject but focused the literature research on the signs and symptoms of pharyngitis, when we focused it on CDRs. Findings were slightly different in terms of articles reviewed and not different in terms of performance of individual variables.41 The CDR by Attia et al was identified by their systematic research but not the one by Joachim et al.
Lastly, we must question whether physicians will use a CDR at all for a well known and usually banal disease. It might be useful for countries where the RDT use is not recommended in current practice.18 It might also well interest the 50% of physicians who do not use RDTs at all.13 ,16 ,17 It will do so only, however, if the CDR produces accurate results, is useful, is well validated and is easy to use. The heterogeneity of patients between studies might necessitate a validation and a comparison of available CDRs in a single paediatric population. Our results showed that one CDR has a performance that did not produce a false-negative rate for GAS pharyngitis higher than that of the RDT strategy.20 The rule has only 35% Sp; but its use could avoid about six millions of antibiotic prescriptions in American children (<15 years old) when considering that almost 20% of the 300 millions of people in the USA are under 15 and that 96/10002 receive an antibiotic for pharyngitis. However, an external validation in different resource settings may be warranted before generalisation. After validation, this CDR might help physicians focus RDTs on children at higher risk of GAS pharyngitis and therefore decrease antibiotic prescriptions for children in the low-risk group.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
- Data supplement 1 - Online supplement
Footnotes
-
Contributors Study concept and design, and study supervision were carried out by FLM, FD, IP and AM. Acquisition of data was carried out by FLM. Analysis and interpretation of data, and critical revision of the manuscript for important intellectual content were carried out by FLM, FD, AD, IP and AM. Drafting of the manuscript was carried out by FLM, FD and AM. Statistical analysis was carried out by FLM, FD and AD. Le Marechal and Dubos (guarantor) have full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Funding This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.