Primary prevention of rheumatic fever (RF) and rheumatic heart disease (RHD) requires diagnosis and antibiotic treatment of group A β haemolytic streptococcal (GABHS) pharyngitis;^1,2 however, this approach may not be easy to implement in low and middle income countries. In these settings, RHD is still a major public health problem and is the leading cause of cardiovascular morbidity and mortality in children and young adults.^3,4

The World Bank’s main criterion for classifying economies is gross national income (GNI) per capita. Economies are divided according to 2003 GNI per capita. The groups are: low income, $765 or less; lower middle income, $766–3035; upper middle income, $3036–$9385; and high income, $9386 or more.⁵

In high income countries, RF and RHD have been largely controlled since the 1950s. This dramatic decline is attributed partly to antibiotic treatment of streptococcal pharyngitis and partly to improvement in living standards. However, occasional resurgence may occur, as was seen in certain parts of the USA in the mid-1980s.^6,7

In contrast to high income countries, the incidence of RF in low and middle income countries is approximately 5 per 100 000 per year. RHD is a major public health problem and is the leading cause of cardiovascular morbidity and mortality in children and young adults. The prevalence of RHD ranges in low and low-middle income countries from 1.0 to 10 per 1000. The World Health Organisation estimates that approximately 12 million people are affected by RHD/RF globally per year, resulting in about 40 000 deaths annually.^3,4,8 Besides the enormous burden of medical and surgical costs, these illnesses cause hardship to patients and their families, with repeated hospitalisations, disability, and premature death. Our study took place in two lower-middle (Brazil and Egypt) and one middle-income (Croatia) countries with varying population demographics (table 1).⁵

Studies from the United States and Costa Rica have suggested that providing penicillin treatment of streptococcal pharyngitis decreases the frequency of RF and RHD.^9,10 It is generally recommended in North America that diagnosis of streptococcal pharyngitis in children should be confirmed by laboratory testing before treatment.^1,2 However, in many low and middle income countries, bacterial culture and rapid tests are too costly or not feasible, and clinicians must assess the probability of GABHS pharyngitis using clinical assessment only. A variety of studies in North America show that individual signs and symptoms of GABHS pharyngitis are not specific enough to make a confident clinical diagnosis, since the signs overlap with other aetiologies.^11,12 Therefore, clinical decision rules using combinations of selected signs and symptoms have been developed to assist the diagnosis of GABHS pharyngitis in adults and children.^13–16 A few of these clinical decision rules have been validated through studies in the USA and other high income populations, but have not been validated in less developed regions.^17,18

The World Health Organisation (WHO) has two published recommendations for presumptive therapy of GABHS pharyngitis in the absence of microbiological data. The WHO Cardiovascular Disease Program has a list of common signs of GABHS, but does not specify a specific rule for management decisions.³ The WHO Acute Respiratory Infection (ARI) control programme¹⁹ and the WHO IMCI Adaptation Guidelines²⁰ suggest a clinical decision rule for children under 5 years of age; acute streptococcal pharyngitis should be suspected and presumptively treated when pharyngeal exudate plus enlarged, tender cervical lymph nodes are found. A prospective study of 451 children aged 2–13 years in Egypt found this rule to have high specificity (94.0%), but low sensitivity (16.0%), not detecting 84.0% of ill children who had GABHS positive throat cultures.¹⁶

This prospective study was carried out to assess the performance of the WHO clinical decision rule for GABHS pharyngitis in three low and middle income countries.

METHODS

Children were enrolled in three urban paediatric outpatient clinics from September 2001 to August 2003 in Rio de Janeiro (Brazil), Cairo (Egypt), and Zagreb (Croatia). In all three sites, children age 2–12 years presenting to participating outpatient clinics with complaints of cough, rhinorrhoea, sore or red throat were consecutively enrolled unless they reported one of the following: oral antibiotic use within the preceding three days or intramuscularly administered antibiotics within the 28 days prior to the clinic visit; history of previous rheumatic fever or rheumatic heart disease; or presence of another illness requiring hospitalisation. These exclusion criteria were designed to exclude children who were taking antibiotics, and who therefore may have had a modified clinical presentation of streptococcal pharyngitis or be carriers, and those children who were being medically managed such that these children may not have represented the target population.

For the purpose of this analysis, we included only children who presented with complaint of sore throat, to minimise the number of culture positive GABHS carriers. All sites used a common study protocol with standard forms for data collection, translated into the local language. The study protocol was approved by both local and national institutional review boards at each of the clinical sites, the World Health Organisation in Geneva, and the Committee on Human Research at the Johns Hopkins Bloomberg School of Public Health. Informed consent was obtained from the accompanying parent or guardian and child assent was obtained from all participating children age 5 and older. After enrolment, demographic data were recorded and a physical examination was performed by a study physician. Specimens were processed locally in the microbiology laboratories of participating hospitals. Laboratory staff in each site were trained using the WHO Manual for Laboratory diagnosis of group A streptococcal infections.²¹ Site visits were made to each participating laboratory to ensure that standard methods were being used. In all sites, throat cultures were obtained and plated onto 5.0% sheep blood agar plates, incubated at 37.0°C, and examined at 24 and 48 hours for the presence of β haemolytic streptococci and confirmed by bacitracin.²¹ For this study, children in these clinics with a positive throat culture for GABHS were considered to have streptococcal pharyngitis.

In all sites, we conducted a standardisation exercise with participating clinicians to minimise inter-rater variation in identification of clinical signs. The exercise consisted of clinical photographs of pharynges, which were displayed on a computer monitor or a projection screen, and written definitions of common signs of pharyngitis. Study physicians were asked to mark forms containing a list of physical signs, and asked to record each sign as either absent, present, not applicable, or unknown for each clinical photograph. A discussion followed the exercise regarding the correct “findings” for each photograph.

χ² and Student’s t tests were used to assess differences in patient characteristics among sites and age groups (Stata 7.0 statistical software, College Station, TX).²²

For comparison, we calculated a number of summary statistics to assess the predictive performance of the WHO clinical decision rule (presentation with both pharyngeal exudate and enlarged, tender cervical lymph nodes). We used sensitivity (the “true positive rate”) and specificity (“true negative rate”) as the main performance measure. The main advantage of using these measures is that the statistics do not change as the prevalence of disease changes in the population. We also assessed performance through the positive predictive value (PPV) and the negative predictive value (NPV). PPV is the proportion of people with a positive test result who actually have the disease. NPV is the proportion of those with a negative result who do not have the disease). These measures are useful in knowing how many of the patients who test positive (or negative) have (or do not have) the disease; however they have a significant limitation. Predictive values change as the prevalence of disease changes in a population; therefore, it is difficult to compare how the rule performs in different populations with different levels of disease prevalence.²³

RESULTS

Patient characteristics

A total of 2255 children met the enrolment criteria and participated in the study. Of these, 1810 had complaint of sore throat and were included in this analysis. Patient characteristics of the 1810 children presenting with complaint of sore throat varied among countries (table 1). Children recruited in Egypt were younger (mean age 4.9 years) than those recruited in Brazil (mean age 5.5 years) and Croatia (mean age 6.4 years). These differences were statistically significant (Student’s t test, p ⩽ 0.001). The proportion of children below the age of 5 varied among countries: 42.4% in Brazil, 32.0% in Croatia, and 52.8% in Egypt (χ² = 34.5, p ⩽ 0.0001) The proportion of female patients also varied between countries: 50.3% in Brazil, 42.0% in Croatia, and 50.3% in Brazil (χ² = 4.47, p = 0.106). The proportion of children with a positive GABHS culture also differed statistically between countries: 24.6% in Brazil, 42.0% in Croatia, 27.7 in Egypt (χ² = 20.3, p ⩽ 0.0001).

Evaluation of WHO clinical decision rule

In children under the age of 5 years, the WHO clinical decision rules sensitivity for presumptive diagnosis of GABHS pharyngitis was low for all sites. It was 3.6% in Croatia and 4.6% in Egypt. In Brazil the WHO clinical decision rule did not correctly identify a single child with culture proven GABHS. In all three sites, there was no statistically significant difference in the sensitivity of the rule between age groups (above and below 5 years of age). In both age groups, sensitivity was consistently low (3.6–8.5%) and specificity was uniformly high (93.8–97.4%).

Positive predictive value (PPV) varied by age group. In children under 5 years of age, PPV values were 0.0% in Brazil, 33.3% in Croatia, and 29.6% in Egypt; in children 5 years and older PPV was higher: 57.1% in Brazil, 44.4% in Croatia, and 40.0% in Egypt (table 2).

Figure 1 displays both sensitivity and specificity of the WHO clinical decision rule in four countries for children ages 2–12 years, and of several published reports of other clinical decision rules for streptococcal pharyngitis.^{15,16,19,24–26}

DISCUSSION

This is the first multiregional validation of the WHO clinical decision rule. The development of a proposed clinical decision rules has three stages: (I) development of the clinical decision rule; (II) initial validation in a clinical setting; (III) validation in new settings with different patients and clinicians. It is not clear how the WHO clinical decision rule was developed or derived (stage I).¹⁹ We have carried out a prospective stage III multiregional validation of the WHO streptococcal pharyngitis clinical decision rule, which confirms the data from a stage II study carried out in Egypt.¹⁶

Although the same clinical enrolment criteria were used in all sites, there were a number of differences observed in patient characteristics (table 1), including percentage of patients with positive GABHS throat culture. Sensitivity and specificity were used as the main measure of the performance of the rule, since these are unaffected by prevalence of disease. Positive predictive value (PPV) and negative predictive value (NPV) were also assessed because these indices are of interest to clinicians, even though they vary with prevalence.

There were several limitations in our study. One limitation concerned misclassification. We used the WHO recommended bacitracin disc method for presumptive identification of GABHS.²¹ Colonies of group A streptococci are often indistinguishable from those of other groups of β haemolytic streptococci, especially groups C and G, and serological methods are required for definitive identification. Therefore, it is possible that a percentage of those patients who were presumptively diagnosed with GABHS were colonised with groups C or G streptococci.

Another limitation of the study was that it is possible that some culture positive patients included in our analysis were GABHS carriers who did not have true infection. It is estimated that approximately 10.0–30.0% of patients who have a positive throat culture for GABHS may be carriers.^2,27,28 A positive throat culture for culture proven GABHS without serologic confirmation (raised or rising titres of antistreptolysin O) does not distinguish between a carrier state and acute GABHS infection. Serologic testing is expensive and logistically impractical for GABHS diagnosis and therefore is rarely used. In order to minimise inclusion of these carriers in the study, we excluded those patients who did not complain of sore throat on presentation.

We designed the study to assess the performance of the WHO clinical decision rule for GABHS pharyngitis in wide range of children who presented to paediatric clinics with sore throat. This clinical decision rule was developed for children under 5 years of age; however, it is the only specific guideline given by WHO for antibiotic treatment of streptococcal pharyngitis in children. We therefore have analysed the performance of the guideline in children below 5 years, and older children. As expected, the characteristics of the patients (age, sex, and prevalence of GABHS) varied among countries. In none of the sites did the WHO rule perform well in terms of sensitivity, and sensitivity was low in both age groups (below 5 years, and 5 years and older). This rule was relatively specific in all countries and age groups. In comparison to previously published clinical rules, the WHO clinical decision rule appears to exhibit a substantially lower sensitivity and higher specificity (fig 1).^13–16,19 Due to its very low sensitivity, the use of this decision rule in an area where rheumatic fever and rheumatic heart disease are common may not be appropriate either for individual patients or for public health policy.

In countries where rheumatic fever and rheumatic heart disease are still important health burdens, physicians are likely to favour increased sensitivity, to avoid missing a true case of streptococcal pharyngitis. Knowing the local prevalence of rheumatic fever and rheumatic heart disease in a population, clinicians and policy makers will have to judge the best balance between potential over-treatment (higher sensitivity) and under-treatment (higher specificity) of GABHS pharyngitis.

Conclusion

The performance of the WHO clinical decision rule for GABHS pharyngitis was prospectively assessed in clinic patients from both low and middle income countries. In these populations, the current WHO guideline has a high specificity but low sensitivity, and misses 91.5–100% of all children who have culture proven GABHS pharyngitis who would not be treated. Our data suggest that the characteristics of the current WHO clinical decision rule are not ideal for the low resource regions with high rheumatic heart disease incidence for which it is designed.

In regions with limited resources and high rates of rheumatic heart disease, there is a need for a modified clinical decision rule with higher sensitivity and a locally defined adequate specificity.