Figures
Abstract
Background
Dengue diagnosis is complex and until recently only specialized laboratories were able to definitively confirm dengue infection. Rapid tests are now available commercially making biological diagnosis possible in the field. The aim of this study was to evaluate a combined dengue rapid test for the detection of NS1 and IgM/IgG antibodies. The evaluation was made prospectively in the field conditions and included the study of the impact of its use as a point-of-care test for case management as well as retrospectively against a panel of well-characterized samples in a reference laboratory.
Methodology/Principal Findings
During the prospective study, 157 patients hospitalized for a suspicion of dengue were enrolled. In the hospital laboratories, the overall sensitivity, specificity, PPV and NPV of the NS1/IgM/IgG combination tests were 85.7%, 83.9%, 95.6% and 59.1% respectively, whereas they were 94,4%, 90.0%, 97.5% and 77.1% respectively in the national reference laboratory at Institut Pasteur in Cambodia. These results demonstrate that optimal performances require adequate training and quality assurance. The retrospective study showed that the sensitivity of the combined kit did not vary significantly between the serotypes and was not affected by the immune status or by the interval of time between onset of fever and sample collection. The analysis of the medical records indicates that the physicians did not take into consideration the results obtained with the rapid test including for care management and use of antibiotic therapy.
Conclusions
In the context of our prospective field study, we demonstrated that if the SD Bioline Dengue Duo kit is correctly used, a positive result highly suggests a dengue case but a negative result doesn't rule out a dengue infection. Nevertheless, Cambodian pediatricians in their daily practice relied on their clinical diagnosis and thus the false negative results obtained did not directly impact on the clinical management.
Author Summary
Dengue is a potentially life-threatening viral disease. Symptoms are often not specific hence the importance to confirm the diagnosis during the early stage of the disease. Nevertheless, until recently only specialized laboratories were able to confirm dengue diagnosis. The discovery of the NS1 protein as a marker of infection has allowed the development of point-of-care tests for a rapid diagnosis confirmation. These tests have previously been evaluated by laboratories, but their performances have never been assessed in field conditions. In this study we evaluated the performance of SD Bioline Dengue Duo kit when tests were performed by hospital laboratories staff in a dengue hyper-endemic country. We also assessed the impact of the test results on the clinical management decision. The combination of NS1 test with antibodies detection improved the performance, though discordances on IgM and IgG results were observed between the hospitals and the national reference laboratories. Physicians treated patients according to their clinical diagnosis and did not take negative results into consideration.
Citation: Andries A-C, Duong V, Ngan C, Ong S, Huy R, Sroin KK, et al. (2012) Field Evaluation and Impact on Clinical Management of a Rapid Diagnostic Kit That Detects Dengue NS1, IgM and IgG. PLoS Negl Trop Dis 6(12): e1993. https://doi.org/10.1371/journal.pntd.0001993
Editor: Jeremy Farrar, Oxford University Clinical Research Unit, Viet Nam
Received: August 6, 2012; Accepted: November 16, 2012; Published: December 27, 2012
Copyright: © 2012 Andries et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: The study was supported by the French Ministry of Research (http://www.enseignementsup-recherche.gouv.fr/). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
The World Health Organization estimates that 50 million dengue infections occur annually and approximately 2.5 billion people live in area at high risk of infection. These areas are located in tropical and sub-tropical regions in South East Asia, Africa, Eastern Mediterranean, Western Pacific, Central and South America. The number of reported cases increased approximately 30 times over the last 50 years [1] and this could be in relation to many factors including population growth, urbanization, failure to control mosquito vectors, etc. [2].
Dengue is a viral disease transmitted by Aedes mosquitoes, principally Ae. aegypti. Dengue virus (DENV) belongs to the family Flaviviridae, genus Flavivirus. There are 4 antigenically and genetically distinct serotypes (DENV-1, -2, -3 and -4). In human, the virus can cause a spectrum of illness ranging from asymptomatic infection or self-limiting influenza-like illness (dengue fever or DF) to life-threatening disease associated with vascular leakage, hemorrhage (dengue hemorrhagic fever or DHF), potentially leading to vascular shock (dengue shock syndrome or DSS).
There is currently no specific treatment available for dengue. An early diagnosis is nevertheless very important for efficient clinical management in order to cure or prevent life-threatening complications. In addition, accurate and early diagnosis directs clinical attention to warning signs of an evolution to severe forms and avoids unnecessary use of antibiotics. A range of serological and virological diagnostic methods are available but most of them require specialized laboratory equipment, experienced personnel and are time consuming which is not adapted for a field and point-of-care use. Serological diagnosis by ELISA or rapid diagnostic tests (RDTs) is technically easy to perform and provides fast results but requires most of the time paired sera to definitively confirm the diagnosis [1], [3].
Detection of the NS1 antigen in the blood is a recent and very popular diagnostic method. This viral protein is secreted in the blood and can be detected by ELISA or immunochromatographic tests from the first day of fever and up to 14 days after infection [4]–[7].
The purpose of this study was to evaluate a commercial rapid dengue diagnostic kit, the SD Bioline Dengue Duo device (Standard Diagnostic Inc., Korea), in particular in point-of-care applications, and to evaluate the impact of the results of this combined test on the clinical management decision. The SD Bioline Dengue Duo kit is composed of 2 tests designed to detect DENV NS1 antigen (first test) and anti-DENV IgM/IgG (second test) in serum, plasma or whole blood. The kit evaluation was double. Firstly, the use of the test in the field was for the first time evaluated during a prospective study in 2 Cambodian provincial hospitals. The results obtained in the hospital's laboratories were then compared with those reported with the same samples by a national reference laboratory at Institut Pasteur in Cambodia (IPC). We also investigated how the results of this point-of-care test designed to assist clinical management were perceived and subsequently incorporated into the clinical management decision of physicians from 2 hospitals during a dengue epidemic. Secondly, a more usual retrospective case-control evaluation against reference methods was performed at IPC in order to assess the kit performances in the context of a dengue-endemic South-East Asian country.
Materials and Methods
Patients' recruitment and samples collection for the prospective evaluation
Patients were enrolled in the pediatric wards of Kampong Cham and Takeo provincial hospitals during the 2011 dengue epidemic in Cambodia i.e. between June and October 2011. Patients presenting spontaneously to these hospitals or referred by health centers with a history of fever during the previous 7 days and at least one of the following symptoms: rash or severe headache or retro-orbital pain or myalgia or joint pain or bleeding, were examined by physicians who decided whether or not the child should be hospitalized. When the number of beds available was limited, priority was obviously given to the most severe cases. In each hospital, a maximum of 10 hospitalized patients, randomly selected, were enrolled weekly. Patient's information and clinical data were collected by physicians using a specific case report form and blood samples were taken at the time of hospital admission (early/acute specimen) and discharge (convalescent/late specimen). Patients with incomplete test kit results, missing blood samples and incomplete clinical records were excluded.
Panel of samples used for the retrospective laboratory evaluation
The panel used for the retrospective laboratory evaluation of the kit performances consisted of 157 samples collected in 2011 during the field prospective evaluation and tested negative or positive by the reference methods available at IPC completed with an additional 167 samples selected from IPC's dengue laboratory's biobank (samples collected between 2008 and 2010). Positive samples were selected in order to obtain an evaluation panel as balanced as possible in terms of DENV serotypes, day of collection after onset of fever (DAOF), anti-DENV antibodies titer and immune status (primary/secondary infections). Negative samples were selected from patients presenting with a non-dengue febrile illness and also from pregnant women.
Ethical aspects
For the field prospective evaluation, a written consent was signed by the children's legal representatives before enrolment. This study was approved by the Cambodian National Ethics Committee. The use of stored samples from IPC's biobank was also approved by the Cambodian National Ethics Committee.
Dengue diagnosis
The SD Bioline Dengue Duo kits were provided by Standard Diagnostics (Kyonggi-do, Korea) and tests were performed according to the manufacturer's instructions. For the prospective study, only acute blood samples were tested with the kit in hospitals as well as at IPC.
At IPC, laboratory diagnosis was based on RT-PCR, isolation of DENV after inoculation into mosquito cell lines, detection of anti-DENV IgM and measure of an increase of anti-DENV antibodies titer measured by hemagglutination inhibition assay (HIA) between acute and convalescent sera.
RT-PCR was performed after viral RNA extraction from acute serum samples using QIAmp Viral RNA Mini kit (Qiagen, Hilden, Germany). Either a conventional nested RT-PCR according to Lanciotti et al. [8] protocol and modified by Reynes et al. [9] or a real-time multiplex RT-PCR based on the technique developed by Hue et al. [10] was performed.
DENV was isolated on C6/36 cells and the virus serotype identified by immunofluorescence assay using monoclonal antibodies as described previously [11].
An in-house IgM capture Enzyme-Linked Immuno-Sorbent Assay (MAC-ELISA) was used to detect anti-DENV and anti-Japanese Encephalitis virus (JEV) IgM as describe previously [11]. A result was considered positive for dengue when the optical density (OD) was higher than 0.1 for the DENV IgM and when the OD of the anti-DENV ELISA was higher than the OD of the anti-JEV ELISA.
HIA followed the method described by Clark and Casals [12] adapted to 96-well microtiter plate. Primary or secondary acute dengue infection was determined by HI titer according to criteria established by WHO [13]. In brief, the patient was defined as having a primary infection when the convalescent serum had a HI titer ≤2560 associated with a fourfold rise of the titer between the acute and convalescent sera (collected with a time interval of at least 7 days). When the convalescent serum had an HI titer >2560, the patient was defined as having a secondary dengue infection.
All early samples were tested by PCR, viral isolation, IHA and MAC-ELISA whereas late samples were only tested by HIA and MAC-ELISA.
Confirmed and suspected dengue cases were defined according to WHO guidelines [1]. A confirmed case was defined by a RT-PCR and/or a culture positive result and/or an IgM seroconversion in paired sera and/or a fourfold antibodies titer increase measured by HIA in paired sera. A probable dengue infection was defined by an HI antibody titer >2560 in paired sera without a fourfold increase or IgM positive result in the acute serum [1].
At IPC, technicians were blinded for the results of the kit evaluated as well as for the results of gold standard tests. In hospitals, the staff performing rapid diagnostic tests was blinded for the results obtained with these tests as well as for the results of the gold standard assays.
Hospital case management
Each clinical record contained the complete medical data recorded at the time of admission and the complete follow-up of the patient during the hospitalization (temperature, blood pressure, pulse, diuresis, medical prescriptions, etc.) until discharge. These data were anonymized by the physicians for the purpose of the analysis.
Statistical analysis
Statistical analysis was performed using STATA version 11.0 (StataCorp, College Station Texas, USA). Significance was assigned at P<0.05 for all parameters and were two-sided unless otherwise indicated. Uncertainty was expressed by 95% confidence intervals (CI95).
For the prospective study, agreement between hospital's laboratories and IPC laboratory's data was measured by agreement percentage and Kappa (κ) coefficient.
For the prospective study, sensitivity and specificity obtained when tests were performed at hospitals were compared with those obtained at IPC with McNemar test. Positive and negative predictive values (PPV and NPV) were compared with Fisher exact test. For the retrospective laboratory study and for the analysis of medical records Fisher exact test was used.
During the retrospective laboratory evaluation, sensitivity was calculated according to infecting serotype, DAOF, immune status and antibodies profiles. Four different antibodies profiles were arbitrarily defined according to HIA and MAC-ELISA results: profile 1, low HI titer (<640) and negative MAC-ELISA; profile 2, low HI titer and positive MAC-ELISA; profile 3, high HI titer (≥640) and negative MAC-ELISA; profile 4, high HI titer and positive MAC-ELISA.
Results
Prospective evaluation of the SD Duo kit's performances
Characteristics of the study population.
A total of 162 patients were enrolled (100 patients in Takeo and 62 in Kampong Cham). The NS1 result of one patient and the IgM/IgG results of 4 others patients were not reported by hospitals. These 5 patients were therefore excluded. At IPC, the reference laboratory tests confirmed 85 dengue cases, 41 children were classified as probable dengue infection, and in 31 cases a dengue infection was excluded. Among the 126 patients with confirmed or probable dengue, 32 were classified as DF, 84 as DHF and 8 as DSS. Clinical, virological and demographical information of the population are summarized in Table 1. All the patients enrolled in this study survived and were discharged without complication or sequelae.
Comparison of results between hospital laboratories and Institut Pasteur's laboratory.
For the NS1 test, an agreement of 98.1% and a κ coefficient of 0.96 were obtained (Table 2). For the IgM/IgG tests, an agreement of 68.8% and a κ coefficient of 0.55 were observed (Table 3).
In the hospitals, the overall sensitivity, specificity, PPV and NPV of the NS1 test were 44.4%, 96.8%, 98.2% and 30% respectively and were 45.2%, 96.8%, 98.3% and 30.3% respectively at IPC. Results were not statistically different (Table 4).
For the combined test (antibodies and NS1) the overall sensitivity, specificity, PPV and NPV at the hospitals were 85.7%, 83.9%, 95.6% and 59.1% respectively whereas they were 94,4%, 90.0%, 97.5% and 77.1% respectively at IPC. Sensitivity was significantly higher at IPC (p-value = 0.002). Specificity, PPV and NPV were better at IPC but the differences were not statistically significant (Table 4).
The combination of all the tests significantly improved the sensitivity and NPV of the kit at hospitals (p-value sensitivity <0.001; p-value NPV = 0.001) as well as at IPC (p-values <0.001) with a non-significant decrease of the specificity (p-value at hospitals = 0.12, p-value at IPC = 0.5) and of the PPV (p-value at hospitals = 0.67, p-value at IPC = 1).
Retrospective laboratory evaluation of the SD Duo kit's performances
Samples description.
A total of 166 positive samples and 120 negative samples were included in the retrospective laboratory evaluation of the kit. Eighty five and 81 positive samples as well as 31 and 89 negative samples were obtained from the 2011's prospective study and from the IPC's biobank, respectively. The 41 patients defined as suspect dengue infection by the gold standard methods during the prospective study were also included in the retrospective study. Positive samples included 86 sera with a low HI titer (<640) and 80 sera with a high HI titer (≥640) (51.8% and 48.2%, respectively). The distinction between low and high HI titer groups was established during a preliminary comparative study between HI titers and SD Duo kit IgG results. In the high HI titer group, the correlation between both tests was over 70% which was considered as acceptable (data not shown). Of note, HIA does not only detect IgG but also other immunoglobulin isotypes and as such some low HI titers measured during the early phase of the infection may not contain or only very low quantities of anti-DENV IgGs. DENV serotype was identified in 87.3% of the positive samples: there were respectively 57, 35, 26 and 27 (34.3%, 21.1%, 15.7% and 16.3%) DENV-1, -2, -3 and -4 cases. Forty seven (28.3%) samples were collected 2 days after onset of fever or earlier, 67 (40.4%) between day 3 and day 4, 41 (24.7%) between day 5 and 6, and 11 (6.6%) after the 7th day of illness (Table S1). A total of 19 and 83 patients (11.4% and 50%) were classified as having primary and secondary infections, respectively.
Negative samples included sera obtained from patients infected by a Plasmodium vivax or P. falciparum (22/120, 18.4%), Orientia tsutsugamushi (13/120, 10.8%), Japanese encephalitis virus (16/120, 13.3%), Chikungunya virus (14/120, 11.7%), Hepatitis C virus (10/120, 8.3%), from patients presenting with meningo-encephalitis (6/120, 5%) or non-specific febrile illness of unknown etiology (30/120, 25%) and from pregnant women (9/120, 7.5%).
NS1 test.
The overall sensitivity and specificity of the NS1 test against the reference methods was 58.4% (97/166, CI95 = [40.5–66.0]) and 98.3% (118/120, CI95 = [94.1–99.8]), respectively (Table 5).
Sensitivity was significantly better for DENV-4 than DENV-2 (77.8% vs 48.6%, p-value = 0.01). No differences were observed with the other serotypes (Table 5). Sensitivity was also significantly better in primary than in secondary infections (89.5% vs 43.4%, p-value<0.001) (Table 5). When considering the various antibodies profiles, the analysis demonstrated that a high HI titer (profiles 3 and 4) was associated with a significant decrease in NS1 test's sensitivity (profile 1 vs 3, p-value = 0.005; profile 1 vs 4, p-value<0.001; profile 2 vs 3, p-value = 0.014; profile 2 vs 4, p-value<0.001). The presence of detectable IgM (profiles 2 and 4) did not seem to affect the sensitivity (profile 1 vs 2, p-value = 1; profile 3 vs 4 p-value = 0.185) (Table 6).
The test's sensitivity diminished significantly also when the interval of time between onset of fever and sample collection increased (overall p-value = 0.003). This value varied from 76.6% (36/47, CI95 = [62.0–87.7]) for samples collected before the 2nd day of illness to 45.5% (5/11, CI95 = [16.7–76.6]) for those collected after the 7th day of illness (Figure 1).
Evaluation of the sensitivity of NS1 test, IgM/IgG test and combination of the two tests depending on day of sampling after onset of fever.
Out of the 41 patients with only suspected dengue infection as defined by the reference diagnostic methods used, 12 (29.2%) tested positive by NS1 kit. Out of the 120 negative samples, two weak signals were recorded, one from a patient presenting with a febrile illness of unknown etiology and one from a patients with a JEV infection.
IgM/IgG tests.
The overall sensitivity and specificity of the IgM/IgG tests were 68.1% (113/166, CI95 = [60.4–75.1]and 95% (114/120, CI95 = [89.4–98.1]), respectively (Table 5).
The sensitivity was better with DENV-1 than DENV-4 but this difference was at the limit of significance (70.2% vs 48.2%, p-value = 0.051). No differences were observed with the other serotypes (Table 5). The sensitivity improved significantly when the interval of time between onset of fever and sample collection increased (overall p-value<0.001), from 40.4% (18/47, CI95 = [26.4–55.7]) for samples collected before the 2nd day of illness to 90.9% (10/11 CI95 = [58.7–99.8]) for those collected after the 7th day of illness (Figure 1).
The sensitivity was also significantly better in patients with secondary than primary infections (79.5% vs 42.1%, p-value = 0.003) (Table 5).
All the 41 patients with a suspicion of DENV infection tested positive either by IgM only (6/41, 14.6%), by IgG only (11/41, 26.8%) or by both IgM and IgG (26/41, 63.4%). In 3 non-dengue febrile cases and 3 anti-JEV IgM positive cases the kit gave non concordant results compared to our reference methods. In 2 non-dengue febrile cases both IgM and IgG were positives. In 1 JEV case the IgG test was positive while in 1 non-dengue febrile and 1 JEV cases, IgG test was weakly positive. In the last JEV case, the IgM test was weakly positive.
NS1 and IgM/IgG tests combination.
The overall sensitivity of the NS1/IgM/IgG combination tests was 94.6% (157/166 CI95 = [90.0–97.5]) and the specificity was 94.2% (113/120, CI95 = [88.4–97.6]) (Table 5).
The sensitivity did not vary significantly between the serotypes when compared all together (p-value = 0.868), or 2 by 2, but also not according to the immune status (p-value = 1) or the time interval between DAOF and sample collection (p-value = 0.8) (Table 5, Figure 1).
Impact of the RDTs results on clinical case management in Cambodia
The medical records of 129 patients (82.2% of all patients enrolled) were provided by the two hospitals and subsequently analyzed. All the 66 patients who tested positive for acute dengue infection using the IPC gold standard test were also clinically diagnosed by the physicians as dengue cases, with or without co-infection (63 and 3 patients, respectively). One patient with a laboratory-suspected DENV infection as well as two children who tested negative were clinically diagnosed as non-dengue febrile illness (Table 1).
All patients received a treatment based on WHO 2009 recommendations, i.e., intravenous fluid therapy with 0.9% saline, Ringer's lactate or Ringer's acetate with or without dextrose, paracetamol if fever and oral rehydration solution or other fluids containing electrolytes and sugar when possible. Patients in circulatory shock received dextran, O2 and blood transfusion when necessary. Twenty-nine patients (27.7%) also received antibiotics. The prescription of antibiotics was justified by the phisicians in the medical records of 11 patients because the following diagnoses: 4 dysenteric syndromes with suspicion of typhoid fever, 3 meningitis or meningo-encephalitis, 1 suspicion of nosocomial infection, 2 pharyngitis and 1 bronchiolitis. Among the 90 patients with a positive NS1 and/or IgM and/or IgG test, 17.8% (16/90) were treated with antibiotics. Out of 39 patients who tested negative by the RDT, 13 (33.3%) also received antibiotics. The comparison of antibiotic prescription between both groups was at the limit of significance (p-value = 0.067). There was no difference in the duration of antibiotic therapy between patients with a positive test and those with a negative test (p-value = 0.216).
Among the 16 positive patients who received antibiotics, only 7 (43.7%) had their antibiotic therapy stopped once the point-of-care kit tested positive for dengue. Among the 13 patients with a negative result who received antibiotics, 8 (61.5%) had their antibiotic therapy stopped once the test was performed. The decision to maintain or discontinue the antibiotic therapy was not affected by the result of the RDT (p-value = 0.338).
Discussion
Early management of patients with dengue infection is essential to ensure a favorable evolution of the disease and prevent the occurrence of severe forms. Until recently an early confirmed diagnosis was only achievable in specialized laboratories. The discovery of the NS1 protein as an early marker for DENV infection, especially in RDT format, now allows dengue diagnosis during the early phase of the disease, even in laboratories with limited equipments and human resources. Evaluations are required to ensure that these tests are suitable for diagnosis and clinical management or epidemiological surveillance and outbreak investigations. Different methodologies can be used: laboratory-based evaluations (or retrospective evaluations) and field evaluations (or clinical-based/prospective evaluations) [14]. Retrospective evaluations are easy to perform but tend to overestimate tests accuracy. Prospective evaluations allow determination of PPV and NPV with tests performed on patients in the real clinical settings. However, accuracy of diagnostic tests estimated by prospective evaluations could be biased due to imperfect gold standard in the prospective clinical setting. In our study we combined both prospective and retrospective evaluations. The retrospective part was added in order to better understand the results obtained in the field during the prospective study.
Since the two test kits of the SD Bioline Dengue Duo combo test do not give exactly the same information, the NS1 assay was initially assessed alone in the prospective as well as in the retrospective study. If a positive NS1 test can confirm a dengue diagnosis, this is not the case for IgM and IgG tests as the antibodies remain detectable for months and thus a positive result obtained on a single blood specimen is only suggestive of a dengue infection. Indeed, to confirm an acute dengue infection by serology, an IgM seroconversion or a four-fold increase of IgG antibody titers in paired sera must be demonstrated (which cannot be done with the RDT kit as result is only qualitative) [1]. By evaluating separately, but in parallel, the NS1 test and the serological kit, we estimated the ability of the test to both suggest and confirm a dengue infection.
During the prospective study, the sensitivity of the SD Bioline Dengue Duo NS1 when performed at the hospitals was only 44.5% to confirm dengue infections in children hospitalized for dengue-like illness during the epidemic season. The tests were carried out in laboratories equipped for routine medical biology. Out of the 127 patients included in the prospective evaluation, 70 (54.7%) had an HI titer ≥640 which could probably explains such a poor sensitivity. The retrospective study helps to understand why the sensitivity was limited. It suggested that the presence of high level of anti-DENV HI antibodies in the sample was a major factor for sensitivity decrease. Indeed, while a sensitivity >80% was obtained with samples containing no or low HI antibodies titer (<640), the sensitivity dropped to 37% when the HI titer was ≥640. Almost 86% of the samples with a high HI titer issued from patients with a secondary infection. Since HI titer reflects mainly IgG response, the poor sensitivity observed during secondary infections is probably directly linked to the high IgG titer. Similar observations were already made by other authors. In Vietnam, the same NS1 test demonstrated a sensitivity of 24.6% for samples positive for IgG by GAC-ELISA and a sensitivity of 77.3% in sera negative for IgG [15]. In Colombia, Osario et al. reported an even lower sensitivity (IgG negative: 65.6%, IgG positive: 15.6%) [16]. Of note, the methods used for IgG detection in these evaluations were all different and rather than giving the real performance of the kit, the data indicate a global trend to a lower sensitivity when IgG titers increase.
As others [16], [17], we observed that the sensitivity of this test decreased when the window of time between onset of fever and sampling increased. This was expected since the IgG titer also increased with the time. Finally, a higher IgG titer also characterizes the secondary dengue infections and the better sensitivity of the NS1 in primary infections was also already reported [15]–[17].
The performances of the NS1 test reported here as well as by other retrospectives studies are close to those observed with other commercial NS1 RDTs [15], [18]. A major value of the kit marketed by SD is the combination of the NS1 test with an anti-DENV antibodies detection kit. Indeed, the serological results improved the sensitivity by compensating for the loss of sensitivity usually observed with the NS1 test when used alone in the presence of specific anti-DENV antibodies. During the prospective evaluation, we demonstrated that the addition of IgM and IgG results to the NS1 data was only associated with a slight non-significant decrease of the specificity. However, this result should be interpreted with caution as the number of negative patients included was relatively small. In addition, the relatively low overall performance of the IgM/IgG test could well be partially due to imperfect gold standard tests. In the retrospective study, we did not observed any cross-reactivity with Chikungunya virus, Orientia tsutsugamushi or Plasmodium sp.. However when evaluating the SD Bioline Dengue Duo kit, Blacksell et al. [18] reported 12.2% (10/82) of cross-reactivity with Chikungunya virus, 12.5% (1/8) with Orientia tsutsugamuhi and 100% (1/1) with Plasmodium sp. When evaluating only the IgM part of the kit, Hunsperger et al. [19] reported around 35% of IgM cross-reactivity with malaria as well as some false positive results with leptospirosis, tuberculosis and West-Nile infections.
During the prospective evaluation, the PPV value of the NS1 test was 98.2%, suggesting that the probability to correctly confirm a dengue infection was very high when the test was positive. When the test was used in combination, the PPV decreased only very slightly (NS1/IgM: 96.9%; NS1/IgM/IgG: 95.6%). Consequently, the NPV observed when the tests were performed in the hospitals was only 29% for the NS1 test alone and 56.8% for the combination test. In other words, the probability of truly exclude a dengue infection when the tests were negatives was low. These PPV and NPV results should be regarded with caution as they depend on the dengue disease prevalence that can be extremely different in other contexts and epidemiological situations. In this prospective study, the prevalence of dengue infection was very high (80.3%, 126/157) because the evaluation was performed during the peak epidemic season and only involved dengue suspect patients. Observing high prevalence of dengue infections in suspect patients hospitalized is common in Cambodia (87.8% of average between 2000 and 2008) and in neighboring countries like Vietnam (86.2% during a DENV-4 epidemic in 2002) [20], [21].
On the samples collected during the prospective study, the comparison of the results of the tests performed by technicians in hospital laboratories or by health workers who did not receive any specific training for the use of the kits with the results reported by the staff of the national reference laboratory at IPC demonstrated a moderate agreement with the serological tests and an excellent agreement with the NS1 test. Indeed, 49 discordant results between the hospitals and IPC were observed with the IgM/IgG test out of which 34 (69.3%) were positive at IPC but negative at the hospitals while 13 (26.5%) were negative at IPC but positive at the hospitals. These discrepancies could be explained if the reading was made before the recommended 15 minutes (leading to false negative results) or after the correct time (leading to apparition of unspecific bands) or because of problems with the interpretation of weak signals (faint bands). To evaluate if the issue was the interpretation of the faint bands, these data were removed from the analysis and a better agreement percentage and Kappa coefficient were obtained (82.0% vs 68.8% and 0.73 vs 0.55). A problem of reproducibility could also have accounted for some of the discrepancies observed. Nevertheless, in the case of bad reproducibility an equal number of discrepancies should have been observed in each laboratory which was not the case in our study. Moreover all tests were from the same manufacturing lot. During a malaria RDTs evaluation, misinterpretation of weak signal in the field had already been reported [22]. It was also reported that health workers in the field tend to read the results before the time recommended by the manufacturer [22], [23]. Despite its relative ease to use, the performances of the IgM/IgG RDT are obviously partially person-dependent, hence the importance of providing specific training or at least very clear pamphlets which could guide the health worker in its interpretations and expose the risks of false results when the recommendations are not strictly followed. On the contrary a very good agreement was observed with the NS1 test since the bands in this immunochromatographic device almost always appear very clearly. As the RDTs have a significant cost, promoting the use of these kit does only make sense if the health workers can perform the tests in good conditions, which seems to be sometimes challenging in intensive care units and pediatric wards that are often unable to cope during peak epidemics. Knowing these constraints and limitations, the manufacturer should be encouraged to correct, if possible, the reading issues of the serological test. The outcomes of the patients who were wrongly tested negative by the kit was a matter of concern as RDTs are designed for rapid diagnostic and to assist physicians in their decisions. Dengue is a life-threatening disease that requires specific clinical care. The analysis of the medical records demonstrated that physicians ignored the negative results and followed their clinical instinct as all patients who tested negative by RDT received an intravenous fluid therapy which is recommended in patients with warning signs [1] but which is also often administrated in mild cases to prevent complications. Similar observations were also made in the context of malaria RDTs use. Between 54% and 85% of the patients with negative malaria RDT results were treated with anti-malaria drugs in Nigeria, Tanzania, Burkina Faso, Philippines and Laos [22]–[25]. There are probably several reasons that could explain that physicians did not consider the negative results obtained with the RDT: the habit to rely mostly on clinical intuition explained by a frequent limited access to laboratory tests, some mistrust against a new test, the difficulties to understand the kinetic of the immune response during dengue infection and the significance of NS1, IgM and IgG test results, a high confidence in clinical diagnosis when children present to pediatric wards with dengue-like symptoms during the epidemic season (especially since the national virological surveillance confirms usually more than 80% of the dengue clinical diagnosis) [20], the fear that a misdiagnosed dengue infection evolves towards a DHF or a DSS while these complications are pretty easy to prevent with simple clinical management, etc. The SD Bioline dengue Duo test could have a better utility in smaller medical care structures, like health care centers and dispensary where the proportion of dengue among all febrile diseases is lower (e.g., 12% of all the febrile episodes in Kampong Cham province, 2006–2008) [11] and where routine hematology (e.g., hematocrit, platelet count) that could help to orientate the diagnosis are not often available.
One of the advantages to perform a rapid confirmatory diagnostic of dengue in the context of febrile illness is to avoid the unnecessary use of antibiotics. In the context of Cambodia, it seems the RDT results did not have a significant impact on the decision to start or discontinue an antibiotic therapy.
In an endemic country, especially in the context of an epidemic, it seems that the sensitivity of the NS1 RDT alone is too low and that only positive results should be taken into consideration. Nevertheless, the performances of the combined kits are good and these kits appear to be a useful tool for the clinicians as they can quickly confirm the diagnosis of dengue and therefore contribute to the an optimal clinical management of the cases and avoid an unnecessary use of antibiotics or other drugs which is important in the context of a developing country with limited resources.
In conclusion, we observed that for a patient presenting with dengue-like symptoms in a dengue-endemic/epidemic region, a NS1 positive result obtained with the SD Bioline Dengue Duo kit confirms a dengue diagnosis, an IgM and/or IgG positive result highly suggests dengue infection but a negative result doesn't rule out a dengue infection. We have also demonstrated that the performances of the test in the field were lower than the ones obtained in the more experienced hands of technicians working in a national reference laboratory. This suggest that even for a point of care test theoretically designed to be used by untrained staff, there is still a significant improvement of the performance of the test to expect if a proper training and a quality assurance program can be implemented. With the time, the trust of the physician will probably increase if the accuracy of the test improves. In general, manufacturers should always bear in mind that the ultimate goal of the RDTs is essentially to be used as a point-of-care test or in support of epidemiological investigation and as such should be easy to use, stable at room temperature but also not posing reading difficulties unless they can provide proper training and organize quality programs. More prospective field evaluations are still necessary now to better assess the interest to use such point-of-care tests in the real conditions that justified their development and to address some of the questions and concerns raised by this study.
Supporting Information
Table S1.
Composition of the panel of samples used for retrospective evaluation. Description of the panel of positive samples used for the laboratory retrospective evaluation of SD Bioline Dengue Duo Kit.
https://doi.org/10.1371/journal.pntd.0001993.s001
(DOCX)
Author Contributions
Conceived and designed the experiments: ACA VD PB. Performed the experiments: ACA VD SO KKS VT BY PLT. Analyzed the data: ACA VD PB. Contributed reagents/materials/analysis tools: CN RH KKS VT PLT. Wrote the paper: ACA VD PB.
References
- 1.
WHO/TDR (2009) Dengue: guidelines for diagnosis, treatment, prevention and control. 3rd edition.
- 2. Gubler DJ (2011) Dengue, Urbanization and Globalization: The Unholy Trinity of the 21st Century. Trop Med Health 39 (4 Suppl) 3–11.
- 3. Guzman MG, Halstead SB, Artsob H, Buchy P, Farrar J, et al. (2010) Dengue: A continuing global threat. Nature Reviews Microbiology S7–16.
- 4. Young PR, Hilditch PA, Bletchly C, Halloran W (2000) An antigen capture enzyme-linked immunosorbent assay reveals high levels of the dengue virus protein NS1 in the sera of infected patients. J Clin Microbiol 38: 1053–1057.
- 5. Alcon S, Talarmin A, Debruyne M, Falconar A, Deubel V, et al. (2002) Enzyme-linked immunosorbent assay specific to Dengue virus type 1 nonstructural protein NS1 reveals circulation of the antigen in the blood during the acute phase of disease in patients experiencing primary or secondary infections. J Clin Microbiol 40: 376–381.
- 6. Wang SM, Sekaran SD (2010) Evaluation of a commercial SD dengue virus NS1 antigen capture enzyme-linked immunosorbent assay kit for early diagnosis of dengue virus infection. J Clin Microbiol 48: 2793–7.
- 7. Hu D, Di B, Ding X, Wang Y, Chen Y, et al. (2011) Kinetics of non-structural protein 1, IgM and IgG antibodies in dengue type 1 primary infection. Virol J 8: 47.
- 8. Lanciotti RS, Calisher CH, Gubler DJ, Chang GJ, Vorndam AV (1992) Rapid detection and typing of dengue viruses from clinical samples by using reverse transcriptase-polymerase chain reaction. J Clin Microbiol 30: 545–551.
- 9. Reynes JM, Ong S, Mey C, Ngan C, Hoyer S, et al. (2003) Improved molecular detection of dengue virus serotype 1 variants. J Clin Microbiol 41: 3864–3867.
- 10. Hue KD, Tuan TV, Thi HT, Bich CT, Anh HH, et al. (2011) Validation of an internally controlled onestep real-time multiplex RT-PCR assay for the detection and quantitation of dengue virus RNA in plasma. J Virol Methods 177: 168–73.
- 11. Vong S, Khieu V, Glass O, Ly S, Duong V, et al. (2010) Dengue Incidence in Urban and Rural Cambodia: Results from Population-Based Active Fever Surveillance, 2006–2008. PLoS Negl Trop Dis 4 (11) e903 .
- 12. Clarke DH, Casals J (1958) Techniques for hemagglutination and hemagglutination-inhibition with arthropod-borne viruses. Am J Trop Med Hyg 7: 561–573.
- 13.
WHO/TDR (1997) Dengue Hemorrhage Fever: diagnosis, treatment prevention and control. 2nd edition.
- 14. Artsob H, Pelegrino JL, Buchy P, Cardosa MJ, Devi S, et al. (2010) Evaluation of diagnostic tests: dengue. Nat Rev Microbiol (12 Suppl) S30–8.
- 15. Tricou V, Vu HTT, Quynh NVN, Nguyen CVV, Tran HT, et al. (2010) Comparison of two dengue NS1 rapid tests for sensitivity, specificity and relationship to viraemia and antibody responses. BMC Infect Dis 10: 142.
- 16. Osorio L, Ramirez M, Bonelo A, Villar L, Parra B (2010) Comparison of the diagnostic accuracy of commercial NS1-based diagnostic tests for early dengue infection. Virol J 7: 361.
- 17. Wang SM, Sekaran SD (2010) Early Diagnosis of Dengue Infection Using a Commercial Dengue Duo Rapid Test Kit for the Detection of NS1, IGM and IGG. Am J Trop Med Hyg 83: 690–695.
- 18. Blacksell SD, Jarman RG, Bailey MS, Tanganuchitcharnchai A, Jenjaroen K, et al. (2011) Evaluation of six commercial point-of-care tests for the diagnosis of acute dengue infections: The need for combining NS1 antigen and IgM/IgG antibody detection to achieve acceptable levels of accuracy. Clin Vaccine Immunol 18: 2095–101.
- 19. Hunsperger EA, Yoksan S, Buchy P, Nguyen VC, Sekaran SD, et al. (2009) Evaluation of commercially available anti-dengue virus immunoglobulin M tests. Emerg Infect 15: 436–440.
- 20. Huy R, Buchy P, Conan A, Ngan C, Ong S, et al. (2010) National dengue surveillance in Cambodia, 1980–2008: Insights on epidemiological and virological trends and impact of vector control intervention. Bull WHO 88 (9) 650–7.
- 21. Buchy P, Luong VV, Toan BK, Mai TTX, Glaziou P, et al. (2005) Secondary Dengue virus type 4 infections in Vietnam. Southeast Asian J Trop Med Public Health 36: 178–185.
- 22. Rennie W, Phetsouvanh R, Lupisan S, Vanisaveth V, Hongvanthong B, et al. (2007) Minimising human error in malaria rapid diagnosis: clarity of written instructions and health worker performance. Trans R Soc Trop Med Hyg 101: 9–18.
- 23. Uzochukwu BSC, Onwujekwe E, Ezuma NN, Ezeoke OP, Ajuba MO, et al. (2011) Improving Rational Treatment of Malaria: Perceptions and Influence of RDTs on Prescribing Behaviour of Health Workers in Southeast Nigeria. PLoS ONE 6 (1) e14627 .
- 24. Reyburn H, Mbakilwa H, Mwangi R, Mwerinde O, Olomi R, et al. (2007) Rapid diagnostic tests compared with malaria microscopy for guiding outpatient treatment of febrile illness in Tanzania: randomised trial. BMJ 334: 403.
- 25. Bisoffi Z, Sirima BS, Angheben A, Lodesani C, Gobbi F, et al. (2009) Rapid malaria diagnostic tests versus clinical management of malaria in rural Burkina Faso: Safety and effect on clinical decisions. A randomized trial. Trop Med Int Health 14: 491–498.