Article Text
PDF
Abstract
Objectives To compare point-of-care (POC) systems in two different periods: (1) before 2010 when all high-risk patients were offered POC management for urogenital gonorrhoea by Gram stain examination; and (2) after 2010 when only those with symptoms were offered Gram stain examination.
Methods Retrospective comparison of a Gram stain POC system to all high-risk patients (2008–2009) with only those with urogenital symptoms (2010–2011) on diagnostic accuracy, loss to follow-up, presumptively and correctly treated infections and diagnostic costs. Culture was the reference diagnostic method.
Results In men the sensitivity of the Gram stain was 95.9% (95% CI 93.1% to 97.8%) in 2008–2009 and 95.4% (95% CI 93.7% to 96.8%) in 2010–2011, and in women the sensitivity was 32.0% (95% CI 19.5% to 46.7%) and 23.1% (95% CI 16.1% to 31.3%), respectively. In both periods the overall specificity was high (99.9% (95% CI 99.8% to 100%) and 99.8% (95% CI 99.7% to 99.9%), respectively). The positive predictive value (PPV) and negative predictive value (NPV) before and after 2010 were also high: PPV 97.0% (95% CI 94.5% to 98.5%) and 97.7% (95% CI 96.3% to 98.6%), respectively; NPV 99.6% (95% CI 99.4% to 99.7%) and 98.8% (95% CI 98.5% to 99.0%), respectively. There were no differences between the two time periods in loss to follow-up (7.1% vs 7.0%). Offering Gram stains only to symptomatic high-risk patients as opposed to all high-risk patients saved €2.34 per correctly managed consultation (a reduction of 7.7%).
Conclusions The sensitivity of the Gram stain is high in men but low in women. When offered only to high-risk patients with urogenital symptoms, the cost per correctly managed consultation is reduced by 7.7% without a significant difference in accuracy and loss to follow-up.
Statistics from Altmetric.com
Introduction
Point-of-care (POC) management involves on-the-spot rapid testing with a result on the same day as the sample is obtained. It allows clinicians to make a presumptive diagnosis and offer direct treatment, counselling and contact tracing. POC systems are important to break the chain of infection at the shortest notice and can reduce ongoing transmission due to immediate treatment and fewer patients lost to follow-up.1–5 Nonetheless, due to inferior test characteristics compared with reference tests, relying solely on POC results can also lead to overtreatment and missed infections.6
At present there is no accurate and simple POC test for urogenital gonorrhoea. Light microscopic detection of Gram-negative diplococci in genital smears is a rapid and cheap procedure to diagnose urogenital gonorrhoea presumptively, but it requires a laboratory environment and trained technicians.7
At the Sexually Transmitted Infection (STI) clinic of the Public Health Service of Amsterdam, Gram stains are used as a POC test for urethral gonorrhoea in men and cervical gonorrhoea infections in women.
In January 2010 the indication for a Gram stain was restricted because of a steady yearly increase in new consultations as well as structural understaffing of the POC laboratory. Before February 2010, all high-risk patients were offered a Gram stain POC test to diagnose urogenital gonorrhoea presumptively but, after February 2010, only symptomatic high-risk patients were offered presumptive Gram stain testing.
We compared retrospectively the effects of changing the POC system for urogenital gonorrhoea before and after 2010. The research question was the extent to which patient management and costs were affected by restricting the indication for Gram stain analysis to symptomatic high-risk patients only. We compared the diagnostic accuracy, loss to follow-up, the percentage of correctly treated infections and of presumptively treated infections and the costs of diagnostics.
Methods
Study design, selection of patients and patient management
From 1 January 2008 to 31 December 2009 (the 2008–2009 period), cervical or urethral smears were obtained from patients who were identified as ‘high-risk’ patients irrespective of signs or symptoms. These smears were used for light microscopic examination and, if intracellular Gram-negative diplococci were found, the patients were treated directly with ceftriaxone 500 mg parentally (with lidocaine), as well as azithromycin 1000 mg orally to treat presumed co-infection with chlamydia.
All Gram stains were confirmed with a culture for which the result followed 1 week later. In case the culture was positive and the patient had not received treatment at the first visit, he/she was recalled to the clinic for treatment and contact tracing.
A patient was considered ‘high-risk’ if one of the following criteria was met: having STI-related complaints, notified of a STI by a sex partner, paid for sexual contact, men who had sex with men or uninsured patients from sub-Sahara Africa.8 If none of these criteria were met, the patient was considered ‘low-risk’ and assigned to the short screening protocol. Low-risk patients were not tested with a Gram stain but were tested only with a nucleic acid amplification test (NAAT) for which the result was available 1 week later. For this reason, low-risk patients were excluded from this study.
Because of a yearly increasing workload (from 12 000 patients/year in 2006 to 25 921 patients/year in 2009) in combination with structural understaffing at the POC laboratory, it was decided to restrict the indication for a Gram stain in 2010. From 12 February 2010 to 31 December 2011 (the 2010–2011 period), Gram stains were offered only to high-risk patients with signs or symptoms suspected of urogenital gonorrhoea.
Symptoms or signs warranting a Gram stain test in men were: dysuria, high urinary frequency, urethral discharge, scrotal pain and/or a red irritated meatus urethrae upon physical examination and/or urethral discharge. In women the indications for a Gram stain examination were: abnormal vaginal discharge, dysuria, vaginal itching, irregular vaginal bleeding, a red inflamed cervix or signs of pelvic inflammatory disease.
Consultations with a missing or failing Gram stain or culture result in both periods were excluded from the analysis. Consultations occurring on days when the Gram stain examination was not available because of lack of laboratory staff were also excluded. Some patients visited the STI clinic more than once and were offered a new standard testing procedure each time. The number of patients used in our calculations refers to the number of consultations and not to the number of unique patients. More information about patient management can be found in the online supplement.
Loss to follow-up, confirmed gonorrhoea treated and time of treatment
The number of patients lost to follow-up was calculated as the proportion of consultations in which no medication for gonorrhoea was provided at our clinic within 12 weeks after the definite diagnosis of urogenital gonorrhoea out of all culture-positive (and false-negative Gram) consultations requiring treatment. Excluded from this analysis were consultations in which treatment was given because of a partner with gonorrhoea or in which treatment was given for anorectal gonorrhoea.
The proportion of confirmed and treated urogenital gonorrhoea infections was calculated as the proportion of all urogenital gonococcal infections confirmed by culture that were treated at the STI outpatient clinic out of all urogenital gonorrhoea infections confirmed by culture.
The proportion of gonorrhoea infections treated on the day of the first visit was calculated as all urogenital gonococcal infections confirmed by culture that were treated at the first visit (because of a positive Gram stain, a partner with a gonorrhoea infection or a presumptively treated anorectal gonorrhoea infection) out of all urogenital gonorrhoea infections confirmed by culture. Delayed treatment was defined as all gonorrhoea infections confirmed by culture that were treated during the second visit.
Costs per consultation and per correctly managed consultation
Costs were estimated from a health services perspective and included direct staff time (salary plus benefits), supplies, overheads and treatment drugs (preparing the cefriaxone formulation). Azithromycin was excluded as it is also used to treat chlamydia. Costs for the patient such as loss of productivity due to waiting hours were not accounted for.
We calculated the cost per consultation and the cost per correctly managed consultation. Incorrect management of gonorrhoea was defined as delayed treatment (treatment after the first visit), no treatment at all (loss to follow-up after a positive culture) or overtreatment (based on a false-positive Gram stain result). The remaining consultations fell into the group of correct management. More information about the cost calculations can be found in the online supplementary data and table S1.
Statistical analysis
Statistical analyses were done using Stata/SE V.12.1 for Windows. Differences in proportions between the two study periods with 95% CIs were calculated by performing tests for the equality of proportions using large sample statistics (normal approximation) in the two study periods. Exact binomial CIs were calculated.
Results
In the 2008–2009 period 50 753 consultations were performed, of which 29 006 (57.2%) were considered high-risk (figure 1). After exclusion, the final analysis set consisted of 10 952 (21.6%) high-risk consultations. In the 2010–2011 period 52 408 consultations were performed, of which 27 214 (51.9%) were considered high-risk. After exclusion the final analysis set consisted of 25 524 (48.7%) high-risk consultations.
Flowchart of the management of urogenital gonorrhoea, STI Outpatient Clinic, Amsterdam, The Netherlands, 2008–2011. *One patient with a positive Gram stain and culture did not receive treatment, because he did not wait for the Gram stain result at the initial visit and could not be reached. †No action required: already treated for a presumptive diagnosis of anorectal gonorrhoea at the first visit or because of a proven gonorrhoea-infected partner. ‡Received their final result (online or telephone) and told us that they were treated by their general practitioner (GP). §Received their final result (online or telephone) but it was unknown if they were treated elsewhere. ¶Unknown if they were treated elsewhere (5 patients received their final result and 2 patients could not be reached for notification).
Although the percentage of the final analysis set as a proportion of all consultations is much higher in the 2010–2011 period, the general characteristics such as sex, age, nationality and various risk factors mentioned during the consultations for the patients included and excluded from analyses are comparable (see online supplementary table S2).
Diagnostic accuracy
The sensitivity of the Gram stain was 87.2% (95% CI 83.3% to 90.4%) in the 2008–2009 period and 84.8% (95% CI 82.3% to 87.1%) in the 2010–2011 period (p=0.29). In men the sensitivity of the Gram stain was high in the two time periods 95.9% (95% CI 93.1% to 97.8%) and 95.4% (95% CI 93.7% to 96.8%), respectively; p=0.72), whereas in women the sensitivity of the Gram stain was low 32.0% (95% CI 19.5% to 46.7%) and 23.1% (95% CI 16.1% to 31.3%), respectively), with no significant difference between the two periods (p=0.21). In both periods Gram stain microscopy had a high specificity regardless of sex (≥99.8%).
The positive predictive value (PPV) and negative predictive value (NPV) were high for both sexes in both periods: PPV 97.0% (95% CI 94.5% to 98.5%) and 97.7% (95% CI 96.3% to 98.6%), respectively (p=0.50); NPV 99.6% (95% CI 99.4% to 99.7%) and 98.8% (95% CI 98.5% to 99.0%), respectively. More information on the diagnostic accuracy is given in online supplementary table S3.
Confirmed gonorrhoea treated, same day treatment, overtreatment and loss to follow-up
There was no significant difference in loss to follow-up between the two periods (7.1% vs 7.0%) and the proportion of infections with confirmed urogenital gonorrhoea that were treated at the outpatient clinic was high in both periods (99.5% vs 99.0%). In the 2008–2009 period the percentage of infections that were treated on the day of the first visit was higher than in the 2010–2011 period (92.3% vs 86.3%). In both periods, few patients were overtreated (3.0% vs 2.3%, table 1).
Proportion of confirmed culture-positive urogenital Neisseria gonorrhoeae (NG) infections treated and untreated, receiving same day treatment at first visit and delayed treatment at the second visit, loss to follow-up and overtreatment by sex (STI Outpatient Clinic, Amsterdam, The Netherlands, 2008–2011)
Costs per consultation and per correctly managed consultation
The average cost per consultation was €30.15 in the 2008–2009 period and €27.75 in the 2010–2011 period, a saving of €2.40 per consultation (a reduction of 8%). In the 2008–2009 period, 99.7% of the consultations involving a urogenital gonorrhoea infection were managed correctly compared with 99.4% in the 2010–2011 period. The cost per correctly managed consultation in the two periods was €30.25 and €27.91, respectively, a saving of €2.34 per correctly managed consultation (a reduction of 7.7%, table 2).
Cost per consultation and per correctly and incorrectly managed consultation of urogenital gonorrhoea (STI Outpatient Clinic, Amsterdam, The Netherlands, 2008–2011)
Discussion
The change in POC management for urogenital gonorrhoea to offer Gram stain analysis only to symptomatic high-risk patients (as opposed to all patients irrespective of symptomatology) resulted in a cost reduction of 7.7% per correctly managed consultation, with comparable sensitivity (87.2% vs 84.8%), specificity (99.9% vs 99.8%), PPV (97.0% vs 97.7%), NPV (99.8% vs 99.5%) and loss to follow-up cases (7.1% vs 7.0%). In both periods, very few patients (3.0% vs 2.3%) were overtreated at a negligible cost to the clinic.
A striking finding is the low sensitivity of the Gram stain for urogenital gonorrhoea for women in both periods (32.0% vs 23.1%), yet the PPV and NPV for women were high in both periods (>97%). The high PPV can be explained by the low number of false-positive Gram stains. The NPV is high, despite a low sensitivity, because of the low prevalence of gonorrhoea among high-risk female patients. The low sensitivity of Gram stain microscopy in women in our study confirms the results of earlier studies.9 ,10 Nonetheless, because of the presence of other infections such as candidiasis, trichomoniasis and bacterial vaginosis, we continue to offer a Gram stain to symptomatic high-risk women.
During the study periods, culture was used as a reference test for urogenital gonorrhoea. However, current guidelines recommend a NAAT as the preferred test. In settings where a NAAT is used as the reference test, the diagnostic accuracy outcomes are likely to be different. The lower sensitivity of the culture may have caused an overestimation of both the sensitivity and specificity of the Gram stain in both periods.
The strength of our study is the large number of consultations analysed in the two study periods. It should be noted that we selected a high-risk population in a large inner city STI clinic, so these results might not be representative of other populations.
We limited the cost analysis to expenses for the clinic and did not include the costs and burden of disease for the patient and the community at large. POC systems can prevent extra visits to the clinic with additional financial gain for the patient. Moreover, same day treatment for false-positive test results can cause allergic and gastrointestinal reactions, antibiotic resistance, unnecessary costs and anxiety for the patient.11 ,12
In conclusion, the Gram stain smear POC system was more cost-effective when offered only to symptomatic high-risk patients. Further research is needed to investigate other public health outcomes of POC management such as ongoing transmission, cost to the patient and antibiotic resistance due to undertreatment or overtreatment.
Key messages
-
Microscopic analysis of Gram-stained urethral and cervical smears is the most reliable point-of-care test (POC) to date for the presumptive management of urogenital gonorrhoea.
-
The Gram-stained smear POC system is accurate for the presumptive management of urogenital gonorrhoea in high-risk men but not in high-risk women.
-
When offered only to high-risk patients with urogenital symptoms, the cost per correctly managed consultation is reduced by 7.7% without a significant difference in accuracy and loss to follow-up.
Acknowledgments
The authors thank Dewi Usmany, Arjen Speksnijder, Caspar Signet and Ineke Linde of the Public Health Laboratory for providing data on diagnostic costs, Sander Hoogenkamp of the financial department of the Public Health Service of Amsterdam for providing data on human resources costs and Ente Rood of the Royal Tropical Institute for help with setting up the design of the study.
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
- Data supplement 1 - Online supplement
Abstract in Dutch
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
- Abstract in Dutch - Online abstract
Footnotes
-
Handling editor Jackie A Cassell
-
Contributors HJCdV, MSvR and MS were responsible for the design of the study. MSvR and MB collected and interpreted the data on the diagnostic outcomes. SA did the statistical analysis of the diagnostic outcomes. KV collected and analysed the data on costs. MB drafted the paper. HJCdV, MS and WRF supervised the overall study. All authors reviewed and approved the final article.
-
Funding This study was financed by the Research and Development Fund of the Public Health Service of Amsterdam, no 2382.
-
Competing interests None.
-
Ethics approval All the data used in this study were collected as part of routine management and anonymised before the analysis so ethical approval was not considered necessary.
-
Provenance and peer review Not commissioned; externally peer reviewed.