Analysis of the potential for point-of-care test to enable individualised treatment of infections caused by antimicrobial-resistant and susceptible strains of Neisseria gonorrhoeae: a modelling study

Objective To create a mathematical model to investigate the treatment impact and economic implications of introducing an antimicrobial resistance point-of-care test (AMR POCT) for gonorrhoea as a way of extending the life of current last-line treatments. Design Modelling study. Setting England. Population Patients accessing sexual health services. Interventions Incremental impact of introducing a hypothetical AMR POCT that could detect susceptibility to previous first-line antibiotics, for example, ciprofloxacin or penicillin, so that patients are given more tailored treatment, compared with the current situation where all patients are given therapy with ceftriaxone and azithromycin. The hypothetical intervention was assessed using a mathematical model developed in Excel. The model included initial and follow-up attendances, loss to follow-up, use of standard or tailored treatment, time taken to treatment and the costs of testing and treatment. Main outcome measures Number of doses of ceftriaxone saved, mean time to most appropriate treatment, mean number of visits per (infected) patient, number of patients lost to follow-up and total cost of testing. Results In the current situation, an estimated 33 431 ceftriaxone treatments are administered annually and 792 gonococcal infections remain untreated due to loss to follow-up. The use of an AMR POCT for ciprofloxacin could reduce these ceftriaxone treatments by 66%, and for an AMR POCT for penicillin by 79%. The mean time for patients receiving an antibiotic treatment is reduced by 2 days in scenarios including POCT and no positive patients remain untreated through eliminating loss to follow-up. Such POCTs are estimated to add £34 million to testing costs, but this does not take into account reductions in costs of repeat attendances and the reuse of older, cheaper antimicrobials. Conclusions The introduction of AMR POCT could allow clinicians to discern between the majority of gonorrhoea-positive patients with strains that could be treated with older, previously abandoned first-line treatments, and those requiring our current last-line dual therapy. Such tests could extend the useful life of dual ceftriaxone and azithromycin therapy, thus pushing back the time when gonorrhoea may become untreatable.

1. Introduction (page 6, lines 19-25): the way this sentence is phrased could be interpreted as meaning that empirical treatment at the time patients first present could increase opportunities for transmission. It would be helpful if this was clarified. 2. Methods (page 10, lines 4-6): the proportion of the patients treated on the same day would have been because of gram negative diplococci being seen on microscopy or because they attended as a contact of someone who had been diagnosed with GC. What do you mean by 'epidemiological signs'? 3. Discussion (page 15, lines 29-30): the proportion of women treated on the same day as testing is also lower because of the poor sensitivity of detection of gonorrhoea in endocervical and urethral smears from women. 4

GENERAL COMMENTS
The authors use a simple spreadsheet model to estimate the reduction in the number of ceftriaxone treatments through introduction of a point-of-care antimicrobial resistance test for gonorrhoea in England. The study does not give any real explanation of why this number is of scientific importance, noting only that it is a 'first step' (page 15, line 52) and an improved dynamical model is also under review (page 14, line 42), and indeed it is noticeable that the abstract conclusions do not in any way refer to the abstract results.
Below are general additional comments on this paper.
Title -'impact' suggests that the authors are looking at how the number of cases of antimicrobial-resistant gonorrhoea may reduce. Can the authors use a title that better reflects their study.
Page 7 line 9 "most patients could be treated with an older oral first-line therapy which could potentially extend the life of ceftriaxone" -it seems that the fundamental premise of this paper is that reducing use of ceftriaxone will reduce evolution of strains resistant to this drug. This seems likely, but it is not inconceivable that strains that have developed resistance to one form of treatment could be better placed to evolve resistance to other drugs (e.g. if the drugs are related). Can the authors comment on this.
Page 9 line 19 -"…previous study Turner" is a typo.  Table 2 -at present this is a meaningless set of numbers. How does the assumption of £25 per test affect the costs? Is £25 a potentially realistic number? (and I note that it is different to the £50 used in one of the supplementary reports by the same author). Either provide a breakdown of the costs (how much of the increase is due to the cost of the new test) or a sensitivity analysis given a range of plausible costs. As it stands I cannot see a reason for presenting 'costs' in this paper at all, as the 'costs' are based on an assumed cost of a non-existent test and do not include any long-term benefits (for example how reduced prevalence of drug-resistant gonorrhoea prevents future costs) and a naive reader will just see that this test is extremely expensive and therefore should not be considered.
Strategy 2 -I cannot see the point of strategy 2 -the authors themselves only give a footnote in Table 1 and don't mention it in their results section.
Other questions: Model assumptions --Sensitivity and specificity of the POCT AMR -this is assumed 100%. Can the authors comment on how realistic that is? -Model sensitivity analysis: can the authors either comment on the sensitivity of their model to the input parameters, or else contextualise their results (for example if these parameters represent the gonorrhoea epidemic in England in 2014, say that this is the context of the results).
-Number of gonorrhoea cases -has this remained fairly static in England in years other than 2014?

Reviewers comments from submission to Sexually Transmitted Infections
Reviewer: 1 1. My main comment is that the authors do not seem to have taken into consideration the impact of sensitivity and specificity of the AMR POCT -is this assumed to be 100%?
We have assumed 100% sensitivity / specificity for simplicity.
We have clarified this in the text in 2. It isn't very clear to me how the authors assumed that this will work in practice. I believe that this paper would have benefited from the input from a Genito-urinary physician. My interpretation is that all patients presenting to a GU clinic have a POCT for CT/GC (irrespective of risk/whether they are symptomatic or not) and only those who test positive will have separate AMR POCTs for ciprofloxacin and penicillin. Although the authors have made brief reference to this, it is a major limitation that how long the AMR POCT takes to process and whether patients will be willing to wait for this result has not been factored into this modelling. The Cepheid POCT for chlamydia and GC takes 90 minutes to process and a previous study of the impact of this test on patient management in a clinic setting found only 3/19 men waited for their results (6 were positive). (Harding-Esch et al. ISSTDR 2015). If an initial POCT needs to be performed followed by an AMR POCT, then the length of time a patient has to wait will be extended.
It is true that introducing delays in treatment to wait for results of susceptibility testing might require re-arrangement of current arrangements for providing prescriptions. The trade-offs between preserving ceftriaxone against potential delays in treatment requires further consideration in the context of a transmission dynamic model. However we believe the problem of drug resistance is severe enough that such issues require full assessment.
We have clarified in the text as follows: We assume that results of point of care diagnostics can be provide within the clinical consultation, e.g. if patients provide samples for testing on arrival at a GUM clinic and then wait for an appointment or return later in the day. It is possible that this would result in delays to treatment for symptomatic individuals and sexual contacts, but we do not consider this further here as we are exploring the potential of theoretical new tests.

Methods section -para 3, last 2 lines
And also raised this as a discussion point Discussion -Strengths/weaknesses compared with other studies, final para We also assume that results of point of care diagnostics can be provided within the clinical consultation. This is not currently possible unless the patient provides samples on arrival then waits or returns for an appointment later. The Cepheid GeneXpert has a turnaround time of about 90 minutes which was previously found to result in the majority of men (16/19) not waiting for their results (6 were positive) 26 .
Including the ref given (thank  you) 3. Prior to the introduction of NAATs in routine care, even when we had antibiotic sensitivities available prior to treatment, and where the GC was sensitive to ciprofloxacin and/or penicillin, we still used the recommended first line treatment to avoid increased levels of resistance. The authors mention that re-introduction of ciprofloxacin would likely have this effect. However, they do not mention that it could consequently lead to increased resistance to penicillin (if used) and azithromycin as well. This could be covered.
We expand on the consequences of re-using older drugs.

Discussion, section Strengths/weaknesses wrt other studies Para 2
Similarly re-using other drugs would also result in increases in resistance observed, including increasing selection for plasmids conferring multidrug resistance. (Figure 1, page 20) however, MSM account for 70% of gonorrhoea diagnoses in men (Health Protection Report Vol 10 No 22 -8th July 2016).

Only the heterosexual male pathway is illustrated
We included MSM and women in the calculation of the total cost and total number of treatments given (each pathway considered separately, then summed) -this has been clarified in the text in the methods sectionmanagement strategies.
5. Presumably multi-site infection, with the need for testing at multiple sites for AMR, has not yet been factored into this model.
No this hasn't been factored in and may also increase costs especially for MSM -added a point to discussion 1. Introduction (page 6, lines 19-25): the way this sentence is phrased could be interpreted as meaning that empirical treatment at the time patients first present could increase opportunities for transmission. It would be helpful if this was clarified.
We have substantially reworded the introduction to provide greater clarity and this sentence has been deleted.
2. Methods (page 10, lines 4-6): the proportion of the patients treated on the same day would have been because of gram negative diplococci being seen on microscopy or because they attended as a contact of someone who had been diagnosed with GC. What do you mean by 'epidemiological signs'? Sentence changed to: Methods: Management strategies …."Some patients are managed on the same day, either due to symptoms and positive microscopy or as contacts of infected individuals, others wait for lab results, resulting in some unnecessary treatment and some delays to treatment or loss to follow-up." 3. Discussion (page 15, lines 29-30): the proportion of Thank you -amended sentence women treated on the same day as testing is also lower because of the poor sensitivity of detection of gonorrhoea in endocervical and urethral smears from women.
However, this proportion is lower for women due to the higher percentage of asymptomatic infections and from poorer sensitivity of detection of gonorrhoea in endocervical and urethral smears.
This sentence was deleted during rewriting of the introduction and more appropriate referencing of current treatment guidelines and GRASP reports given instead. The use of culture to detect Neisseria gonorrhoeae is not mentioned in this manuscript. Using this, the patients with delayed management could also receive alternative treatment without the development of POCT.

5.
Yes this is true, although current practice is mainly PCR NAAT tests with culture done after diagnosis.
Page 3, Lines 42-44: What if the POCT AMR could detect susceptibility to both ciprofloxacin and penicillin? it would be very interesting to see the proportion of ceftriaxone reduction combined for ciprofloxacin and penicillin.
We agree that multiplex or more complex tests to detect combinations of resistance/susceptibility would be interesting, however this study was designed around the most likely tests to be developed (ciprofloxacin is due to chromosomal, single point mutations so is the easiest to detect through PCR for example).
Page 3, Line 48: "no positive patients remain untreated": I find this to optimistic. Because this hypothetical POCT AMR has extraordinary discriminatory characteristics, no patients leave the clinic untreated. I would not see this as a main finding but a very logical, hypothetical result.
We have added discussion of potential loss of patient during wait for results (see reviewer 1) in the discussion. Agree this is a limitation but this is intended to be a theoretical exercise to stimulate future research Key messages, Page 5, lines 5-9: please add that the inability to discern resistant strains is about the fact that we are not possible to do this at the same time as the moment of sampling.

Clarification added
 Most strains of Neisseria gonorrhoeae in the UK are susceptible to older, abandoned first-line treatments, but characterisation of the resistance/susceptibility profiles of infection is not available at the time of diagnosis and treatment.
Page 6, line 9: please describe the source of the data that 34,958 cases were reported. Does this only concern sexual health clinic data or also data from other health care provider like GP's? This introduction has been reworked and clarified including addition of Figure 2 which shows the options more clearly and a new paragraph Page 6, line 58: please add a reference for the data on resistant ciprofloxacin/penicillin (20-40% resistant).
This has been deleted in the new introduction.
Page 7, line 3: I do not agree that doctors lack the means: they can wait until AMR results are available. In fact there are two problems: 1) there is not a good POCT for gonorrhoea available and 2) for those patients who are treated because of being notified, having symptoms, a positive gram etc., no AMR data is available.

This has been clarified
General comment on the introduction: as a reader I would like to have some background about the development of POCT and AMR POCT. What is the current status of these tests (do they exists or are they in development or is it purely hypothetical)?
We have added a little more context on potential technologies and tools for acquiring information on AMR profiles more rapidly.

Introduction
No such test currently exists. A promising option based on existing nucleic acid amplification test (NAAT) could be a PCR test for ciprofloxacin resistance as this is conferred in a single, chromosomal mutation 10 . Other technologies could involve direct measurement of live cell responses to the presence of a panel of antibiotics including microfluidic devices, atomic force microscopy, volatile chemical detection or mass spectroscopy.
Computational approaches based on in silico phenotyping based on genotype may also be able to detect new mutations more rapidly than traditional microbiological testing 11-13 .

Methods
Page 7, line 27: In the title of Figure 1 the focus is on heterosexual males. Is this right? In the manuscript, the model is also about MSM and women.
Yes the numbers are illustrated for heterosexual males, but the same pathway is used with slightly different parameters for MSM and women.
Page 7, lines 36-38: I feel like I miss the option of a culture as initial screening tool for gonorrhoea. Is this not done anymore in the UK? Please elaborate on this.
Most testing is NAAT PCR. Microscopy is done in GUM clinics for symptomatic patients, then swabs sent for culture which would take a couple of weeks for AMR testing. This is clarified in Figure 2 (new figure).

Done
Page 9, line 13: Please remove "men who have sex with men" and use only MSM here. The numbers you give are just for heterosexual men -the women are much more likely to have delayed treatment and require a second visit so overall the average is 1.44.
Page 11, lines 9-11: In this line is stated that with ciprofloxacin a 66% reduction in ceftriaxone treatment can be reached. If 37% of the infections are resistant to ciprofloxacin, why is the reduction in ceftriaxone not 63%? The same applies for penicillin on page 12.
The slight difference is because of loss to follow up assumed in the current pathway.
Page 12, line 27-28: 2 days reduction in waiting time for treatment: I would suggest to focus on the group that receives delayed management in scenario 1. See also "Page 10, line 48" above.
We do agree that the most benefit in terms of reduction in time to treatment are the groups currently in the delayed management arm, but we think that is appropriate to calculate the average change in waiting time overall.
Page 12, line 40: please remove "diagnostic". Amended Page 13, lines 17-27: please make use of two sentences to make this long sentence more comprehensible.
Apologies -I couldn't work out which line this referred to Theoretically a POCT could improve PN if both partners are tested at the same time in the clinic or if the reduced delay enables better recall of recent partners or people are more motivated by same day discussion with a health care professional. I am not aware of any specific data on this however -toned down to "potentially improve partner notification" Page 15, line 54: remove "by" in the sentence "Future research investigating by how".
Amended Figure 1, Difference between AMR and non-AMR is not clear. Is this based on the proportion of gonorrhoea tests were a culture has been performed including resistance testing?
This is now Figure 3 In the model we assume a percentage of infections are AMR based on the reported percentage in GRASP, given in supplementary Table A1. This is clarified in the title and we have also referred to Figure 3 in the management scenarios definition in methods.  We have clarified the scientific importance of the study in the introduction first paragraph Title -'impact' suggests that the authors are looking at how the number of cases of antimicrobial-resistant gonorrhoea may reduce. Can the authors use a title that better reflects their study.

Analysis of the potential for point-ofcare test to enable individualised treatment of infections caused by antimicrobial-resistant and susceptible strains of Neisseria gonorrhoeae
Page 7 line 9 "most patients could be treated with an older oral first-line therapy which could potentially extend the life of ceftriaxone" -it seems that the fundamental premise of this paper is that reducing use of ceftriaxone will reduce evolution of strains resistant to this drug. This seems likely, but it is not inconceivable that strains that have developed resistance to one form of treatment could be better placed to evolve resistance to other drugs (e.g. if the drugs are related). Can the authors comment on this.
We have added a comment on evolution of multidrug resistance (see review 1 response) Page 9 line 19 -"…previous study Turner" is a typo. Amended  This has been added (see reviewer 2 response) Negative reductions in mean time to treatment are surely double negatives.
Agree "-" removed (and I note that it is different to the £50 used in one of the supplementary reports by the same author). Either provide a breakdown of the costs (how much of the increase is due to the cost of the new test) or a sensitivity analysis given a range of plausible costs. As it stands I cannot see a reason for presenting 'costs' in this paper at all, as the 'costs' are based on an assumed cost of a non-existent test and do not include any longterm benefits (for example how reduced prevalence of drug-resistant gonorrhoea prevents future costs) and a naive reader will just see that this test is extremely expensive and therefore should not be considered.
We used both £25 and £50 in supplementary reports.
Our most up to date understanding is that a new test would have to cost in the order of £25 for widespread use, unless subsidised, e.g. by government.
Although the test doesn't exist, many companies are working on additions to current POCT PCR based tests which detect specific genetic markers of resistance and are likely to have equivalent cost to existing tests of this type.
We have edited table 2 to also reflect that we are considering a work case number, ignoring savings from treating fewer people and using cheaper antibiotics or fewer consultations Strategy 2 -I cannot see the point of strategy 2 -the authors themselves only give a footnote in Table 1 and We have included it for completeness and to illustrate that most of the don't mention it in their results section.
benefits described can be achieved without the AMR POCT , but just with a POCT -We feel this is an important point to make -Model assumptions, sensitivity and specificity of the POCT AMR -this is assumed 100%. Can the authors comment on how realistic that is?
We have assumed 100% specific and sensitive. Although this may not be realistic we assume that a new test would have to have at least equivalent performance to current NAAT tests to be implemented.
-Model sensitivity analysis: can the authors either comment on the sensitivity of their model to the input parameters, or else contextualise their results (for example if these parameters represent the gonorrhoea epidemic in England in 2014, say that this is the context of the results).
We have added context to the results and also to the method section.

First line, Results
We modelled a snapshot of GUM attendance, gonorrhoea diagnosis and prevalence of resistance to ciprofloxacin and penicillin based on the situation in England, 2014 17 . Under current treatment guidelines for 1.4 million people attending GUM per year we estimate -Model assumptions, number of gonorrhoea caseshas this remained fairly static in England in years other than 2014?
Sentence added to introductionnumber of gonorrhoea cases has been rising, especially in MSM where levels of resistance tend to be higher.

GENERAL COMMENTS
In this study the authors use a simple spreadsheet model to quantify the benefits, including doses of ceftriaxone saved, of introducing different hypothetical point-of-care tests for gonorrhoea in England. Although the model used has limitations, in particular not including onwards transmission of gonorrhoea or any dynamics of drug resistance, this represents an important first step in quantifying the potential benefits of such point-of-care tests. Given the necessity of developing strategies to address gonorrhoea resistance to ceftriaxone and azithroymycin, the benefits suggested by this simple model warrant further investigation of the subject.
In general this paper is clearly written, and the modelling well described. The only point which remains somewhat unclear is the hypothesised cost of £25 per test. Can the authors supply a reference to support whether this would be a realistic figure, for example by comparison to other similar point-of-care tests.

Jo Gibbs
University College London, UK

GENERAL COMMENTS
This is an assessment of the treatment and economic impact of the introduction of hypothetical antimicrobial point of care tests for gonorrhoea which detect ciprofloxacin and penicillin resistance. This has not been previously assessed and is therefore of interest to the journal. I am not a mathematical modeller and am unable to comment in detail on the methodology employed. However, as the journal's target audience is not mathematical modellers, what I can offer is the opinion of a sexual health & HIV clinical academic. I have previously reviewed this article for STI and am pleased to see that some of my recommendations and comments have been acted upon. However, I have still have some concerns about this model and would like the authors to address the following comments: