Objectives We aimed to construct widely useable summary measures of the net impact of antibiotic resistance on empiric therapy. Summary measures are needed to communicate the importance of resistance, plan and evaluate interventions, and direct policy and investment.
Design, setting and participants As an example, we retrospectively summarised the 2011 cumulative antibiogram from a Toronto academic intensive care unit.
Outcome measures We developed two complementary indices to summarise the clinical impact of antibiotic resistance and drug availability on empiric therapy. The Empiric Coverage Index (ECI) measures susceptibility of common bacterial infections to available empiric antibiotics as a percentage. The Empiric Options Index (EOI) varies from 0 to ‘the number of treatment options available’, and measures the empiric value of the current stock of antibiotics as a depletable resource. The indices account for drug availability and the relative clinical importance of pathogens. We demonstrate meaning and use by examining the potential impact of new drugs and threatening bacterial strains.
Conclusions In our intensive care unit coverage of device-associated infections measured by the ECI remains high (98%), but 37–44% of treatment potential measured by the EOI has been lost. Without reserved drugs, the ECI is 86–88%. New cephalosporin/β-lactamase inhibitor combinations could increase the EOI, but no single drug can compensate for losses. Increasing methicillin-resistant Staphylococcus aureus (MRSA) prevalence would have little overall impact (ECI=98%, EOI=4.8–5.2) because many Gram-positives are already resistant to β-lactams. Aminoglycoside resistance, however, could have substantial clinical impact because they are among the few drugs that provide coverage of Gram-negative infections (ECI=97%, EOI=3.8–4.5). Our proposed indices summarise the local impact of antibiotic resistance on empiric coverage (ECI) and available empiric treatment options (EOI) using readily available data. Policymakers and drug developers can use the indices to help evaluate and prioritise initiatives in the effort against antimicrobial resistance.
Statistics from Altmetric.com
- antimicrobial resistance
- composite index
- cumulative antibiogram
- device-associated infection
- burden of resistance
Strengths and limitations of this study
Provides novel and clinically meaningful summary measures of our current ability to provide empiric coverage, and of the available stock of antibiotics.
Uses widely available cumulative antibiogram data.
Accounts for variation in resistance, the relative clinical importance of pathogens, and the availability of drugs among infection types and contexts.
Does not account for variation in toxicity, cost, or in vivo efficacy among drugs that are considered appropriate for empiric therapy.
Does not account for variation in the propensity of pathogens to acquire resistance.
Efforts to understand the epidemiology and burden of antibiotic resistance have largely focused on a few sentinel multidrug-resistant organisms,1–3 but this strategy is increasingly unfeasible given the increasing prevalence and diversity of resistance. There is a need to summarise and understand the net clinical impact of resistance—accounting for variation in strain prevalence and drug availability—to focus attention on the most pressing problems.
Strategies for minimising the impact of antibiotic resistance on patients include introducing new drugs, preventing the spread of threatening strains, and reducing unnecessary antibiotic use.3–9 A central challenge is that strategies may be effective and appropriate in one context, but not in another, depending on the incidence and virulence of resistant strains, the type of infections, the consequences of failed treatment, and the availability of treatment options. In order to better understand variation in the effectiveness of stewardship strategies, we need to better understand variation in the impact of resistance. We propose two indices10–12 that can be used to summarise the local impact of resistance on empiric treatment of common bacterial infections, and to assess threats and interventions in that context. The indices account for resistance, drug availability and the relative importance of pathogens, and can be easily calculated from readily available cumulative antibiogram data.13
We concentrate our attention on the failure of empiric therapy because it is the most common problem caused by resistance in many contexts.14 ,15 Empiric therapy may fail microbiologically (ie, related to bug-drug mismatch) because resistance is prevalent, because clinicians prescribe antibiotics contrary to available evidence, or because patients do not follow prescriptions. Although poor treatment decisions, patient adherence and resistance each deserve attention, the problems require different solutions, so we do not include poor treatment or adherence in our indices. Our approach differs in this respect from the drug resistance index (DRI) proposed by Laxminarayan and Klugman.10 The DRI can be interpreted as the probability of inadequate treatment given observed drug use.11 In contrast, we estimate potential empiric coverage by assuming that the empiric therapy that is most likely to succeed is selected, informed by cumulative antibiograms, Gram stains, and knowledge of the likely causative organism(s) for each clinical syndrome.16–18 This allows investigation of new drugs, new resistant strains and other cases in which drug-use data are not available.
We propose two complementary indices: an Empiric Coverage Index (ECI) to measure available empiric coverage of common infections, and an Empiric Options Index (EOI) to measure the empiric value of the current stock of antibiotics on the understanding that antibiotic resources are inevitably diminished by use.19 Together, the ECI and EOI measure current difficulties and vulnerability to increasing resistance, providing a clinically meaningful summary of antibiotic resistance that can be used to assess the real and potential impacts of threats and interventions. As an example, we consider device-associated infections in North American intensive care units (ICUs), but the indices can be adapted for any population for which cumulative antibiogram data are available.13
Materials and methods
Common infections and their causes
The impact of resistance depends on the incidence of infections and their causes, but these data are not readily available at most institutions. Instead, we have a general understanding of the causes of infections. Economists solve an analogous problem using classic fixed-basket consumer price indices.20 These indices measure the average price of a standard basket of consumer goods weighted by the relative importance of each good, ignoring variation in the set of goods consumed in order to focus on price variation. Following a similar logic, we construct a standard basket of infections in order to examine variation in resistance. Our basket consists of common infections, with weights indicating infection frequency.
Proceeding with a critical care example, we construct a basket of device-associated infections consisting of central-line associated bloodstream infections (CLABSI), catheter-associated urinary tract infections (CAUTI), and ventilator-associated pneumonia (VAP). These represent a substantial portion of serious bacterial infections in ICUs,21 ,22 and relatively good surveillance and aetiology data are available worldwide.2 ,23–25 Infections in the basket are weighted using data from the US National Healthcare Safety Network (figure 1).23 Surveillance of CLABSIs suggests that aetiology is similar in Canada and the US.23 ,24 Indices can be easily constructed for other contexts by defining an appropriate basket of infections, and the basket can be updated as new surveillance data becomes available. Local aetiology data can also be used where available.
Resistance to available drugs
As an example, we used the 2011 cumulative antibiogram from the ICUs of a large Toronto teaching hospital, constructed according to guidelines of the Clinical and Laboratory Standards Institute (CLSI).13 ,26 The antibiogram includes the first clinical isolate from each patient, as recommended by CLSI to guide initial empiric therapy.13 Blood, respiratory, urine and skin and soft-tissue infections were included. Resistance data are widely available in this format.13
To determine the set of drugs available for empiric treatment of device-associated infections, we excluded unsuitable and unavailable options from Magiorakos et al27 (table 1). Although aminoglycosides are avoided in most circumstances due to toxicity and poor efficacy, they have been included because they are used in empiric therapy for severe infections.28 Aminoglycosides were considered individually because many strains are resistant to only a subset of drugs in the class.29 We also include drugs that may be reserved for treatment of difficult infections, including carbapenems, linezolid, daptomycin and antipseudomonal cephalosporins,30 ,31 and then consider a scenario omitting these drugs. Cross-resistance within other classes is common but not universal, so the selection of class representatives is somewhat subjective. We selected class representatives from the antibiogram26 to construct indices using available data (table 1).
Antibiotic susceptibility testing was not done when resistance was intrinsic (table 2), implied by other test results (table 3), or known to be rare or common.26 These gaps in the data were filled using general knowledge (see online supplementary appendix A).26 ,27 ,32–35
Index construction—empiric coverage and loss of options
Index construction begins by calculating coverage, defined as the proportion of infections of each type in the basket (figure 1) that are susceptible to each available drug (figure 2, online supplementary appendix B). Empiric therapy of severe device-associated infections often includes two drugs, but combination antibiograms15 ,41 ,42 are not readily available at most institutions. To get a rough, conservative measure of the coverage provided by drug combinations, we assume that treatment is informed by Gram stains.18 We also assume that each infection is caused by a single species.
We then define the ECI for common device-associated infections as the percentage of infections in the basket (figure 1) that are covered by empiric therapy, provided that each infection is treated with the drug that is most likely to provide coverage (see online supplementary appendix B). In cases where coverage is uncertain due to missing resistance data (table 4), we estimate a range, assuming susceptibility in the best case and resistance in the worst case. Coverage in a ‘utopian’ scenario is obtained by assuming that all species are 100% susceptible, except in cases of intrinsic resistance. This scenario shows what has been lost due to acquired resistance.
Antibiotics are a resource that is depleted by use.1 ,3 ,19 ,34 Our EOI measures the value of available treatment options from this perspective. The EOI is constructed on the assumption that repeated use of drugs leads to resistance, so there are a finite number of patients that can be successfully treated before a drug becomes completely ineffective. The EOI is then defined as the number of patients that can be covered by a set of drugs, relative to the number that can be covered by a single fully effective drug for each syndrome (figure 3, online supplementary appendix B). For simplicity, we assume that a linear relationship between drug use and resistance applies to all drugs, that the set of drugs that provide highest coverage are used in equal proportions, and that the total number of infections remains constant over time. Our EOI formulation allows antibiotic mixing to slow the spread of resistance (see online supplementary appendix B),43 but in this analysis, we assume the only advantage of using multiple drugs is that we use less of each.
The Empiric Therapy Indices (ETI) R package includes methods for calculating the indices. Find instructions for installation and use at https://github.com/josie-hughes/EmpiricTherapyIndices/blob/master/Example.R.
The ECI measures our ability to provide empiric coverage of common bacterial infections within a population. An ECI of 100% indicates that at least one drug provides full coverage of each infection type in the basket, and an ECI of 0% indicates no empiric coverage. The current ECI at our site remains high (98%) because susceptibility to vancomycin, linezolid, meropenem and the aminoglycosides remains high (figure 4). However, the calculation includes drugs that are often reserved for difficult infections or those associated with a high risk of complications.30 ,31 If we chose to restrict the use of carbapenems, linezolid, daptomycin, antipseudomonal cephalosporins and aminoglycosides for empiric therapy, the ECI would be 86–88% (figure 4). Uninformed by a Gram-stain, treating with a single non-reserved drug further reduces the ECI to between 69% and 77%.
The index accounts for the importance of resistant strains as causes of infection, so is most threatened by common pathogens that are resistant to the few drugs that still provide good empiric coverage; for example, plasmid-mediated carbapenem resistance among Enterobacteriaceae poses a major threat to the ECI (100% KPC in figure 4). Pan-drug-resistant A. baumannii are similarly resistant to remaining empiric drugs, but would have less impact (100% PDR Acinetobacter in figure 4) because A. baumannii causes fewer infections (figure 1). The ECI is not altered by increasing resistance to drugs that provide less coverage; for example increasing MRSA prevalence would not alter the ECI or the empiric effectiveness of vancomycin and linezolid (100% MRSA in figure 4).
The EOI measures the value of the set of available antibiotics, including drugs that provide less coverage. The EOI reaches a maximum value equal to the number of available treatment options if all drugs provide full coverage, and can therefore be interpreted as the equivalent number of fully effective treatment options. We assume that more treatment options are better because drug use leads to resistance. We also assume that partially effective drugs have some value, but that a single drug providing 100% coverage is better than two drugs providing 50% coverage. Thus, a higher EOI is assigned when the best available coverage is high, all else being equal (compare rows in figure 5). Essentially, a low ECI indicates an urgent problem with failing empiric therapy, while a low EOI indicates treatment vulnerability. Without acquired resistance, the EOI in our example ICU would have been 9.4 at our site, but now it is 5.3–5.9 (utopian vs current scenarios in figure 4). We can still provide coverage, but 37–44% of treatment potential measured by the EOI has been lost (1−5.3/9.4=0.44) as resistance has developed to nearly all available drugs (figure 2). This analysis assumes the only advantage of using multiple drugs is using less of each. If mixing also slows the spread of resistance,43 then redundancy is more valuable and our losses are even more concerning (see online supplementary appendix C figure C.1).
New antibiotics can reduce vulnerability and increase the EOI, but not all new antibiotics improve our ability to empirically cover common infections. For example, dalbavancin, oritavancin and tedizolid are closely related to existing drugs and do not provide additional coverage,3 ,4 so would have no impact on the EOI. Ceftolozane-tazobactam and ceftazidime-avibactam provide additional coverage of multidrug-resistant Gram-negatives.37 ,44 ,45 Provided they prove useful for CLABSIs and VAPs,3 ,46 both drugs could increase the EOI by providing good coverage of Gram-negative infections, but neither drug would alter the ECI at our site because amikacin and meropenem remain effective (figure 4). However, in a context where carbapenemase (KPC)-producing Enterobacteriaceae are common, ceftazidime-avibactam could mitigate their impact on the ECI (100% KPC with and without ceftazidime-avibactam in figure 4).
Empiric therapy and the EOI are most threatened by common pathogens that are resistant to the few drugs that still provide good empiric therapy. For example, S. aureus is a common cause of disease (figure 1), but MRSA and resistant CoNS have already rendered many drugs relatively ineffective for empiric therapy of Gram-positive infections (figure 2), so further increasing the prevalence of MSRA would not have a large impact (100% MRSA in figure 4 and online supplementary supplementary appendix C figure C.2), even though S. aureus causes more disease (figure 1), because VRE threatens the empiric effectiveness of vancomycin. Increasing aminoglycoside resistance among Gram-negative strains could substantially reduce the EOI (Gram-negatives 100% resistant to aminoglycosides in figures 4 and online supplementary appendix C figure C.2) because aminoglycosides are among the few drugs that provide good empiric coverage of Gram-negative infections.28
Efforts to understand the epidemiology and burden of resistance have largely focused on a few sentinel multidrug-resistant organisms.1–3 However, the selection of focal strains does not always reflect clinical significance.47 As an alternative, we propose indices that summarise the impact of resistance on empiric therapy. These indices can draw attention to strains such as aminoglycoside-resistant Gram-negatives which threaten empiric therapy28 but do not receive much attention. We do not know the collective impact of delayed therapy if aminoglycosides lose effectiveness but we suspect it may be comparable to other more widely recognised threats.1 ,14
We examined the potential impacts of new drugs and resistant strains on our indices in the intensive care units (ICUs) of a Toronto hospital. Results confirm the indices are reasonable. Empiric therapy is most threatened by common pathogens that are resistant to the few drugs that currently provide good empiric coverage. These considerations are not new,1 ,3 ,5 ,6 ,8 ,16 ,47 but our indices make the assessment process explicit, transparent, quantitative and easily repeatable.
The major advantage of our approach over alternative index-based approaches10–12 is investigation of scenarios for which drug use data are not available. Our indices can thus provide insight into the potential impact of threats and interventions in a local context. For example, in the context of the ICUs of a large Toronto hospital in 2011, increasing prevalence of MRSA does not pose a major threat to empiric therapy because MRSA is already common. However, this conclusion is context-dependent; before MRSA became common,48 it posed a major threat to the empiric effectiveness of several important drug classes. The value of the new drug ceftazidime-avibactam is also context dependent; the drug would not substantially alter our ECI or our current ability to provide empiric coverage, but could be invaluable where carbapenemase (KPC)-producing Enterobacteriaceae have reduced the empiric effectiveness of carbapenems and aminoglycosides.3
Identifying threats is useful, but the pressing question is what to do about them. Broadly, index values can be improved by adding new drugs or reducing resistance to available drugs. Rapid identification of the infecting species18 could also improve index values by converting syndrome-based infection baskets (eg, VAP) to organism-based baskets (eg, Pseudomonas VAP), but if rapid tests preclude phenotypic resistance testing, the change could also reduce our ability to predict empiric coverage. Unfortunately, we do not know enough about the effectiveness of many stewardship and infection control strategies,5–9 ,47 ,49 in part because it is unclear what outcomes are most relevant, and in part because strategies may be effective in one context, but not in another. We hope our indices prove to be relevant outcome measures that help resolve apparent differences by explicitly acknowledging context-dependence.
Our example indices are based on a basket of device-associated infections that we believe to be representative in our context.23 However, other infections are important, and the causes of infections can vary substantially over space and time.21–23 ,50–54 We encourage users to choose an appropriate basket of infections for their context using local data where available, and caution against using our coverage estimates to guide clinical decisions. The weighted-incidence syndromic combination antibiogram (WISCA)15 ,41 ,42 is a more appropriate measure of coverage in a clinical context because it accounts for local variation in disease aetiology and reports the coverage provided by drug combinations. In vivo efficacy, side effects, previous drug exposure and other patient risk factors, previous culture results, rapid diagnostic results, administration route, costs and ecological impacts should also be considered.6 ,16 ,18 ,55–57
We also encourage users to keep assumptions in mind when interpreting results. For example, variation in the toxicity, effectiveness and ecological impacts of drug are important when considering the potential value of new drugs3 ,4 for empiric therapy, but our indices do not include this information. Neither ceftazidime-avibactam nor ceftolozane-tazobactam would substantially alter the ECI in our ICUs because other drugs still provide good empiric coverage in this context. However, ceftolozane-tazobactam may be preferable to aminoglycosides due to considerations of toxicity and efficacy28 and may provide an alternative to carbapenems that helps reduce selection for carbapenem resistant strains. Ceftazidime-avibactam should be reserved for treatment of carbapenem-resistant infections.
Despite simplifying assumptions, index-based approaches can yield useful information about the burden of resistance; Ciccolini et al11 successfully predicted the appropriateness of empiric therapy of complicated urinary tract infections using a drug resistance index (DRI).10 Poor treatment decisions and consideration of allergies, side effects and ecological impacts all reduce empiric coverage, so we expect the actual percentage of appropriate empiric therapy to be lower than our ECI.14 ,15 DRIs take these other considerations into account,10–12 providing complementary information to our ECI. One could distinguish inappropriate treatment attributable to resistance from inappropriate treatment attributable to poor drug choices and other considerations by comparing the ECI and DRI.
In our example of ICUs, the ECI remains high. This does not mean we are free of resistance problems. Remaining drugs are often more expensive,3 less effective28 or more toxic28 than alternatives, and some should perhaps be reserved for treatment of difficult infections.30 ,31 Administering multiple drugs to cover Gram-negative and Gram-positive bacteria increases costs and risks. Restricting the use of reserved drugs would reduce the ECI by more than 10%, implying that we now rely on reserved drugs for empiric therapy and that drug restriction would reduce the empiric coverage. Treating with a single non-reserved drug would further reduce the ECI, implying that dual therapy is also now necessary. Resistance testing, surveillance, infection control, prevention and stewardship interventions are also costly.3 ,5–9 ,13 Empiric and definitive therapies are often inappropriate,14 ,15 and difficult multidrug-resistant infections do occur.1 Our resistance rates are conservative because they use microbiology data from first isolates only, and therefore under-report resistance acquired during ICU stay which may or may not be due to antimicrobial therapy.13 The ECI only measures the impact of resistance on our ability to empirically cover initial infections.14
History suggests that the development of resistance to widely used drugs is inevitable;1 ,3 ,34 ,48 antibiotics are a resource that continue to be diminished by use.19 Although we can still provide empiric coverage in our ICUs, the losses are concerning, and measuring them is important. However, the value of our current stock of antibiotics depends on future development of resistance, which in turn depends on many factors,58 making measurement difficult. In order to construct the EOI, we assumed a very simple treatment model and a very simple linear relationship between drug use and resistance that applies equally to all drugs. The EOI formulation allows mixing to slow the spread of resistance,43 but we ignored the possibility in this analysis because parameter values are unknown. Increasing the value of redundancy would make our losses even more concerning. This simple linear model also likely undervalues drugs that provide full coverage because the effects of selection are generally smaller when the frequency of resistance is low.59 The use of one drug can select for resistance to others, but it is difficult to predict the precise relationships. Our simple model assumes resistance to each drug class develops independently. It would be interesting to investigate more realistic models58 but they would necessarily be less general. Our assumptions yield an EOI that depends only on coverage and is easy to calculate, while still capturing the understanding that more treatment options are better because drug use leads to resistance.
The indices we present here apply only to ICUs in North America, but our approach can easily be generalised to other contexts by defining an appropriate basket of infections and set of available drugs. Unfortunately, we do not expect a high ECI in all contexts. Across the globe, resistance rates vary widely, and the prevalence of carbapenem-resistant K. pneumoniae already exceeds 50% in some places3 which would substantially reduce the ECI. Drug availability also varies because the high cost of newer drugs can be prohibitive.3 We generally also expect a lower ECI for surgical prophylaxis60 and uncomplicated outpatient infections because none of the drugs that provide good empiric coverage in our ICUs are used for prophylaxis or empiric primary care. Options for treatment of severe hospital infections in North America are dwindling, but we can still provide empiric coverage, provided we use at least two drugs, aminoglycosides, and expensive last-resort drugs. Outside of ICUs and in many other parts of the world, empiric therapy is no longer adequate.
JSH thanks her family for patience, encouragement and support. The authors thank Allison McGeer, Warren McIsaac, Thomas Wong, Jun Wu, Lynora Saxinger, Jennifer Grant, Troy Day, Xi Huo, Chaim Bell, Marilyn Steinberg, and members of the MSH-UHN Antimicrobial Stewardship Team for helping translate medicine into math and vice versa. This work was presented on a poster at ID Week 2015, October 7–11, San Diego CA.
Contributors AMM, JW and AH developed the research question. JSH led the development of the indices and the writing. All authors provided invaluable guidance and feedback throughout.
Funding This work was supported by the Canadian Institutes of Health Research and the Natural Sciences and Engineering Research Council of Canada (grant number 446610-3).
Competing interests We do not have commercial or other associations that might pose a conflict of interest.
Ethics approval Mount Sinai Hospital Research Ethics Board.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Table 4 contains example antibiogram data.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.