Article Text

Download PDFPDF

Effectiveness of implementation interventions in improving physician adherence to guideline recommendations in heart failure: a systematic review
  1. Deepti Shanbhag1,
  2. Ian D Graham2,
  3. Karen Harlos3,
  4. R. Brian Haynes4,
  5. Itzhak Gabizon5,
  6. Stuart J Connolly5,
  7. Harriette Gillian Christine Van Spall4,5
  1. 1 Bachelor of Health Sciences Program, McMaster University, Hamilton, Ontario, Canada
  2. 2 School of Epidemiology and Public Health, University of Ottawa, Ottawa, Ontario, Canada
  3. 3 Department of Business and Administration, University of Winnipeg, Winnipeg, Manitoba, Canada
  4. 4 Department of Medicine and Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada
  5. 5 Population Health Research Institute, McMaster University, Hamilton, Ontario, Canada
  1. Correspondence to Dr. Harriette Gillian Christine Van Spall; Harriette.VanSpall{at}phri.ca

Abstract

Background The uptake of guideline recommendations that improve heart failure (HF) outcomes remains suboptimal. We reviewed implementation interventions that improve physician adherence to these recommendations, and identified contextual factors associated with implementation success.

Methods We searched databases from January 1990 to November 2017 for studies testing interventions to improve uptake of class I HF guidelines. We used the Cochrane Effective Practice and Organisation of Care and Process Redesign frameworks for data extraction. Primary outcomes included: proportion of eligible patients offered guideline-recommended pharmacotherapy, self-care education, left ventricular function assessment and/or intracardiac devices. We reported clinical outcomes when available.

Results We included 38 studies. Provider-level interventions (n=13 studies) included audit and feedback, reminders and education. Organisation-level interventions (n=18) included medical records system changes, multidisciplinary teams, clinical pathways and continuity of care. System-level interventions (n=3) included provider/institutional incentives. Four studies assessed multi-level interventions. We could not perform meta-analyses due to statistical/conceptual heterogeneity. Thirty-two studies reported significant improvements in at least one primary outcome. Clinical pathways, multidisciplinary teams and multifaceted interventions were most consistently successful in increasing physician uptake of guidelines. Among randomised controlled trials (RCT) (n=10), pharmacist and nurse-led interventions improved target dose prescriptions. Eleven studies reported clinical outcomes; significant improvements were reported in three, including a clinical pathway, a multidisciplinary team and a multifaceted intervention. Baseline assessment of barriers, staff training, iterative intervention development, leadership commitment and policy/financial incentives were associated with intervention effectiveness. Most studies (n=20) had medium risk of bias; nine RCTs had low risk of bias.

Conclusion Our study is limited by the quality and heterogeneity of the primary studies. Clinical pathways, multidisciplinary teams and multifaceted interventions appear to be most consistent in increasing guideline uptake. However, improvements in process outcomes were rarely accompanied by improvements in clinical outcomes. Our work highlights the need for improved research methodology to reliably assess the effectiveness of implementation interventions.

  • heart failure
  • change management
  • quality in health care

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

View Full Text

Statistics from Altmetric.com

Strengths and limitations of this study

  • While previous reviews have evaluated implementation interventions, to our knowledge, this review is the first to examine interventions to improve heart failure care, and to identify contextual factors associated with implementation success.

  • We conducted an extensive search of nine databases and include 38 studies spanning nine implementation intervention categories.

  • A limitation of our review is that moststudies (n=28) used observational or quasi-experimental designs, which are subject to bias and confounding. Only 10 studies were randomised controlled trials.

Introduction 

Heart failure (HF) has a prevalence of approximately 10% in the elderly, and is a common cause of hospitalisation and death in older adults.1 Patients diagnosed with HF have a 30% risk of mortality at 3 years, and those hospitalised for HF face a substantially higher risk.1 Patients with HF are classified as having reduced ejection fraction (ie, ≤40%) or preserved ejection fraction (ie, >50%).2 Evidence-informed treatments can improve clinical outcomes in HF, and recommendations surrounding their use are published in clinical practice guidelines.2–5 Class I/level A recommendations are supported by strong evidence, and are associated with reduced hospitalisation and mortality. Class I recommendations include the assessment of heart function and provision of self-care education for all patients with HF; for patients with reduced ejection fraction, class I recommendations also include specific pharmacological and device therapies.2 However, studies show that the uptake of these guidelines by physicians into routine clinical practice remains slow and inconsistent.6–8

Implementation interventions are designed to bridge the gap between evidence and practice, and are broadly classified at the provider, organisational or health system levels. Interventions may be single or multifaceted.9 Implementation success also depends on the intervention development process and organisational context. While previous reviews have evaluated implementation interventions,10–12 none, to our knowledge, have evaluated interventions within HF care or identified contextual factors associated with implementation success.

Accordingly, the primary objective of our review was to examine the effectiveness of implementation interventions in increasing physician adherence to the specified HF guideline recommendations. Our secondary objectives were to assess the effect of implementation interventions on clinical outcomes, and to identify process and contextual factors that influence implementation success.

Methods and analysis

The systematic review protocol is registered in the International Prospective Register of Systematic Reviews (PROSPERO: CRD42015017155), and published in a peer-reviewed journal.13 The only deviation from the protocol was the inclusion of uncontrolled before-after studies.

Eligibility criteria

We included trials evaluating one or more interventions aimed at improving physician adherence to class I HF guidelines, relative to usual care. Interventions were categorised by level (ie, provider, organisation or system level) and type (ie, education, decision support, financial incentives) according to the Cochrane Effective Practice and Organisation of Care (EPOC) taxonomy.9

Outcomes

While implementation interventions were targeted towards healthcare providers, outcomes were measured at the level of the patient (eg, number of patients receiving guideline-appropriate care). Primary outcomes were process indicators, defined as measures that assess guideline-consistent activities undertaken by a provider.14 The primary outcomes included the proportion of eligible patients with HF who: were prescribed a guideline-recommended pharmacological treatment such as β-blockers, ACE inhibitors (ACEI), angiotensin II receptor blockers (ARB) or mineralocorticoid receptor antagonists (MRA); were referred for implantable cardioverter defibrillator (ICD) and/or cardiac resynchronisation therapy (CRT) consideration; were provided self-care education at discharge; and/or had their left ventricular ejection fraction (LVEF) quantified. Secondary outcomes were clinical outcomes such as HF-related hospitalisations, readmissions and mortality. In the absence of HF-specific clinical outcomes, we extracted and reported all-cause clinical outcomes.

Study design

We included randomised controlled trials (RCT), cohort studies (with comparisons), controlled and uncontrolled before and after studies, and interrupted time series studies.

Study selection

We searched for all English language articles published since 1990 in MEDLINE, EMBASE, HEALTHSTAR, CINAHL, The Cochrane Library, The Campbell Collaboration, The Joanna Briggs Institute Evidence-based Practice Database, The Agency for Healthcare Research and Quality Evidence-based Practice Centers’ Research Reports, and the University of York Centre for Reviews and Dissemination Database. Our primary search strategy used the following terms: heart failure, guideline adherence, practice guideline, evidence-based medicine, implement (online supplementary appendix 1). Our secondary search included terms for each of the different EPOC intervention types and heart failure (online supplementary appendix 2). Two authors independently screened titles and abstracts, and then assessed select full-text articles according to the eligibility criteria.

Supplementary file 1

Supplementary file 2

Data extraction and management

Two authors independently extracted details about study design, statistical analysis, intervention, patient and provider characteristics, follow-up and outcomes using the EPOC Data Collection Checklist.9 In addition, the Process Redesign framework was used to extract and synthesise details on the intervention development process, and relevant contextual factors.15

Assessment of risk of bias

In addition to identifying the limitations inherent within specific study designs, two authors independently applied design-specific criteria to assess the internal validity of studies retained for analysis. We used the criteria outlined in the EPOC Data Collection Checklist to evaluate RCTs, cluster RCTs, controlled before-after studies and interrupted time series studies.9 For cluster RCTs, we used the additional criteria of recruitment bias, loss of cluster and incorrect analysis according to the Cochrane Handbook for Systematic Reviews of Interventions.16 For cohort studies, we used the Cochrane Collaboration’s tool to assess risk of bias in cohort studies.17 For uncontrolled before-after studies, we used the National Institute of Health’s quality assessment tool for before-after studies with no control group.18 Because our goal was to assess internal validity, we did not use tool criteria pertaining to applicability or external validity, precision and quality of reporting. We categorised studies as low risk of bias if one criterion was not satisfied, medium risk if two to three criteria were not satisfied and high risk if more than three criteria were not satisfied.

Data synthesis

We classified the implementation interventions according to the level targeted (provider, organisation and system) and the type of intervention (eg, education, decision support, audit and feedback, financial) using the EPOC taxonomy.9 An abbreviated version of the EPOC taxonomy is presented in table 1. We explored the suitability of a meta-analysis of the results within each intervention category by first assessing clinical heterogeneity at face value on the basis of included patient populations, settings (inpatient/outpatient), intervention types and outcome measures. We then assessed statistical heterogeneity using the I2 statistic, defining substantial heterogeneity as I2>75%. For studies not suitable for meta-analysis, we narratively synthesised results.19 20 We performed vote counting for each outcome measure in each EPOC intervention category, by noting the number of studies reporting significant improvements compared with those with no significant improvements.

Table 1

Cochrane Effective Practice and Organisation of Care taxonomy

Contextual factors

Context generally refers to the physical, social, political and economic influences on healthcare practices.21 We used the Process Redesign framework to systematically evaluate contextual factors that may influence the effectiveness of implementation interventions.15 The Process Redesign framework classifies context into categories: outer setting, inner setting and characteristics of individuals and teams. The inner context refers to the structural characteristics of the clinical setting (eg, inpatient, outpatient, community-based care, academic status), networks and communications, culture and climate. The characteristics of individuals and teams more specifically refer to professional roles, responsibilities and authority within the organisation. The outer context refers to factors related to the broader social, political and economic environment in which the intervention is applied. We considered processes that introduced and adapted the intervention to the organisation as part of the intervention, rather than the context. An abbreviated and modified version of the framework is presented in table 2.

Table 2

Adapted Process Redesign framework

Results

Identification, screening and selection of studies

Our systematic search produced 3742 unique articles, of which 3590 were excluded on the basis of title and/or abstract review. We assessed 152 full-text articles, of which 38 studies met eligibility criteria. We excluded articles that: were abstracts, protocols or letters (n=17); did not test implementation interventions (n=26); did not focus on patients with HF (n=4); had no comparator group (n=6); or had no outcomes of interest (n=61) (see figure 1).

Figure 1

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram of study selection. HF, heart failure.

Characteristics of included studies

Setting

A majority of the studies were conducted in the USA (n=26), and the remainder in Europe (n=10) and Australia (n=2). Sixteen studies were conducted in inpatient settings, twenty-one in outpatient settings and one involved care in both settings (table 3).

Table 3

Summary of studies evaluating strategies for the implementation of heart failure (HF) clinical guidelines

Types of implementation interventions

Thirteen studies offered interventions directed at the level of healthcare providers, 18 at the organisation level, three at the health system level and four across multiple levels. Provider-level interventions included: audit and feedback (n=4 studies),22–25 reminders (n=5),26–30 education (n=2)31 32 and a combination of these (n=2).33 34 Organisation-level interventions included: changes in medical records systems (ie, adaptations to existing systems on the basis of organisational need) (n=4),35–38 clinical multidisciplinary teams (n=8),26 39–45 clinical pathways (n=5)46–50 and continuity of care (n=1).51 System-level interventions included: financial incentives for providers (n=1)52 and financial incentives for institutions (n=2).53 54 Four studies offered interventions across multiple levels. A common feature across all six multifaceted interventions was the use of audit and feedback (table 3).

Study design

Among the 38 studies included, 10 were RCTs. Five were randomised at the level of patients,26 39 40 44 46 and five were cluster randomised by practice or hospital.22 23 31 33 41 Twenty-three studies used quasi-experimental designs: three were controlled before-after studies,32 41 53 two were interrupted time series studies34 35 and 18 were uncontrolled before-after studies.24 25 27–30 34–38 42 43 47 48 55 56 Four studies used a retrospective cohort design,45 49 50 52 while one used a combination of retrospective and prospective cohort designs51 (see table 3).

Risk of bias

Most studies had a medium risk of bias according to  design-specific criteria (online supplementary appendix 3). Five patient-level RCTs,26 39 40 44 46 and four of the five cluster RCTs had a low risk of bias.23 31 33 46

Supplementary file 3

Quality of reporting

We evaluated the quality of reporting in RCTs using the Consolidated Standards of Reporting Trials statement, including the extension for cluster RCTs. Among the five RCTs, four did not provide information on the methods of randomisation or allocation concealment.26 39 44 46 None of the five studies reported the precision of effect size estimates or provided relative effect sizes in addition to absolute risk differences.26 39 40 44 46 Among the five cluster RCTs, four did not provide information on the methods of randomisation or allocation concealment,22 31 33 41 three did not describe eligibility criteria,20 21 29 three did not provide sample size calculations22 33 41 and four did not provide intra-cluster correlation values.22 23 31 41

Outcomes reported

Thirty-seven studies reported the proportion of patients prescribed recommended medications (ie, ACEI/ARBs, β-blockers, MRAs); 30 studies reported prescription of indicated medications at any dose,22 24–26 28 29 33–40 42 44–50 52–54 57–59 and 12 reported prescriptions of medications at target doses.26 31 33 39 41 43–45 47 48 51 56 Other studies reported: patient self-care education prior to discharge (n=9)23 27 38 42 46 54 55 57 59; referrals for ICD/CRT (n=2)30 55; and LVEF assessments (n=11).27 33 34 38 46 47 49 53 54 57 58 In addition to these primary outcomes, 11 studies reported clinical outcomes such as mortality, hospitalisation and readmission rates.26 32 39 40 42 44–46 49 51 55 I2 calculations produced a value greater than 80% for most categories of interventions, precluding the possibility of a meta-analysis. Therefore, the studies were synthesised narratively.

Effectiveness of implementation interventions

A summary of study outcomes is presented in table 3. A majority of studies (n=32, 84%) reported significant improvements in at least one primary outcome.

Prescription of indicated medications

Reminders, clinical pathways, changes in medical records systems and multifaceted interventions were commonly associated with an increase in guideline-recommended prescriptions. In four studies that reported prescriptions of more than one indicated medication, significant improvements were observed in the prescription of β-blockers and MRAs, but not in the prescription of ACEIs. In these studies, the prescription rates at baseline for ACEIs were substantially higher than those of β-blockers or MRAs, ranging from 78.0% to 86.3%.28 34 36 48

Reminders

Two of four studies on reminders within electronic medical records (EMR) reported a significant increase in the per cent of patients prescribed an indicated medication.28 29 One study in which reminders were unsuccessful had suboptimal intervention fidelity; stratification by actual use of the reminder system revealed a significant improvement in prescription rates.27

Clinical pathways

Four of five studies on clinical pathways reported a significant increase in the per cent of patients prescribed an indicated medication.47–50 The single study that reported no significant change was an RCT in a remote community hospital, in contrast with the urban and/or teaching hospital settings of other clinical pathway studies.

Medical records systems

All four studies evaluating changes to EMRs reported significant increases in the per cent of patients prescribed an indicated medication.35–38 In each of these interventions, existing EMRs were enhanced by addressing identified limitations (table 3).

Combination interventions

Two studies evaluated combinations of provider-level interventions. A combination of education with audit and feedback did not significantly increase the per cent of patients prescribed an indicated medication,33 while a combination of education, reminders, and audit and feedback did.34

Four studies combined implementation interventions across different levels of the EPOC taxonomy.55–59 Two studies combined clinical pathways with audit and feedback; one reported a significant increase in the per cent of patients prescribed an indicated medication.55 Another study that combined a computerised order set, reminders, audit and feedback, financial incentives and provider educational meetings also reported a significant increase in the per cent prescribed an indicated medication.57 Finally, an intervention that fostered hospital-community integration using a combination of reminders, education for providers, audit and feedback, discharge summaries and patient follow-up by pharmacists58 reported a significant increase in β-blocker prescriptions in-hospital, and in all medications 6 months after discharge.

Prescription of target dose medications

Clinical multidisciplinary team interventions were consistently successful in increasing prescription of target dose medications, with five of six studies reporting significant improvements for this outcome.26 39 43–45 The five successful clinical multidisciplinary team interventions—including three RCTs26 39 44—involved nurses or pharmacists initiating or titrating medications according to a protocol. Among these studies, the absolute increase in proportion of patients prescribed target dose ACEIs ranged from 10% to 25.1%.39 43–45 The absolute increase in proportion of patients prescribed target dose β-blockers ranged from 23.9% to 43%.26 44 45 In contrast, an unsuccessful intervention tasked pharmacists with improving prescribing practices, without clearly defining the mechanism to do so.41

One of two studies47 48 evaluating clinical pathways reported a significant increase (from 6% to 13%) in prescription of target dose β-blockers.48 Of the two studies evaluating multifaceted interventions, an intervention combining education with audit and feedback reported a significant improvement (from 44% to 72%) in the prescription of target dose ACEIs,33 while a comprehensive intervention combining education, reminders, audit and feedback and clinical pathways did not report significant improvements.56 In the successful multifaceted intervention, feedback was focused strictly on medication dosing for individual patients.33

A study evaluating a continuity of care intervention, including the provision of instructions for medication titration to the outpatient general practitioner, reported a significant improvement (from 38% to 51%) in the prescription of target dose β-blockers within 6 months of discharge.51

Provision of patient self-care education

Only nine studies reported on the provision of self-care education to patients. Three multifaceted intervention studies reported this outcome measure, with a significant improvement in each case.55 57 59 Provision of patient education also increased with a reminder system,27 a clinical multidisciplinary team42 and a clinical pathway.46 In contrast, interventions that did not produce significant improvements included audit and feedback23 and changes to medical records systems.38 One study, on financial incentives, did not report statistical significance.

LVEF assessment

Eleven studies reported the per cent of patients who received an LVEF assessment. All three clinical pathway studies, including an RCT, reported significant improvements in this outcome.46 47 49 Of the two studies evaluating institutional financial incentives,53 54 only one reported significant improvements.53 Only one of three studies34 57 58 evaluating multifaceted interventions that included audit and feedback as well as reminders reported significant increases in LVEF assessment.34 Education,32 reminders27 and changes in medical records systems38 did not significantly increase LVEF assessment

ICD/CRT referral

Only two studies measured the per cent of indicated patients who received an ICD/CRT referral. These studies evaluated a reminder intervention,30 and a multifaceted intervention combining reminders, clinical pathways, education, and audit and feedback,55 respectively, with significant improvements reported in each case.

Evidence from RCTs

Very few RCTs were available for most intervention types; none were available for medical records system changes or financial incentives. Five RCTs evaluated the effect of clinical multidisciplinary teams on overall prescription rates26 39 40 44 and target dose prescriptions.26 39 41 44 Among these, two of four reported significant improvement in overall prescription rates,26 44 and three of four reported significant improvements in target dose prescriptions.26 39 44 Two RCTs evaluated audit and feedback interventions,22 23 with no significant improvements in the reported outcomes. An RCT evaluating education31 reported significant improvements for all outcomes measured, while an RCT assessing reminders26 reported no significant improvements. The RCT evaluating a clinical pathway46 significantly increased patient self-care education,46 and the RCT assessing a multifaceted intervention significantly increased the prescription of some target dose medications.33

Clinical outcomes

While five of the six studies reporting all-cause mortality successfully improved process outcomes, only two reported a significant decrease in mortality: an RCT evaluating a clinical pathway46 and a before-after study assessing a multifaceted transitional care intervention.58

While all six studies reporting all-cause hospitalisation or readmission rates improved process outcomes,32 39 42 46–48 significant improvements in the clinical outcomes were only reported in two: a multidisciplinary team study42 and a clinical pathway study.48 Both studies used a before-after design with medium risk of bias. There was no improvement in two studies assessing clinical pathways,46 47 one assessing multidisciplinary interventions,39 and one assessing an educational intervention.32

While three of four studies reporting HF-related hospitalisations or readmissions14 34 improved process outcomes, none reported significant improvements in the HF-related clinical outcomes.

Process of implementation

Six studies reported provision of preliminary training, education and resources to introduce clinicians to the implementation intervention and encourage utilisation; in each case, interventions were effective in improving at least one process outcome (table 3).23 27 40 47 48 Nine studies assessed barriers to guideline implementation at baseline and adapted the interventions accordingly.18 30 33 37 42 46 51 57 This was associated with implementation success for all interventions, with the exception of audit and feedback.46 Seven studies used an iterative process, where the programme was regularly updated on the basis of institutional requirements and user feedback.28 34 36 40 51 56 59 An iterative intervention development process was associated with implementation success across the range of interventions in which it was reported.

Contextual factors

Online supplementary appendix 4 presents the contextual factors influencing implementation interventions among the included studies. 

Supplementary file 4

Inner setting

Five interventions that improved at least one process outcome reported leadership support from either the department or hospital level.28 34 41 56 57

Outer setting

In nine US studies,28–30 36–38 42 56 59 there were pre-existing initiatives by the Centers for Medicare and Medicaid Services or The Joint Commission, including financial reimbursements or accreditation on the basis of HF readmission rates, and public reporting of quality of care data. These contextual factors encouraged organisations to implement interventions to improve guideline adherence. This is in contrast to the lack of success observed when financial interventions were used as the implementation intervention itself.

Discussion

In this systematic review, we assessed the effectiveness of implementation interventions aimed at improving physician adherence to class I HF guideline recommendations. We synthesised our findings narratively as the variation in study design, intervention and outcomes across studies precluded meta-analysis.

We found that a majority (84%) of the 38 studies reported significant improvements in at least one process outcome. A process outcome commonly reported across studies and interventions was the proportion of patients prescribed an indicated medication: 12 studies reported on the prescription of ACEIs,22 27 29 37 38 42 46 49 50 53 54 57 two on the prescription of β-blockers,26 47 12 on the prescription of ACEIs and β-blockers,24 32–35 39 40 45 48 52 58 and four on the prescription of ACEIs, β-blockers and MRAs.25 28 44 55 Electronic medical system interventions were associated with significant improvements in the prescription of at least one medication in 100% of studies (4/4 studies),35 37 38 60 followed by clinical pathways (80%, 4/5 studies),47–50 multifaceted interventions (66%, 4/6 studies)34 55 57 58 and reminders (50%, 2/4 studies).28 29 Very few studies on education or audit and feedback reported this outcome, making direct comparisons with other interventions challenging. However, on the whole, the results across a number of studies suggest that educational seminars30 and audit and feedback20 21 are minimally effective in isolation. Audit and feedback appears to be an important component of multifaceted interventions, however,34 55 57 58 and it is possible that factors such as the type of feedback and cointerventions to address gaps in care can influence its effectiveness.61

Results from RCTs reinforced overall findings that clinical multidisciplinary teams, with clear predefined responsibilities, seem to be especially effective in titrating patients to their target dose.26 39–41 44 These findings are important; despite evidence of dose-related improvements in hospitalisation and mortality, only a small proportion of patients with HF receive an appropriate dose of evidence-informed medications.62–64 A study using registry data from 21 European and Mediterranean countries from 2011 to 2013 found that while ACEIs, β-blockers and MRAs were used in 92.2%, 92.7% and 67.0% of patients, respectively, only 30% of these patients received medications at the target dosage.65

In general, improvements in process outcomes as a result of implementation interventions were rarely accompanied by improvements in clinical outcomes. In some studies, the gap between process and clinical outcomes may be attributed to insufficient statistical power to detect improvements in clinical outcomes.13 25 33 The gap may also be explained by study designs that did not account for background trends or adjust for confounding variables. Finally, HF clinical outcomes are multifactorial, and depend on the prescription of appropriate medications, the patient’s adherence to these medications, and follow-up care.32 The studies that showed a trend towards reduction in HF-related readmissions, although not significant, are those that addressed more than one of these factors.40 42

The context in which an implementation intervention is applied can influence its success.61 66 The limited contextual details available in the included studies made it difficult to identify facilitators of implementation efforts. In general, support of organisation leaders, and external policies and incentives for guideline adherence seemed to be associated with guideline uptake. These findings are consistent with results from a 2011 study that used iterative, formal discussions with leaders in patient safety and healthcare systems to identify leadership involvement and external factors (eg, financial or performance incentives or patient safety regulations) as context domains important to quality improvement initiatives.67

Consistent with existing literature,61 68 our results did not demonstrate a clear relationship between the number of intervention components and intervention success. An extensive review by Grimshaw et al concluded that while multifaceted interventions are not inherently more effective than single interventions, they may be more effective when built on a comprehensive assessment of barriers.60 69 70 Among the studies on multifaceted interventions in our review, the four studies that reported significant improvements in medication prescription rates carefully considered barriers at baseline and sought user feedback throughout the intervention development process.34 55–58

Our results are concordant with recently published findings from the American Heart Association’s comprehensive Get With The Guidelines-HF programme, which used a combination of educational approaches, multidisciplinary teams and public hospital performance reporting to improve care.71 The intervention was carefully adapted and introduced at each hospital site through collaborative discussions of barriers and solutions, and iterative plan-do-study-act cycles prior to the intervention phase.72

There were a number of limitations to our review. First, the variation in interventions, settings, study designs and outcome measures precluded meta-analyses, and in turn, our ability to draw substantive conclusions regarding specific implementation strategies and their comparative effectiveness. We chose to use a ‘vote counting’ approach to synthesis. While this method is useful in presenting an initial description of the trends found across studies, it is limited by the fact that it assigns equal weight to studies of varying sample sizes, effect sizes and significance levels.73

Another limitation was the methodological quality of the primary studies. Most studies used observational and quasi-experimental study designs. Quasi-experimental and observational designs possess some inherent risks of bias. In uncontrolled before-after studies, which formed the majority of studies in this review, temporal trends or sudden changes make it difficult to attribute the observed effects to the intervention alone. A time series design increases confidence with which the observed effect can be attributed to the intervention; however, it does not protect against simultaneous events that may influence the intervention effect. Controlled before-after studies can protect against these effects, but cannot match groups on the basis of unknown confounders. We found that most quasi-experimental and observational studies possessed at least a medium risk of bias. Though almost all included RCTs demonstrated low risk of bias, they were largely applied in the evaluation of multidisciplinary team interventions, and less so to the evaluation of other implementation interventions.

A minority of studies in this review (10 of 35 studies) were RCTs, considered the gold standard in establishing a causal link between an intervention and its outcome. Indeed, RCTs are an uncommonly used methodology in implementation studies. In a recent systematic review of implementation interventions for the management of intensive care unit delirium, only one of the 21 studies was an RCT, 16 were before-after studies and the remaining were cohort studies.74 In another review on implementation interventions to improve the use of pain management assessments for hospitalised patients, only three of the 23 studies were controlled clinical trials, and the remaining 20 were uncontrolled before-after or time series studies.75 While randomised trials are robust in methodology, they pose a number of logistical challenges that may make them suboptimal for implementation research; they are expensive and time consuming, often requiring years to complete.76 Changes in healthcare delivery are often implemented under internal and external pressures that seek to resolve an institutional problem in the shortest time possible. Under such circumstances, quasi-experimental designs are often felt to be most feasible.76 77 A solution may be found in pragmatic clinical trials—such as the stepped wedge cluster RCT—which can offer the methodological benefits of randomisation while being sensitive to the challenges of implementation research.78

Another limitation was that many studies failed to provide adequate details on the intervention, context, barriers, facilitators or fidelity to the intervention. A review by Proctor et al explores the reporting challenges in implementation research in significant detail. It offers a theoretical discussion of principles for naming, defining and specifying implementation interventions.79

Suggestions for future studies

We identify a number of ways in which future research on the effectiveness of implementation interventions may be strengthened. First, there is a need for implementation interventions to be evaluated using more robust study designs that also account for the pragmatic challenges of implementation research. Furthermore, reporting of studies should adhere to standardised guidelines in order to better facilitate comparison between interventions. An example of reporting guidelines is the Quality Improvement Minimum Quality Criteria Set, which spans the spectrum of intervention characteristics and contextual factors.80 Implementation research in HF may also benefit from more careful consideration of the contextual factors that influence implementation success. Finally, in addition to examining process outcomes, the direct impact of implementation interventions on clinical outcomes should be examined more consistently.

Conclusions

In this review, the heterogeneity of interventions, study designs and outcomes limited our ability to draw substantive conclusions regarding the comparative effectiveness of implementation interventions. Trends observed across the included studies suggest that effective implementation interventions include EMR systems, clinical multidisciplinary teams, clinical pathways and multifaceted interventions that include audit and feedback. There is a need for higher quality research to assess the effectiveness of implementation interventions on HF care processes and on clinical outcomes, and for the use of standardised reporting guidelines. Future work in the area should also include a closer examination of the organisational and external implementation context in order to better facilitate targeted application of implementation strategies.

Acknowledgments

We thank Quazi Ibrahim for his help in assessing whether the study was amenable to meta-analysis.

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
  18. 18.
  19. 19.
  20. 20.
  21. 21.
  22. 22.
  23. 23.
  24. 24.
  25. 25.
  26. 26.
  27. 27.
  28. 28.
  29. 29.
  30. 30.
  31. 31.
  32. 32.
  33. 33.
  34. 34.
  35. 35.
  36. 36.
  37. 37.
  38. 38.
  39. 39.
  40. 40.
  41. 41.
  42. 42.
  43. 43.
  44. 44.
  45. 45.
  46. 46.
  47. 47.
  48. 48.
  49. 49.
  50. 50.
  51. 51.
  52. 52.
  53. 53.
  54. 54.
  55. 55.
  56. 56.
  57. 57.
  58. 58.
  59. 59.
  60. 60.
  61. 61.
  62. 62.
  63. 63.
  64. 64.
  65. 65.
  66. 66.
  67. 67.
  68. 68.
  69. 69.
  70. 70.
  71. 71.
  72. 72.
  73. 73.
  74. 74.
  75. 75.
  76. 76.
  77. 77.
  78. 78.
  79. 79.
  80. 80.
View Abstract

Footnotes

  • Contributors HGCV and IDG conceived the study, and all authors contributed to the study design. DS informed the search strategy, extracted and synthesised study data, and drafted and edited the manuscript. IDG, KH, RBH and SJC contributed intellectual input and edited the manuscript for critical content. IG contributed to the search strategy and extracted study data. HGCV informed the search strategy, synthesised and analysed study data, drafted and edited the manuscript, obtained research funding, and supervised the conduct of the study.

  • Funding This study was funded by the Ontario’s Ministry of Health and Long Term Care (MOHLTC) Health System Research Fund and the Canadian Institutes of Health Research (CIHR). IDG receives support from a CIHR Foundation Grant. HGCV receives salary support from the MOHLTC and Hamilton Health Sciences Early Career Research Award.

  • Competing interests None declared.

  • Patient consent Not required.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data sharing statement All relevant study data can be found in the online supplementary files.

  • Author note This work is dedicated to the memory of Aubrey Ignatius Van Spall, beloved father and friend.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.