Article Text

Download PDFPDF

Built to last? The sustainability of health system improvements, interventions and change strategies: a study protocol for a systematic review
  1. Jeffrey Braithwaite,
  2. Luke Testa,
  3. Gina Lamprell,
  4. Jessica Herkes,
  5. Kristiana Ludlow,
  6. Elise McPherson,
  7. Margie Campbell,
  8. Joanna Holt
  1. Australian Institute of Health Innovation, Macquarie University, Sydney, Australia
  1. Correspondence to Professor Jeffrey Braithwaite; jeffrey.braithwaite{at}mq.edu.au

Abstract

Introduction The sustainability of healthcare interventions and change programmes is of increasing importance to researchers and healthcare stakeholders interested in creating sustainable health systems to cope with mounting stressors. The aim of this protocol is to extend earlier work and describe a systematic review to identify, synthesise and draw meaning from studies published within the last 5 years that measure the sustainability of interventions, improvement efforts and change strategies in the health system.

Methods and analysis The protocol outlines a method by which to execute a rigorous systematic review. The design includes applying primary and secondary data collection techniques, consisting of a comprehensive database search complemented by contact with experts, and searching secondary databases and reference lists, using snowballing techniques. The review and analysis process will occur via an abstract review followed by a full-text screening process. The inclusion criteria include English-language, peer-reviewed, primary, empirical research articles published after 2011 in scholarly journals, for which the full text is available. No restrictions on location will be applied. The review that results from this protocol will synthesise and compare characteristics of the included studies. Ultimately, it is intended that this will help make it easier to identify and design sustainable interventions, improvement efforts and change strategies.

Ethics and dissemination As no primary data were collected, ethical approval was not required. Results will be disseminated in conference presentations, peer-reviewed publications and among policymaker bodies interested in creating sustainable health systems.

  • sustainability
  • health systems improvement
  • interventions
  • complex systems
  • systematic review
  • study protocol

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

  • Defining sustainability is challenging, making it difficult to develop inclusion criteria.

  • The protocol is multifaceted, with pluralist methods being deployed to identify useful articles.

  • An updated systematic review in this area is much-needed and will be a useful reference for clinicians, policymakers and researchers.

  • The search strategy has been refined by building on the search strategies of previous systematic reviews.

Introduction

Rationale

Health systems are facing a battery of formidable challenges. Populations are ageing1–4; there is a rising prevalence of chronic conditions5–8; complex patients have multiple comorbidities9–12; new technologies are creating new models of care13 14; 20% or more of healthcare spending is wasteful15; the role of the patient is changing with a growing ‘consumer culture’ and demand for patient-centred healthcare models16–19; there is pressure to increase standards of patient safety and quality of care20–23; the costs of care are rising,24 25 driven in part by high prices for new cancer and orphan drugs26–28; and there are increased fiscal pressures to pay for everything medicine can do.29 30 Every health system is striving for solutions that find and deploy viable methods to meet growing demands while capitalising on new technologies and ensuring that core processes of care remain of high quality.31 However, the problem is complex. Health system sustainability—the capacity to deliver affordable, cost-effective outcomes over time—requires numerous stakeholders, multiple approaches and coordinated actions undertaken across various system components.32 33 Sustainable health systems are ones that have sufficient resources to meet their objectives and are able to adapt to a changing environment34; in short, they keep up with developments, or leap-frog hurdles. One way in which policymakers, decision-makers and healthcare management try to achieve the sustainability goal is through the implementation of improvements, interventions and change strategies.

While older reviews have been conducted on this topic,35–43 a synthesis of the more recent evidence, regarding how disparate programmes and interventions are achieving sustainability and how they might contribute to or help inform system sustainability, is absent. Therefore, we propose a systematic review with an in-depth focus on the sustainability of such improvement programmes.

Defining sustainability

Sustainability is poorly defined in the literature,35 42 43 which has hindered the development of a consensus, evidence-based, operational paradigm for research and evaluation.43 44 A seminal report released by the World Commission on Environment and Development in 1987 articulated ‘sustainable development’ as that which ‘meets the needs of the present without compromising the ability of future generations to meet their own needs’ and as a ‘process of change in which the exploitation of resources, the direction of investments, the orientation of technological development, and institutional change are all in harmony and enhance both current and future potential to meet human needs and aspiration’.45 This transdisciplinary conceptualisation of sustainability construes it as a multidimensional dynamic interplay of economic, social and ecological factors.42

Regarding the sustainability of improvement programmes in healthcare, a focus on innovation and organisational development has led to the conceptualisation of sustainability as the ‘ongoing delivery of health programmes, which may be measured by the longevity of independent projects, or how well programmes become institutionalised in organisations or health and social systems’ (p1580).44 This approach has been criticised for promoting the continuation and institutionalisation of health programmes with insufficient prioritisation of enduring health outcomes. Gruen et al44 suggest that sustainability instead requires ‘ongoing cycles of reflection, planning, and action’ (p1587).44 Hudson and Vissing43 argue that health benefits may be better achieved through alternate programmes or treatments, therefore requiring the constant evaluation and evolution of existing programmes and interventions.43 They contend that a blinkered adherence to programme maintenance may fail to promote population health.

Envisaging sustainable interventions as static tools fails to take into account the complex adaptive nature of healthcare systems.43 44 46 Within a complex adaptive system framework, sustainable interventions can be better seen as another variable47 that acts on, and responds to, the dynamic system. We can potentially refine and improve interventions over time, to sustainably meet contextual needs and maintain desirable patient outcomes.33

Earlier this decade, Wiltsey Stirman et al35 noted that the current body of sustainability research is limited by a lack of working definitions and models of sustainability to guide researchers.35 In their review of sustainable interventions, 65% of studies did not provide an operational definition of sustainability, whereas those that did frequently cited Scheirer’s (2005) definitions, which are based on earlier work of Shediac-Rizkallah and Bone.48 Scheirer49 describes three separate operational definitions for interventions that promote sustainability: (1) the continued health benefits for individuals beyond the initial funding period; (2) the continuation of programme activities within an organisation; and (3) the continued ability of a community to develop and deliver health promotion programmes.49 In a later paper, Scheirer and Dearing47 defined sustainability as ‘the continued use of program components and activities for the continued achievement of desirable program and population outcomes’ (p2060).47 In our review we will consider an amalgam of Scheirer’s49 and Scheirer and Dearing’s47 definitions. We have selected these characterisations of sustainability based on an understanding of health systems as complex adaptive systems and the prioritisation of health outcomes alongside the maintenance of programmes or programme elements.

Prior reviews of sustainable health interventions and programmes

Several reviews have investigated the sustainability of interventions and programmes and their effects on outcomes, typically looking at different areas or levels of the health sector.35–44 Some have focused on sustainability in specific regions, such as Canada and the USA,36 or sub-Saharan Africa.37 Others have looked at specific types of programmes or interventions, such as chronic disease programmes and interventions,38 39 medical professionals’ adherence to clinical practice guidelines,40 and the influence of interventions on sustaining culture change.41 Approaches to achieving programme sustainability have also been investigated, without examining outcomes.42 43

Gruen et al44 conducted a broader systematic review looking at both empirical studies and conceptual frameworks of health programme sustainability.44 They focused on health programmes assessed over a defined period. The authors identified factors they believed to be associated with the programmes’ sustainability. These factors include programme design elements (eg, stakeholder involvement), organisational setting characteristics (eg, favourable organisational culture) and environmental features (eg, community engagement). The authors developed a conceptual framework for sustainability planning grounded in sustainability science, which regards health programmes as complex systems.

Likewise, Wiltsey Stirman et al’s35 review took a more expansive approach to studying sustainable interventions, while maintaining a focus on empirical studies. Without limiting their review by context, the authors examined a broad scope of studies to assess the sustainability of interventions, the outcomes they provided, and their influences in a variety of countries and health settings.35 They revealed a ‘fragmented and underdeveloped’ body of research suffering from a lack of methodological rigour and definitional consensus (p13).35 The authors note that the absence of validated measures, of programme monitoring post implementation and of real-time observations has also affected the evidence base. Five years later, with growing pressure on health systems, and increased interest in sustainable healthcare, there is a need to establish the current state of the evidence.

Objectives

Following Wiltsey Stirman and colleagues,35 the objective of our review is to provide an account of the sustainability of interventions, improvement efforts and change strategies across health settings. We aim to analyse research conducted since Wiltsey Stirman et al’s 2012 review in order to provide an updated synthesis of the literature in the past 5 years. As figure 1 shows, considerable growth in publications focused on sustainability in healthcare has occurred between 2013 and 2016, supporting the need for an updated review of the evidence.

Figure 1

Publication titles containing the words ‘health’ and ‘sustainability’, 1978–present. Adapted from Hudson and Vissing (2013), using data from Google Scholar.

Following Wiltsey Stirman et al, the review will be guided by the following research questions: (1) For the change strategy or intervention studied, has sustainability been defined and deployed in accordance with the evidence? (2) At what levels and units of analysis has it been studied? (3) What research methods have been used? (4) Over what time periods? (5) What outcomes have been reported in the empirical literature? (6) What were the findings? (7) What has research told us to date about influences on sustainability? (8) Were health outcomes sustained with continuation of the change strategy or intervention?

This systematic review will provide an essential contribution by synthesising the most relevant and up-to-date literature in this area. It seeks to provide important information for decision-makers, researchers, health professionals, clinicians and patients interested in collaborating on sustainable interventions, programmes and improvement efforts.

Methods

Eligibility criteria

Guided by previous reviews,35 37 46 49 50 studies will be included if they report on either the status of an ongoing intervention, programme or improvement, or the continued health benefits after the initial programme period, or programme funding, ends. Similar to Wiltsey Stirman and colleagues, there is no specified time frame between programme or funding completion, and assessment of outcomes. Rather, each study will be evaluated on a case-by-case basis. Studies that provide evidence on factors that influence sustainability will be included regardless of whether this was the primary aim of the study.

Outcome measures

Outcome measures will include objective measures of sustainability, such as improved health and safety,35 44 or cost reduction with sustained quality over time.38 Indicators of sustainability are expected to be highly heterogeneous, and consequently multiple methods of measuring sustainability will be considered.

Report characteristics

Following earlier reviews,35 37 46 49 50 publications will be assessed against the following inclusion criteria: English-language, peer-reviewed, primary, empirical research articles published after 2011 in scholarly journals, for which the full text is available. No restrictions on location will be applied. In order to provide a comprehensive review of the peer-reviewed evidence, grey literature will be excluded.

Information sources and search strategy

Our search terms, as detailed in the search strategy (table 1), are intended to cover a wide range of terminology used to define, measure and study sustainability. Search terms will be applied to the databases CINAHL, EMBASE and Ovid MEDLINE. These databases were selected due to their specific focus on biomedical, health system, allied health and nursing research. Healthcare-related subject headings (eg, healthcare delivery) will be employed to limit the search to healthcare settings.

Table 1

Search strategy

Additional search methods will be conducted to reduce the likelihood that relevant articles are overlooked. Applying a snowballing approach, a hand search of bibliographical references of key systematic reviews will be conducted, and experts in the field will be contacted for advice on potential studies for inclusion. Additionally, a title search will be conducted using the Scopus and Web of Science databases, which include articles from medicine and health sciences, in addition to the arts, humanities and social sciences.

Study records

Data management

Using the strategy specified in table 1, and informed by the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols statement, the initial search will be carried out by three researchers (JHe, KL and EM). These researchers will also examine the reference lists of pertinent reviews and contact appropriate experts in the field for advice on potentially relevant articles. Data will be imported into an EndNote library by LT and duplicates will be deleted.

Selection and data collection processes

To ensure consensus on the retained articles, abstracts from 5% of the EndNote library will be randomly assigned for assessment by pairs of reviewers (EM and JHe; KL and LT; GL and JHe) against the inclusion criteria. Inter-rater agreement rates will be calculated for each pair using Cohen’s kappa. Any discrepancies between authors concerning the inclusion or exclusion of articles will be discussed by all reviewers as a group, with JB as the arbitrator, until a consensus is reached. Each researcher will then independently review the remaining abstracts against the inclusion criteria.

Following this process, included abstracts will be randomly assigned to the reviewers for a full-text review against the inclusion criteria. A data extraction sheet will be used to record relevant information from included studies and reasons for exclusion for omitted studies (online supplementary file 1). It is expected that this process will begin soon after publication of the protocol, and we are scheduling to complete by mid-2018.

Supplemental material

Data items

The data extraction sheet will record article details, definition of sustainability (if provided), context and setting, number of sites, type of study, details of improvement or intervention, assessment period, measures of sustainability, and key findings for individual studies.

Outcomes and prioritisation

Following Wiltsey Stirman et al,35 and in line with Scheirer’s, and Scheirer and Dearing’s definitions of sustainability,47 49 outcomes refer to the ongoing impact or health benefits of interventions, programmes, change strategies and improvement efforts that continue after initial implementation efforts or cessation of funding. Priority will be given to studies that address sustainability over a longer time frame. For example, studies assessing the sustainability of an improvement intervention over years, as opposed to months, will provide more valuable information about sustainability and its long-term effects. Other studies to be prioritised include those that provide a working definition of sustainability and those that report on multiple sustainability outcomes.

Risk of bias in individual studies

Where appropriate, study bias will be assessed using a risk of bias template, specifically the Cochrane Collaboration’s tool for assessing risk of bias, adapted from the Cochrane Handbook for Systematic Reviews.51 Articles will be independently assessed and classified as ‘high’ or ‘low’ risk of bias. Consideration of bias will be given when interpreting the results of the review.

Data synthesis

Based on previous systematic reviews of this type,35 44 52 a quantitative meta-analysis of data may not be feasible. In the event that it is possible, a random-effects model will be used.53 Depending on the findings from the literature review, a scoping meta-review may also be undertaken.54

Where meta-analysis is not appropriate, data will be summarised using a narrative synthesis approach.55 The synthesis will focus on the overall evidence for sustained effectiveness of interventions, programmes and change strategies, including barriers and facilitators to their sustainability and the outcomes they produce. Articles will be grouped and discussed according to similarities and differences in their setting, participants, the research methods (eg, quantitative, qualitative or mixed method; cross-sectional vs longitudinal) and the results obtained. Possible areas of comparison include differences between micro and macro interventions, short-term and long-term programmes, and between low-income, middle-income and high-income countries. Results will be used to determine factors associated with sustainability.35 44

Meta-biases

In publishing this protocol we aim to avoid publication bias or selective outcome reporting by detailing our search and inclusion criteria, and by employing a data extraction form.56 Publication bias will also be limited by searching the reference lists of key systematic reviews and with the use of snowballing techniques to locate articles that may not have been detected in the database searches.53

Confidence in cumulative evidence

We will assess the quality of evidence using an appropriate assessment tool, such as the Grading of Recommendations Assessment, Development and Evaluation approach.57 Each study will be categorised by level of quality, in accordance with the chosen assessment tool.

Conclusion

The challenge of creating and maintaining a sustainable health system is an enduring problem faced by all health system stakeholders, including politicians, funders, providers, insurers, policymakers, taxpayers and patients. Ageing populations and increasing demands for services present substantial challenges to the affordability of healthcare systems, making the need for an urgent solution all the more necessary. We do not know enough about how interventions, programmes and improvement efforts, especially recent ones, are contributing to sustainability, nor the effect that they may have on system durability. The proposed review will provide a contemporary synthesis of the factors that influence the sustainability of interventions, improvement efforts and change strategies in health settings. It is anticipated that this review will be of value to researchers, policymakers and others interested in contributing to sustainable improvements in health settings and ultimately in health system performance.

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
  18. 18.
  19. 19.
  20. 20.
  21. 21.
  22. 22.
  23. 23.
  24. 24.
  25. 25.
  26. 26.
  27. 27.
  28. 28.
  29. 29.
  30. 30.
  31. 31.
  32. 32.
  33. 33.
  34. 34.
  35. 35.
  36. 36.
  37. 37.
  38. 38.
  39. 39.
  40. 40.
  41. 41.
  42. 42.
  43. 43.
  44. 44.
  45. 45.
  46. 46.
  47. 47.
  48. 48.
  49. 49.
  50. 50.
  51. 51.
  52. 52.
  53. 53.
  54. 54.
  55. 55.
  56. 56.
  57. 57.

Footnotes

  • Contributors JB conceptualised the study and leads the team’s work. LT and GL drafted the initial manuscript and search strategy, assisted by KL and JHe. Important contributions to refine and improve the manuscript were provided by JB, KL, JHe, EM, JHo and MC.

  • Funding This work was supported by the NHMRC Partnership Centre in Health System Sustainability (Grant ID 9100002) and NHMRC Program Grant APP1054146.

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.