You are here

Article Text

Article menu
XML
Health services research
Protocol
Investigating the cost-effectiveness of health information technologies: a systematic review protocol
  1. Aziz Sheikh1,2,
  2. Ulugbek B Nurmatov1,
  3. Kathrin Cresswell3,
  4. David Bates1,2
  1. 1eHealth Research Group, Centre for Population Health Sciences, The University of Edinburgh Medical School, Edinburgh, UK
  2. 2Division of General Internal Medicine and Primary Care, Brigham and Women's Hospital/Harvard Medical School, Boston MA, USA
  3. 3The School of Health in Social Science, The University of Edinburgh, Edinburgh, UK
  1. Correspondence to Professor Aziz Sheikh; aziz.sheikh{at}ed.ac.uk

Abstract

Introduction There is a need to develop new, more cost-effective models of healthcare and in this vein there is a considerable international interest in exploiting the potential offered by major developments in health information technologies (HITs). Very substantial investments are, as a result, now being made globally, but these still probably only represent a fraction of the investments needed if healthcare is to make the transition from the paper to the digital era. Investing greater resources is, however, inherently challenging and unpopular at a time of financial austerity and this is furthermore complicated by the thus far variable evidence of health benefits and demonstrable short-term to medium-term returns associated with investments in HITs.

Objectives Building on our related systematic overviews investigating the impact of HITs, we now seek to estimate the cost-effectiveness of HITs and as a secondary aim to identify potentially transferable lessons in relation to how to realise returns on investments in these technologies.

Methods We will conduct a systematic review to identify the empirical evidence base surrounding the return on investments from implementing HITs. Two reviewers will independently search major international databases for published, unpublished and on-going experimental and quasi-experimental studies of interest published during the period 1990–2013. These searches of bibliographic databases will be supplemented by contacting an international panel of experts. There will be no restriction on the language of publication of studies. Studies will be critically appraised using the Critical Appraisal Skills Programme (CASP) Economic Evaluations checklist. In view of the anticipated heterogeneity in intervention investigated, study design and health system contexts, we will undertake a descriptive, narrative and interpretative synthesis of data.

Ethics and dissemination Ethical approval is not required.

Results These will be presented in one manuscript. The protocol is registered with the International Prospective Register for Systematic Reviews (PROSPERO) CRD42013005294.

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/

https://doi.org/10.1136/bmjopen-2013-003737

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • We will systematically identify and critically appraise the available evidence and grey literature on this important policy question and identify research in progress.

    • The main limitations are those that are common to all systematic reviews, namely the possibility of failing to identify all relevant evidence and the need to ensure that the synthesis of evidence is appropriate and meaningful.

    Introduction

    Health systems globally face substantial challenges associated with, among other issues, rapidly changing demographic profiles (in particular, ageing populations and the reduced proportion of the population, ie, economically active), increasing numbers of people living with long-term conditions and associated spiralling healthcare costs, and the substantial, persistent variation in the quality and safety of care provision.1 ,2

    There is, in response to these challenges, considerable policy interest in the potential of health information technologies (HITs) to support fundamental and far-reaching transformations of health systems, but achieving change on the scale required is, as our recent work and that of others has clearly demonstrated, challenging, time-consuming and expensive.3–6 Policy makers are therefore increasingly facing major challenges. These include questions surrounding the justification of substantial investments in HIT, particularly in times of austerity and unprecedented cuts in public investment, and associated pressures to achieve returns on investment in the short to medium terms.7 ,8

    For example, as in many other parts of the world, the USA and the UK governments have already committed to making the transition from paper-based health systems to digital health economies.9 ,10 Substantial investments have thus been made through the Health Information Technology for Economic and Clinical Health (HITECH) Act in the USA and the National Programme for Information Technology (NPfIT) in the UK.11 These however represent first-wave investments in basic underpinning technologies and infrastructures associated with electronic health record (EHR) functionality. Very substantial subsequent investments will be needed in order to build on these infrastructures and utilise data held within them to redesign health systems.4 ,12

    Given these challenges, there is a pressing need to better understand issues relating to the cost-effectiveness of HITs and also to identify strategies of proven value in enhancing economic returns on investments in HITs. This systematic review builds on our recent systematic overviews of the evidence investigating the impact of HITs,4 ,13 and is the first step in a related phased programme of work investigating approaches to enhancing returns on investments in HITs.

    Methods and analysis

    Design

    We will undertake a systematic review of the published, unpublished and in-progress empirical research literature.

    Eligibility criteria

    Interventions of interest

    We are interested in a broad range of HIT, which include tools to support the management of data (eg, EHRs), support and enhance clinical decision- making (eg, computerised physician order entry and computerised decision support systems) and provide care at a distance (eg, telehealthcare).4 ,13 ,14

    Study design

    The following study designs will be potentially eligible for inclusion:

    • Systematic reviews, randomised controlled trials (RCTs), quasi-RCTs, controlled clinical trials, controlled before-after-studies, interrupted time series and before-after or cohort type evaluations, undertaken with formal health economic evaluations from the perspective of healthcare providers.

    • Economic modelling studies.

    Outcome measures

    • Primary outcome

      • Estimates of the range of cost-effectiveness of HITs.

    • Secondary outcomes

      • Description of HITs that have been subjected to formal economic evaluation, the geographical, policy and clinical contexts in which these deployments have been made, the time frames over which the economic evaluations have been undertaken and the lessons learned.

      • Insights into interventions that are potentially cost-effective based on economic modelling studies.

    Search methods

    We will search the published empirical and grey literature from 1990 until 2013 for work investigating the returns on investment in HITs. This start date has been chosen to ensure that we capture the period when policy makers particularly became interested in the potential of HIT and also because the technologies that existed before this time period are likely to have only limited relevance to contemporary health systems.4 ,13 We will search the following major biomedical databases: The Cochrane Library (and its associated databases, namely: Cochrane Database of Systematic Reviews; Cochrane Central Register of Controlled Trials (CENTRAL); Cochrane Methodology Register; Database of Abstracts of Reviews of Effects (DARE); Health Technology Assessment Database; and the NHS Economic Evaluation Database), MEDLINE, EMBASE, Google Scholar, ClinicalTrials.gov and Current Controlled Trials. Specific search strategies will be employed for each database; an example of a search strategy developed for MEDLINE is shown in online supplementary appendix 1.

    Study selection

    Titles and abstracts of studies identified from the searches will be checked by two investigators. The full text of all retrieved potentially eligible studies will be independently assessed against the above criteria by these two investigators. The investigators will decide which of the studies satisfy the inclusion criteria and the methodological quality of eligible studies will be recorded (discussed below). Any disagreements will be resolved by discussion between the investigators, with referral to a third member of the research team, if necessary.

    Quality assessment and analysis

    Formal quality assessment of eligible studies will be independently undertaken by two reviewers using the Critical Appraisal Skills Programme Economic Evaluations checklist.15 Disagreements will be resolved through discussion, with arbitration by a third reviewer if agreement cannot be reached through discussion.

    Data extraction

    Data will be abstracted onto a customised data extraction sheet by two independent reviewers; any disagreements will be resolved through discussion, with arbitration by a third reviewer if necessary. Variables to be extracted will include: author and year; title of the study; country of origin; healthcare setting; HIT application(s); estimates of costs and cost-effectiveness; time frame over which evaluations are undertaken; lessons learned. Key findings from each study will be summarised and presented in summary tables.

    Data analysis

    A quantitative synthesis is likely to prove inappropriate due to the heterogeneity of technologies, study designs and care contexts. Data will therefore be descriptively summarised and narratively synthesised. We will follow four main steps in conducting an interpretive synthesis of our findings: (1) noting the range of functions and uses of existing systems; (2) developing a preliminary synthesis of the findings of included studies; (3) exploring relationships in the findings and (4) exploring the potential transferability of findings to other contexts and settings.16 There are likely to be challenges associated with integrating heterogeneous data sources, due to for example, differences in interventions studied, methods, outcomes, study population and context.17 We will attempt to address these by judging the relevance of heterogeneity, and the quality of evidence and consistency of findings of included studies.17

    Acknowledgments

    The authors wish to record their appreciation to Andrew Stoddart, health economist at the Lothain Health Services Research Unit, University of Edinburgh for his advice.

    References

    1. ↵
      1. Kohn LT,
      2. Corrigan JM,
      3. Donaldson MS
      . eds. To err is human: building a safer health system. Washington: National Academy Press, 1999.
    2. ↵
      Institute of Medicine. Crossing the quality chasm: a new health system for the 21st century. Washington, DC: National Academy Press, 2001.
    3. ↵
      1. Chaudhry B,
      2. Wang J,
      3. Wu S,
      4. et al
      . Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med 2006;144:742–52.
      OpenUrlCrossRefPubMedWeb of Science
    4. ↵
      1. Black A,
      2. Car J,
      3. Beyer K,
      4. et al
      . The impact of eHealth on the quality and safety of healthcare: a systematic overview of the literature. PLoS Med 2011;8:e1000387.
      OpenUrlCrossRefPubMed
    5. ↵
      1. Sheikh A,
      2. Cornford T,
      3. Barber N,
      4. et al
      . Implementation and adoption of nationwide electronic health records in secondary care in England: final qualitative results from prospective national evaluation in “early adopter” hospitals. BMJ 2011;343:d6054.
      OpenUrlAbstract/FREE Full Text
    6. ↵
      1. Robertson A,
      2. Bates DW,
      3. Sheikh A
      . The rise and fall of England's National Programme for IT. J R Soc Med 2011;104:434–5.
      OpenUrlFREE Full Text
    7. ↵
      1. Robertson A,
      2. Cornford T,
      3. Barber N,
      4. et al
      .; NHS Care Records Service Evaluation Team. The NHS IT project: more than just a bad dream. Lancet 2012;379:29–30.
      OpenUrlPubMed
    8. ↵
      1. Cresswell K,
      2. Sheikh A
      . Electronic health record technology. JAMA 2012;307:2255–6.
      OpenUrlPubMed
    9. ↵
      1. Blumenthal D
      . Wiring the health system—origins and provisions of a new federal program. N Engl J Med 2011;365:2323–9.
      OpenUrlCrossRefPubMedWeb of Science
    10. ↵
      Department of Health. Delivering 21st century IT support for the NHS: national strategic programme. London: Department of Health, 2002.
    11. ↵
      1. Morrison Z,
      2. Robertson A,
      3. Cresswell K,
      4. et al
      . Understanding contrasting approaches to nationwide implementations of electronic health record systems: England, the USA and Australia. J Healthcare Eng 2011;2:25–41.
      OpenUrl
    12. ↵
      1. Bitton A,
      2. Flier LA,
      3. Jha AK
      . Health information technology in the era of care delivery reform: to what end? JAMA 2012;307:2593–4.
      OpenUrlCrossRefPubMedWeb of Science
    13. ↵
      1. McLean S,
      2. Sheikh A,
      3. Cresswell K,
      4. et al
      . The impact of telehealthcare on the quality and safety of care: a systematic overview. PLoS ONE 2013;8:e71238.
      OpenUrl
    14. ↵
      1. Haas P,
      2. Sembritzki J
      . The European eHealth initative—objectives and solutions. European eHealth Initiative: http://subs.emis.de/LNI/Proceedings/Proceedings91/GI-Proceedings-91-2.pdf (accessed 29 Jul 2013).
    15. ↵
    16. ↵
      1. Cresswell K,
      2. Majeed A,
      3. Bates DW,
      4. et al
      . Computerised decision support systems for healthcare professionals: an interpretative review. Inform Prim Care 2012;20:115–28.
      OpenUrl
    17. ↵
      1. Mulrow C,
      2. Langhorne P,
      3. Grimshaw J
      . Integrating heterogeneous pieces of evidence in systematic reviews. Ann Intern Med 1997;127:989–95.
      OpenUrlPubMedWeb of Science
    18. PROSPERO. http://www.crd.york.ac.uk/prospero (assessed 29 Jul 2013).
      1. Liberati A,
      2. Altman DG,
      3. Tetzlaff J,
      4. et al
      . The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med 2009;6:e1000100.
      OpenUrlCrossRefPubMed

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

      Files in this Data Supplement:

    Footnotes

    • Contributors AS and DB conceived the idea for this study. AS developed the methods, and together with UBN, KC and DB drafted this protocol.

    • Competing interests AS is supported by The Commonwealth Fund, a private independent foundation based in New York City. The views presented here are those of the author and not necessarily those of The Commonwealth Fund, its directors, officers, or staff.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement The systematic review protocol is registered with the PROSPERO International Prospective Register of Systematic Reviews (http:// www.crd.york.ac.uk/prospero)18 CRD42013005294 and reported using Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) guidelines19 in the peer-reviewed literature. The systematic review will be in one manuscript and shared with others.