Objectives Review available evidence for impact of electronic health records (EHRs) on predefined patient safety outcomes in interventional studies to identify gaps in current knowledge and design interventions for future research.
Design Scoping review to map existing evidence and identify gaps for future research.
Data sources PubMed, the Cochrane Library, EMBASE, Trial registers.
Study selection Eligibility criteria: We conducted a scoping review of bibliographic databases and the grey literature of randomised and non-randomised trials describing interventions targeting a list of fourteen predefined areas of safety. The search was limited to manuscripts published between January 2008 and December 2018 of studies in adult inpatient settings and complemented by a targeted search for studies using a sample of EHR vendors. Studies were categorised according to methodology, intervention characteristics and safety outcome.
Results from identified studies were grouped around common themes of safety measures.
Results The search yielded 583 articles of which 24 articles were included. The identified studies were largely from US academic medical centres, heterogeneous in study conduct, definitions, treatment protocols and study outcome reporting. Of the 24 included studies effective safety themes included medication reconciliation, decision support for prescribing medications, communication between teams, infection prevention and measures of EHR-specific harm. Heterogeneity of the interventions and study characteristics precluded a systematic meta-analysis. Most studies reported process measures and not patient-level safety outcomes: We found no or limited evidence in 13 of 14 predefined safety areas, with good evidence limited to medication safety.
Conclusions Published evidence for EHR impact on safety outcomes from interventional studies is limited and does not permit firm conclusions regarding the full safety impact of EHRs or support recommendations about ideal design features. The review highlights the need for greater transparency in quality assurance of existing EHRs and further research into suitable metrics and study designs.
- health informatics
- health & safety
- health policy
- quality in health care
- risk management
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Strengths and limitations of this study
Scoping review to identify the gaps in research on assessing the impact of electronic health records) on patient safety.
Only interventional clinical studies were included.
Limitation of search to terms from a previously validated authoritative search strategy.
Exclusion of observational and feasibility studies.
Caring for patients with complex conditions safely and competently mandates having access to the right information at the right time.1 Ineffective sharing of information between providers and patients seriously impedes the quality and safety of patient care and is a leading cause of adverse events in hospital.2 Harm from medical care is common, has a significant associated morbidity and mortality and affects the mental health of staff as well as the financial performance of institutions.3 A small number of categories of patient harm account for the bulk of adverse events.4 Most interventions aimed at reducing harm have included introducing a digital health record while restructuring the patient documentation and communication.5
It is widely accepted wisdom that the introduction of comprehensive systems for documentation and communication such as electronic health records (EHRs) should improve the safer delivery of care. Mortality improves after implementation of EHRs in smaller non-teaching hospitals.6 The number of reported adverse events changes after implementation of EHRs with ‘meaningful usage’ functionality7 but it is unclear whether changes are due to improved practice or changed event reporting. There are technical standards for EHR implementation and metrics for meaningful usage have focused on technical and efficiency aspects but not safety outcomes.8 There is hence the need to review the existing evidence for this specific aspect of care at a time of increasing spread of EHRs.
The objective of this scoping review is to map key concepts as a basis for a deeper understanding on the effects of electronic record systems on commonly used clinical safety metrics while identifying gaps in our current knowledge to inform design of future research and the design of more effective EHRs.
Scoping reviews are a traceable method of ‘mapping’ areas of research and highlighting gaps in the literature for future research.9 Scoping reviews are a useful tool in the ever-increasing arsenal of evidence synthesis approaches and require rigorous and transparent methods to ensure that the results are trustworthy.10 We used OMalley’s and Arksey11 framework for undertaking a scoping review. This methodology summarises the evidence available on a topic in order to convey the breadth and depth of that topic by mapping the existing literature in a field of interest in terms of the volume, nature and characteristics of the primary research and identify gaps in the existing literature. In line with the methodology of scoping reviews, a formal evaluation of the quality of the studies was not undertaken.
The review included the following five key phases1: identifying the research question,2 identifying relevant studies,3 study selection,4 charting the data and5 collating, summarising, and reporting the results. A detailed review protocol can be obtained from the primary author on request.
A checklist for the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews can be found in online supplemental appendix 1.
This review was guided by the question: ‘How do patients admitted to hospital (P) benefit from implementation of an EHR (I) compared with patients not exposed to this or exposed to a different technology (C) in relation to commonly used outcome measures of safe care(O).’ Our PICO12 search strategy for identifying and selection of studies is outlined below. The studies were divided into categories based on similarities in their main objectives/findings and the themes discussed.
Data sources and search strategy
The initial search was undertaken in March 2019 on studies published between January 2008 and December 2018, in the following databases: PubMed (including MEDLINE) and Embase, the European Trials Register, the Australian New Zealand Clinical Trials Register, the International Standard Randomised Controlled Trial Register and the Cochrane Library with supplementary searches on Google. The databases were selected to be comprehensive and to cover a broad range of disciplines. No limits on language, subject or type were placed on the database search. The initial search was conducted in March 2019 with the supplementary searches run in December 2019.
We used a validated algorithm from a literature review on search terms for studies on patient safety13 that was subsequently used by an authoritative systematic review of interventions to reduce adverse events in hospital.14 Online supplemental appendix 2 provides a sample listing of the search query terms tailored to the specific requirements of each database.
Fourteen topics of patient safety were identified in the review,14 including adverse drug events, infection, delirium, adverse event after hospital discharge or clinical handover, fall, adverse event in surgery, cardiopulmonary arrest, venous thromboembolism, staffing, pressure ulcer, mechanical complication and underfeeding, clinical pathway, safety culture, external inspection. EHRs were defined according to the National Centre for Biotechnological Information as Media that facilitate transportability of pertinent information concerning patients illness across varied providers and geographical locations.15
Study selection process
The study initial selection for inclusion was based on the title and abstract of the studies that were reviewed to preclude waste of resources in procuring articles that did not meet the minimum inclusion criteria. Two of the authors (CS and GT) reviewed titles, references and abstracts generated by the original search against the agreed inclusion and exclusion criteria. When the title and abstract provided insufficient information to determine the relevance, a full-text copy of the article was retrieved and reviewed. For the final selection, a full-text copy of each study was examined to determine if it fulfilled the inclusion criteria. The references of eligible studies were manually checked to identify additional relevant studies that were missed in the database searches (snowballing). The studies were reviewed for their research design and internal validity and the references of the selected studies were manually checked to identify additional relevant studies that were missed in the database search.
Inclusion criteria: Record systems can be applied to inpatient or outpatient settings as well as to systems in community, primary or secondary care. This review focuses on medical record systems that are being used to support care of adult patients admitted to hospital wards. The review included publications identified in any language that reported experimental interventions in clinical trials that tested how records influenced patient safety. Only studies comparing two interventions or an intervention against usual or standard care were included. Studies excluded at this phase if they were found to not meet the eligibility criteria
Exclusion criteria: Study protocols, case series, technical descriptions, conference abstracts and studies limited to primary care records, outpatient care and highly specialised environments such as cardiac catheterisation laboratories, operating rooms or day-case units were excluded. Systematic reviews have been undertaken to document the safety impact of electronic prescribing systems. Studies examining the effects of interventions after hospital discharge were outside of the scope.
In order to validate the search strategy, additional searches were undertaken against the name of commonly used EHR vendors from the USA and UK identified from a Google search of EHRs companies. In order to assure the capture of important themes additional searches against the names of a sample of 12 major providers of electronic records was undertaken (online supplemental appendix 2). A total of 451 studies were screened. Four clinical trials that fulfilled inclusion criteria were identified. One of these16 reviewed safety alerts about gastrointestinal prophylaxis in a population that included inpatient and outpatient. The study did not allow to differentiate between the two groups and the study was thus excluded. Supplementary searches identified one further trial.17
Each article that met the study eligibility criteria was abstracted by using a standardised form based on a template by the Cochrane Collaboration.18 The data were extracted from the studies using an extraction tool that included the following items: article identifiers (authors, year of publication, objective); study identifiers (sample size, design, country, length of follow-up, inclusion and exclusion criteria); setting and population; outcome measures.
We organised the study characteristics in a tabular form. The identified studies were summarised according to key themes based on similarities of their main intervention and metrics and mapped against the 14 safety topics.
Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of the research. The study was not formally registered.
Patients or the public involvement
Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this literature review.
Role of the funding source
The Health Foundation provided funding for the study through an improvement science fellowship (CS). The funding agency did not participate in study conception, data collection, analyses, manuscript preparation, the decision to submit the manuscript for publication or any other part of the study.
The initial searches identified 60 articles for full-text review in the scoping review and further analyses. Twenty-four papers met the eligibility and inclusion criteria and underwent a full-text abstraction (table 1). Because of heterogeneity of the study designs, participants and outcome measures, a meta-analysis was not feasible. The flow of articles through identification to final inclusion is represented in figure 1.
General characteristics of included studies
The studies originated from a number of countries: 18 from the USA, three from Switzerland and one each from Australia, Belgium and Korea. The studies involved general hospital wards areas, critical care19 20 and laboratory settings.21 Studies almost exclusively originated from academic medical institutions.
Eleven studies were randomised controlled trials; 13 studies were observational before-and-after studies or parallel group studies comparing electronic records with paper records20–25 and other electronic records.25 ,26 The methodological quality of the studies was not formally assessed in line with the framework of scoping reviews.
The majority of studies involved only a single institution, some involved a group of hospitals and in one study, the authors reported from one geographical region.27 The small number of multicentre studies involving between 223 28 and 2929 hospitals. The study duration ranged from a single month to 5 years with most studies lasting 6–18 months.
The studies examined interventions created by installing new electronic systems, changes delivered within an existing system and changes between different electronic systems.
The unit of examination were patients, hospitals units, pathology specimens and categories of healthcare professionals: nurses, physicians, prescribers.
Processes by which EHRs aimed to effect changes in safety outcomes
The majority of studies used interventions that created information aimed to influence the behaviour of physicians or prescribers, one study was aimed at nurses and no study was aimed at patients. The interventions included randomisation that was delivered at hospital, clinical units, clinician or patient levels. The comparative studies reviewed changes in adverse event reporting in hospitals implementing EHRs to those that did not implement EHR or in clinical departments preimplementation and postimplementation. Alerts were created for a random sample of patients or for a random sample of clinicians. Most studies reported on compliance with processes associated with safe care. Only a limited number of studies reported on actual adverse events or harm.24 25 27 29–32
Metrics of impact
Results were mapped against the 14 predefined topics of patient safety (table 2): Significant evidence was identified for the topic of adverse drug events and limited evidence for the topics of clinical handover, venous thromboembolism, clinical pathways, pressure ulcers and falls. No evidence was identified for seven of the predefined topics.
Identified studies were linked to safety themes. The patient safety themes identified included (1) the use of electronic notifications as reminders or alerts,22 25 31 33–39 (2) electronic notifications specifically in relation to medication safety,19 20 22 23 26–28 31–36 40–43 (3) communication between teams,27 28 44 (4) prevention and treatment of infections,19 22 24 and (5) harm caused by the architecture of the EHRs.29 45 46
Theme (1): Electronic reminders: Automated notifications were used to alert prescribers to good practices in prescribing of antibiotics,19 22 24 43 prevention of falls and hospital acquired pressure ulcers,25 oral anticoagulants,35 thrombosis prophylaxis31 38 and nephrotoxic medications.33 36
Best practice alerts for prescribing of antibiotics on general wards22 elicited only a response in 19% of prescribers in one study, with most of the responders following the advice that resulted in a reduction in the number of broad-spectrum antibiotics prescribed.
A study in a medical intensive care unit used checklists for antibiotics in the EHRs.19 These checklists were more effective on their own when compared with additional face-to-face prompting by a dedicated resident in changing the antibiotics to empirical antibiotics. Adverse events were not reported. The length of stay in the intensive care unit and standardised mortality rates were not different between the intervention and control groups.
The electronic reminders for clinicians to prescribe oral anticoagulants in patients with stroke and atrial fibrillation35 resulted in a relative improvement in the rates of appropriate prescribing from 16% to 22%, however, the adverse effects were not reported.
The computer-generated alerts about rising creatinine levels that indicated acute kidney injuries resulted in a significantly higher rate of repeat creatinine laboratory checks.36 There was a small increase in the subgroup of surgical ward patients in the number of renal consults ordered and in subsequent dialysis sessions. The primary combined outcomes of maximum creatinine rise, dialysis or death at 7 days, however, did not change.
Implementation of risk assessments for falls and hospital acquired pressure ulcers led to improved documentation rates25: Falls rates did not change and the rate of hospital acquired pressure ulcers dropped continuously over the period of the investigation but no step-change after implementation of the EHR.
An electronic protocol for the clearance of the cervical spine after mechanical trauma resulted in improved documentation.37 A falls-prediction algorithm47 created a notification tool for falls prevention—this was tested against a non-matched control group.
Theme (2): Medication safety: The studies included reconciliation of medications,27 28 anticoagulants,31 35 antibiotic prescribing,19 22 24 acute kidney injury,33 36 calculating and monitoring of correct dosage32 42 43 and error-reporting.20 23 48 The effects on patient outcomes were either not reported or small and limited to subgroups of patients.
Theme (3): Communication between teams: Medication reconciliation on admission to the hospital was the focus of two studies.27 28 The reconciliation on hospital admission led to no measurable impact on safety outcomes. The electronic handover was related to a reduction in clinician reported ‘non-routine events’.44
Theme (4): Infection: The prescribing practice of antibiotics19 22 24 was examined. Significant impact on patient outcomes was reported in one study with a fall in only one of several examined nosocomial infections.24 A list of indwelling devices generated by the EHR was used to inform multi-disciplinary rounds with some evidence of lower exposure to risk.39 The evidence was lacking on surrogate metrics describing the clinical course of infections such as the patients’ white cell count, level of C reactive protein or vital signs.
Theme (5): Harm caused by the EHR: The potential harm caused by introduction of the EHR was measured through a novel ‘retract-and-reorder’ tool45 46 that captured when clinicians prescribed corrected prescriptions and were reordered again for other patients. The majority of these events were likely near-misses. A reduction of harm from ‘wrong patient’ orders were attempted through the repeat of identity checks/verification45 and a reduction in the number of maximum opened patient records.46 A summary nationally reported measure of patient harm was used in another study to quantify the impact of transitions between medical records.29
Additional gaps in understanding of impact of EHRs on safety outcomes
Studies reported limited explanatory context required to fully understand the likelihood of an impactful implementation such as staff workload, patient satisfaction, staff satisfaction or health economic outcomes. Staff satisfaction was measured in a single study44 and only one study reported a patient-reported outcome measure: Adverse events collected through telephone interviews in the study on electronic discharge notifications were not specified and not affected by the intervention.30
We found limitations in measurement of attributable harm at the patient level: A study examining the effect of a Health Information Exchange on adverse drug events found only 37 adverse events in 381 patients27: All reported adverse events were characterised by temporary symptoms (eg, pain) or temporary organ dysfunction (eg, a rise in creatinine), and none caused serious or permanent harm. A study using electronic alerts for acute kidney injury36 examined events such as the administration of contrast in patients-at-risk without clinical validation of the preventability of these events.
There was some degree of innovative functionality specific to electronic systems in relation to safety outcomes: An EHR specific ‘retract-and-reorder’ measure45 46 and a ‘patient safety composite measure’ for a selected validated summary indicator, the 'Patient Safety and Adverse Events Composite for the International Classification of Diseases' (PSI-90) 29 were described. We were unable to identify a single trial using personal health records (PHRs) or patient portals in a hospital that reported on safety outcomes.
This is the first scoping review, to the authors’ knowledge, to systematically evaluate the impact of EHR interventions on patient safety metrics in hospital. We found little published evidence for positive effects of EHRs on safety metrics that commonly feature in the literature such as hospital acquired infection, medication safety, allergies, falls. The review identified some evidence for a meaningful impact of EHRs in hospitals on surrogate outcomes that was largely restricted to changes in hospital prescribing practices. Limited follow-up periods might have been too short to capture the lasting effects beyond the immediate implementation period.
The review did not examine studies in primary care or paediatrics. Mortality was not included as a primary safety outcome as it depends on a large number of variables including the patient casemix but there are indications that patient mortality improves in a subgroup of hospitals that have implemented EHRs.6
Direct comparative clinical studies of EHRs by different vendors were missing. We were only able to identify two studies that directly compared EHRs. The first, a non-clinical study tested the safety processes in a simulated environment,49 and demonstrated large differences in the number of computer keyboard clicks and the time required to perform basic work tasks, and the second, an observational audit study that compared the prescription errors between two EHRs.50
We found no evidence for EHR related patient engagement at any level. Patients have been called the first line of defence or the ‘smoke alarm’ to raise alerts about potential patient harm and are able if invited to do so, to play a key role in monitoring their safety across the health continuum.51 52 PHRs held by patients might provide an obvious tool for enhanced patient safety but the evidence for a safety impact in primary care is limited to medication safety.53 The American Veterans Administration Healthcare system has undertaken a robust evaluation of their PHR that indicates a better adherence to treatment plans but little data on whether this adherence leads to safer or cost-effective care54 and patients’ active contribution to documentation systems in hospital is likely to enhance care.55 56
Our scooping review has several limitations. First the search strategy was limited to safety outcomes predefined by a group of experts4 and we focused exclusively on EHRs. It is not clear whether other safety relevant outcomes could have been found in other studies of EHRs. Second, we focused on interventional studies to obtain a higher graded evidence and it is possible that safety outcomes are described in observational studies. Third, there is an understanding that monitoring systems for specific diseases that can be displayed through an EHR might be of benefit for safety outcomes such as measuring blood sugar levels in patients with diabetes57 or the electrocardiogram in patients with a coronary event.58 For unselected patient groups, there is evidence for the value of systems’ monitoring of vital signs that might be linked to an EHR or have their own recording systems; these authors have illustrated an impact on relevant clinical and safety outcomes59–61 although with some methodological challenges.62 Fourth, the studies identified in this review used exemplar conditions and applications of electronic records. Frameworks to classify safety incidents in a broader, real world context63 64 are missing. Fifth, the number of studies identified was small and despite using a robust, systematic search strategy we were unable to generate a hierarchy of effective or ineffective EHR interventions. The comparison between EHR systems is difficult given the lack of operational and interoperative standards,65 the lack of transparent data by the vendors and even in a simulated environment straight comparisons are exceedingly rare.49 Sixth, the overwhelming number of studies originated in the USA which is highly influenced by the US healthcare regulatory and reimbursement schemes that are rather different from other healthcare systems. Finally, scoping reviews are not intended to assess the quality of the literature analysed. Nevertheless, this scoping review provides a comprehensive overview of the existing research and has clearly identified key themes and challenges for broader research which is needed.
EHRs can be used in many different ways in different hospitals. Linking the EHR intervention to a specific outcome might therefore be challenging even where process changes are the endpoints. Randomised trials might not be the most appropriate methodology for EHR evaluation and other generic service interventions because the effects at system level might be too diffuse. Carefully designed observational and adaptive interventional studies are needed to allow appropriate evaluation of service and policy interventions in this area.66
The authoritative peer-reviewed search strategy deployed to identify publications reporting on patient safety topics uses a mix of process and outcome measures. Definition of these is subject to interpretation—that is, organisational culture could be used as an outcome measure as part of the quadruple aim or as a process that facilitates better quality of care for patients. Conceptually it would, however, be difficult to identify changes in outcomes without a model of change that does not involve some measure of change in process. Outcomes will of course depend on fidelity of implementation of processes but the absence of changes in safety critical processes is therefore likely to signify an absence in changes in safety outcomes.
The implementation of EHRs has got safety implications well beyond the scope of this review which range from the reliability of software and hardware, design or systems and user interfaces and risk of abuse and fraud.67 We have also not examined the broader context of implementations: evidence suggests that nurses working in hospitals with no EHRs report poorer quality of care and patient safety68 and cultural context and trust might modify impact.69–71
Clinicians at the coal-face of care complain bitterly about poorly designed and supported EHR systems, which have unsuitable interfaces,72 add to workload, and fail to respond to change requests in a timely manner.73 EHR’s are reported be the number one reason for clinician burnout and dissatisfaction.74 Given the enormous investment costs in the development and deployment of the technology and the emerging evidence of the negative effects of EHR on clinician burnout,75 76 the lack of reported benefits in sustainable and measurable safety outcomes is surprising. We share the concerns of others that there is largely ‘anecdotal evidence of the fundamental expected benefits and risks relating to the organisational efficiency resulting from the storage and management facilities within the EHR and thus the potential for secondary uses’.77 Health information systems designed for and by a clinical teams using a technology that enables real-time adaptation might provide greater efficiency for the staff in decreasing the time to complete standard tasks.78
Unstructured and fragmented information is at the core of countless serious adverse events and the link between fragmented information and patient harm is well established in the literature.79 Human factors and ergonomics design is part of the safety assurance of medical devices80 but not the commonly used EHRs.
The EHRs are among the most expensive capital investments that health system leaders undertake with cost for an installation by a single organisation up to a billion dollars81 despite the absence of evidence for cost-effectiveness,82 and routine complaints about the deleterious effects of implementation on clinicians and their workflow.83 EHRs have been introduced with an expectation of workflow and safety improvements that have failed to materialise.84 An Australian study demonstrated that systematic errors in the usage of EHRs are common, and the audited files of 629 patients admitted to hospital were found to contain 493 errors related to the EHR and accounted for 42% of prescription errors,85
Our review outlines a rich area for several key research questions, including the need to develop a clearer description of EHR interventions, using uniform and validated outcomes measures, and attending to care provider’s needs, attitudes and training.86 Given the erosion of trust in the data safety of large projects, smaller pilots in multiple locations might be needed to develop EHR systems that aid patients, professionals and policy-makers.87 Enormous amounts of data relevant to patient safety are collated within EHRs. It is likely that hospitals and vendors are undertaking internal reviews of safety outcomes for purposes of audit, quality improvement, internal quality assurance or research. Given the size of the investment in EHRs and the adverse public health impact of patient safety it would seem that these type of datasets should be made public for research and quality assurance.
The clinical consequences of EHR use for patients might be considerable but the available studies suggest a limited understanding about the safety or potentially harmful outcomes following the implementation of EHRs. The literature contains inadequate evidence to guide policy or a digital strategy for healthcare jurisdictions in how best to select and implement EHRs.
Our review highlights an urgent need for greater transparency in quality assurance of existing EHRs and further research into suitable outcome metrics and appropriate study designs.
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information.
Patient consent for publication
This study did not involve human material or human data, so an ethics approval was not needed.
Correction notice This article has been corrected since it first published. The provenance and peer review statement has been included.
Contributors CS and PB designed the study; CS and GT screened the articles, CS, GT and PB synthesised and interpreted the data; CS, GT and PB drafted and revised the manuscript; all authors approved of the submitted version to be published; all authors agreed to be accountable for all aspects of the work.
Funding This work was supported by for an Improvement Science Fellowship from The Health Foundation, London, UK (Unique Award Reference Number: AIMS 109820).
Competing interests CPS is partially funded through an Improvement Science Fellowship to investigate the use of Personal Health Records on patient safety.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.