Article Text

Protocol
Estimating the frequency of inpatient adverse events using a two-step retrospective chart review: a study protocol
  1. Syed Sabih ul Hassan1,
  2. Fozia Asif1,
  3. Ghazal Haque1,
  4. Farwa Ayub1,
  5. Tahir Munir2,
  6. Fauzia Khan2,
  7. Sameen Siddiqi3,
  8. Asad Latif1,2,3
  1. 1Centre for Patient Safety, The Aga Khan University, Karachi, Sindh, Pakistan
  2. 2Department of Anaesthesiology, The Aga Khan University, Karachi, Sindh, Pakistan
  3. 3Department of Community Health Sciences, The Aga Khan University, Karachi, Sindh, Pakistan
  1. Correspondence to Dr Asad Latif; asad.latif{at}aku.edu

Abstract

Introduction Adverse events are a major cause of patient harm in the hospitalised setting. Low-income and middle-income countries account for a disproportionate share of the global burden of adverse events. However, patient safety research is still centred around high-income countries and high-resource health systems. The methods and data produced from these efforts are ill-suited to low-income and middle-income systems due to the social and technical differences between these settings. We aim to use our pilot-tested, locally developed methodology to estimate the frequency and characteristics of adverse events in hospitalised patients in a lower-middle-income country to inform patient safety policies and initiatives.

Methods and analysis This multi-centre study will employ a two-step chart review methodology to identify adverse events in a representative sample of patients admitted at five hospitals between 1 January 2019 and 31 December 2019. The first step will include assessing patient files against a list of triggers to detect adverse events and the second step will involve an in-depth review of the events to capture pertinent characteristics. The triggers have been adapted from validated tools used in other studies. The reviewing team will be trained on the use of research tools and operational definitions to ensure that data are collected uniformly. The main outcome of interest is the rate at which adverse events occur in hospitalised patients. Further analysis will look to identify and quantify associations between the main outcome of interest and a variety of variables such as patient age and gender using tests of independence and regression techniques.

Ethics and dissemination This study protocol has been approved by the Ethics Review Committee at Aga Khan University (Reference number: 2023-6324-24566). The findings of this study will be published in a peer-reviewed journal and disseminated to the public through national and international conferences, workshops, websites and social media.

  • Health & safety
  • Quality in health care
  • Retrospective Studies
http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

STRENGTHS AND LIMITATIONS OF THIS STUDY

  • This is a large, multicentre study involving different types of hospitals which will enhance the generalisability of the results.

  • This study benefits from its design being tailored to the local healthcare setting by the use of an iterative approach based on the results of two separate pilots.

  • By training data collectors, adapting prevalidated research tools, and performing quality checks at regular intervals, this study will ensure the collection of accurate and reliable data.

  • This study is limited by using patient records as its sole source of information as the reviewers will be restricted by the availability and quality of information documented.

Introduction

Patient safety continues to present a formidable challenge to healthcare systems worldwide. At the crux of this issue are adverse events (AEs), which refer to instances of harm caused by medical care rather than underlying disease processes.1 The magnitude of this problem is staggering, and despite concerted efforts over the past decades, a significant quality gap persists, demanding urgent attention.2 Globally, unsafe care is responsible for millions of deaths each year, with both direct and indirect economic costs estimated to be in the trillions of US dollars.3 Even in high-income countries (HICs) with robust healthcare systems, 1 in 10 healthcare encounters result in an AE, with outcomes ranging from increased length of hospital stay to permanent disability and even death.3 However, the landscape of unsafe care in low-income and middle-income countries (LMICs) is far more imposing with nearly three-fourths of global disability-adjusted life-years lost due to iatrogenic harm being attributed to AEs in hospitalised patients in LMICs.4 Moreover, estimates indicate that one in four patients are harmed while receiving hospital care in LMICs, resulting in 134 million AEs annually and up to 2.6 million deaths per year.5

The WHO has released an action plan for the current decade, outlining the necessary measures to eliminate avoidable harm in healthcare.2 With zero patient harm as the goal, the WHO stresses the importance of generating a robust body of locally relevant data on unsafe care to understand why patient safety incidents occur, and ultimately, translating this into effective harm reduction and risk mitigation strategies. Additionally, these data highlight the scale and scope of this pervasive issue, which is a necessary step in raising awareness and securing support from stakeholders ranging from patients to policymakers. However, substantial disparities persist in the global landscape of health research, whereby HICs hold a dominant position in terms of research output and funding, significantly disadvantaging LMICs.6 7 The data derived from research endeavours in HICs exhibit limited applicability in LMICs due to inherent social and technical differences in their respective healthcare systems, as well as disparities in research priorities and interests that shape study design and methodological choices.8 Moreover, locally generated data have higher engagement rates with local stakeholders and so foster sustainable support for proposed solutions and initiatives.9 Therefore, LMICs need to design their own contextually informed studies to generate data with which to formulate patient safety policies and solutions as well as increase awareness and support for patient safety initiatives and advocacy groups.

Over the past three decades, various methodologies have been developed to identify AEs occurring during inpatient care. These approaches have included chart reviews, voluntary reporting systems and patient surveys. A pivotal contribution to this field was made by the Harvard Medical Practice Study, which published its findings in a seminal paper in 1991, establishing a two-step retrospective chart review as the gold-standard methodology for estimating the frequency of inpatient AEs.10 11 Building on this foundation, the Institute for Healthcare Improvement developed the Global Trigger Tool, which could detect 10-fold more AEs compared with other methods like voluntary reporting.12 Despite these advancements, initiatives to enhance patient safety in LMICs remained notably scarce. Consequently, the WHO published a methodological guide aimed at addressing patient harm in resource-constrained settings.13 Among its key elements is guidance on identifying AEs in data-poor hospitals through a variety of methods. However, it is crucial to recognise that each of these approaches has its own strengths and limitations, and requires adaptation to local healthcare settings to ensure the collection of accurate and reliable data.

The principal aim of this study is to estimate the incidence of AEs in hospitalised patients in a LMIC. We will be using a two-step approach to review patient records in order to identify and characterise AEs. We hope that this study will be an initial step towards the development of evidence-based patient safety solutions in the region. Additionally, we seek to provide future researchers with a locally developed methodology for the estimation of patient harm as well as a baseline rate of AEs against which to gauge the effectiveness of future quality improvement interventions.

Methods and analysis

Study design

This multi-centre study will employ a cross-sectional design and will involve a retrospective review of the medical records of all patients admitted between the 1 January 2019 and 31 December 2019 at a private, tertiary care hospital in phase 1, and four of its affiliated secondary care hospitals in phase 2, to identify any AEs experienced by inpatients during this time. The project is planned to begin on 1 August 2023 and we anticipate that it will be completed by 31 December 2024.

Study setting

The five hospitals chosen for this study are located in an LMIC in South Asia and they collectively provide care to over 100 000 inpatients annually.

Sample size and sampling technique

For the sample size calculation, the parameters used are as follows: an anticipated AE rate of 25.2%,14 precision of 2.5%, level of significance at 5% and an anticipated drop-out rate of 10%. The tertiary care hospital had 56 839 patients admitted in 2019, requiring a sample size of 1250 patients. This will be divided in proportion to the number of patients who were admitted to each clinical service. Similarly, all four secondary care hospitals admitted a combined total of 45 474 patients and so the combined sample size required is 1250 patients. This will be split among all four secondary care hospitals in proportion to how many patients were admitted to each hospital, and then each hospital will divide its sample in proportion to the number of patients admitted to each clinical service. The sample size distribution by hospital is shown in table 1. The sample size calculation was carried out using R.

Table 1

Sample size distribution

A list of patients, sorted by the date of admission, will be drawn for each clinical service. Systematic sampling using every kth (total number of admissions for the clinical service/total sample size for the service) patient will be used to achieve the required sample sizes.

Definitions

We define an AE as ‘unintended physical injury resulting from or contributed to by medical care that requires additional monitoring, treatment, or hospitalisation, or that results in death’.1

Data collection tools

Screening tools (tools IA and IB)

The screening tools are divided into two sections: patient characteristics and trigger lists. In addition to demographic information such as age and gender, patient characteristics also include diagnoses, comorbidities, and procedures performed which are coded according to the International Classification of Diseases, Ninth Revision, Clinical Modification.15 The burden of comorbidities will be quantified using the Charlson Comorbidity Index16 and the Paediatric Comorbidity Index.17 Tool IA will be used to screen for AEs in adult inpatients and can be found in online supplemental material tool IA. The trigger list used in Tool IA was adapted from the Global Trigger Tool developed by the Institute for Healthcare Improvement.18 Tool IB will be used to screen for AEs in paediatric inpatients and the trigger list used was adapted from the Paediatric Trigger Tool.19 Tool IB can be found in online supplemental material tool IB.

AE assessment tool (tool II)

Tool II will be used to delineate the characteristics of the AEs found, including the severity of the outcome, which will be classified according to The National Coordinating Council for Medication Error Reporting and Prevention’s Medication Error Index.20 Tool II is adapted from the work of Letaief et al21 and can be found in online supplemental material tool II. A single patient file may contain evidence of multiple AEs and every AE will have a corresponding Tool II applied.

Team composition and training

The investigating team will comprise reviewers and adjudicators. Reviewers will be individuals with at least an undergraduate degree in clinical sciences (physicians, nurses, etc). We have no restrictions on specialties and years of practice. However, those with specific experience with chart review or quality and safety will be given preference. Adjudicators will be patient safety leaders with a postgraduate qualification and multiple years of experience in a substantial role in patient safety or quality improvement. Each member of the reviewing team will undergo a total of 10 hours of training before beginning chart reviews. The training will comprise 2-hour sessions every week where the reviewers will be introduced to the study protocol, taught essential definitions, and instructed on how to apply the research tools. They will be trained on 10 test cases developed by the adjudicators. We hope these cases will allow the reviewers to familiarise themselves with the use of our research tools as well as highlight any areas of concern that need to be addressed by the adjudicators. A short pilot will then be conducted where each reviewer will be asked to review three test cases and the adjudicators will grade their performance to determine if the tools have been appropriately and consistently applied. Reviewers that are underperforming will have to undergo further training.

Data collection approach

The data collection will employ a two-stage review, as recommended by the WHO methodological guide for collecting data on AEs using retrospective record review.13 During stage 1, the team will review the medical records of the patients admitted during the study period and will assess the medical records against a list of triggers present in the screening tools. Triggers are specific events that allow for the selection of medical records with a high probability of containing AEs. We will be using separate screening tools for adult and paediatric patients; Tool IA for adults and Tool IB for paediatric patients. Each positive trigger will be checked for its association with an AE. During stage 2, our team will review the medical records that screened positive during stage 1. For each of these records, an in-depth review will be carried out using Tool II, the AE assessment tool. During this stage, reviewers will be encouraged to seek advice from adjudicators whenever they need additional knowledge to assess whether an AE has occurred and to assess its preventability. The hospitals use a combination of physical patient files in addition to electronic databases for information such as pharmacy orders and laboratory investigations to maintain detailed patient records. The Health Information Management Services are responsible for data governance and ensuring that all files are coded and catalogued according to hospital policy. We will be using the physical files and all available electronic databases to conduct our record review. To improve efficiency and data quality, the research tools will be converted from paper-based instruments into electronic templates on the browser-based software REDCap22 so that they may be accessed on laptops/computers/tablets. Furthermore, after every 250 records, the team will collectively review a sample of the records entered to ensure the accuracy and consistency of the data being collected.

Data analysis plan

We will be classifying AEs based on systems such as the one developed by Southwick et al.23 The confidence scores for preventability and causation will be dichotomised with positive responses being those that have a score above three and negative responses being those that have a score of 3 or less.

The main outcome measure will be the rate of AEs in the inpatient setting. The rate will be reported as the number of events per admission/patient days as appropriate. Furthermore, we will be looking at the frequency of the different types of AEs along with their outcomes, locations, preventability and contributing factors. In addition to reporting descriptive statistics, we will examine the univariate association between our main outcome measure and a range of variables, including age, gender, length of stay and type of admission, using χ² tests for categorical variables, and t-tests and analyses of variance for continuous measures, with non-parametric equivalents used as appropriate for ordinal variables. Multivariable analysis will be carried out using advanced regression techniques as appropriate and the outcomes will be reported as the ORs with 95% CIs. Data analysis will be carried out using R and STATA version 14.1.

Patient and public involvement

Patients and/or the public were not involved in the design, conduct, reporting or dissemination plans of this research.

Discussion

Patients incur significant harm during the course of receiving medical and surgical care. It is imperative to study these instances of harm to identify error-prone systems and generate insights that may inform future mitigation strategies. These studies have to be designed with respect to the social and technical context in which the AEs occur because their epidemiology varies based on the setting and study population. Initially, this study was based on the methodology employed by Letaief et al.21 However, an initial pilot conducted by the investigating team demonstrated that the methodology captured a low number of AEs. This discrepancy suggested that the triggers used were not well suited to our specific setting, possibly due to differences in the types of AEs encountered. To address this issue, a comprehensive literature review was conducted to identify more suitable trigger lists. The Global Trigger Tool and Paediatric Trigger Tool were selected and used for a second pilot that showed a significantly improved rate of capture of AEs. However, as has been previously noted by Hibbert et al,24 the Trigger Tools did not go beyond identifying the AEs and failed to capture pertinent details such as the preventability of events. To remedy this, we included an adapted version of the AE review tool used by Letaief et al into our methodology to allow for a thorough characterisation of all the AEs identified. We believe this two-step approach, of using a trigger tool to identify events and a second tool to capture their pertinent details, will allow us to address a significant gap regarding the regional burden of iatrogenic harm by providing an accurate estimate of the rate at which AEs occur in the inpatient setting along with their associated characteristics.

A notable strength of the study is that this is a multicentre study that uses five hospitals that differ in type, size, services offered, level of care and patient population. This serves to increase the generalisability of the results and so ensures applicability across similar settings. Moreover, by using standardised, well-defined triggers, training the reviewers beforehand, and performing quality checks throughout we expect the data collected to be highly reliable. However, we are anticipating that the rate we will obtain will still be an underestimation due to the fact that we are limited to using only previously documented information and we expect that poor documentation practices will hinder our attempts at faithfully recreating patient encounters. Moreover, we expect to miss out on AEs that manifest after the patient has been discharged unless the outcome was serious enough to warrant another encounter with health services. Furthermore, differing documentation practices between hospitals, such as the use of paper or electronic charts, may affect the AEs identified by the investigating team. We recommend that to overcome these shortcomings chart reviews should be paired with other approaches such as patient interviews as this combination would allow for a more holistic assessment of the patient journey.

In LMICs in the region, patient safety is not currently a national priority. This is the result of a near-complete lack of actionable data available for advocacy groups and government organisations. We hope to take the first step in highlighting the scale and scope of the problem, and in doing so, start a national conversation about this pervasive issue. Moreover, by publishing this methodology, we aim to enable other researchers to pursue similar projects in the region. Ultimately, we envision that the results of this study will be instrumental in informing institutional policies and mitigation strategies aimed at addressing the issue of inpatient AEs.

Ethics and dissemination

Ethical considerations

This study protocol has been approved by the Ethics Review Committee at Aga Khan University (reference number: 2023-6324-24566). We will generate unique identifiers against which all findings will be documented and every effort will be made to maintain the anonymity of the patients whose files are being reviewed. Data will be stored on encrypted, institution-approved cloud services which will restrict access to only the investigating team. Any changes to the study design will promptly be submitted for review to the ethics review committee.

Dissemination

The findings of this study will be published in a peer-reviewed journal and presented at academic conferences, both nationally and internationally. Furthermore, we plan to publicise the findings of our study through workshops, websites and social media to galvanise regional patient safety movements.

Ethics statements

Patient consent for publication

References

Supplementary materials

Footnotes

  • Contributors All persons listed as authors met the International Committee of Medical Journal Editors (ICMJE) authorship criteria. All authors have made significant contributions and revisions to the manuscript and have provided complete assent for publication. AL and FA were responsible for the conceptualisation of the study and for supervising all activities. SSuH, GH, FA, FK and SS made major contributions toward designing the methodology. SSuH, FA and GH were responsible for conducting the pilots and analysing the results. TM was responsible for calculating the sample size and formulating the data analysis plan. Each author is responsible for the content and has read and approved the final manuscript.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests None declared.

  • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.