Article Text
PDF
Abstract
Introduction Medication errors (MEs), which occur commonly in the perioperative period, have the potential to cause patient harm or death. Many published recommendations exist for preventing perioperative MEs; however, many of these recommendations conflict and are often not applicable to middle-income and low-income countries. The goal of this study is to develop and disseminate consensus-based recommendations for perioperative medication safety that are tailored to country income level.
Methods and analysis The primary site of this mixed-methods study is Massachusetts General Hospital/Harvard Medical School. Participants include a minimum of 108 international medication safety experts, 27 from each of the World Bank’s four country income groups (high, upper-middle, lower-middle and low-income). Using the Delphi method, participants will rate the appropriateness of candidate medication safety recommendations by completing online surveys using RedCAP. We will use Condorcet ranking methods to prioritise the final recommendations for each country income group. We will execute a comprehensive dissemination strategy for the recommendations across each country income group. Finally, we will conduct semistructured interviews with our participants to evaluate the initial adoption and implementation of the recommendations in each country income group.
Ethics and dissemination This study was approved by the Human Research Committee/Institutional Review Board at Partners Healthcare (2019P003567). Findings will be published in peer-reviewed journals and presented at local and international conferences.
Trial registration number NCT04240301.
- health & safety
- protocols & guidelines
- quality in health care
- adult anaesthesia
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
Robust mixed-methods study design to include a large number of participants.
Recommendations that target the entire perioperative medication use process.
First medication safety initiative to include equal representation from each of the World Bank’s four country income groups, allowing for generalisability of recommendations internationally, regardless of country income level.
World Federation of Societies of Anesthesiologists endorsed the study and deemed it to be a priority.
While we will translate all study documents and have interpreters available in each of the WHO’s six official languages (Arabic, Chinese, English, French, Russian and Spanish), language barriers may impact the participation of experts who are not fluent in any of these languages.
Introduction
Perioperative medication errors (MEs) have the potential to cause serious patient harm. Growing evidence indicates that MEs and adverse medication events are as common in the perioperative setting as they are in other hospital environments.1 2 However, medication use in the perioperative setting presents particular challenges to patient safety. The delivery of medications in the operating room usually bypasses standard safety checks, such as electronic physician order entry systems that include clinical decision support and alerts, approvals by pharmacists and double-checks by nurses prior to medication administration. Furthermore, the high-stress, time-sensitive nature of work in the operating room can contribute to high rates of MEs and errors of greater severity in the operating room compared with other clinical settings. In the operating room, syringe swaps, ampoule swaps and wrong dose errors can cause serious harm.3 In fact, the most frequently cited critical adverse events in anaesthesia are MEs.4–6 Surprisingly, after decades of decline, the worldwide death rate during anaesthesia is once again increasing,7 and MEs may be a contributing factor.
Many published recommendations exist for reducing the incidence of perioperative MEs,8–10 some of which have been endorsed by national or international professional organisations such as the Anesthesia Patient Safety Foundation,8 the European Board of Anaesthesiology,10 and the Australian and New Zealand College of Anaesthetists.11 Often these recommendations offer conflicting advice—for example, some recommend that all syringes be labelled (even when possible in emergency situations),11 whereas others endorse preparing and immediately administering a medication without a label if the syringe does not leave the provider’s hand.10 It is imperative to standardise and optimise the recommendations for safe medication use.
Many of the existing recommendations that aim to prevent perioperative MEs are not feasible in middle-income and low-income countries. While MEs may be similar in type and number between high-income and middle-income or low-income countries, the interventions needed to improve medication safety may differ between these groups due to financial and resource constraints. For example, one common recommendation to prevent syringe swaps in high-income countries is the use of prefilled syringes that couple with point-of-care bar code scanning and clinical decision support systems. This recommendation is not currently affordable in low-income countries.9 Instead, providers in low-income countries could use a two-person verification approach for high risk medications, and focus on the use of extra vigilance when reading the labels on syringes and vials.9
Currently, no clear recommendations for perioperative medication safety exist that are tailored to country income level, or that consider a hospital’s existing processes and technologies. Our study will address this gap by creating the first set of recommendations that are specifically tailored to the World Bank’s four country income groups: high, upper-middle, lower-middle and low-income countries.12 Consequently, this study has been endorsed and deemed a priority by the World Federation of Societies of Anesthesiologists (WFSA). Our specific aims are to:
Develop consensus-based recommendations for perioperative medication safety.
Prioritise the recommendations by their level of clinical importance and feasibility of implementation in high, upper-middle, lower-middle and low-income countries.
Disseminate the recommendations.
Evaluate the initial adoption of the recommendations in each country income group.
Methods and analysis
Study design
The goal of this mixed-methods study is to create consensus-based recommendations for perioperative medication safety. The study will be conducted in five parts. First, we will develop a set of candidate recommendations using an extensive review of the literature. Recommendations will address the entire medication use process (ordering, dispensing, preparing, administering, documenting and monitoring) in preoperative holding areas, operating rooms and post-anaesthesia recovery areas. Second, we will use the extensively studied Delphi method11 13–16 to achieve consensus on the candidate perioperative medication safety recommendations. The Delphi method was developed by the RAND Corporation (Santa Monica CA) and the Univeristy of California Los Angeles (UCLA, Los Angeles CA). Third, we will prioritise the recommendations for implementation in each of the four country income groups. Fourth, we will disseminate these recommendations. Finally, we will use semistructured interviews with a grounded theory analysis approach to assess the initial adoption of the recommendations. Our methodological approach for each of these five activities is presented separately below.
Study population and recruitment
Our expert panel will consist of a minimum of 108 members, 27 from each of the World Bank’s four county country income groups: high, upper-middle, lower-middle and low-income (as defined by the World Bank Atlas method).12 Each of the four groups of 27 expert panel members will be comprised anaesthesiologists (n=17), surgeons (n=2), operating room nurses (n=2), nurse anaesthetists (n=2), pharmacists (n=2) and medication safety experts (n=2).
To recruit anaesthesiologist participants, the study team will contact each of the 134 national societies of anaesthesiologists that are members of the WFSA. These member organisations represent anaesthesiologists from over 150 countries, including those from 45 (32.6%) high-income, 42 (30.4%) upper-middle income, 34 (24.6%) lower-middle income and 17 (12.3%) low-income countries.17 Each national society leadership will be asked to recommend members who are local experts in medication safety, for our research team to contact to personally ask to participate in this study. This process will be repeated to recruit participants from surgical, nursing and pharmacist national professional societies.
Part 1: development of candidate recommendations by literature review
Our research team performed an extensive literature search to identify publications containing recommendations for perioperative medication safety. Our search included PubMed (MeSH terms Drug/Medication Error, Drug/Medication Safety, Operating Room, Anesthesia), and an internet search for recommendations released by national agencies and professional societies such as Anesthesia Patient Safety Foundation,8 the European Board of Anaesthesiology,10 the Australian and New Zealand College of Anaesthetists11 and the WHO.18 We created a database of all published recommendations, deleting duplicate recommendations. This resulted in a final set of 133 recommendations, in the following categories: standardisation (77, 57.9% of recommendations), technology (9, 6.8% of recommendations), medication use process (42, 31.6% of recommendations) and culture (5, 3.8% of recommendations). These recommendations will serve as the candidate recommendations for our first round Delphi survey.
Part 2: development of medication safety recommendations using a Delphi approach
Round 1: we will use the extensively studied RAND-UCLA Delphi method13–16 to achieve consensus on the perioperative medication safety recommendations, using the preliminary candidate recommendations as a starting point. We will modify the Delphi method to use electronic surveys in order to allow participants to participate from remote locations. We will develop electronic surveys using RedCAP (Nashville, Tennessee, USA), that will be sent to expert panel members by email to ask them to rate the appropriateness of each candidate recommendation on a 9-point scale, with a score of 1 denoting inappropriate and 9, appropriate. If a panel member rates a recommendation 6 or below, they will be asked to provide feedback (free text) to improve the recommendation. Recommendations with a median rating of 1–3 without any disagreement among panel members will be discarded. Recommendations with median ratings of 7–9 without disagreement will be included in our final recommendations. Those with median ratings of 4–6 or any median with disagreement among panel members will be considered uncertain. For non-English-speaking participants, surveys will be translated into the six official WHO languages: Arabic, Chinese, English, French, Russian and Spanish.
Due to the unique medication safety hazards and a paucity of literature on medication safety in low-income and middle-income countries, the survey will also ask participants to provide additional recommendations that we may not have captured in the survey. Recommendations suggested by five or more of our expert panel members from a single country income group will be included as candidate recommendations in the second round Delphi survey.
Round 2: a second electronic survey will be sent to the panel asking members to rate any new recommendations that were suggested by more than five participants from a single country income group. They will also be asked to rerate the recommendations that were considered uncertain based on the results of the first survey. These uncertain recommendations will be revised for the second survey based on comments we receive from respondents on the first survey. While the rating process for the second survey will be identical to the first survey, deidentified comments and feedback from panel members who rated a recommendation 6 or less in the first survey will be included for all panel members to consider as they re-assess their ratings in the second survey. After the second survey, recommendations with median ratings of 1–3 without disagreement will be discarded and those with median ratings of 7–9 without disagreement will be included in our final recommendations. If more than 10 recommendations remain with median ratings of 4–6 or any median rating with disagreement among panel members, a third and final Delphi survey will be created, with the same process as the round 2 survey.
Data analysis
Power calculation
Our expert panel will consist of a minimum of 108 members, 27 from each of the four country income groups. Using a two-sided CI for one proportion test, a sample size of 27 participants in each of the four country income groups (n=108 total in all four groups) would yield a 95% CI with a width equal to 0.3, assuming the sample proportion of 0.85. This resulting 0.3 two-sided CI width is equal to a CI having the lower limit of 0.66 and the upper limit of 0.96. Thus, our study will be sufficiently powered with the sample size n=108 for a 95% chance that the true population rate of agreement will lie between 66% and 96%, assuming the sample agreement rate among our expert panel members is 85%.
Each of the four groups of 27 expert panel members will consist of 17 anaesthesiologists and 2 non-anaesthesiologists professionals from each of the following categories: surgeons, operating room nurses, nurse anaesthetists, pharmacists and medication safety experts. Thus, each non-anaesthesiologist professional will have a total sample size of 8 in all four country income groups. The sample size of 8 non-anaesthesiologist professionals will yield a power of 0.8 to detect a minimum detectable effect size d=1.5 using a two independent sample t-test with alpha=0.05. The effect size corresponds to a mean difference among professions in rating of a recommendation of 1.5-points, assuming an SD of 1 rating point.
Interpercentile range adjusted for symmetry method
Due to the large size of our expert panel, we will use the Rand Corporation’s interpercentile range adjusted for symmetry (IPRAS) method to assess agreement between panel members on each survey question during the Delphi analyses.13 Briefly, IPRAS involves comparing the actual interpercentile range (10th–90th percentile) of survey ratings to the IPRAS. The interpercentile range is smaller when score distributions are asymmetric than when they are symmetric. Thus, disagreement occurs when the actual interpercentile range is larger than the IPRAS; all other cases will be classified as agreement.
Inclusion of additional recommendations
New recommendations suggested by five or more of our expert panel members from a single country income group in the first round Delphi survey will be included as candidate recommendations in the second round Delphi survey. If there is a 30% chance of a participant suggesting a new recommendation, there would be 27×0.3 = 9 newly suggested recommendations on average per country income group. Since five of our expert panel members from a single country income group will need to suggest the same recommendation for it to be included in the Delphi survey, we hypothesise that 1–2 new recommendations from each country income group could be added to our total 133 candidate recommendations, for a total of 3–6 new recommendations (a 2%–4% increase). This will not affect our panellist sample size estimation or statistical analytic method choices.
Part 3: prioritisation of recommendations by their level of clinical importance for implementation in each of the country income groups
A final electronic survey will be sent to participants asking them to rank each of the recommendations selected by the Delphi method on a scale from 1 to n=number of accepted recommendations, according to its importance as a next step in improving medication safety in each of low, lower-middle, upper-middle and high-income country groups. When ranking by importance, participants will consider the anticipated reach, effectiveness, adoption, implementation and maintenance for each recommendation (RE-AIM framework).19–21 A rank of 1 will denote the recommendation(s) with the highest importance considering these five RE-AIM dimensions and N, the lowest importance. Panel members will be allowed to have tied rankings within a country income group and will not be required to rank all the recommendations. Unranked recommendations will be assigned the lowest rank, N.
Data analysis
RE-AIM framework
We will use the extensively studied RE-AIM framework, which defines the impact of an intervention as the product of its reach (proportion of the target population that participates), efficacy (success rate if implemented as intended; positive outcomes minus negative outcomes), adoption (proportion of settings/practices that adopt the intervention), implementation (extent to which the intervention is completely implemented as intended) and maintenance (extent to which the intervention is sustained over time).21 22 The product of these five dimensions is called the public health impact score, and has been used to determine which interventions are worth sustained investment, and which will work in real-world environments.20 RE-AIM dimensions can also be assessed at multiple points in time to track the impact of an intervention. In part 3, we will use the RE-AIM framework to prioritise each recommendation by its importance as a next step in improving medication safety in each of the four country income groups. In part 5, we will use the RE-AIM framework as a practical measure of how well the recommendations work in real-world settings in each income group, and to assess the impact of the recommendations on global public health. This framework is well-suited for healthcare innovation projects because it focuses on the validity of the intervention (in this case recommendations), and guides the planning, conduct, evaluation and maintenance of implementation of each recommendation.
Individual rankings
For each country income group, we will calculate the mean rank assigned by the panel to each recommendation. Recommendations with a higher mean rank can be interpreted as having a higher importance than those with a lower mean rank. To explore differences in rankings between panel members from different country income groups, we will use the Kruskal-Wallis test for each rule, with the Bonferroni correction to adjust for multiple hypothesis testing. P values lower the 0.05/N will be considered statistically significant where N is the number of recommendations ranked.
Overall group ranking
We will use a Condorcet ranking method called the Crowd Ranking Kit23 24 to achieve an overall group ranking of recommendations for each of the four country income groups. Briefly, for each possible pair of candidate recommendations within an income group, we will determine whether at least as many panel members prioritised recommendation A over B as prioritised recommendation B over A. The Condorcet Winner25 is the recommendation that is prioritised pairwise to all other recommendations. Condorcet cycles occur when there is no clear preference among recommendations (eg, the majority prioritise recommendation A over B, B over C and C over A).25 26 The Crowd Ranking Kit method combines individual rankings into a unique hierarchy of ranked Condorcet cycles, which defines the ordering and gives a ranked list of group preferences for each country income group.
Graphical representation
With X survey responses, we will graph the importance of each recommendation for each country income group in X-dimensional space, with the coordinates for each recommendation being its rank assigned by the X panel members. Unranked recommendations will be assigned the lowest rank n=number of recommendations. We will illustrate the relative positions of the N recommendations in the X-dimensional space using a statistical method often used in psychological and behaviour research called ordinal multidimensional scaling (MDS),27–29 which maps points in X-dimensional space to points in two-dimensional space. We will define goodness of fit as 1-Kruskal’s Stress so that 100% represents a perfect representation of the relative positions of the recommendations in two-dimensional space. Fits higher than 80% are acceptable and fits higher than 90% are very good.30
For each country income group (and overall), we will identify clusters of recommendations that are ranked similarly by panellists using hierarchical cluster analysis methods.31 32 We will indicate the identified clusters on the two-dimensional feasibility map of the recommendations. This sequential application of MDS and cluster analysis is common in behavioural research.33–35 On the two-dimensional map, the horizontal axis is the axis that most separates the recommendations in X-dimensional space. Given that our data points are rankings of importance, this access will represent the importance of implementing the recommendation in the given country income group.
Part 4: dissemination, diffusion and adoption of the recommendations
While diffusion is the informal, peer-mediated, de-centralised spread of innovation, dissemination is a more planned, formal, centralised approach to adoption of innovation.36 Adoption of the prioritised recommendations for each of the four country income groups will be encouraged by disseminating and diffusing the information using the following strategies:
Endorsement of our recommendations by the WFSA.
Descriptions and links to our recommendations from the WFSA website.
Peer-reviewed manuscripts describing our methodology and our recommendations.
Presentations at influential national and international conferences, including the American Society of Anesthesiologists, Anesthesia Patient Safety Foundation and World Congress of Anesthesiologists.
Presentations at hospital grand rounds in the USA and globally.
Outreach to professional societies not only in high-income countries, but also in middle-income and low-income countries, to encourage use of the recommendations. This will include a description of the recommendations in their native language.
Part 5: evaluation of initial adoption using semistructure interviews
Approximately 12 months after we disseminate the recommendations, we will conduct one-on-one semistructured interviews with each participant to evaluate the initial adoption and implementation of the recommendations in each of the country income groups. The semistructured interviews will be audio recorded and conducted either in-person or by telephone/video conference.
The semistructured interview instrument shown in figure 1 will be pilot tested and iteratively revised with our project steering committee members. It is based on the extensively studied RE-AIM framework,19 21 22 and includes: introductory comments, questions with generic probes, questions with specific probes, final/summary questions and closing statements.
Preliminary interview instrument. This will serve as the basis for the final semistructured interview instrument to be used in part 5: evaluation of initial adoption using semistructure interviews. The preliminary instrument will be pilot tested and iteratively revised with our project steering committee, to arrive at the final interview instrument.
Data analysis
Grounded theory analysis
In order to achieve thematic saturation (the point at which no new themes emerge from the interview data), we will interview at least 80 participants, n=20 from each of the four country income groups. We will follow the grounded theory approach to interpret qualitative data.37–40 Audio recordings of semistructured interviews will be transcribed verbatim, reviewed/corrected for transcription accuracy and removal/masking of identifying information, and entered into ATLAS.ti software (Berlin, Germany) for coding and analysis. Our study team will regularly review themes and emergent findings with the project steering committee. We will repeatedly look at alternative explanations for interpretations of data, and rule them out or modify our initial interpretations. We will code data into categories based on emergent themes. This iterative, analytic and reflective process will be conducted as interview transcripts become available, allowing for modification of the coding scheme as well as assuring that thematic saturation in reached.
Patient and public involvement
Patients and the public were incorporated into our study design in several ways. First, recommendations for perioperative medication safety that are in the public domain were included as candidate recommendation for this study. Second, we incorporated national professional societies (for anaesthesiologists, surgeons, nurses, nurse anaesthetists and pharmacists) from around the world into our recruitment strategy, as described in the Study population and recruitment section. Third, by making our recommendations publicly available, we will involve patients and the public in the diffusion and dissemination of our recommendations.
Ethics and dissemination
This study was approved by the Human Research Committee/Institutional Review Board at Partners Healthcare (2019P003567). The study will be overseen by a multidisciplinary, international steering committee, including members from North America, Latin America, Europe, New Zealand, Asia and Africa.
This project will create and disseminate the first consensus-based recommendations for perioperative medication safety that are tailored to country income level, using the World Bank’s four country income groups: high (includes the USA), upper-middle, lower-middle and low-income.12 16 The project will include the entire medication use process (ordering, dispensing, preparing, administering, documenting and monitoring) in preoperative holding areas, operating rooms and post-anaesthesia recovery areas. The resulting recommendations will make surgery safer for patients not only in high-income countries but also in upper-middle, lower-middle and low-income countries around the world.
By creating the first set of recommendations that are specifically tailored to a country’s income and resource level, we will facilitate the successful dissemination and diffusion of our recommendations. Greenhalgh and colleagues performed an extensive literature review on adoption of innovation in health care36 and found that diffusion and dissemination programmes have been most effective when they:
Incorporate potential adopters’ needs and perspectives, with particular attention to the cost–benefit trade-off. We will achieve this in part 2 via the Delphi method, which will incorporate the perspectives and recommendations of all participants, with equal representation from high, upper-middle, lower-middle and low-income countries.
Tailor different strategies to different demographic, structural and cultural groups, which we will achieve in part 3: prioritise the recommendations by their level of clinical importance for implementation in high, upper-middle, lower-middle and low-income countries.
Use appropriate communication style and channels, which we will achieve via (a) diffusion that is facilitated by our large, international group of participants and (b) dissemination that is facilitated by the participation of and endorsement by the WFSA.36
These strategies will ensure widespread diffusion and dissemination of our recommendations, and thereby change the culture of medication safety, which currently leaves low resource countries behind. Our inclusion of targeted recommendations for these areas will provide a valuable resource for policy-makers, hospital administrators and frontline healthcare providers to reduce the incidence of MEs and their associated patient harm. Also, our prioritisation of the recommendations will allow easy identification of the most important recommendations for future evaluation by randomised controlled trial. Finally, adoption of the recommendations will make surgery and anaesthesia safer both in the USA and around the world.
The recommendations generated by this project will be shared globally via publication in peer-reviewed journals, and descriptions and links on the websites of the WFSA and other professional organisations. Our findings will also be presented at national and international meetings, and we will use grass-roots diffusion methods via our large, international group of participants.
This project is important because it will make surgery safer for patients globally by creating and disseminating the first consensus-based recommendations for perioperative medication safety that are tailored to country income level. Widespread diffusion and dissemination of the resulting recommendations has the potential to change the culture of medication safety, which currently leaves low resource areas behind. Our inclusion of targeted recommendations for these areas will provide a valuable resource for policy-makers, hospital administrators and frontline healthcare providers to reduce the incidence of MEs and their associated patient harm.
References
Footnotes
Twitter @chrischirp, @AdrianGelb
Contributors KCN contributed to study design, statistical plan, drafting and revising the manuscript. AFM contributed to study design, statistical plan, and revising the manuscript. SDS contributed to drafting and revising the manuscript. CP contributed to study design, statistical plan and revising the manuscript. HD contributed to the statistical plan and revising the manuscript. JW contributed to study design, statistical plan and revising the manuscript. AWG contributed to study design, statistical plan and revising the manuscript. BAO contributed to study design, statistical plan and revising the manuscript.
Funding This work was supported by a grant from the Agency for Healthcare Research and Quality (AHRQ, Rockville, MD, #1K08HS024764-01). The content is solely the responsibility of the authors and does not necessarily represent the official views of the AHRQ. The AHRQ did not have any role in the design and conduct of the study; collection, management, analysis and interpretation of the data; or preparation, review or approval of the manuscript.
Competing interests KCN receives author royalties from UpToDate, Inc (Waltham MA). AFM has shares in Safersleep LLC (Auckland, New Zealand) and chairs its Board. JW received speaker honoraria from the Anesthesia Patient Safety Foundation (Rochester, Minnesota, USA) and the Aspen Institute (Aspen, Colorado, USA). AWG receives consulting fees from Masimo Inc (Irvine, California, USA) and Haisco Pharmaceutical (Shannan, China). He is also Secretary of the World Federation of Societies of Anesthesiologists (London, UK). BAO serves on the Board of Directors of the Institute for Safe Medication Practices (ISMP) Canada (Toronto, Canada).
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.