Article Text
PDF
Abstract
Objectives The Medication Safety Thermometer (MedsST) is a medication safety data collection tool, which has been used by over 100 UK healthcare organisations to enable measurement of medication safety for improvement purposes. This study aimed to explore whether, and how, data collected by the MedsST have been used in organisations to facilitate medication safety improvements.
Design Routine MedsST data collected between October 2013 and July 2016 were analysed using Run charts. Identified changes were investigated using interviews with staff from each hospital trust. The interviews were analysed using a framework based on Normalisation Process Theory, focusing on use of the MedsST and its data.
Setting Three National Health Service hospital trusts in the North West of England, which have used the MedsST for the longest period.
Participants Eight interview participants, purposely sampled based on their involvement with the MedsST, included pharmacists, pharmacy technicians and nurses.
Results Improvement was often at ward level and focused on particular areas of medication safety, led by clinical champions. The most sustainable improvements involved changes to systems, such as introducing new guidelines. Although some improvement occurred, internal communication about improvements was poor, and large amounts of data remained unused, often due to a lack of ownership of data review and use.
Conclusions Simply collecting data is not sufficient; a system of data collection, review and use for improvement is required. Issues with such systems may have been recognised and averted if implementation theory had been used in the early stages of national development and implementation. However, implementation theory could be used within organisations to fix issues locally, particularly to increase ward-level ownership of this system, which could lead to considerable improvements.
- medication safety
- quality improvement
- patient safety
- measurement
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This is the first study to use Medication Safety Thermometer (MedsST) data with qualitative interviews to explore whether quality improvement has occurred in different hospitals, with specific examples demonstrating how.
A small sample size was used consisting of three sites with the most experience with the MedsST. While the small sample size does not allow generalisation to occur, this study demonstrates potential for successful quality improvement using MedsST data.
Using implementation theory, it was found that one of the causes for data being unused was that more focus had been given to implementing data collection, rather than the holistic system of collection, reviewing and use of data.
Only one member of nursing was interviewed, compared with five pharmacy staff; however, this was representative of staff using the MedsST who have predominantly been pharmacy staff.
Audio-recording of conducted interviews did not occur; however, notes from two interviewers were merged, and meanings of notes and quotes from participants were clarified with participants after interviews, addressing potential inaccuracies.
Background
Failures within healthcare systems serve as reminders of the need to focus on improving patient safety, for example, the much-publicised failures at the Mid-Staffordshire National Health Service (NHS) Foundation Trust in the UK.1 2 Reports have consistently identified medication safety as a priority area of improvement within healthcare systems.2 3 It has been recommended that improvements are not just made in response to serious incidents but that routine measurement should occur to enable organisations to proactively seek potential problems before they lead to significant incidents.3
Since 2013, several English healthcare organisations have used a tool called the Medication Safety Thermometer (MedsST) to routinely measure medication safety.4 5 The most recent version of the MedsST5 has three steps measuring: process errors (such as medication omissions), triggers of harm (such as hypoglycaemic attacks in patients with diabetes) and actual harm from medication (ie, establishing whether a high [>6] International Normalised Ratio (of prothrombin time of blood coagulation) for a patient on anticoagulants was due to medication issues or a patient’s deteriorating condition).4 Step 1 data are collected for all patients. If a patient is receiving any high-risk medicines (anticoagulants, injectable sedatives, opioids or insulin), then the data collector is prompted to move to step 2. Step 2 involves assessing each patient’s high-risk medicines and aims to detect potential problems. If a trigger of potential harm is detected, the potential harm and the patient should be discussed in step 3 through a multidisciplinary huddle between a nurse, pharmacist and doctor. The discussion should determine whether harm has been caused by medication issues and the level of harm occurred. Associated learning (ie, how to improve practices to prevent reoccurrences) from the discussion and whether an incident report has been completed must also be recorded in step 3. According to the national guidance, collected data from the MedsST and other safety thermometers can then be used to ‘obtain a baseline, understand variation within your organisation or across multiple organisations, set an improvement goal and measure improvement’.6
The design, development and implementation of the MedsST have been reported in a previous study,4 and a second qualitative study evaluating the MedsST’s implementation has been conducted using Normalisation Process Theory (NPT).7 The latter study found that staff using the MedsST displayed understanding of why medication safety measurement is required (coherence) and engagement with the tool (cognitive participation). Conversely, there was a lack of awareness regarding how to review the use of the tool and its data (reflexive monitoring) and to scale up use of the tool for medication safety improvement (collective action).7 Nonetheless, all healthcare staff interviewed displayed interest in improving medication safety and many had attempted to learn from other organisations but with limited success. The study suggested that only some hospitals had support from quality improvement departments with analysis of MedsST data and only some organisations used data for board level reports. One participant from a hospital who had only recently started using the tool suggested that data should be used for ‘bench-marking through the back door’ to compare organisations.7 However, there was no evidence of this happening currently and many participants were unsure whether data were even being used and how they could be used, which was also found in our previous study.5
To date, only one study has reported how MedsST data have been used as part of further improvement work.8 Phippen et al describe how data can be used in residential care settings to assist prioritisation of patients requiring urgent medicines reconciliation. However, the MedsST has predominantly been used in hospital settings and little is known about whether the data collected are being used.7
To address this knowledge gap, this study aimed to identify whether MedsST data have been used to influence and measure improvements in medication safety in hospitals that were ‘early adopters’ of the MedsST, and if so, how. Early adopters of the MedsST were hospitals who joined the national programme during the alpha-testing phase, between January 2013 and March 2013.7
Method
The MedsST was piloted and implemented through Manchester Academic Health Science Centre (MAHSC), and the tool has since been used by over 100 English healthcare organisations. MAHSC was a partnership between The University of Manchester and six NHS healthcare organisations. The three NHS hospital trusts within MAHSC, which are the focus of this study, have used the MedsST since its inception in 2013 and were ‘alpha-sites’ (contributing to the development of the MedsST during initial ‘alpha’ testing).4 7
A mixed-methods study design was used, with two stages: identification of Special Cause Variation (SCV) by analysis of existing data, followed by interviews to investigate SCV and explore how the MedsST and its data were used at each hospital (online supplementary appendices 1–4). Interviews were analysed using NPT, and case studies were created to give specific examples of how data had been used for improvement. Although this is a research study, it is sharing knowledge of quality improvement initiatives. Therefore, reporting of this study is in line with the Standards for Quality Improvement Reporting Excellence (SQUIRE) criteria for reporting quality improvement work, and a SQUIRE checklist has been provided in the online supplementary material.9
Supplementary file 1
Supplementary file 2
Stage 1: identification of SCV
MedsST data that had been collected between October 2013 (when the MedsST was nationally rolled out)4 and June 2016 were reviewed. First, openly accessible MedsST data collected by staff at each of the hospitals were downloaded from www.SafetyThermometer.nhs.uk.5 Data were analysed at both ward and organisational levels. Throughout the data collection period, data analysts checked the accuracy of data and worked with hospital staff where issues had arisen. For example, if astronomical data points occurred (see table 1), this would be investigated to see whether any issues with data collection or input had occurred.
Rules for identifying Special Cause Variation (SCV) in run charts
Time-series analysis was performed using run chart methods10 using Excel 2013. Standard criteria for SCV and system shifts (table 1) were used to determine whether observed changes were due to specific assignable causes.10–12 New control limits and centre lines were calculated when a system shift was observed. SCVs were noted and used to develop questions to include in interviews.
Stage 2: interview data collection and analysis
Qualitative interviews were used to investigate the highlighted variation and contextual factors regarding how the MedsST and its data had been used. A purposive sample of staff (n=8) from the three hospitals was interviewed, including a combination of leads (senior members overseeing the use of the tool) and users (frontline staff collecting data). Originally, this study had aimed to interview senior staff only to get a snapshot of the way that data had been used across the hospitals, but it was discovered that most improvement occurs at ward level, and therefore, we expanded our inclusion criteria to recruit ward-level staff. This was difficult as many ward-level staff do not have frequent access to emails, unlike senior staff, and it was challenging to identify the ward-level staff involved with MedsST use. Snowball sampling was used where the identified contacts were unavailable or to contact staff on specific wards where SCV had occurred. A range of staff were recruited including pharmacists, nurses and pharmacy technicians. In the UK pharmacy technicians are healthcare staff who are registered with the UK General Pharmaceutical Council (GPhC) and have relevant qualifications that are accredited by the GPhC.
Interviews were conducted by the first author and assisted by a project manager. An approximate interview schedule was used, which included specific questions about SCV variations and questions regarding contextual factors, such as the types of wards that used the MedsST. Interviews were not recorded as our previous study exploring the implementation of the MedsST found that staff were more comfortable discussing the use of the MedsST when not recorded.7 However, notes were taken by both interviewers, and summaries of participants’ own interviews and the final study report draft were sent to participants to clarify meanings and for approval. Not recording interviews help to maintain a more conversational interview tone and make the frontline staff more comfortable to help prevent inadvertently inhibiting responses.13
As mentioned above, a previous study highlighted that staff displayed understanding of (coherence) and engagement with (cognitive participation) the MedsST.7 However, engagement often declined due to a lack of understanding about how to review the data and use of the MedsST (reflexive monitoring) and how to scale-up use (collective action) of it. Therefore, this study focused on questions related to the latter two of the four constructs of NPT, and the framework in table 2 was used to analyse data.7 Notes from interviews were coded by hand, categorised and similar categories grouped into themes emerging from data. The deductive approach was used, where data analysed were constantly compared with the descriptive themes displayed in table 2.
Descriptive themes and their definitions
The hospitals and wards were given pseudonyms of A-C and 1–11 respectively. Quotes were chosen to illustrate a range of varying opinions about each theme. Words in parentheses have been added to quotes to clarify meaning, and ellipses (…) have been used to indicate the removal of unrelated text or information that may lead to identification of participants.
Ethical consideration
According to the NHS research ethics decision tool14 and the ethics guidance of the main author’s university,15 this work was service evaluation rather than research; therefore, approval from an NHS Research Ethics Committee or the University’s Ethics Review Panel was not required.
The university uses the following criteria for determining whether service evaluations require ethical review:
‘Data are collected without personal identifiers, the participants are not asked for confidential or sensitive information, the issues being researched are not likely to upset or disturb participants.
The research involving interviews with participants on subjects deemed to be within their professional competence.’
The work reported here meets the above criteria and sought to evaluate an existing service. No personal or upsetting questions were asked, and the MedsST does not collect patient-identifiable data. Verbal consent for interviewing was obtained from all participants.
Patient and public involvement
Patients were not involved in the development of the research question, outcome measures or study design. A study report with the results have been shared with each participating organisation, and a presentation summarising results has been presented to MAHSC leaders at the MAHSC Population Health and Implementation Domain Meeting.
Results
The results have been separated into ‘MedsST data’ and ‘interview data’. MedsST data describe the quantitative data that were analysed (online supplementary appendices 1–4), and interview data provide results of the interviews using quotes for illustration purposes. The case studies created have been presented in box 1.
Case studies to illustrate how data were used
Case study 1: the general surgery ward of hospital A had regularly reviewed the MedsST data and displayed them for staff, patients and visitors to consult. On this ward, data were collected consistently by the same ward manager, who ensured that they would be working on days where data were scheduled to be collected, leading to ownership of collection, review and use of data. The ward manager had identified issues with medication safety and he proactively worked with the ward pharmacist to raise awareness of these issues within the team. This activity commenced in summer 2015 and led to improvements, such as increased rates of medicines reconciliation within 24 hours (figure 1), which had increased by 11.6%, whereas the overall rate within the hospital only improved by 2.5% within the study period.
Case study 2: the participants from hospital B reported investigating triggers of harm (step 3) by performing multidisciplinary huddles with junior doctors, pharmacists and nurses involved in the care of that patient. Data from step 3 and indcident reports were analysed. Two of the lessons learnt included:
Patients with diabetes under palliative care were triggering step 3 due to hypoglycaemic attacks. By combining MedsST data and information from incident reports, it was found that the reason these patients were having hypoglycaemic attacks was that, despite their reduced nutritional intake, insulin doses had not been adjusted. Increased awareness of reviewing and reducing insulin doses for patients with diabetes under palliative care.
Elderly patients were triggering step 3 due to signs of overanticoagulation. Investigation revealed that the hospital’s guidance for anticoagulation was not appropriate for very elderly and frail patients. A new guideline specifically for elderly and frail patients was developed with ‘toned down’ loading doses.
Case study 3: in hospital C, two wards had shown a significant reduction in medication omissions from 40% to 26% (figure 2). The MedsST lead was unaware of any changes; however, the lead pharmacist for these wards reported that the multidisciplinary ward team had made a group effort to improve reporting and foster a culture of ‘learning from mistakes’. In addition, the ward’s lead clinician had obtained funding for the pharmacist on these two wards to improve medication safety. Therefore, the pharmacist had become aware of MedsST data and used online resources to educate themselves about how to use MedsST data, so that they could work closely with the multidisciplinary team to improve patient safety. This included attending routine patient safety meetings from summer 2014 to report medication safety incidents and interpret MedsST data. The pharmacist also contributed to nurse teaching sessions and had directed the focus of these to the biggest medication safety issues, such as omissions:
‘Lactulose is usually used for constipation; however, in liver disease it is also used for patients with encephalopathy … Many of the refused omissions were due to patients refusing lactulose and saying they did not have constipation and didn’t like the taste. And the nurses would think that’s fine, even though it could potentially cause harm to the patient. However, now after the extra teaching sessions they know that the patient may need lactulose to reverse encephalopathy and have the knowledge to help educate patients’. (Participant 8, pharmacist)
MedsST data
Data were collected on 29 mainly medical and surgical wards across the three hospitals, using both the MedsST application (app) and paper-based collection forms. The run chart presenting aggregated data from all three hospitals showed the following SCV (presented using absolute values):
The rate of medication omissions had reduced by 15% (from 40% to 25%).
The rate of medicines reconciliation initiated within 24 hours of patient admission had improved by 7% (from 78% to 85%).
The proportion of patients with a trigger of harm had reduced by 1.5% (from 2.0% to 0.5%).
Online supplementary appendix 1 presents MedsST process measure data for each hospital. Individual ward-level data from each hospital has also been provided (online supplementary appendices 2–4). Medication omissions, medicines reconciliation initiation and allergy status data have been presented at both organisational and ward levels. Step 2 data (triggers of harm) have only been provided at organisational level, as these data were not collected by many wards. Reasons for this were explored in the interviews.
Small system shifts were observed in all measures at organisational levels; however, at ward levels, a handful of wards displayed greater system shifts. This suggested that improvement often occurred at ward levels, and a small selection of wards were driving SCVs at organisational level. For example, in hospital C, overall omissions had reduced from 34% to 32%; however, five wards showed no reduction over time. The decrease was driven by six wards, with the biggest reduction of 23% in ward C10 (from 38% to 15%).
Interview data
A total of eight participants were interviewed, including pharmacists (n=3), pharmacy technicians (n=4) and a nurse. The lack of participation was due to a low response rate from nurses as it was difficult for them to take time off from their patient-facing roles. Pharmacy staff were involved with MedsST use at all organisations and had more availability for meetings after their ward rounds; therefore, they were more likely to be selected for interviews. Interview data have been presented under each theme from the framework presented in table 2.
Collective action
Scaling up
Healthcare staff had scaled up the use of the MedsST and its data in different ways. For example, personal relationships between ward sisters were used to scale up its use. Scale-up of data collection had been led by clinical ‘champions’ (frontline users who acted to forward the implementation process). This was observed at both organisational and ward levels and relied on champions’ ‘enthusiasm’ for MedsST use.
There are differences in views about the MedsST between wards. For example, Ward … shows major improvement and omissions seem to be … (very low) – this may be because … (the ward manager) is very enthusiastic about using the MedsST. (Participant 1, pharmacist)
Impetus was lost when champions left organisations. In one organisation, a ‘champion’ who was also a medication safety officer (MSO) had retired, and participants from this organisation reported ‘waiting’ for the appointment of a new to take ownership of using data for improvement (a designated member of staff who supports medication error reporting and learning).16
After … (the Medication Safety Officer) left, not much has happened with data, we are waiting for the new Medication Safety Officer … to start. (Participant 3, pharmacy technician)
Time and money
A range of staff had acted as champions and taken ownership of the use of the MedsST and its data, including those who were involved with the MedsST as part of their main role, such as MSOs. However, there were also voluntary champions who had taken ownership of the MedsST in addition to their main roles. For example, a nurse ward manager had arranged their working pattern to ensure they would be working on days where data were scheduled to be collected; they had also used their non-working time, including lunch breaks, to print data to display on the wards for all staff and patients (box 1, case study 1 and figure 1). This was sustainable at ward level, as a ward manager was focusing on their own data. However, it was less sustainable if one person, such as a non-ward-based pharmacist, focused on several wards’ data, in addition to their routine work, and often these staff had used their own time to conduct further improvement work. Even where funding was provided for using MedsST data for related improvement work, it created an extra burden on top of day-to-day duties and the champion needed to have a personal interest to volunteer to be involved. Furthermore, good professional relationships with senior colleagues were required for obtaining funding for improving medication safety.
Run chart 1 shows medicines reconciliation initiation data collected monthly on ward A6 of hospital A between October 2013 (national roll-out of the Medication Safety Thermometer) and August 2016. The blue line represents the proportion of patients on the ward whose medicines reconciliation had been initiated at the point of survey. The green line represents the proportion of patients whose medicines reconciliation status was undocumented at the time of survey. The red line represents the mean, which has been adjusted using Special Cause Variation rules (see table 1).
The lead clinician said to me (the ward pharmacist) ‘Do you want this money to improve medication safety on your wards?’. And I said ‘Yes’. Otherwise he would have given it to … (another pharmacist). (Participant 8, pharmacist)
Organisations’ complex systems created barriers for more time-efficient MedsST use; for example, where individuals had to prebook iPads and acquire relevant permissions prior to data collection, even though it was reported iPads saved staff ‘hours of time’ (participant 4, pharmacy technician). Additionally, ‘Wi-Fi dropout (connectivity issues)’ also acted as a barrier to use of the iPads.
Education and training
Many staff reported training themselves on how to use MedsST using online resources but that there was ‘no formal training’ within hospitals. Contrarily, it was highlighted that MedsST collection training had merged into nurse and preregistration pharmacist induction training at two organisations.
(In terms of training, there is) nothing formal at any time…Training (for using the MedsST) is included in pre-reg(istration) pharmacists’ induction on an annual basis. (Participant 6, pharmacy technician)
In one case, a ward pharmacist had used online resources to learn how to interpret MedsST data. This was in response to colleagues, including doctors and nurses, enquiring about what the MedsST data meant, after it was sent out to wards via email. This indicated a lack of education about the MedsST across hospitals, for those not involved with data collection.
Ward-based staff reported that it had been ‘difficult’ to attend the WebExes due to the timing and other work commitments and mainly pharmacists had attended the monthly WebExes during the testing stages.
Reflexive monitoring
Use of the MedsST and its data
Marked variations were reported between how data were collected and used at organisations. For example, different members of staff collected data at different organisations, including preregistration pharmacists, pharmacists, pharmacy technicians, pharmacy nurses and ward managers. Generally, ownership and engagement with use of data increased if those who had collected it were more senior, that is, ward managers or MSOs. Participants from hospitals A and B reported that the data collected were discussed in some senior patient safety meetings in which safety across the hospital was discussed. However, in hospital C, data were more likely to be reviewed in local patient safety meetings that discussed patient safety on particular wards.
Most collected data were unused for improvement purposes, unless staff involved with data collection had taken ownership of reviewing and using the data for further improvement work. For example, hospital B’s Medication Safety Team who were responsible for ensuring MedsST data were collected had analysed step 3 data regarding triggers of harm to see why most reports of harm were related to certain medicines (box 1, case study 2). There was also evidence of staff not involved with MedsST data collection using data in hospital C, but this was rare. For example, in hospital C, a specialist hepatology pharmacist (who had not been involved with data collection) had taken ownership of data collected by lower grade pharmacy staff to facilitate improvements in medication safety, after becoming aware of the existence of MedsST data (box 1, case study 3). Identification of SCV of two of the hospitals wards led to interviews with the specialist hepatology ward pharmacist who was asked about the SCV (see box 1, case study 3 and figure 2). However, the hospitals MSO was unaware of the improvement work undertaken by the specialist hepatology pharmacist suggesting poor communication about improvement work
Run chart 2 shows omissions data collected monthly on ward C2 of hospital C1 and C2 of hospital C between October 2013 (national roll-out of the Medication Safety Thermometer) and August 2016. Omissions refer to medication administration omissions and exclude omissions due to valid clinical reasons. The blue line represents the proportion of patients on the wards with one or more omission(s) of medicine(s) at the point of survey. The red line represents the mean, which has been adjusted using Special Cause Variation rules (see table 1).
Where funding had been allocated to improve patient safety, it was often ‘reactive improvement’ to show senior staff improvements had been made in response to events that highlighted issues with medication safety. For example, an external inspection in one hospital had identified issues with the medication omissions, and according to one participant, the hospital staff had become more interested in how to show medication safety improvement following the inspection.
We had an inspection, and one inspector happened to be a pharmacist, (therefore was particularly) interested in medication safety. He had a chat with a patient who said they were self-administering because of the nurses… (who had missed his medication administration). The inspector had fed back these issues to the hospital which led to other staff asking me how to use the MedsST data. (Participant 8, pharmacist)
Reviewing and amending use of the tool
It was reported that frontline staff felt they received ‘little or no feedback’ on the results of the collection. Many staff reported uncertainty about whose ‘job’ it was to review data, and staff were unsure who they could contact for support with interpreting data.
If someone could feed that back to me and explain it (data) as well… (or) maybe someone from the… I don’t know, (maybe) the QI (Quality Improvement) team? (If they) could attend the ward and… help me pick out some (information). (Participant 2, nurse)
Communication issues existed, and a lack of feedback to staff was highlighted on many wards. Some individual wards demonstrated a high degree of local commitment to staff feedback, with some wards displaying data for staff and patients to view. However, this appeared to be sporadic.
We display our results on our (ward’s) whiteboard… I don’t think anywhere else (any other wards) do… next door definitely don’t display their results. (Participant 2, nurse)
Awareness raising of medication safety issues at ward level seemed to have contributed to improvements on individual wards, but this relied on at least one member of staff to understand and interpret the data to others, and it was reported some feedback methods had been ‘useless’ (participant 2, nurse), for example, feedback given via email despite several staff having no access to emails.
The tool was adapted within the three hospitals in various ways. For example, at one hospital, many wards did not collect step 3 data, and the hospital had not formally implemented step 3 of the MedsST. Staff described the reason for this as ‘harm from medication would be “covered” by the hospital’s incident reporting system’ (participant 7, pharmacist). In the other two hospitals, both data collection systems ran in parallel.
Discussion
This study highlights that it is possible to use routine medication safety data for improvement in hospitals but that collected data often remains unused by hospital staff. There has been more focus solely on data collection, rather than the full system of data collection, review and use. Healthcare staff are familiar with how data are collected and submitted but less familiar with how the data are fed back and used for further improvement. It has been difficult to understand exactly why the implementation of this system has not succeeded due to poor theoretical underpinning of its implementation. However, there are some examples where this system has been successfully implemented and has contributed to improvements, mainly at ward level. Improvements made to systems, rather than individual behaviour, have been more sustainable.
Prior to national roll-out, the MedsST had undergone two stages of testing; alpha and beta testing.4 The three organisations, within this study, have used the MedsST throughout these stages and have previously had financial incentives attached to collecting data and evidence of using data (April 2013–March 2014).4 Despite these financial incentives and early use of the tool, a system of collecting, reviewing and using data has not been fully implemented across the organisations. Therefore, it is possible that organisations who have started using the tool more recently have had greater issues with the implementation of the above described system. While a significant amount of effort has gone into the development of the tool and collecting data, it seems that there has been less focus on the implementation of the reviewing and use of data aspects of the system. This had led to large amounts of data remaining unused and many staff not involved with data collection were unaware of the existence of data.
A wide range of healthcare data are collected in UK hospitals, ranging from patient feedback17 to catheter data,11 and various studies have found that data are often underused and not interpreted correctly.7 18–20 As the quantity of various types of data collected within NHS hospitals grows,18 it is vital that staff are able to access and interpret the collected data to ensure an efficient use of resources.
There are issues with implementing innovations to improve patient safety without ensuring that staff fully understand the innovations and how to use them to aid improvement of safety.21 If there is a lack of understanding about how to use an intervention, there is a risk of decline of use of the intervention, leading to difficulties in demonstrating added value of the intervention. Early evaluation of new tools, such as the MedsST, can help avert issues with declining use as evaluations can lead to better understanding of associated system mechanisms and contexts.21
A variety of quality improvement methodologies have been used to evaluate and refine the tool while it was developed. For example, Plan–Do–Study–Act cycles have been useful for iteratively improving the MedsST.4 However, additional quality improvement and research methods would benefit scale up of the system of data collection, review and use. In particular, there has been a lack of focus on the use of implementation theories, models and frameworks for evaluating this system.22 If implementation theory had been used during the national roll-out period, it could have aided evaluation of what worked and what did not work in terms of the system of data collection, use and review. There are still opportunities for hospital staff to work with researchers to use implementation theory at local levels to aid scale-up of this system, for example, by reviewing the current system and making improvements (eg, training staff to use data for quality improvement) and scaling up these improvements using implementation frameworks. Several Academic Health Sciences Centres (AHSCs) have been established in the UK in the last decade23; they are ‘a constellation of functions and organizations committed to improving the health of patients and populations through the integration of their roles in research, education, and patient care’.24 AHSCs provide an ideal opportunity for greater collaboration between hospitals and universities to work together to improve the system of medication safety data collection review, using research such as implementation theory. For example, by encouraging further education and training on quality improvement methods across the healthcare system for staff, students and patients. If staff involved with the MedsST have a better understanding of QI methods, it is likely they will be able to identify SCV on their graphs and may be more likely to undertake further improvement work.
Other healthcare systems considering introducing the MedsST or similar tools must focus on the system of data collection, review and use from early stage of introducing the tool, rather than implementing data collection alone. This includes UK hospitals considering adopting the MedsST as well as national healthcare organisations who have started using the MedsST more recently such as Hamad Medical Corporation in Qatar (personal communication, Y Jani, 2018). Additionally, organisations implementing data collection tools similar to the MedsST must also be cautious about focusing on the implementation of the aforementioned system, for example, organisations implementing the ‘All Wales Safety Thermometer’ in Wales25 and the ‘Patient Safety Thermometer’ in Saudi Arabia.26
While data have often remained unused, it is encouraging that healthcare professionals not directly involved with the MedsST are curious about the MedsST and its data. Further education and training is required to ensure all staff are aware of the MedsST within hospitals, as medication safety improvement requires a multidisciplinary effort7 from frontline staff, senior staff, quality improvement departments (if existent) and others. Furthermore, it should be made clear how each member of staff can have a role in the collection of data and its use to aid improvement. There was evidence of some discrete initiatives, for example, information about the MedsST was included in the introductory training for all nurses at hospital B and the induction pack for preregistration pharmacists at hospital C. Further work is required to investigate the impact of these changes. It is vital that organisations learn from each other, and lessons learnt from these changes and other related improvement work must be shared both internally within hospitals and externally. Since 2017, hospital trusts in Greater Manchester have started to merge.27 The merger plans may provide a greater opportunity for wards to share improvement strategies, if the correct system-level changes are introduced, for example, introducing communication channels between the large number of wards that will result from the merger could aid scaling-up of improvements.
For the lessons learnt to be transferable between settings, it is important that all changes to systems are clear and transparent. For example, there are often no clear procedures for training to use the MedsST, and staff may not even realise that training is occurring, making it difficult to tell others how training is occurring. Some UK organisations have introduced guidelines for how to collect MedsST data7; it may be useful to also introduce guidelines for how to review and use data. The guidelines could also provide contact details about who to contact if there are issues or to get help with data interpreting, such as a hospitals’ quality improvement team, as frontline staff highlighted they were unsure who they could get help from.
Clinical champions played a fundamental role in scaling up the use of the MedsST; however, there are issues introduced with relying on individual behaviour change, for example, when hospital B’s data were negatively impacted after the MedsST lead left organisation. It was reported that the former MedsST lead played a significant role in encouraging both pharmacy and clinical staff to take an interest and thus appreciate the benefits of regular data collection. Furthermore, most champions were more focused on scaling up MedsST data collection. If champions were supported to also provide feedback and action plans for use of data to their colleagues, it may aid the system of using data for improvement. Providing feedback to peers promotes collective behaviour change, and previous research has found that if feedback is provided by supervisors or colleagues, they are more effective, particularly if feedback is given verbally and in written form, and an action plan is provided.28 29 Senior staff should also receive feedback from champions about how to improve MedsST use by helping to overcome barriers such as the lack of resources to collect data, such as introducing more efficient systems for providing iPads for data collection.
Traditionally, safety in hospitals has been defined as the absence of harmful incidents or events, in line with ‘Safety I’ approaches,30 and hospitals therefore focus on identifying the causes of negative events and eliminating their reoccurrence. There is evidence of medication safety data being used in this way, for example, when senior staff started to review data in response to a negative incident reported by a hospital inspector. While this reactive data use is important and can help lead to system improvements, there should also be proactive use of data.
There was also evidence of Safety II approaches to use of data, where participants spoke about exploring ‘what is going right’ with medication safety30 and which wards had shown improvement. Previous research has showed how Safety Thermometer data (for other areas of patient safety) can be used to identify positive deviants within organisations.31 Senior staff may use MedsST data in a similar way to identify wards who are positive deviants31 within their hospitals and share positive practice to other wards before the negative incidents occur.
Conclusion
The collection of routine data can enable hospitals to measure the trajectory of change and aid improvement but only if data collection is part of a system of data collection, review and use for improvement. While using collected medication safety data may sound like an obvious next step after collecting data, this study has highlighted that it is not simple as it seems due to barriers such as a lack of communication between staff. Visible improvements were made across many participating wards and discrete improvements on wards. Increased focus on the implementation of the complete system of data collection, review and use would have been beneficial in the early stages of the MedsST’s development. However, individual organisations can improve this system at local levels, particularly by learning from positive deviants. Further multidisciplinary work is required between research departments and organisations to support the ‘champions’ who have been using MedsST data to aid improvement of medication safety to share and disseminate their work. Changes made to systems, rather than just individual behaviour, must be encouraged to make improvements more sustainable.
Acknowledgments
The authors would like to thank Bridget Armour (BA) and Haelo data analysts for assistance with data collection and management. The authors would also like to acknowledge all MedsST data collectors and interview participants from the three hospitals.
References
Footnotes
Contributors All authors have made substantial contributions to the conception or design of the work and have been involved in acquiring, analysing or interpreting the data. All authors have been active in drafting, critically reviewing and revising the manuscript and have approved the final version to be published. Concept: all authors. Data collection: PR. Data analysis: PR, MPT, DA and AH. Article drafting: PR, MPT, DA and GP. Critical revision: all authors.
Funding PR is a PhD student funded by Haelo.
Competing interests The main author (PR) is a PhD student funded by Haelo who previously facilitated and managed the development of the MedsST. However, PR was predominantly based at and supported by University of Manchester.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement The MedsST data for all organisations, which have been referred to in this study, are available online (www.safetythermometer.nhs.uk). To access a dedicated dashboard presenting the data in graphical format, the ’Medication' tab must be selected, followed by the ’analyse data' button. Since this study was conducted, management of the MedsST data has been transferred from Haelo to the Quality Observatory team at South, Central and West Commissioning Support Unit on behalf of National Health Service Improvement, who can be contacted for more recent raw data. In terms of qualitative data, short interview excerpts are available within the text. Participants in our in-depth interviews consented to participate in the study. which stated that identifying information would remain confidential. However, they did not provide consent to have their full interview notes made publicly available, which would potentially lead to identification of participants and their organisations. Many of the quotes used have been amended with ellipsis (…) to remove the identifiable data.
Patient consent for publication Not required.