You are here

  • Home
  • Archive
  • Volume 12, Issue 1
  • Quality improvement in a crisis: a qualitative study of experiences and lessons learned from the Irish National Ambulance Service response to the COVID-19 pandemic

Article Text

Article menu
Qualitative research
Original research
Quality improvement in a crisis: a qualitative study of experiences and lessons learned from the Irish National Ambulance Service response to the COVID-19 pandemic
  1. Eithne Heffernan1,
  2. Dylan Keegan1,
  3. Bridget Clarke2,
  4. Conor Deasy2,3,
  5. Cathal O'Donnell4,
  6. Philip Crowley5,
  7. Angela Hughes6,
  8. Andrew W Murphy1,
  9. Siobhán Masterson1,4
  1. 1Discipline of General Practice, Clinical Science Institute, School of Medicine, National University of Ireland Galway, Galway, Ireland
  2. 2National Ambulance Service, Health Service Executive, Dublin, Ireland
  3. 3Department of Emergency Medicine, Cork University Hospital, Cork, Ireland
  4. 4Clinical Directorate, National Ambulance Service, Health Service Executive, Limerick, Ireland
  5. 5Health Service Executive, Dublin, Ireland
  6. 6Quality Improvement Division, Health Service Executive, Dublin, Ireland
  1. Correspondence to Dr Eithne Heffernan; eithne.heffernan{at}nuigalway.ie

Abstract

Objectives The COVID-19 pandemic has produced radical changes in international health services. In Ireland, the National Ambulance Service established a novel home and community testing service that was central to the national COVID-19 screening programme. This service was overseen by a multidisciplinary response room. This research examined the response room service, particularly areas that performed well and areas requiring improvement, using a quality improvement (QI) framework.

Design This was a qualitative study comprising semi-structured, individual interviews. Maximum variation sampling was used. The data were analysed using an established thematic analysis procedure. The analysis was guided by the framework, which comprised six QI drivers.

Setting Response room employees, including clinicians, dispatchers and administrators, were interviewed via telephone.

Results Leadership for quality: participants valued person-oriented leadership, including regular, open communication and consultation with staff. Person/family engagement: participants endeavoured to provide patient-centred care. Formal patient feedback mechanisms and shared decision-making could be beneficial in the future. Staff engagement: working in a response room could affect well-being, though it also provided networking and learning opportunities. Staff require support and teambuilding. Use of improvement methods: improvements were made in a relatively informal, ad hoc manner. The use of robust methods based on improvement science was not reported. Measurement for quality: data were collected to improve efficiency and accuracy. More rigorous measurement would be beneficial, especially formally collecting stakeholder feedback. Governance for quality: close alignment with collaborators and clear communication with staff are essential. Information and communications technology for quality: this seventh driver was added because the importance of information technology specially designed for pandemics was frequently highlighted.

Conclusions The study provides insights on what worked well and what required improvement in a pandemic response room. It can inform health services, particularly emergency services, in their preparation for additional COVID-19 waves, as well as future crises.

  • COVID-19
  • quality in health care
  • qualitative research

Data availability statement

No data are available.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

http://dx.doi.org/10.1136/bmjopen-2021-057162

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • The qualitative design of this study facilitated the collection of rich data about the experiences of staff in a National Ambulance Service (NAS) in the midst of a pandemic.

    • This study was conducted in a recently established COVID-19 Response Room, which oversaw the delivery of a national COVID-19 home and community testing service, and thus provided unique insights on an entirely new service within a NAS.

    • The study employed several techniques for enhancing the rigour of qualitative research, such as peer debriefing and disconfirming evidence analysis.

    • This was a novel application of the Health Service Executive Framework for Improving Quality, which is a healthcare quality improvement (QI) framework, in the context of pandemic-imposed change.

    • A potential limitation of this study was that it used deductive thematic analysis, which could mean that important concepts or patterns that did not fit with the QI framework were overlooked.

    Introduction

    The COVID-19 crisis, which the WHO declared a pandemic in March 2020, has required health services around the world to undergo radical and rapid changes.1 2 COVID-19 is a highly transmissible disease that can have an array of short-term and long-term adverse health consequences, including respiratory, cardiovascular, neurological and psychiatric consequences.3 4 It has necessitated the introduction of numerous healthcare policies and practices, including infection control procedures, screening and vaccination programmes, and telehealth interventions.5 It has also prompted the development of quality improvement (QI) methods and tools suited to evaluating pandemic-imposed change, such as rapid learning cycles and after action reviews.1 6 QI typically entails using a systematic approach to design, test, measure and implement changes to improve the safety, effectiveness and experience of care for the benefit of patients and healthcare workers.7 In times of crisis, QI approaches must be adapted and streamlined so that improvements are implemented quickly without unnecessarily burdening strained services and staff.1 6 However, it is important that the validity and scientific rigour of these QI approaches is preserved, despite the urgency of the crisis, which has seen the rapid dissemination of large volumes of healthcare research of variable quality.8–11

    In Ireland, the first known case of COVID-19 was reported on 29 February 2020.12 In subsequent weeks, the Irish Government introduced various public health measures to curb the spread of the virus, including travel restrictions, closure of schools and non-essential businesses, cessation of large indoor gatherings, contact tracing for potential cases, and stay-at-home and social distancing measures, in addition to increasing funding for the Health Service Executive (HSE) and re-registering former healthcare workers.13 Ireland has since undergone several waves of COVID-19, including a substantial third wave during the 2020 Christmas period.14 Public health measures, including strict lockdown procedures and mask/face covering regulations, were implemented to mitigate these waves.12 At the time of writing, the nation had recently been heavily impacted by the Delta variant, which brought the total number of deaths from COVID-19 to over 5600 individuals, representing a death rate of approximately 1.1%.15 However, Ireland has implemented one of the strongest vaccination campaigns in the European Union, which has resulted in over 89% of the eligible population (ie, those aged over 12 years) being fully vaccinated since its launch in December 2020.16

    The National Ambulance Service (NAS) has been at the forefront of the HSE response to the COVID-19 pandemic in Ireland since the first wave. This required NAS to undergo swift and substantial changes, including the urgent implementation of a COVID-19 testing service, which was central to the national COVID-19 screening programme.17 18 To oversee the delivery of this service, NAS established the COVID-19 Response Room in its headquarters in Dublin. This was the only centre in Ireland dedicated to managing all referrals for COVID-19 tests and organising home testing for patients who cannot attend mass testing centres, as well as on-site testing for individuals living and working in residential facilities, such as nursing homes. It is staffed by a single team who were drawn from a variety of disciplines and professions, including general practitioners (GPs), nurses, administrators, dispatchers and call-takers. Some were recruited from within NAS. Others were recruited from different health service sectors or the Irish Defence Forces and thus had not previously worked in an ambulance dispatch environment. The response room structure and processes have evolved over the course of the pandemic and will continue to evolve while the pandemic endures. Moreover, the response room could be reinstituted in the event of future pandemics or similar emergencies. Therefore, this research aimed to examine the quality of the response room service by identifying practices and innovations that worked well, in addition to those that did not work well, in order to prepare for further COVID-19 waves or new crises, as well as to improve routine service. This examination was guided by a national QI framework: the HSE Framework for Improving Quality.19 The research will inform the preparation of health services, particularly Emergency Medical Services (EMS), for future crises.

    Methods

    Design

    This was a qualitative study that was designed to explore staff experiences and perceptions of the NAS COVID-19 Response Room service. The research paradigm was pragmatism, as this study was a component of a mixed methods evaluation of the NAS response to the pandemic.20 The full protocol of this evaluation was published prior to the present qualitative study being conducted.17 The qualitative method used in this study was the semi-structured, individual interview. This technique entails a discussion between a researcher and a participant that is guided by a flexible interview schedule comprising a shortlist of opening questions and a range of potential follow-up questions and probes.21 The schedule is typically designed to ensure that data are captured relating to core topics while maintaining sufficient flexibility to allow participants to bring their own perspectives and personalities to the fore.22 Amendments can be made to the schedule as the study progresses, including removing ineffective questions or adding probes to explore new issues raised by participants in previous interviews.21 The semi-structured approach enables the researcher to develop rapport, discuss sensitive or complex topics, explore unexpected responses and identify issues valued by the participants.21 23 In this study, the interview schedule was reviewed and, where necessary, refined following each interview, though its core content remained the same across the interviews (online supplemental file 1). The interviews took place between 7 December 2020 and 5 January 2021, when Ireland had entered a third COVID-19 wave.14 By this time, NAS had screened many tens of thousands of patients for COVID-19. Physical distancing regulations and travel restrictions meant that participants were interviewed via telephone. The study has been reported in line with the Standards for Reporting Qualitative Research (online supplemental file 2).24

    QI framework

    This research was informed by the HSE Framework for Improving Quality, which aims to guide the thinking, planning and delivery of healthcare in order to foster a culture of quality that continuously seeks to provide safe, effective and person-centred care.19 It comprises six QI drivers.

    Leadership for quality

    This entails cultivating a culture of continual improvement that places patients at the centre of care planning and delivery and that enables staff to provide safe, effective and compassionate care. It includes setting clear aims, regularly communicating with and listening to patients and staff, seeking evidence about service quality and demonstrating a commitment to a culture of quality.

    Person and family engagement

    This means listening to and learning from patients and their families and involving them in designing, planning and delivering care so that it meets their needs, considers their preferences and focuses on outcomes they value.

    Staff engagement

    This involves supporting teamwork, promoting a culture of respect, creating a healthy workplace, facilitating learning and development, and partnering with staff to design, deliver and improve services.

    Use of improvement methods

    QI should entail using established improvement methods, prioritising proven solutions to reduce harm, focusing on standardisation and availing of the unique knowledge of frontline staff.

    Measurement for quality

    Measuring and analysing data are central to QI by supporting the identification of areas that progressed or underperformed. This includes measuring patient experiences and outcomes, measuring only what matters and building measurement into routine practice.

    Governance for quality

    Boards and executive management teams should implement the structures, processes, standards and oversight needed for safe, person-centred and effective services. This includes clear leadership and accountability, a culture of learning and strong relationships with patients and staff.

    The framework has been applied to hospital services, a national safety programme for pressure ulcers and a social care programme for intellectual disabilities, though it has not previously been used in a pandemic setting.19 25 The current study explored the perceptions and experiences of COVID-19 Response Room staff regarding the six QI drivers.

    Patient and public involvement

    Patients and the public were not involved in this study.

    Participants

    In qualitative research, sampling typically concludes when sufficient information has been gathered to produce rich and convincing account of the phenomenon of interest, which differs from quantitative research, where sampling normally ceases when a predetermined sample size that can produce generalisable results has been achieved.26 27 As such, qualitative research can be conducted with relatively small samples, particularly when the topic is clearly defined and the participants are chosen carefully based on their substantial knowledge and experience of the topic.26 In this study, the target sample was the staff of the NAS COVID-19 Response Room. They were recruited via a study invitation email composed by the university research team and sent on their behalf by a NAS manager to former and current staff. They were informed that, if they agreed to participate, they would take part in a confidential interview with an external researcher from a university. They were also informed that the interview would not be used to evaluate their individual work performance, they would receive a copy of their transcript and they would not be identifiable in any subsequent reports. They could contact the university researchers directly for more information and to arrange participation.

    It was ensured that recruitment closed as soon as it had been determined that the two criteria of saturation and maximum variation had been fulfilled. This minimised the burden placed on the COVID-19 Response Room staff, who were overseeing a national COVID-19 testing service during the third wave of the pandemic in Ireland at the time of the study. Saturation occurs when no new themes or salient patterns that are relevant to the research aims are identified from the data.28 Saturation was assessed through preliminary data analysis and research team discussions. Maximum variation referred to diversity in the key characteristics of gender and role/occupation.28 29 This process led to the recruitment of nine participants, which represents approximately half of the staff in the response room (table 1). One other person, who was a medical doctor, declined participation due to clinical work commitments during the third wave of COVID-19.

    View this table:
    Table 1

    Participant demographic information

    Procedure

    The research team gave the participants an information sheet containing detailed information about the study, as well as their contact details for any questions. Informed, written consent was then obtained. Participants were interviewed at their preferred time and location (eg, home, office). Each interview lasted approximately 1 hour. They were conducted by the first or second author, who were respectively a postdoctoral researcher and a research assistant in a medical school in an Irish university. They had no prior connection to the participants. The first author had formal training and experience in conducting qualitative research, including interviewing healthcare workers. The second author received training from the first author and had previous experience of interviewing patients. They maintained field notes and reflexive notes throughout. They met virtually after each interview to determine whether any amendments were required to the schedule. For example, after the initial interviews, they shortened the schedule slightly to ensure that all core topics were covered within an hour. They also made minor adjustments to enhance the clarity of some questions and identified topics that had arisen during the initial interviews that could be further explored in subsequent interviews. The interviews were audio-recorded and transcribed verbatim. All data collected were treated confidentially and stored securely.

    Data analysis

    Reflexive thematic analysis was conducted in accordance with an established procedure developed by Braun and Clarke (2006; 2019; 2020).30–32 This procedure comprises six main analytical phases: familiarising oneself with the data, generating initial codes, generating initial themes, reviewing themes, defining themes and producing the written report. A theme represents a pattern of response or meaning within the dataset that is salient to the research question.30 In this procedure, the analysis is conducted at a point on the spectrum between primarily inductive analyses, which prioritise data-based meaning, and primarily deductive analyses, which prioritise analyst-based or theory-based meaning.32 33 Inductive analyses are firmly grounded in the data, though they are not purely and exclusively data-driven, as researchers cannot completely free themselves from their assumptions, aims and knowledge of the literature.30 32 Deductive analyses use existing research and theory as a lens through which the data are interpreted, which includes exploring evidence for themes identified by previous research or using an established framework to inform the coding of the data.32 They often include inductive elements, such as inductive coding, generating inductive subthemes within deductive themes, mapping inductive themes onto deductive themes or generating both inductive and deductive themes.33–36 The deductive approach is commonly used to evaluate, explicate, amend or challenge an existing framework, rather than to simply endorse it.36 37 The importance of the framework and its components can increase or decrease as the analysis progresses depending on the researcher’s assessment of its relevance and utility.36 37

    In the current study, the analysis was conducted by the first author and was supported by QSR International NVivo V.12 software. The analysis was primarily deductive, as it was underpinned by the Framework for Improving Quality.19 30 This framework had not been developed for or previously applied to pandemic-imposed change. The analysis explored experiences and perceptions of the QI drivers from the framework within a pandemicresponse room. The framework could be amended as part of the analysis because the aim was to ensure that all key patterns from the dataset were adequately represented, rather than to uncritically support the framework. For example, new drivers could be added to the framework or existing drivers could be re-contextualised or removed from the framework. The analysis began with immersion in the interview recordings and transcripts. Subsequently, the entire dataset, including extracts that appeared to be unrelated to the framework, was coded inductively to ensure that nothing of importance was overlooked and that the meaning communicated by the participants was captured.30 34 The codes were then refined and assigned to relevant themes. Peer debriefing enhanced the trustworthiness of the analysis.38–40 Debriefing entailed the first and second authors meeting to compare their interpretations of a subset of four transcripts. No substantial discrepancies were identified, indicating that the analysis was not restricted to the perspective of a single researcher. Additionally, the research team met regularly to discuss the data analysis. Disconfirming evidence analysis was also used to bolster trustworthiness.38–40 This involved returning to the dataset to ensure that the themes had sufficient evidence and that any pertinent contradictory evidence was reported.38–40 Ultimately, six themes were derived from the QI drivers of the framework. They represented the participants’ experiences and interpretations of these drivers in the context of a pandemic response room. Furthermore, a seventh theme, which was also a QI driver, was added so that all salient patterns from the dataset were captured.

    Results

    There were seven themes that each represented a key QI driver for the COVID-19 Response Room (table 2).

    View this table:
    Table 2

    Thematic analysis summary

    Leadership for quality

    Several participants stated that good leadership is essential in a pandemic response room where staff must frequently adapt to new and amended directives and procedures. In particular, the participants appreciated a leadership style that is person-oriented, rather than task-oriented, which includes ensuring staff feel comfortable sharing ideas and reporting mistakes and looking after staff well-being. Interviewee 1 said: ‘That would be the top thing: that [leaders] are not task-orientated but that they … value people and they … allow people to have a good idea and raise it … and that they are just empathetic’. Interviewee 8 stated:

    The leadership approach … on the clinical team was brilliant … Very understanding, very approachable … very appreciative of everyone’s contribution. [They] would encourage us to voice our opinions and … to continuously assess … As humans, we all are more inclined to hide [errors] if … we are going to be judged … An approach where, rather than scorning someone for a mistake, ask them: “Okay how can we improve this?”… is invaluable … That’s going to build up their skills [and] create a more effective and efficient environment.

    Many reported that the response room manager provided strong leadership, which included securing resources, providing support and boosting morale, thus enabling staff to perform well in a demanding environment. Interviewee 7 explained:

    She was absolutely amazing … She got to know people [and] their personalities … It was majorly busy … but we were well looked after … She would bend over backwards to help people out so they knew they could go to her … If we needed extra resources … the resources were provided … She looked after people, and everybody got their head down and got their work done … and it was always done to the highest standard.

    Additionally, the participants themselves often provided leadership within the response room, such as by training and supporting their peers, as well as becoming self-sufficient and identifying solutions to problems: ‘I’ve had to train [staff about] how the COVID Response Room works … I really enjoy teaching people … They’ll know the system inside-out’ (Interviewee 6). Interviewee 9 said:

    ‘We were able to … manage ourselves at times … Leadership isn’t just management. Leadership is getting people to lead themselves as well … One or two leaders that were in there … would teach you … so you can go off and manage it and then maybe teach … the other people’.

    Communication and consultation with staff was one aspect of leadership that several participants highlighted as an area that was often done well:

    The manager … was very caring and very direct and she allowed you [flexibility] around what you were doing. I never felt I was being told to follow a line I didn’t agree with … I could … make suggestions and she listened … If I found a way of doing it that was better: I did it. (Interviewee 1)

    However, some reported that there remained scope for the response room leaders to improve their consultation with staff. In particular, a more formal means of obtaining and implementing their feedback would be of benefit:

    ‘[It sometimes] felt like a very top-down approach … Whoever is on the ground has most experience with patients … Surely their input should be extremely valuable? … A platform where they can feedback [is needed]. Also maybe an audit … to make sure that input is actually … being taken into consideration’ (Interviewee 8).

    Person and family engagement

    Many reported listening to and learning from patients and families and providing person-centred care: ‘If a family member rang the room [needing] special arrangements, that was certainly done … There was a lot of communication … with patients and their families’ (Interviewee 7). Interviewee 1 recalled:

    This lovely … woman rang … in tears. Her father … was in a nursing home … She spent [his] last hours looking through the window at him … She didn’t know what he died of … The GP wasn’t available and …the laboratory [was] busy … I went on a search … I would ring her: “I haven’t forgotten”. … I eventually got the result … She was enormously grateful that I cared.

    It was difficult to formally gather patient and family feedback and involve them in decision-making during a pandemic: ‘Especially in the first wave … it was so fast-paced … Everyone was running off adrenaline … I don’t think there was time to ask for feedback … I don’t think there was going to be a benefit … when we knew it wasn’t perfect … at the time … It was such a scramble’ (Interviewee 8). However, feedback was often obtained informally: ‘[We] would have taken a lot of calls from patients, a lot of emails … We would have talked to patients to explain … if there was a delay or … ask them how they were doing … That was … the feedback mechanism … but it was … informal’ (Interviewee 8). Most agreed that formal engagement could be beneficial in future pandemics, especially in terms of clarifying the role, capacity and procedure of the testing service:

    In regards to testing … patient feedback could have helped … People are aware of what is involved now but certainly at the beginning … there was a lack of awareness. [Feedback on] how information could have been delivered better or what they thought of the entire process … could have a benefit … A lot of the descriptions that were going out were from medical experts … Not everyone does understand that terminology. (Interviewee 9)

    Staff engagement

    A key element of staff engagement is the creation of a healthy workplace. Several participants reported that well-being can be affected by working in the response room, especially during surges of COVID-19 cases or following difficult calls: ‘In the height of [the pandemic], staff were getting … long, long hours … It tires out the crew … Everyone is very down … You are taking call after call … But it’s what … we love to do … Just to help people’ (Interviewee 6). Several noted that well-being support was available from managers and peers. Interviewee 9 said: ‘I would have had one or two bad phone calls … You were able to take that step back … There wasn’t a shortage of people to go to or resources to look after you’. Interviewee 3 said: ‘You had lots of support … Lots of people checking in on you … There was good collegial interaction’. Adequate staffing is also important for well-being: ‘If we knew we were going to be busy, if we could get [extra] cover: that would be a great help’ (Interviewee 4).

    Another important element of staff engagement is supporting teamwork and facilitating development. Several commented that the response room provided opportunities for learning and variety: ‘I’m really enjoying what I’m doing. It’s … totally different … I’ve learnt an awful lot’ (Interviewee 4). Interviewee 7 said: ‘Being able to help out and feel like you [are] making a difference was good’. There were also networking opportunities: ‘It’s good for the CV: the connections that you are making … You have more time to get to know the staff, the managers, the officers out on the road’ (Interviewee 2). Some noted that additional training, teambuilding and engagement could help staff adjust to the environment and cooperate more effectively. This is especially important for those who were unfamiliar with an ambulance dispatch environment:

    There was segregation: a little bit of … “You are this team, I’m that team”. … At the same time, generally, there was no friction … What struck me was … how regimental it was … There was … a plan for us to have a … training day … but it just didn’t happen … It would have probably helped people to understand … how the structure works, how it’s organised … We would have been able to slot in … faster, (Interviewee 5)

    Interviewee 8 said: ‘Everything … stems back to relationships and communication and just to make sure that everyone is in the same boat and ensure that patient care is the end goal’. Unfortunately, teambuilding can be difficult during a pandemic: ‘I would hope when everything quietens down that we could … sit down and have dinner and … talk about war stories … We are not there just yet’ (Interviewee 4).

    Use of improvement methods

    The participants reported that they had encountered little use or adaption of established and rigorous improvement methods, such as rapid learning cycles, in the response room. Several stated that implementing improvement initiatives can be difficult given the considerable upheaval, unpredictability and increased demands brought about by a pandemic: ‘During the first wave, there was very little time to … look back and see how things could be improved. It was more so a case of … just fighting against the wave … certainly from our level’ (Interviewee 9). Interviewee 6 said: ‘I’m sure there is plenty of ways of improving it and obviously we don’t know [what’s coming]. After Christmas we could be absolutely [inundated] or it could be … calm … We are on the top of the roller-coaster about to tip down … We don’t know what way it’s going to go’.

    Nevertheless, improvements were possible in the response room, though they tended to be made on a relatively informal, ad hoc basis: ‘The clinical team … towards the end of the first peak … created a few standard operating procedures and we created lists … with all the important contacts [and] what we thought worked, what we thought didn’t work … That was a really invaluable document … if they were training anyone’ (Interviewee 8). Many provided examples of improvements they had introduced or in which they had been involved, often out of necessity, during the course of their work. Interviewee 1 said: ‘We had to design a [referral] form for … the nursing homes because we were getting the two hundred medications the elderly clients were on … We are not interested in that. We wanted the bare bones … So we designed that and that made life easier’. Interviewee 7 recalled:

    We would have … hundreds of emails … on a daily basis for all the referrals … Initially, it was two people looking at the same email account … We didn’t know … if the other person had … actioned the email … I set up a process that, if I … actioned it, … I put a little tick beside it … I did … a process flow chart … that was followed then by anybody that took over the email account … They were too important not to know whether they were definitely actioned or not.

    Despite the challenges posed the by pandemic, most were in favour of the implementation of improvement processes, especially more defined and stringent processes, given their importance for managing both current and future crises: ‘It’s good that people are looking to see what worked and what didn’t … That’s always a positive’ (Interviewee 7). Interviewee 8 commented: ‘Continuous assessment … was a good way … to try and find the learnings and findings within the process itself or while doing it’.

    Measurement for quality

    Data were collected and analysed regarding response room activities in order to support QI, especially enhancing efficiency and streamlining, as well as minimising errors and delays. For instance, Interviewee 8 said: ‘The clinical team … were constantly assessing … “Could we do this more effective and efficiently?” … If anything fell through the cracks, [like] a patient had been waiting longer than … expected, [They] wouldn’t … hide that there was some error or issue … They would always assess … if there’s something that was avoidable’. However, the collection and analysis of data for QI was often unsystematic, especially in the early days of the response room, when procedures and resources were still being put in place. For example, Interviewee 5 recalled efforts to record information about patient care:

    NAS were using their [emergency] call system … There was a notes section on that … Otherwise we were using a communications log book [for] sharing information … in relation to the care … being provided … Informally [the manager] did actually call us … to say: “How are we getting on?” … Informally at break times you were talking about how things could improve or better processes.

    Interviewee 6 reported that the measurement of call data, especially response times, became more systematic with time:

    At the start … they used to print off sheets with all the calls and it would get very messy … Now … everything is recorded … We have to make sure that we don’t miss anybody … It’s all about … ways of improving … We have a system now where … we time each call and [measure] the distance of each call to improve … the speed of the swabbing.

    Several participants identified areas where measurement for quality could be improved, including facilitating more rigorous and routine measurement. For example, Interviewee 9 suggested regularly collecting and reviewing appointment data to maximise COVID testing: ‘Make sure that every available opportunity there was to provide a COVID test … was being used … Have a look at was every [test] slot filled … Make sure … people were showing up or [find out] what was going on’. Several recommended formally gathering and assessing feedback from a range of stakeholder groups, including key collaborators: ‘They should [get] feedback … from GPs, from the public health, from the … lab … to try and make sure that we are … meeting the needs’ (Interviewee 5). Patients were another important stakeholder group:

    [In] the patient-centred service that we are trying to provide … their input is hugely important … There was one scenario of a lady … Since she had been referred for a COVID test, the home help stopped coming … I called … to ask: “Was she eating? Could she shower?” … When you are so focused on getting a COVID swab, it’s easy to [overlook] the holistic approach to the patient care … Another reason why patient feedback would be extremely important. (Interviewee 8)

    Staff were also an important stakeholder group: ‘With [this interview], it’s good to see someone on the outside talking to someone on the inside … You’d be wording it differently if you were speaking to someone … in the job … It’s good to see other people … are interested in … what we do’ (Interviewee 6). This measurement would benefit not only the response room but also improve the management of future crises: ‘Be prepared for future public health shocks … That’s absolutely key … I don’t think you can just close the room and say it no longer exists … There’s a lot of … ongoing work to be done … around evaluation of those processes … Other QIs will emerge out of that’ (Interviewee 3).

    Governance for quality

    When the response room was established during the first wave, its standards and processes were still being developed. Interviewee 2 explained:

    When we started off, it was kind of madness … We just had to learn new procedures, put new policies in place … We had to learn new protocols [about] COVID symptoms … We had a lot of people ringing in thinking they had it … It was all up in the air about who gets tested, who doesn’t … We were looking at maybe a thousand calls … a day.

    Interviewee 8 said:

    ‘A few different … areas … were all trying to come together, which was extremely difficult under the circumstances … There was NAS and the clinical team and [staff] back out of retirement … It was difficult … at the start trying to come up with a strategy and … a system, trying to understand all the different software … and talk to external stakeholders’.

    Over time, the governance evolved:

    The COVID room was … a combination of people from the [NAS] college, from the Air Corps, from the [NAS] corporate office … and from [ambulance] control … It was well governed … It was good … once that all settled into place … When it started, we were all learning what we needed to do … We’d never dealt with this before but … it was dealt with very positively. (Interviewee 7)

    One particular area that required improvement was collaboration and coordination with other organisations, including general practices, residential facilities, laboratories and contact tracing: ‘There was no communication between the ‘Test and Trace’ process and ourselves in the early days. [We] asked them for their communication strategy … I went down through it … There’s nothing about the home test … by the National Ambulance Service, so we made that change’ (Interviewee 3). Some felt that, occasionally, communication from management required improvement: ‘Everyone in a nursing home was going to be tested as part of mass screening … We had found that out through watching the news … Communication in that regard was lacking’ (Interviewee 9). At other times, there was good communication: ‘In the morning … you got a handover and … the urgent stuff flagged to you … The directive would come down. The communication was quite clear’ (Interviewee 3).

    Information and communications technology for quality

    Most asserted that suitable information and communications technology (ICT) is essential for optimal performance in the response room. Initially, the response room relied heavily on paperwork: ‘We generated so much paperwork … because the ICT infrastructure wasn’t … sufficient … We must have … damaged a rainforest’ (Interviewee 3). Over time, the ICT infrastructure improved, enhancing efficiency and accuracy: ‘We used to have to handwrite all the lists … We’d write down each call individually … Then we started typing up the lists … That cut the workload a bit and then they brought in a report so basically you can … pull all the calls that are active … That cut down the workload’ (Interviewee 2). Several recommended developing ICT infrastructure specifically for managing pandemics:

    The system was … built for the dispatchers to deal with accidents … around the country but the COVID-19 cases were all going into that as well so the system hadn’t been designed to actually tackle [that] amount of people … A system developed specifically for … pandemics … would help greatly, or at least designed to have a greater capacity. (Interviewee 9)

    Ideally, different health service computer systems and databases should be integrated within the ICT infrastructure, including giving each patient an identification code that can track them across systems:

    Everybody [needs] a unique identifier … There’s no doubt … We had a reliance on two numbers. I don’t think you should rely on two … The systems should have spoken to each other … When somebody went up on [the referrals system] it should have populated … straight through and onto [the NAS] system … We created a lot of work in relation to that. (Interviewee 3)

    Discussion

    This study examined the quality of service in a pandemic response room dedicated to delivering the Irish national COVID-19 home and residential testing programme. It provided unique insights on QI at a time of crisis. Specifically, it explored the experiences and perceptions of pandemic response room staff regarding key QI drivers from the Framework for Improving Quality. The first such driver was leadership for quality. The results underscore the importance of person-centred, rather than task-centred, leadership that entails attending to staff well-being and regularly communicating and consulting with staff, which includes enabling them to openly share ideas and report errors. This finding aligns with best practice recommendations for healthcare leadership in a pandemic, which include distributed leadership and deference to expertise, frequent and open communication, and meeting physical and psychological safety needs.1 41 42 Moreover, the results indicate that any employee can display leadership in this environment by being self-sufficient, using their initiative to solve problems, and by training and supporting colleagues. This somewhat corresponds with the Safety Differently approach to improving safety, which entails creating a culture where staff are viewed as a source of solutions, rather than problems, and where all employees, including frontline workers, are encouraged to take ownership of safety processes.43

    The second QI driver was person and family engagement. The findings show that staff endeavoured to provide person-centred care, including listening to patients and their families and tailoring the service to their individual needs. However, pandemic pressures impeded formal engagement, such as requesting feedback and making shared decisions about care design and delivery. In line with published guidance, formal engagement was recommended for future pandemics.44 45 This engagement should be meaningful, rather than tokenistic, such that it entails partnering with patients to design, plan and produce healthcare and sharing leadership, power and decision-making.44 45 This has the potential to improve the clarity of pandemic-related communications, increase adherence to pandemic isolation measures, and enhance the support provided to caregivers during the crisis.45 The third QI driver was staff engagement. The results suggest that working in a response room can affect well-being, especially during case surges. Previous investigations confirm that the well-being of EMS professionals can be impaired by the pandemic, though they largely focused on paramedics and first responders, rather than dispatch or response room staff.46–49 In the current study, staff reported benefiting from organisational and peer support, as well as networking and learning opportunities. Greater teambuilding and training would have bolstered integration and cooperation, particularly for staff recruited from external organisations. Such activities should be virtual, outdoors or distanced to protect staff during the pandemic.

    The fourth and fifth QI drivers were use of improvement methods and measurement for quality. In the response room, there were efforts to make improvements and to collect data to enhance efficiency and accuracy. However, there was little utilisation or adaptation of robust improvement methods and measurement that had a foundation in improvement science or established QI models. The substantial workload and uncertainty created by the pandemic, especially the first wave, was frequently cited as the reason for this lack of rigour. This corresponds with reports that the rapid conduct and dissemination of healthcare research during the pandemic frequently diminished its quality and integrity.8–10 For example, the COVID-19 crisis increased the use of novel, untested methods and established but less stringent methods.8 9 However, it is possible to employ and adapt valid QI methods and models during a pandemic, especially where there is appropriate governance and resources, as demonstrated by several recent studies that evaluated new telemedicine services and rapid response capabilities.9 50–52 It is also possible to learn from the repository of literature about previous pandemics, including outbreaks of other respiratory viral infections.53 Quality standards must be upheld, even during crises, in the interests of patient safety, resource allocation, and the trustworthiness of clinical research and practice.9 It should be noted that the current study is part of a project examining the NAS response to the pandemic, which represented a move towards a more formal approach to measurement and evaluation.17 This project also entailed assessing the dissemination of newsletters containing pandemic-related evidence and guidance to NAS personnel and evaluating the sharing of pandemic-related information within an international EMS network.

    The sixth QI driver was governance for quality. The findings highlight the importance of close cooperation with collaborating organisations (eg, laboratories, contact tracing and nursing homes) coupled with clear and continual communication with staff. It has previously been recommended that pandemic-related communication between senior leadership and frontline staff should involve establishing a centralised team to review the latest developments and coordinate among departments, disseminating staff-wide information daily and hosting virtual town halls or staff forums approximately two times per week.41 In Ireland, NAS regularly distributed newsletters to staff summarising the latest COVID-19 literature and established a COVID-19 information-sharing network with international EMS representatives.17 Finally, a seventh QI driver was identified in this study: ICT for quality. Specifically ICT that has been designed to manage a pandemic, including processing large case numbers during waves and tracking cases across different databases and systems, is required to optimise efficiency and minimise errors. This finding suggests that the Framework for Improving Quality should be expanded to include this driver, especially when applied to similar contexts.

    A limitation of this study was that, though participants from a range of roles, including clinical staff, were interviewed, a medical doctor was not recruited. A representative of this profession could have provided unique insights. However, increased workloads due to the third wave of COVID-19 in Ireland made it difficult to recruit a physician. Another limitation was that the participants were recruited via an email sent by a NAS manager on behalf of the university research team. The university researchers did not have access to the contact details of current and former response room staff due to data protection concerns. Additionally, NAS advised that response room staff would be unlikely to respond to an external email from the university, especially when dealing with the third wave. However, a disadvantage of this approach is that the participants could have been concerned about their anonymity, which could have affected their willingness to openly discuss their experiences and opinions. In particular, they could have felt uncomfortable providing negative feedback. To help counter this, they were assured that the interviewers were external to NAS, the interviews would be confidential and would not be used for performance evaluation, and the participants would not be identifiable in any reports produced. They also received a copy of their transcript, which provided an opportunity to raise concerns or make clarifications, though none availed of this. Additionally, both current and former response room staff were recruited, as the latter could be especially willing to speak freely. Moreover, the results show that the participants were willing to highlight practices requiring improvement. Nevertheless, at least some participants could have remained reluctant to be completely open during the interviews.

    Another limitation of this study was that it used a predominantly deductive analysis approach whereby the themes were largely derived from the Framework for Improving Quality. Deductive approaches are especially suited to investigations of a highly specific research question or of a phenomenon of interest that can be illuminated by a relevant, existing framework.30 33 54 In contrast, inductive approaches are especially suited to more explorative studies in which the specific research question evolves during the analysis or in which the aim is to develop a new theory or model.30 Consequently, deductive approaches tend to provide a more detailed analysis of a particular aspect of the dataset, while inductive approaches tend to produce a more comprehensive and rich analysis of the entire dataset.30 Additionally, deductive approaches risk being constrained by the selected framework. For example, important themes that do not match the framework could be overlooked. Conversely, inductive approaches typically produce results that are more strongly linked to the participants’ responses. However, in contrast with this perspective, it has been argued that existing frameworks and orienting concepts can sensitise researchers to relevant issues and interpretations that they might not have identified themselves using an inductive approach, resulting in a rich description of the phenomenon.36 Furthermore, no qualitative analysis is entirely inductive because they cannot be conducted in a vacuum isolated from the researcher’s preconceptions, including their knowledge of relevant literature and theory.30 55 56 In both inductive and deductive reflexive thematic analysis, themes are not viewed as residing within the dataset waiting to be discovered by the researcher or to emerge during the analysis.27 31 Instead, the researcher is regarded as having a central and active role in the construction of themes at the intersection of the data, the analytic process, the research aims and the researcher’s subjectivity, theoretical understanding, training, and experience.27 31

    In the current study, efforts were made to minimise the risk of important themes being overlooked due to the deductive nature of the analysis. Specifically, the entire dataset was coded openly, including responses that seemed unrelated the Framework for Improving Quality.30 34 This led to amendment of the framework through the addition of a seventh QI driver, ‘Information and communications technology for quality’. However, it remains possible that some key concepts or patterns were missed. An alternative approach (eg, inductive thematic analysis, grounded theory) would likely have produced different results that were more data-driven, rather than theory-driven. Despite this limitation, deductive analysis has become an established approach in healthcare research, including QI and emergency care research.37 57–60 For example, a recent study applied an existing health psychology framework to the analysis of interviews with first response volunteers in order to identify barriers to automated external defibrillator use.60 Ultimately, both deductive and inductive analyses are acceptable provided that the researcher is transparent about which approach is used and what its implications and limitations are so that the findings can be interpreted and contextualised appropriately.30 35 54

    Conclusion

    This study examined the quality of service in pandemic response room that coordinated the Irish national COVID-19 home and residential testing programme. It provided unique insights on staff experiences in a NAS in the midst of a crisis and represented a novel application of the Framework for Improving Quality in the context of pandemic-imposed change. The findings indicate that the framework, which comprises six QI drivers (eg, staff engagement, governance), should be expanded to include a seventh driver (ie, ICT), especially when used to address pandemics and similar emergencies. The findings also suggest that, while there were many areas that performed well within the response room (eg, person-centred leadership, patient-centred care), there were also areas requiring improvement. In particular, there was a need for a more formal and rigorous approach to patient and staff engagement and the implementation of QI methods and measurement. This follows calls for research that improves the rigour and quality of improvement practices and evaluations.11 Additionally, there have been calls for clinical research and practice during the pandemic to balance the urgent implementation of change and rapid conduct of studies with the preservation of quality and integrity.9 This study can inform EMS and other health services in their QI activities and in preparation for and response to future public health crises.

    Data availability statement

    No data are available.

    Ethics statements

    Patient consent for publication

    Ethics approval

    This study involves human participants and was approved by the COVID-19 National Ethics Research Committee, Ireland (Reference number: 20-NREC-COV-025). Participants gave informed consent to participate in the study before taking part.

    Acknowledgments

    The authors would like to thank the staff of the National Ambulance Service of Ireland and the Irish Air Corps who contributed to this research.

    References

    1. ↵
      1. Shah A,
      2. Pereira P,
      3. Tuma P
      . Quality improvement at times of crisis. BMJ 2021;373:n928. doi:10.1136/bmj.n928pmid:http://www.ncbi.nlm.nih.gov/pubmed/33975873
      OpenUrlFREE Full Text
    2. ↵
      1. World Health Organization
      . Coronavirus disease 2019 (COVID-19) situation report - 51. Geneva, Switzerland, 2020.
    3. ↵
      1. Hu B,
      2. Guo H,
      3. Zhou P, et al
      . Characteristics of SARS-CoV-2 and COVID-19. Nat Rev Microbiol 2021;19:141–54.doi:10.1038/s41579-020-00459-7pmid:http://www.ncbi.nlm.nih.gov/pubmed/33024307
      OpenUrlPubMed
    4. ↵
      1. Leung TYM,
      2. Chan AYL,
      3. Chan EW, et al
      . Short- and potential long-term adverse health outcomes of COVID-19: a rapid review. Emerg Microbes Infect 2020;9:2190–9.doi:10.1080/22221751.2020.1825914pmid:http://www.ncbi.nlm.nih.gov/pubmed/32940572
      OpenUrlPubMed
    5. ↵
      1. Fisk M,
      2. Livingstone A,
      3. Pit SW
      . Telehealth in the context of COVID-19: changing perspectives in Australia, the United Kingdom, and the United States. J Med Internet Res 2020;22:e19264. doi:10.2196/19264pmid:http://www.ncbi.nlm.nih.gov/pubmed/32463377
      OpenUrlCrossRefPubMed
    6. ↵
      1. Fitzsimons J
      . Quality and safety in the time of : design better, learn faster. Int J Qual Health Care 2021;33doi:10.1093/intqhc/mzaa051pmid:32400860
      OpenUrlCrossRefPubMed
    7. ↵
      1. Jones B,
      2. Vaux E,
      3. Olsson-Brown A
      . How to get started in quality improvement. BMJ 2019;364:k5408. doi:10.1136/bmj.k5437pmid:http://www.ncbi.nlm.nih.gov/pubmed/30655245
      OpenUrlFREE Full Text
    8. ↵
      1. Jung RG,
      2. Di Santo P,
      3. Clifford C, et al
      . Methodological quality of COVID-19 clinical research. Nat Commun 2021;12:943. doi:10.1038/s41467-021-21220-5pmid:http://www.ncbi.nlm.nih.gov/pubmed/33574258
      OpenUrlPubMed
    9. ↵
      1. Lipworth W,
      2. Gentgall M,
      3. Kerridge I, et al
      . Science at warp speed: medical research, publication, and translation during the Covid-19 pandemic. J Bioeth Inq 2020;17:555–61.doi:10.1007/s11673-020-10013-ypmid:http://www.ncbi.nlm.nih.gov/pubmed/32840838
      OpenUrlPubMed
    10. ↵
      1. Spec A,
      2. Schwartz IS
      . Balancing scientific rigor with urgency in the coronavirus disease 2019 pandemic. Open Forum Infect Dis 2020;7:ofaa304.doi:10.1093/ofid/ofaa304pmid:http://www.ncbi.nlm.nih.gov/pubmed/32855988
      OpenUrlCrossRefPubMed
    11. ↵
      1. Dixon-Woods M
      . How to improve healthcare improvement—an essay by Mary Dixon-Woods. BMJ 2019;367:l5514. doi:10.1136/bmj.l5514pmid:http://www.ncbi.nlm.nih.gov/pubmed/31575526
      OpenUrlFREE Full Text
    12. ↵
      1. Cazelles B,
      2. Nguyen-Van-Yen B,
      3. Champagne C, et al
      . Dynamics of the COVID-19 epidemic in Ireland under mitigation. BMC Infect Dis 2021;21:735. doi:10.1186/s12879-021-06433-9pmid:http://www.ncbi.nlm.nih.gov/pubmed/34344318
      OpenUrlPubMed
    13. ↵
      1. Kennelly B,
      2. O'Callaghan M,
      3. Coughlan D, et al
      . The COVID-19 pandemic in Ireland: an overview of the health service and economic policy response. Health Policy Technol 2020;9:419–29.doi:10.1016/j.hlpt.2020.08.021pmid:http://www.ncbi.nlm.nih.gov/pubmed/32923355
      OpenUrlCrossRefPubMed
    14. ↵
      1. Health Protection Surveillance Centre
      . Weekly report on the epidemiology of COVID-19 in Ireland: week 1, 2021. Dublin, Ireland, 2021.
    15. ↵
      1. Central Statistics Office
      . COVID-19 deaths and cases, series 34. Cork, Ireland, 2021.
    16. ↵
      1. COVID-19 Ireland - Ireland's COVID-19 Data Hub, Government of Ireland
      . COVID-19 Ireland - Ireland’s COVID-19 Data Hub, Government of Ireland. Daily Vaccination Headline Figures. Dublin, Ireland, 2021.
    17. ↵
      1. Masterson S,
      2. Heffernan E,
      3. Keegan D, et al
      . Rapid response and learning for later: establishing high quality information networks and evaluation frameworks for the National Ambulance Service response to COVID-19 - the ENCORE COVID project protocol. HRB Open Res 2020;3:68.doi:10.12688/hrbopenres.13149.2pmid:http://www.ncbi.nlm.nih.gov/pubmed/33537553
      OpenUrlPubMed
    18. ↵
      1. Department of the Taoiseach
      2. Government of Ireland
      . Ireland’s national action plan in response to COVID-19 (Coronavirus). Dublin, Ireland, 2020.
    19. ↵
      1. Quality Improvement Division
      2. Health Service Executive
      . Framework for improving quality in our health service - Part 1: introducing the framework. Dublin, Ireland, 2016.
    20. ↵
      1. Johnson RB,
      2. Onwuegbuzie AJ
      . Mixed methods research: a research paradigm whose time has come. Educ Res 2004;33:14–26.
      OpenUrl
    21. ↵
      1. DeJonckheere M,
      2. Vaughn LM
      . Semi-structured interviewing in primary care research: a balance of relationship and rigour. Fam Med Community Heal. 2019;7:e000057.
    22. ↵
      1. Barrett D,
      2. Twycross A
      . Data collection in qualitative research. Evid Based Nurs 2018;21:63–4.doi:10.1136/eb-2018-102939pmid:http://www.ncbi.nlm.nih.gov/pubmed/29858278
      OpenUrlFREE Full Text
    23. ↵
      1. Ives J,
      2. Damery S
      . Qualitative data collection. In: Walker DM, ed. An introduction to health services research. London: Sage, 2014.
    24. ↵
      1. O'Brien BC,
      2. Harris IB,
      3. Beckman TJ, et al
      . Standards for reporting qualitative research: a synthesis of recommendations. Acad Med 2014;89:1245–51.doi:10.1097/ACM.0000000000000388pmid:http://www.ncbi.nlm.nih.gov/pubmed/24979285
      OpenUrlCrossRefPubMed
    25. ↵
      1. Quality Improvement Division
      2. Health Service Executive
      . Report of the review of the application of the framework for improving quality in our health services: learning to guide future approaches. Dublin, Ireland, 2018.
    26. ↵
      1. Cleary M,
      2. Horsfall J,
      3. Hayter M
      . Data collection and sampling in qualitative research: does size matter? J Adv Nurs 2014;70:473–5.doi:10.1111/jan.12163pmid:http://www.ncbi.nlm.nih.gov/pubmed/24450874
      OpenUrlCrossRefPubMed
    27. ↵
      1. Braun V,
      2. Clarke V
      . Conceptual and design thinking for thematic analysis. Qual Psychol 2021.doi:10.1037/qup0000196
    28. ↵
      1. Kuper A,
      2. Lingard L,
      3. Levinson W
      . Critically appraising qualitative research. BMJ 2008;337:a1035. doi:10.1136/bmj.a1035pmid:http://www.ncbi.nlm.nih.gov/pubmed/18687726
      OpenUrlFREE Full Text
    29. ↵
      1. Patton M
      . Qualitative evaluation and research methods. Beverly Hills, CA: Sage, 1990: 169–86.
    30. ↵
      1. Braun V,
      2. Clarke V
      . Using thematic analysis in psychology. Qual Res Psychol 2006;3:77–101.doi:10.1191/1478088706qp063oa
      OpenUrlCrossRef
    31. ↵
      1. Braun V,
      2. Clarke V
      . Reflecting on reflexive thematic analysis. Qual Res Sport Exerc Health 2019;11:589–97.doi:10.1080/2159676X.2019.1628806
      OpenUrl
    32. ↵
      1. Braun V,
      2. Clarke V
      . One size fits all? What counts as quality practice in (reflexive) thematic analysis? Qual Res Psychol 2020:1–25.
    33. ↵
      1. Byrne D
      . A worked example of Braun and Clarke’s approach to reflexive thematic analysis. Qual Quant 2021:1–22.
    34. ↵
      1. Fereday J,
      2. Muir-Cochrane E
      . Demonstrating rigor using thematic analysis: a hybrid approach of inductive and deductive coding and theme development. Int J Qual Methods 2006;5:80–92.doi:10.1177/160940690600500107
      OpenUrl
    35. ↵
      1. Nowell LS,
      2. Norris JM,
      3. White DE
      . Thematic analysis: striving to meet the trustworthiness criteria. Int J Qual methods 2017;16:1609406917733847.
    36. ↵
      1. Macfarlane A,
      2. O’Reilly-de Brún M
      . Using a theory-driven conceptual framework in qualitative health research. Qual Health Res 2012;22:607–18.doi:10.1177/1049732311431898pmid:http://www.ncbi.nlm.nih.gov/pubmed/22203386
      OpenUrlCrossRefPubMedWeb of Science
    37. ↵
      1. Morden A,
      2. Ong BN,
      3. Brooks L, et al
      . Introducing Evidence Through Research "Push": Using Theory and Qualitative Methods. Qual Health Res 2015;25:1560–75.doi:10.1177/1049732315570120pmid:http://www.ncbi.nlm.nih.gov/pubmed/25656415
      OpenUrlCrossRefPubMed
    38. ↵
      1. Yardley L
      . Demonstrating validity in qualitative psychology. In: Smith JA, ed. Qualitative psychology: a practical guide to research methods. London: Sage Publications, 2008: 235–51.
    39. ↵
      1. Creswell JW,
      2. Miller DL
      . Determining validity in qualitative inquiry. Theory Pract 2000;39:124–30.doi:10.1207/s15430421tip3903_2
      OpenUrlCrossRefWeb of Science
    40. ↵
      1. Morse JM
      . Critical analysis of strategies for determining rigor in qualitative inquiry. Qual Health Res 2015;25:1212–22.doi:10.1177/1049732315588501pmid:http://www.ncbi.nlm.nih.gov/pubmed/26184336
      OpenUrlCrossRefPubMed
    41. ↵
      1. Kuy S,
      2. Gupta R,
      3. Correa R
      . Best practices for a Covid-19 preparedness plan for health systems. NEJM Catal Innov Care Deliv 2020.
    42. ↵
      1. Staines A,
      2. Amalberti R,
      3. Berwick DM, et al
      . COVID-19: patient safety and quality improvement skills to deploy during the surge. Int J Qual Health Care 2021;33doi:10.1093/intqhc/mzaa050pmid:32400870
      OpenUrlCrossRefPubMed
    43. ↵
      1. Dekker S
      . Safety differently: human factors for a new era. London: CRC Press, 2014.
    44. ↵
      1. Ocloo J,
      2. Matthews R
      . From tokenism to empowerment: progressing patient and public involvement in healthcare improvement. BMJ Qual Saf 2016;25:626–32.doi:10.1136/bmjqs-2015-004839pmid:http://www.ncbi.nlm.nih.gov/pubmed/26993640
      OpenUrlAbstract/FREE Full Text
    45. ↵
      1. Aboumatar H
      . Three reasons to focus on patient and family engagement during the COVID-19 pandemic. Qual Manag Health Care 2020;29:176–7.doi:10.1097/QMH.0000000000000262pmid:http://www.ncbi.nlm.nih.gov/pubmed/32590495
      OpenUrlPubMed
    46. ↵
      1. Prezant DJ,
      2. Zeig-Owens R,
      3. Schwartz T, et al
      . Medical leave associated with COVID-19 among emergency medical system responders and firefighters in New York City. JAMA Netw Open 2020;3:e2016094. doi:10.1001/jamanetworkopen.2020.16094pmid:http://www.ncbi.nlm.nih.gov/pubmed/32706380
      OpenUrlPubMed
    47. ↵
      1. Hogan C,
      2. Williams J,
      3. Rees N
      . Practitioner experiences in epidemics and pandemics in relation to COVID-19. Int Paramed Pract 2020;10:79–92.doi:10.12968/ippr.2020.10.4.79
      OpenUrl
    48. ↵
      1. Friedman MS,
      2. Strayer RJ
      . Prehospital care at the epicenter of a pandemic: the New York City EMS response. Acad Emerg Med 2020;27:797–801.doi:10.1111/acem.14045pmid:http://www.ncbi.nlm.nih.gov/pubmed/32531844
      OpenUrlPubMed
    49. ↵
      1. Rees N,
      2. Smythe L,
      3. Hogan C, et al
      . Paramedic experiences of providing care in Wales (UK) during the 2020 COVID-19 pandemic (PECC-19): a qualitative study using evolved grounded theory. BMJ Open 2021;11:e048677. doi:10.1136/bmjopen-2021-048677pmid:http://www.ncbi.nlm.nih.gov/pubmed/34140344
      OpenUrlAbstract/FREE Full Text
    50. ↵
      1. Levy N,
      2. Zucco L,
      3. Ehrlichman RJ, et al
      . Development of rapid response capabilities in a large covid-19 alternate care site using failure modes and effect analysis with in situ simulation. Anesthesiology 2020;133:985–96.doi:10.1097/ALN.0000000000003521pmid:http://www.ncbi.nlm.nih.gov/pubmed/32773686
      OpenUrlPubMed
    51. ↵
      1. Gilbert AW,
      2. Billany JCT,
      3. Adam R, et al
      . Rapid implementation of virtual clinics due to COVID-19: report and early evaluation of a quality improvement initiative. BMJ Open Qual 2020;9:e000985. doi:10.1136/bmjoq-2020-000985pmid:http://www.ncbi.nlm.nih.gov/pubmed/32439740
      OpenUrlAbstract/FREE Full Text
    52. ↵
      1. Tenforde AS,
      2. Iaccarino MA,
      3. Borgstrom H
      . Telemedicine during covid-19 for outpatient sports and musculoskeletal medicine physicians. PM&R 2020;12:926–32.
      OpenUrl
    53. ↵
      1. Saunders-Hastings P,
      2. Reisman J,
      3. Krewski D
      . Assessing the state of knowledge regarding the effectiveness of interventions to contain pandemic influenza transmission: a systematic review and narrative synthesis. PLoS One 2016;11:e0168262.
    54. ↵
      1. Kiger ME,
      2. Varpio L
      . Thematic analysis of qualitative data: AMEE Guide No. 131. Med Teach 2020;42:846–54.doi:10.1080/0142159X.2020.1755030pmid:http://www.ncbi.nlm.nih.gov/pubmed/32356468
      OpenUrlPubMed
    55. ↵
      1. Malterud K
      . Qualitative research: standards, challenges, and guidelines. Lancet 2001;358:483–8.doi:10.1016/S0140-6736(01)05627-6pmid:http://www.ncbi.nlm.nih.gov/pubmed/11513933
      OpenUrlCrossRefPubMedWeb of Science
    56. ↵
      1. Tuckett AG
      . Applying thematic analysis theory to practice: a researcher’s experience. Contemp Nurse 2005;19:75–87.doi:10.5172/conu.19.1-2.75pmid:http://www.ncbi.nlm.nih.gov/pubmed/16167437
      OpenUrlCrossRefPubMed
    57. ↵
      1. Dunham AH,
      2. Dunbar JA,
      3. Johnson JK, et al
      . What attributions do Australian high-performing general practices make for their success? applying the clinical microsystems framework: a qualitative study. BMJ Open 2018;8:e020552. doi:10.1136/bmjopen-2017-020552pmid:http://www.ncbi.nlm.nih.gov/pubmed/29643162
      OpenUrlAbstract/FREE Full Text
    58. ↵
      1. Hamilton J,
      2. Verrall T,
      3. Maben J, et al
      . One size does not fit all: a qualitative content analysis of the importance of existing quality improvement capacity in the implementation of releasing time to care: the productive Ward™ in Saskatchewan, Canada. BMC Health Serv Res 2014;14:1–14.doi:10.1186/s12913-014-0642-x
      OpenUrlCrossRefPubMed
    59. ↵
      1. Tang J,
      2. Toma M,
      3. Gray NM, et al
      . Pharmacist and data-driven quality improvement in primary care (P-DQIP): a qualitative study of anticipated implementation factors informed by the theoretical domains framework. BMJ Open 2020;10:e033574. doi:10.1136/bmjopen-2019-033574pmid:http://www.ncbi.nlm.nih.gov/pubmed/32114469
      OpenUrlAbstract/FREE Full Text
    60. ↵
      1. Smith CM,
      2. Griffiths F,
      3. Fothergill RT, et al
      . Identifying and overcoming barriers to automated external defibrillator use by GoodSAM volunteer first responders in out-of-hospital cardiac arrest using the theoretical domains framework and behaviour change wheel: a qualitative study. BMJ Open 2020;10:e034908. doi:10.1136/bmjopen-2019-034908

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • Contributors All of the authors meet the ICMJE criteria for authorship. SM and EH designed the study with input from CD, CO, PC, AH and AM. EH, DK, SM and BC recruited the participants. EH and DK acquired the data. EH analysed the data with assistance from DK. EH drafted the manuscript. EH, DK, SM, BC, CD, CO, PC, AH and AM contributed to the interpretation of the data, critically revised the manuscript for important intellectual content, approved the final version and agreed to be accountable for all aspects of the work. EH acted as the guarantor for the work.

    • Funding This work was supported by the Health Research Board of Ireland [COV19-2020-111].

    • Competing interests None declared.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.