Article Text

Original research
Qualitative evaluation of a mandatory provincial programme auditing emergency department return visits
  1. Lucas B Chartier1,2,
  2. Hanna Jalali3,
  3. M. Bianca Seaton4,
  4. Howard Ovens5,6,
  5. Bjug Borgundvaag5,6,
  6. Shelley L McLeod6,7,
  7. Katie N Dainty4,
  8. Olivia Ostrow8,9
  1. 1Emergency Medicine, University Health Network, Toronto, Ontario, Canada
  2. 2Division of Emergency Medicine, Department of Medicine, University of Toronto Faculty of Medicine, Toronto, Ontario, Canada
  3. 3University of Toronto Faculty of Medicine, Toronto, Ontario, Canada
  4. 4Research and Innovation, North York General Hospital, Toronto, Ontario, Canada
  5. 5Department of Emergency Medicine, Sinai Health System, Toronto, Ontario, Canada
  6. 6Department of Family and Community Medicine, University of Toronto Faculty of Medicine, Toronto, Ontario, Canada
  7. 7Schwartz/Reisman Emergency Medicine Institute, Sinai Health System, Toronto, Ontario, Canada
  8. 8Department of Pediatrics, University of Toronto Faculty of Medicine, Toronto, Ontario, Canada
  9. 9Department of Emergency Medicine, The Hospital for Sick Children, Toronto, Ontario, Canada
  1. Correspondence to Dr Lucas B Chartier; Lucas.Chartier{at}uhn.ca

Abstract

Objective The objective of this qualitative study was to evaluate the perceived impact and value of the Return Visit Quality Programme (RVQP), a mandatory province-wide emergency department audit programme.

Design We employed an interpretive descriptive qualitative approach with maximum variation sampling to ensure diverse representation across several geographical and institutional factors. RVQP programme leads were invited to participate in semistructured interviews and snowball sampling was used to reach non-lead physicians to capture the perspectives of those working within the programme.

Setting In Ontario’s RVQP, participating emergency departments must audit their return visits resulting in admission to identify issues that can be addressed through quality improvement initiatives.

Participants Between June and August 2018, we interviewed 32 participants (local programme leads and non-lead physicians) from 23 out of the 86 participating centres.

Results Participants’ perceived impact and value of the programme was associated with the existence (or absence) and nature of the local quality improvement culture, the implementation approach of the programme within their emergency departments, and key aspects of the programme pertaining to medicolegal concerns and resource availability.

Conclusions This study of an innovative, large-scale programme aimed at promoting continuous quality improvement in emergency departments showed that while its perceived impact has been meaningful, there are key structural and operational elements that support and hinder this aim. Healthcare leaders should consider these findings when looking to implement large-scale audit or quality improvement programmes.

  • accident & emergency medicine
  • audit
  • quality in health care

Data availability statement

All data relevant to the study are included in the article or uploaded as online supplemental information.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

  • We used an interpretive descriptive qualitative approach for this study.

  • We developed a coding structure and triangulated the respondents’ interviews with their emergency department setting.

  • We then grouped, compared and refined our analytical categories through an inductive, iterative approach.

  • The findings can be used by health system leaders to better implement large-scale quality improvement programmes.

Introduction

Emergency medicine is practised in a fast-paced, high-risk environment where adverse events can occur and be consequential.1 2 Compounding this risk is the episodic, broad and unpredictable nature of emergency medicine, which makes it challenging to give targeted provider feedback on these adverse events. While audit and feedback programmes have been shown to improve emergency physician performance,3 auditing every emergency department (ED) case is not practical. To obtain actionable information for individual and system-level feedback and quality improvement (QI), it is crucial to identify a subset of patients that will yield greater learning and QI opportunities. Unplanned return visits to the ED have been considered an indicator of quality of care.4 5 While the overall rate at which people return to the ED is not always indicative of the quality of care,6–9 focusing on patients who had return visits leading to hospital admission (RVAs) has been shown to help identify specific cases that may have a greater rate of adverse events or quality issues amenable to improvement.10–12

Innumerable programmes have been created in recent years to improve the quality of care provided in the healthcare system, using a variety of implementation strategies and with variable sustainability results.13 14 Audit and feedback programmes have previously shown to have positive effects on patient safety and improved care, and all were highly influenced by the context they were placed in.15 16 Some studies have focused on audit and/or feedback in the ED, but they have typically evaluated small initiatives or been discrete, retrospective analyses of programmes rather than focusing on ways to improve programme development, impact and sustainability.6 7 9 11 Some have attempted to implement financial incentives, including with incentivising the practice of doing what is ‘right’ as opposed to simply doing more.7 12 In emergency medicine particularly, larger-scale QI initiatives have leaned heavily towards improving wait times and length of stay, with fewer initiatives and less measurement focused on other domains of quality, such as effectiveness and safety.17

The ED Return Visit Quality Programme (RVQP) was launched in 2016 in Ontario, Canada, as a large-scale programme meant to promote continuous QI through the routine auditing of RVA cases in EDs.18 Participating hospitals must conduct a set number of audits each year to identify the underlying cause(s) of these RVAs, which may lead to the identification of system-level quality gaps that can be addressed with QI initiatives. All EDs with >30 000 patient visits per year are mandated to participate as part of a provincial funding programme, and lower-volume EDs can participate on a voluntary basis. The programme is intended to provide a focused mechanism through which ED clinicians can receive data on their patients who returned to the ED and reflect on their individual practices, and it gives ED leaders a systematic approach for discovering quality issues that are amenable to local improvement opportunities.18

Evaluating healthcare quality programmes to identify QI barriers and enablers is recommended as a valuable source of learning and a strategy to improve programme results.19 Without formal evaluation, the opportunities for system and programme learnings applicable to others may be lost, and the likelihood of programme sustainability may be diminished. To our knowledge, the RVQP is the largest mandatory audit programme in emergency medicine. To maximise what healthcare and policy leaders looking to develop similar initiatives can learn from the development and conduct of such a large-scale programme, we conducted a qualitative evaluation of the impact the RVQP has had on promoting continuous QI. The objective of this qualitative study was to interview key stakeholder groups at participating centres across the province to determine factors that have supported and challenged the implementation of the RVQP over its first 3 years, determine the perceived impact and value of the programme, and identify areas for actionable change to ensure the programme’s ongoing success and sustainability.

Methods

Evaluation design

We employed an interpretive descriptive qualitative approach, described by Thorne as a non-categorical methodology that emerged in response to a call for an alternative way of generating grounded knowledge relating to clinical practice and that aimed to move qualitative inquiry to a more abstract form of interpretation beyond the level of description.20 The product of an interpretive description is a coherent conceptual description that taps thematic patterns and commonalities believed to characterise the phenomenon that is being studied and also accounts for the inevitable individual variations within them.21 This approach is particularly useful for unpacking the assumptions made around the complex issues inherent to healthcare delivery and QI, as well as for assessing stakeholder perspectives on new programme implementation efforts.22

This manuscript conforms to the Consolidated Criteria for Reporting Qualitative Research and Standards for Reporting Qualitative Research.23 24 Participants gave informed consent before taking part in this study.

Patient and public involvement

Patients and the public were not involved in this study. We will disseminate the study results to participants through the sharing of presentations and manuscripts created for dissemination.

Setting

Over the first 3 years of the RVQP, 86 hospitals in Ontario, Canada participated: the province’s largest 73 hospitals participated as part of a mandatory pay-for-results programme, and 13 smaller-volume hospitals participated in the RVQP on a voluntary basis.25–27 Participating hospitals were provided with local data regarding patients who had RVAs within 72 hours of their initial ED visit for any diagnosis, or within 7 days for sentinel diagnoses of acute myocardial infarction, subarachnoid haemorrhage or paediatric sepsis that were paired with relevant diagnoses on the initial visit (eg, chest pain, headache and fever, respectively). These sentinel diagnoses were selected based on previously published literature confirming that these diagnoses have higher rates of adverse events.10 28–32 Hospitals are required by the provincial Ministry of Health to conduct a minimum of 50 audits per year to identify the underlying causes for these RVAs, which may lead to the identification of system-level quality gaps that can be addressed with tailored QI projects.33 Participating hospitals each submit an annual report of the audit results, signed by the organisation’s chief executive officer after presentation to the Quality/Safety Committee of the Board, to a provincial governmental agency overseeing the programme. The performances of hospitals are not assessed or compared, nor is there an intention to decrease the rates of RVAs, given the potential for unintended consequences of such an approach.10 34 In the first 3 years of the programme, data were available on 13 559 664 ED visits in the province, with 125 698 (0.93%) of those being RVAs within 72 hours and 847 (0.006%) of them being 7-day sentinel RVAs. As part of the RVQP, local teams of ED providers conducted 12 852 detailed audits.

Sampling and recruitment

Within the cohort of participating hospitals, we designed a multi-item sampling matrix to ensure representation across several geographic and institutional experiences (urban vs rural, based on population size; academic vs community, based on university affiliation; mandated vs voluntary, based on programme requirement). Each of the ED RVQP local leads (whether physicians or administrators) were invited via email to participate in a telephone interview. Leads agreeing to participate were asked to contact the research assistant directly to set up an interview time. One reminder email was sent to non-responders, after which if no response was received it was assumed there was no further interest in participating. A snowball sampling approach was then used to reach non-lead physicians in order to capture the penetration of the programme at the ground level and its impact on local QI efforts. Snowball sampling is a qualitative sampling tactic that involves the researchers accessing potential informants through recommendations or connections by other informants.35

Data collection

Individual in-depth interviews were conducted by telephone to allow physicians from different geographic regions to participate without the barriers of travel. The research assistant (HJ) conducted the interviews with support from a PhD-trained qualitative researcher (BS), using semistructured interview guides to provide broad topic areas to guide the discussion. The research assistant had no prior relationship with any of the participants, is not involved in the conduct of the RVQP itself, and notified potential participants that their consent or refusal to participate in this study would remain anonymous and have no bearing on their participation in the mandated RVQP.

The study team developed semistructured interview guides based on a review of the literature, input of team members with expertise in QI, emergency medicine, and qualitative research, as well as the objectives of this study (online supplemental appendix 1). Questions were designed to explore participants’ awareness and knowledge of the RVQP, to elicit their perspectives on the implementation of the programme in their EDs, and to understand how it has informed local QI projects and its impact on local QI culture. The study team reviewed and refined the interview guides through three iterations prior to the commencement of data collection and revised them again after the review of the first four interviews had been completed to ensure the topics of interest were being adequately explored. The research assistant also memoed throughout the interview process to keep track of nuances between interviews, new and developing ideas, and analytic thinking. These were transcribed and included in the analysis.

Data collection continued until no new ideas or concepts were identified in subsequent interviews. Through preliminary analysis and discussion the study team agreed on that point at which thematic saturation was reached.36 The interviews were audio-recorded and an external transcription service transcribed them verbatim. All transcripts were deidentified during transcription and checked for accuracy by the research assistant prior to analysis.

Data analysis

The data were analysed through using an interpretive analytic framework and according to standard thematic analysis techniques as described by Braun & Clark.37 Contextual factors of the ED setting such as setting, size and participation requirements were also considered.

Two members of the study team (HJ and BS) independently reviewed the interview transcripts and interviewer memos to generate a list of descriptive codes to represent the data. Five authors then collaboratively developed and finalised a master-coding framework. Two members of the study team independently attached the codes to segments of the text in a sample of five transcripts through line-by-line readings, to ensure consistent application of the coding framework and discuss any discrepancies. Once there was agreement and consistency, the remainder of the transcripts were coded by the research assistant.

Through an inductive and iterative process, the descriptive codes were then grouped into broad topic-oriented categories, compared, refined and formulated into fewer analytical categories. Conceptual themes were inductively derived from analysis among and between individual interviews. Versions of the analysis were reviewed with the study team at regular intervals. The final analytical framework, representing themes that reflect patterns and regularities of responses in the coded transcripts, was discussed among all authors until consensus was reached on its validity and applicability. During these consultations, the team also discussed our respective reflexivity and potential biases and ensured we could recalibrate when our interpretations began to diverge from the data in any way. NVivo V.12 Pro by QSR International (Doncaster, Australia) was used to manage the data.

Results

Between June and August 2018, 32 participants (56.1% of the 57 individuals contacted) were interviewed, including 21 RVQP programme leads (15 physicians and 6 nursing/management leaders) and 11 non-lead physicians, from 23 EDs across the province of Ontario. The 23 hospitals represent 26.7% of RVQP participating centres in Ontario and included various geographic locations and institutional experiences. More details on the characteristics of the sample can be found in table 1. The interviews averaged 30 min in length (range 18–45 min).

Table 1

Description of the centres and participants interviewed

Overall, we found that all participants saw value in the programme as well as in the process of auditing quality of care, both for individual education and self-improvement and for system level to improvement. Additionally, most participants reported specific improvement outcomes related to their local settings, such as access to care, documentation or quality of care. However, there were both successes and challenges for site leaders in the implementation of this province-wide programme, creating mixed feelings about the continued utility of the RVQP. Our analysis showed that the perceived impact of this programme was rooted in three crucial themes: the existing QI culture, the approach to socialising the programme within the department, and key characteristics of the programme that both contribute to and detract from its value. Representative quotes for each theme are included below, and additional ones are provided in box 1.

Box 1

Exemplar interview quotes, broken down by themes

Existing QI culture

‘It was kind of just like me, the manager, and the chief kind of whipping through these and trying to figure it out as we go on top of being so busy and trying to get all of our other things done in the moment rather than looking at the past.’ (Interviewee 14-C)

‘It’s also prompted us to really look at the why. Without the program, we probably wouldn’t have figured out the why. I think that’s been a helpful part of it. Sometimes you need a programprogramme to prompt you really to have to look.’ (Interviewee 19-C)

‘I feel like the person who does this needs to have system thinking, right? You can’t just assign this task to anybody who volunteers. I think to do it well people need some training with regards to cost analysis, and understanding what it truly means to do a chart audit for the purpose of QI’. (Interviewee 08-B)

Local implementation and socialisation

‘Sometimes you can’t be up-to-date with everything that’s going on and having these review cases—it’s a non-threatening way of learning to improve your practice. So, I think it’s got tremendous value. And has really good impact.’ (Interviewee 09-A)

Key programme attributes and challenges

‘I think there’s value because there’s specific indicators that bring us data that we might have not otherwise picked up.’ (Interviewee 20-C)

‘What we're hoping to do is identify system issues and processes that we can improve that overall benefit the patients in our department.’ (Interviewee 11-B)

Medicolegal concerns and communication

‘The only barrier that came up was one quarter was delayed fairly significant because there were some questions as to whether or not we were opening up physicians to liability. So, it had to go through whether or not it was going to be part of our Quality of Care Information Protection Act process just to make sure that we weren't putting it out there that an error had been made or something like that so that this was a positive experience as opposed to a negative one.’ (Interviewee 13-B)

Resources limitations

‘If we had somebody, or hours, or money dedicated to it, then that would definitely justify it, I guess, because right now we’re just doing it on top of all the rest of our duties.’ (Interviewee 14-C)

‘Unfortunately, a community hospital doesn't have the same infrastructure and manpower resources to carry out these audits as some of the bigger academic centers. So, it’s just based on how hospitals fund [their EDs]. There’s a great disparity there.’ (Interviewee 17-B)

Existing QI culture

Participants often mentioned the recent emphasis on QI in healthcare as a driver behind the development of some type of structured approach to addressing quality and performance improvement in their EDs. It was clear the existence (or lack) of a baseline QI infrastructure at participating hospitals was influential in the experience of RVQP implementation and resultant action at each ED. We learnt that existing QI infrastructure was vastly different between centres, and it typically included some of the following components: incident or safety reporting systems for tracking complaints, patient safety events and/or RVAs; individuals or teams providing QI leadership; and a focus on analysing flow metrics such as wait times, ED length of stay and time to hospital admission.

The perceived impact of the RVQP seemed to be directly dependent on the existing level of support for QI initiatives prior to programme implementation. For some hospitals without QI infrastructure or a defined structure for performance review, the RVQP was seen as an additional burden on already-stretched ED managers, and the resulting impact to work ratio was seen as low. Many participants discussed that prior training in the principles of QI was often an important factor associated with a centre observing impact and continued engagement with the programme. Institutions with minimal QI expertise were felt to have a weaker ability to support system-level thinking and introduce effective change. In contrast, at EDs with previously established QI committees and resources for QI projects and data review, the RVQP was seen as an additional improvement opportunity and created more data to support systemic changes for quality issues that had already been identified. For example, one participant shared:

[The program] lets me go and sit down with the chief of medical imaging and say, ‘We identified in our return visit audits that there are issues between the preliminary reads overnight and the final reads’, [and ask for] expanded hours of staff radiologist coverage. (Interviewee 11-B)

However, even for some hospitals with fairly robust safety programmes (eg, incident report systems), the RVQP did not appear to be seen as adding value given the existing workload of provincial reporting and limited perceived impact.

Local implementation and socialisation

There was significant variability in how the RVQP was implemented and socialised within ED teams, and this often could be categorised into one of two approaches. The first approach was more of a ‘centralised’ implementation where the ED chief, manager and/or director completed all the required chart audits with the assistance of someone in a supportive administrative role and disseminated the results to front-line staff at a team meeting or departmental rounds. The second approach was a ‘distributed’ implementation where ED leadership created a shared process for completing the chart audits with other physicians and healthcare team members, and the results were disseminated to front-line staff through additional systems of education, support, and at times individual feedback to the providers involved. An example of the distributed implementation described was:

We then have a rotating doc every month […] review those cases, and then they bring that review to our monthly combined meeting. We set aside between half an hour and 45 minutes for review of that every month. (Interviewee 12-B)

The way the ED leadership introduced and explained the programme to the healthcare teams was highly influential on the teams’ perceptions of its purpose and value. The non-leads we interviewed in centres where the RVQP had been implemented through a centralised approach were generally not aware of the programme’s purpose or its local/provincial impact, beyond the fact that chart audits were being completed in their EDs. Participants in this group typically mentioned being concerned with education and self-improvement based on their own cases, rather than overall system-level change and improvement. Others exhibited anxiety about the programme based on a perceived fear that the government was auditing their practices.

When a more distributed approach was used to implement the RVQP locally, non-lead team members were much better able to articulate the purposes and goals of the programme and its effects on their centres. They understood it was not intended as just an audit of their personal practices but also, more importantly, as a way to identify system-level problem areas for improvement.

Key programme attributes and challenges

Participants noted various elements specific to the RVQP programme that made it of value for their teams, especially with regard to how it was structured and how they were supported. These included the RVQP tools provided to individual centres, including preformatted audit templates and instructions on how to complete them appropriately; the education that supplemented the programme’s launch; the programme requirement for wide dissemination of the results within each hospital (ie, yearly results must be presented to the hospital’s quality committee of the board and signed by the chief executive officer); and the aim of the programme which brought forward quality and system-level issues in emergency medicine as opposed to a focus only on provider and time-based metrics such as patient flow and throughput. One participant shared the following:

And this is actually the first time that they pushed us to look at—quality. […] Like, we're all focused on [flow] metrics right now, and getting people through. And so this was a good initiative. (Interviewee 10-B)

Conversely, there were significant challenges external to the RVQP that may have decreased the potential perceived impact of the programme for some centres. Several providers expressed medico-legal concerns with the RVQP chart audit process. Since patients from the audited cases had a greater likelihood of having suffered an adverse event, some providers were hesitant about formally auditing, writing down, and submitting such information to the governmental agency managing the RVQP. For example, the following was a concern voiced by a participant:

I'd like a really good assurance that government watching is not a personal watching on me, […] that I'm the guy that had all 17 patients return. And that it doesn't become a quality assessment of me, personally. […] And then I think Emerge docs might be more interested in participating. (Interviewee 10-A)

Participants also mentioned the importance of having a dedicated person at their individual sites (ie, with a formal position, resources, and funding) for the RVQP given the significant time commitment involved. This would potentially increase the benefits of the programme and diminish some of the frustrations associated with it, but most centres did not have the funds to create such a position. Relatedly, some participants mentioned the inability to act on adverse events and quality issues uncovered due to hospital-wide funding cuts and their leadership’s prioritisation of other projects.

Discussion

We completed a qualitative evaluation of the largest mandatory audit programme for ED RVA cases, with 32 participant interviews from 23 diverse centres across the province of Ontario. Given the mandatory nature of the programme for the province’s largest EDs, it was not participation but rather the perceived impact and value of the programme that differentiated the various centres. In our study, we saw that perceived impact was underpinned by three themes: the setting in which it was introduced; its existing QI infrastructure and the implementation approach of the programme within each ED; and programme characteristics. Health system leaders aiming to enhance the engagement of their teams and the impact of their programmes may thus want to focus on making benefits and unperceived needs more readily apparent to the end-users.

There was a wide spectrum of QI infrastructure in the participating EDs prior to the implementation of the RVQP, and this had a significant impact on centres’ experiences. Our data suggest the more a given ED team was engaged with QI work in general, the more likely they were to view the programme as useful and have it lead to improvements in their own ED. This was often associated with an existing QI infrastructure including dedicated resources, expertise, and engagement, so the ED teams felt confident in tackling the quality issues that were identified through the programme. These findings confirm other studies’ findings that proper infrastructure, support, and expertise must be available prior to implementing new programmes to ensure their success.38 39 For health system leaders looking to develop similar mandatory programmes, ensuring the alignment between programme requirements and local capabilities, expertise level and incentivisation thus may increase likelihood of successful engagement.

In terms of local implementation and socialisation of the RVQP, we learnt that some centres chose a centralised approach to conduct audits, which often led to poorer understanding by non-leads of the goals of the programme. Alternatively, other centres employed a more distributed and multidisciplinary approach, which resulted in a greater understanding of the aims of the programme by the entire healthcare team. Engaging physicians in operational and improvement projects has indeed been demonstrated to lead to better performance and results.40 41 For audit programmes to be effective, leaders should recommend an implementation process that is distributed and includes both front-line providers and individuals with QI training/education.

Participants shared that the RVQP worked well with regard to the supporting materials, education, programme accountability and the reporting structure. However, challenges included medicolegal concerns, privacy issues, and funding limitations. In Ontario, the Quality of Care Information Protection Act allows ‘health professionals to have open discussions about critical incidents involving patient care and QI matters in general […] without fear that the information will be used against them.’42 However, this act remains poorly understood and inconsistently used by hospitals, which may contribute to greater concern than is warranted (and therefore, diminish the learning potential for providers).43 Better communication and supporting materials describing these issues might have mitigated some of these medico-legal concerns and led to more active engagement by some end-users. As a result of these study results, the RVQP programme developed supporting materials and included these topics in communications with programme participants. For health system leaders looking to develop audit and feedback programmes, our findings reaffirm the necessity to account for medicolegal issues that are of local relevance. Creating educational opportunities to alleviate these concerns is important, as these issues could otherwise detract from and compromise the success of new initiatives.

Limitations

This study is not without limitations. We interviewed 32 participants from 23 diverse centres out of 86 eligible EDs, so our results may not be generalisable to other centres that did not participate. However, given the multi-item sampling matrix that was defined prior to recruitment to ensure representativeness of programme participants and centres, we believe our results to be representative. Additionally, we achieved thematic saturation prior to the last few interviews, and we felt no further meaningful insights were likely to be gleaned from additional interviews.

Another limitation of our evaluation is that the RVQP started only 3 years prior to our study, which may have limited the penetration and impact of the programme in this relatively short time frame for a large-scale programme. For example, several centres reported an iterative approach to programme implementation between the first and second years as they gained a deeper understanding of the programme. As a result, it may be relevant and instructive to re-evaluate the RVQP after a number of years to determine whether its goals are being achieved.

Conclusions

To promote a culture of continuous QI, hospitals should value and build organisational support for QI programmes and initiatives. The RVQP is one innovative and large-scale programme that enables hospitals, particularly EDs, with this cultural direction. This study, evaluating the perceived impact of the RVQP’s aim of promoting continuous QI in EDs, showed that key structural and operational elements support or hinder this primary aim. Health system leaders should consider the findings of this study when looking to implement large-scale QI programmes.

Data availability statement

All data relevant to the study are included in the article or uploaded as online supplemental information.

Ethics statements

Ethics approval

The research ethics board at Mount Sinai Hospital in Toronto, Ontario, reviewed the study protocol and granted a waiver of approval for this QI study (2018-05-05).

Acknowledgments

The authors wish to acknowledge the support of members of the Return Visit Quality Program Working Group (Lee Fairclough as co-chair, Dr. Lisa Calder, Dr. Jonathan Dreyer, Lisa Fuller, Emily Hayes, Delaney Hines, Sudha Kutty, Dr. Michael Schull, Kaeli Stein, and Ivan Yuen), Danielle Porplycia, and Carol Hilton.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • LBC and HJ are joint first authors.

  • KND and OO are joint senior authors.

  • Twitter @chartierlucas

  • Presented at Canadian Association of Emergency Physicians 2019 Scientific Assembly. Halifax, Nova Scotia, Canada.

  • Contributors LBC and OO initially designed the study and all authors helped refine the methodology used. HJ, KND and MBS conducted the interviews and analyzed the data. All authors contributed to the interpretation of the data. LBC wrote the first draft of the manuscript. All authors provided critical revisions incorporated in the final manuscript, and they have reviewed and approved the final draft.

  • Funding This work was supported by Health Quality Ontario (HQO), a provincial governmental agency, which provided an unrestricted grant for the conduct of this study. HQO did not contribute to the design, conduct, analysis or reporting of the study.

  • Competing interests Three of the authors (LBC, HO and OO) sit on the Working Group that supports Health Quality Ontario in managing the Return Visit Quality Program. None of them are remunerated for this work. None of the other authors have competing interests to declare.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.