Article Text

Original research
Differences in emergency nurse triage between a simulated setting and the real world, post hoc analysis of a cluster randomised trial
  1. Stefan Morreel1,
  2. Veronique Verhoeven1,
  3. Hilde Philips1,
  4. Jasmine Meysman2,
  5. Ines Homburg2,
  6. Diana De Graeve2,
  7. KG Monsieurs3,4
  1. 1Department of Family and Population Health, University of Antwerp Faculty of Medicine and Health Sciences, Antwerpen, Belgium
  2. 2Department of Economics, University of Antwerp, Antwerpen, Belgium
  3. 3Emergency Department, Universitair Ziekenhuis Antwerpen, Edegem, Belgium
  4. 4ASTARC, University of Antwerp, Antwerpen, Belgium
  1. Correspondence to Dr Stefan Morreel; Stefan.Morreel{at}uantwerpen.be

Abstract

Objectives In the TRIAGE trial, a cluster randomised trial about diverting emergency department (ED) patients to a general practice cooperative (GPC) using a new extension to the Manchester Triage System, the difference in the proportion of patients assigned to the GPC was striking: 13.3% in the intervention group (patients were encouraged to comply to an ED or GPC assignment, real-world setting) and 24.7% in the control group (the assignment was not communicated, all remained at the ED, simulated setting). In this secondary analysis, we assess the differences in the use of the triage tool between intervention and control group and differences in costs and hospitalisations for patients assigned to the GPC.

Setting ED of a general hospital and the adjacent GPC.

Participants 8038 patients (6294 intervention and 1744 control).

Primary and secondary outcome measures proportion of patients with triage parameters (reason for encounter, discriminator and urgency category) leading to an assignment to the ED, proportion of patients for which the computer-generated GPC assignment was overruled, motivations for choosing certain parameters, costs (invoices) and hospitalisations.

Results An additional 3.1% (p<0.01) of the patients in the intervention group were classified as urgent. Discriminators leading to the ED were registered for an additional 16.2% (p<0.01), mainly because of a perceived need for imaging. Nurses equally chose flow charts leading to the ED (p=0.41) and equally overruled the protocol (p=0.91). In the intervention group, the mean cost for patients assigned to the GPC was €23 (p<0.01) lower and less patients with an assignment to the GPC were hospitalised (1.0% vs 1.6%, p<0.01).

Conclusion Nurses used a triage tool more risk averse when it was used to divert patients to primary care as compared with a theoretical assignment to primary care. Outcomes from a simulated setting should not be extrapolated to real patients.

Trial registration number NCT03793972.

  • primary care
  • accident & emergency medicine
  • organisation of health services

Data availability statement

Data are available on reasonable request. Given the privacy policy of the iCAREdata database, the authors are not allowed to share the used database. Sharing this database would potentially harm the privacy of the included patients. The Belgian Data Protection Authority does not allow the authors to share the raw data. The authors are, however, able to deliver a selection of variables and the outputs of their statistical software upon reasonable request. Most of the studied data is available to researchers worldwide after following the application procedures of iCAREdata (see www.icaredata.eu or contact icare@uantwerpen.be).

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

  • Post hoc analysis of the TRIAGE trial in which patients were diverted from an emergency department to primary care.

  • This cluster randomised trial offered the opportunity to assess the difference in use of the same triage tool between real-world triage and simulated triage.

  • This trial was not designed to measure the outcomes of this post hoc analysis.

  • This is a single-centre study in the context of the Belgian healthcare system, potentially limiting the generalisability of the results.

Introduction

Worldwide, initiatives for collaboration between emergency departments (EDs) and sites for primary out-of-hours care (called general practitioners cooperatives or GPCs) have been installed to improve patient care, staff satisfaction, and to reduce crowding at the ED. In these initiatives, both services are located on the same site or in proximity of each other. They implement triage, defined as the sorting out and classification of patients or casualties to determine priority of need (urgency classification) and proper place of treatment (assignment to ED or GPC).1 The efficiency of such a triage has been studied in simulated settings (written case scenarios or live cases), retrospective case studies or in prospective interventional studies describing the situation before and after implementation of triage.2–5 A comparison of live cases versus paper case scenarios revealed a lower intraclass correlation for urgency classification in live triage assessments as compared with paper cases. Paper case scenarios generally receive lower triage scores (less urgent) than live cases.6 One of the few well validated and widely implemented triage systems is the Manchester Triage System (MTS). A multicentre prospective observational study revealed a sensitivity of 0.47–0.87 and a specificity of 0.84–0.94.7 In a meta-analysis, the agreement regarding written scenario assessment was substantial while it was almost perfect for assessment of real live cases.8

In 2018, a Cochrane review concluded that the current evidence concerning primary care professionals providing non-urgent care in EDs was insufficient to draw conclusions for practice or policy.9 It is unknown how well the efficiency of triage observed in simulated settings compares to everyday practice (the real world).10 Some studies found that the presence of a general practitioner (GP) leads to an improvement in the effectiveness and quality of care at the ED and is less expensive than the usual care method, as GPs use fewer resources than do usual ED staff.11–14 Again, it is unknown whether a simulated experiment can predict such a cost reduction.

The TRIAGE (Triaging and Referring in Adjacent General and Emergency Departments) trial studied the efficiency and safety of a newly developed extension (called eMTS) to the original MTS assigning low risk patients to ED or GPC. This trial was executed during weekends and bank holidays (from here on we refer to weekends and bank holidays as weekends). An eMTS triage results in three parameters: reason for encounter (presentational flow chart, eg, ‘abdominal pain’), discriminator (property of the complaint, eg, ‘mild pain’) and urgency category (ranging from one or to five). Each combination of parameters is linked to an ED or GPC assignment. Weekends were randomly allocated to the intervention group (patients were encouraged to follow their assignment to ED or GPC) or the control group (the assignment was not communicated, all remained at the ED, a simulated setting). This intervention led to the safe diversion of 9.5% of the included patients.15 In the intervention group, 838/6374 patients (13.3%) were assigned to the GPC, in the control group this was almost twice as much: 431/1744 (24.7%). We hypothesise that this remarkable difference was caused by a difference in use of the triage tool between intervention and control weekends.

The objective of this secondary analysis was to assess the differences in the use of the triage tool between the intervention and the control group of the TRIAGE trial. As a secondary objective, the difference in hospitalisations and costs between the intervention and control group for low urgency patients was studied.

Methods

We refer to the original article on the TRIAGE trial for details on the participants, the intervention and the study design.15 Another paper explores the characteristics of patients non-compliant to a GPC advice.16 Below we only describe those aspects important for the current article.

Study design

This study is a post hoc analysis of a cluster randomised controlled trial (RCT) executed from 1 March 2019 to 30 December 2019. This trial was randomised into 10 control and 27 intervention weekends. The intervention was triage by a nurse using a new extension to the MTS assigning low-risk patients to the GPC and all other patients to the ED. During intervention weekends, patients were encouraged to follow this assignment while it was not communicated during control weekends (all patients remained at the ED). In the intervention group, the triage had immediate consequences which is comparable to real world circumstances. The virtual triage in the control group had no consequences which is comparable to a simulated setting with live cases. The primary outcome was the proportion of patients assigned to and handled by the GPC during intervention weekends (9.5%, 95% CI 8.8% to 10.3%). The trial was randomised for the financial analysis (under review) and the secondary outcome: the proportion of patients assigned to the GPC during intervention (13.3%, 95% CI 12.5% to 14.2%) and control weekends (24.7%, 95% CI 22.7% to 26.8%).

Study tool

The study tool for the TRIAGE trial was based on the MTS (V.3.6), a validated tool for prioritisation.7 17 When using the MTS, the nurse starts by choosing a presentational flow chart(eg, chest pain). A flow chart consists of a list of terms called discriminators (eg, mild pain), the presence of which has to be checked top-down.

The eMTS adds site of treatment (ED or GPC) to this system in 42/53 flow charts, the remaining nine flow charts always lead to the ED. In 18 flow charts, additional discriminators led patients in urgency categories four or five to an assignment to the ED (when at least one supplementary discriminator is applicable) or GPC (when no supplementary discriminators are applicable). These additional discriminators were either already present in the original MTS (eg, mild pain) or newly introduced for the eMTS (eg, ‘right lower abdominal pain’ in the presentational flow chart for abdominal pain). See figure 1 for an example. In 26 flow charts, urgency category four and/or five were directly linked to an assignment to the GPC or ED. In all flow charts with possible assignment to the GP, the nurses had the option to choose for the discriminator ‘GP Risk’ linked to an assignment to the ED. ‘GP Risk’ was defined as an unspecified risk to assign the patient to the GPC according to the opinion of the triaging nurse, or because of age less than 3 months. Nurses were instructed to use this parameter when they thought there was a good reason to keep the patient at the ED, but this reason was not specified in the eMTS.

Figure 1

Example of the Manchester Triage System presentational flow chart for abdominal pain in children with the studied extension.29 ED, emergency department; GP, emergency department; GPC, general practice cooperative; PV, per vaginam. Image based on Emergency Triage: Mackway-Jones K, Marsden J, Windle J, Manchester Triage Group. Emergency triage. Third edition. Ed, 2014, ISBN 9781118299067 p. 66 with kind permission.

The eMTS was built into the ED’s computer decision support system (E.care ED 4.1). After choosing a flow chart and a discriminator, this system showed the advice ‘Assign to GPC’ when applicable. Afterwards, the nurse had to register the final assignment he/she had given (intervention weekends) or considered the most appropriate (control weekends). Nurses working at the study ED with a degree in emergency medicine and at least 1 year of experience at the study hospital were allowed to triage, they all participated in the study. These nurses followed a training on using the eMTS, patient communication skills focusing on refusal of the assignment and the study protocol.

Participants

Patients with an available national insurance number triaged by a nurse at the ED were included. Patients already admitted to the study hospital, those arriving by an ambulance staffed with a doctor or nurse, and patients referred to the ED by a doctor or nurse were excluded from the TRIAGE trial. Patients with a missing final assignment were also excluded from the current study. For the cost analysis, patients who were hospitalised were excluded because their invoice does not reflect their true costs at the ED, as it was not possible to differentiate their outpatient costs at the ED and the cost of their subsequent hospitalisation.

Setting

ED of a general hospital (AZ Monica, Deurne) with an annual census of 37 000 patients, and the adjacent GPC with an annual census of 10 000 patients (open during weekends only). Before 2019, ED triage in Belgium only involved urgency classification for prioritisation; patients were only assigned to primary care in experimental settings.18–20

Data collection

The following study tool parameters were collected: MTS flow chart (52 flow charts); MTS urgency level (1–5), chosen MTS discriminator (200 terms), computer-generated assignment (ED or GPC) and nurse-selected assignment (ED or GPC). Due to limitations of the used software at the ED, the collection of the discriminators was incomplete. When the nurse chose an original MTS discriminator (eg, ‘mild pain’), it was registered correctly. But when the nurse wanted to choose a specific newly introduced eMTS discriminator (eg, ‘right abdominal pain’) or the non-specific newly introduced discriminator ‘GP Risk’, they had to click the option ‘Not assigned to the GPC because of a specific newly introduced eMTS discriminator or GP Risk’. Afterward they were asked to write down a free-text motivation why they had chosen this term, but that field was not obligatory. In the results section, the specific newly invented eMTS discriminators and ‘GP Risk’ are reported together as additional discriminators in the eMTS linked to the ED. For the cost analysis, the total invoice cost and costs per cost category (physician fees, medical imaging, technical procedures, non-refundable items and medication) were studied. The number of hospitalisations was calculated using these invoices.

After reading all free-text motivations concerning overruling the protocol and the choice of additional discriminators in the eMTS linked to the ED, authors VV and SM independently divided them into categories. Afterwards, authors VV and SM reached consensus on the categories to use and the classification of all free texts.

Data were collected using iCARE data (Improving Care And Research Electronic Data Trust Antwerp), a Belgian database for out-of-hours care.21 22 The data for the costs were obtained directly from the financial department of the studied ED and GPC and were linked to the medical data based on admission time, sex and ZIP-code.

Outcomes

The first outcome of this study was the proportion of patients with study tool parameters leading to an assignment to the ED in the intervention and control weekends. The following parameters were studied: reason for encounter registered in the MTS as a presentational flow chart (9 flow charts always leading to ED vs 43 with possible assignment to the GPC), urgency category (1–3 always lead to ED, 4 or 5 might lead to the GPC) and discriminator (1197 discriminator-flow chart pairs linked to the ED, 175 to the GPC). The motivations for choosing additional discriminators in the eMTS linked to the ED and for overruling were studied using free-text fields. This outcome is the proportion of patients for which the nurse overruled a computer-generated assignment to the GPC in favour of the ED. A subanalysis for the most frequently used presentational flow chart (limb problems) was made using the same outcomes.

As a proxy for the need of specialist/secondary care in the group of patients assigned to the GPC, the number of hospitalisations and costs were studied. The total costs were studied as well as the different cost categories. The invoices were divided into the share of the invoice paid for by the patient, and the share of the cost refunded by the national health insurance (government costs).

Analysis

A χ2 test was used to compare proportions between intervention and control weekends. A two-sample Wilcoxon rank-sum (Mann-Whitney) test was used to assess the difference in the mean costs and distribution of costs for the intervention and control group. Density histograms were created to illustrate the skewness of the cost data. Data were analysed using JMP Pro V.15.0 (SAS institute) and Stata V.17.0 (StataCorp). The significance level for all tests was set at 0.05.

Patient and public involvement statement

A lay person volunteering at the ED of a hospital not participating in this study was involved in the study design, she gave advice about the study protocol and tool. An advisory board with stakeholders from EDs, GPCs and universities gave advice about the study design and discussed the interim analysis and gave feedback on the results.

Results

Population

In the TRIAGE trial, 9964 patients were assessed for eligibility, of which 1806 patients were excluded mainly because they were already triaged by a healthcare professional prior to arrival in the ED.15 For this paper, patients with a missing final assignment (N=338/8158, 4%) were also excluded leading to a population of 8038 (6294 intervention and 1744 control). The baseline characteristics of the patients in the intervention and in the controlgroup were similar (see online supplemental table 1).

Influence of the intervention on the selection of study tool parameters

See figure 2 for a summary. Nurses equally chose flow charts always leading to the ED for the control and the intervention group (2,5% vs 2.1%, p=0.41). However, an additional 3.2% of the patients in the intervention group were classified in a higher urgency category (mandatory assignment to the ED) as compared with the control group (p=0.02). For an additional 17.3% of the patients within urgency categories four and five, a discriminator leading to the ED was selected in the intervention group (p<0.01). Among those discriminators leading to the ED, newly introduced eMTS discriminators were selected more frequently than those already available in the original MTS (85.0% vs 15.0%) regardless of the allocation to intervention or control weekends (p=0.78).

Figure 2

Influence of the intervention on the selection of study tool parameters. ED, emergency department; GPC, general practice cooperative; ICT, Information and Communications Technology.

The motivation for choosing an additional discriminator in the eMTS linked to the ED was registered for 347/2207 (15.7%, see table 1) in the intervention and 39/534 (7.3%) in the control group. The most frequent motivation was the presence of a predefined eMTS discriminator, mainly the need for imaging according to the nurse. Out of the 202 patients with imaging as a motivation, 160 (79%) had a radiology cost on their invoice.

Table 1

Motivations for choosing an additional discriminator in the eMTS linked tothe ED (N=347) in the intervention group

Influence of the intervention on overruling the protocol

The nurses overruled the automated eMTS assignment of the software to the GPC in favour of the ED for 3.9% of the patients, there was no significant difference between intervention and control weekends (p=0.91). No motivations for overruling were given during control weekends while this information was available for 95/147 (64.6%) during intervention weekends. See table 2 for these motivations. The vast majority of these motivations was either organisational or medical.

Table 2

Motivations for over ruling the automated eMTS assignment (N=95)

Influence of the intervention on the use of the flow chart for limb problems

There was no significant difference in the proportion of patients triaged with the flow chart limb problems (513/1740 or 29.5% vs 1691/6238 or 27.1%, p=0.05) or the selection of the three highest urgency categories (76/513 or 14.8% vs 252/1439 or 14.9%, p=0.96). There was, however, a marked difference in the selection of discriminators within urgency categories four and five. The additional discriminators in the eMTS linked to the ED (nurse perceived need for medical imaging, baby below 3 months or the unspecified GP Risk) were selected for an additional 15.2% of the patients (273/437 or 62.5% vs 1049/1351 or 77.7%, p<0.01) in the intervention group. The only original discriminator in the MTS linked to the ED for this presentational flow chart was ‘deformity’ defined in the MTS as abnormal angulation or rotation. ‘Deformity’ was selected equally in both groups (51/437 or 11.7% vs 180/1351 or 13.3%, p=0.37).

Influence of the intervention on hospitalisations in the study Hospital for patients assigned to the GPC

During intervention weekends,13 (1.0%) out of the 838 patients assigned to the GPC were hospitalised. During control weekends this proportion was significantly higher: 20/431 (1.6%, p<0.01).

Influence of the intervention on the costs for patients and government for patients assigned to the GPC

For this analysis, 1146 patients were included. Four patients were excluded because they had an unlikely low invoice, 33 were excluded because they were hospitalised, and the invoice was missing for 86 patients. Online supplemental appendix figures 1 and 2 illustrate the skewed distribution of the costs and indicates that the costs are more concentrated around zero during intervention weekends. The mean total cost during intervention weekends was €56 while this was €79 during control weekends (p<0.01). The Wilcoxon rank-sum test confirms that the distribution of costs among patients assigned to the GPC differs between intervention and control weekends. This difference was mainly driven by decreased use of medical imaging and technical procedures (p<0.01) during intervention weekends. See table 3 for details.

Table 3

Costs for patients assigned to the GPC

Discussion

In this secondary analysis of the TRIAGE trial, we analysed how emergency nurses used the triage tool differently for the intervention and the control group, which led to a remarkable difference in the proportion of patients assigned to primary care (13% in the intervention vs 25% in the control group). We found that nurses did not choose more flow charts leading to the ED as compared with all other flow charts, but they classified more patients as urgent (plus 3.1%) and they selected more discriminators linked to the ED (plus 16.2%). The motivation for choosing additional study tool discriminators leading to the ED were mostly the nurse-perceived need of medical imaging or medical reasons not specified in the protocol. The nurses did not overrule the protocol more often in the intervention group but when they did, they registered the reason for over-ruling more rigorously. These reasons mainly concerned organisational issues. The number of hospitalisations and the costs for patients assigned to the GPC were lower in the intervention weekends.

The strength of this study lies in the unique opportunity found in a cluster RCT making it possible to study the differences between an intervention and control group. The high number of included patients in a real-world setting is another strength. The study design has important limitations as well: the trial was not designed to study the outcomes of this paper. On the contrary, the marked difference in triage between the intervention and control group was an unexpected finding. Due to ICT limitations, it was not always possible to know which additional discriminators of the eMTS were chosen (either newly invented eMTS discriminators or the nonspecific GP risk). The collected free-text values were only available for a minority of the patients where an additional eMTS discriminator was chosen. Also, it was registered in only 64.6% of the intervention patients where the protocol was overruled even though this was explicitly requested. This might cause an important bias. Finally, due to an ICT error during the first two intervention weekend, the chosen discriminator remains unknown for 5% of the patients within urgency categories 4 and 5.

A previous study revealed that paper case scenarios generally receive lower triage scores (lower urgency) than live cases.6 As our intervention group is comparable to the real world, it seems logical that nurses triaged even more risk averse in the intervention group as compared with the control group (similar to live cases). Our study does not allow to definitely answer the question why the nurses classified more patients as urgent by selecting other discriminators during intervention weekends. It is likely that they did this either consciously or subconsciously because of a desired ED outcome and thus triaged more risk averse. Previous qualitative research concerning the same trial reveals some reasons why nurses are sometimes reluctant to assign a patient to the GPC (Meysman J, Morreel S, Lefevere E, et al. Triaging and Referring In Adjacent General and Emergency departments (the TRIAGE-trial): A process evaluation of medical staff experiences in a nurse-led triage system. Submitted for publication). Some nurses reported that they found it very difficult in the beginning to refer patients to the GPC. They gained trust in the system after reassurance that low-risk patients they diverted were not sent back. One nurse compared the control group to ‘playing poker for chips’ and the intervention group to ‘playing poker for money’. Nurses also indicated that it is time-consuming and complex to divert patients to the GPC even though the ED was only 50 m away. Another possibility is the influence of the study hospital. When diverting patients to the GPC, the ED loses income. On the other hand, the intervention led to a relatively small but significant decrease in the workload which at times was very welcome. Another reason why nurses triaged differently in intervention weekends might be found in the theory of planned behaviour: whether or not a subjective norm (some patients should go to the GPC) leads to an intention (I want to divert these patients) and a behaviour (I have given the advice to go to the GPC) depends on the attitude of the nurse (how important is it to me to give this advice?) and the amount of perceived behavioural control (what are the consequences of this diversion?).23 This attitude and the perceived behavioural control were probably different in the intervention group, because in the control group the nurse’s decision had no influence on the patient’s treatment.

The most frequently chosen eMTS discriminator was the perceived need for medical imaging. Even when not specified in the eMTS, nurses often used the non-specific discriminator GP risk with medical imaging as motivation. This need is subjective but, in most cases (79%), the physician did order medical imaging. The studied tool can be improved by providing more specific guidelines on the need for medical imaging especially for the presentational flow chart for limb problems, for example, by implementing the Ottawa knee, midfoot and ankle rules which have been validated to rule out fractures at the ED24 25 and can be used by ED nurses.26

The admission rate of patients with an assignment to the GPC was low but significantly higher during control weekends. More medical imaging and technical procedures were used for patients with an assignment to the GPC during control weekends. The question whether or not this reflects a true medical need or a difference in clinical behaviour between ED and GPC physicians cannot be answered by the current study leaving the question open whether or not the more risk averse triage during intervention weekends leads to over triage. It also remains unknown whether this more risk averse triage during intervention weekends increased patients’ safety.

The control group of the TRIAGE trial can be regarded as simulated circumstances comparable to triage research using paper-based scenarios or retrospective observational studies. This study proves that the theoretical results of such research should be interpreted cautiously as the nurses are likely to triage more risk averse when it really comes down to diverting patients and not only writing down a theoretical assignment. This difference between simulated and real-world experiments is new for the research about triage but is well known in other fields of research. For example, laboratory experiments may both understate and exaggerate the importance of social preferences.27 In lottery experiments, researchers found that research subjects in laboratory circumstances typically underestimate the extent to which they will avoid risk in the real world.28

Conclusion

Triage nurses classify more patients as urgent and select more discriminators linked to the ED when they actually have to divert patients to primary care as compared with a theoretical assignment to primary care. Researchers should be aware that outcomes from a simulated triage setting should not be extrapolated to the real world.

Data availability statement

Data are available on reasonable request. Given the privacy policy of the iCAREdata database, the authors are not allowed to share the used database. Sharing this database would potentially harm the privacy of the included patients. The Belgian Data Protection Authority does not allow the authors to share the raw data. The authors are, however, able to deliver a selection of variables and the outputs of their statistical software upon reasonable request. Most of the studied data is available to researchers worldwide after following the application procedures of iCAREdata (see www.icaredata.eu or contact icare@uantwerpen.be).

Ethics statements

Patient consent for publication

Ethics approval

Ethical clearance waiving individual informed consent was obtained from the ethics committee of Antwerp University Hospital (reference 18/37/410) and from the local ethics committee of AZ Monica Deurne (reference 367). For this secondary analysis, an amendment was approved (reference 3374).

Acknowledgments

The authors would like to thank Eva Lefevere and all participating healthcare providers at the emergency department of AZ Monica Deurne, the staff of the general practice cooperative Antwerp East, and specially Sander Naeyaert, Ragna Verlent, Joo-Ree Melis, Marc Timmermans, Arnoud Bonemeyer, Edwin Vanbeveren, Lotte Fivez and Guido Michielsen for their extensive help and advice.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • Contributors All authors were equally involved in the design of the study and funding acquisition except IH who was not yet involved. SM and HP were responsible for the data collection. During the formal analysis, SM took the lead with support of VV. IH analysed the hospital invoices in close collaboration with SM. Authors HP, JM, DDG and KGM supervised the analysis. SM made a first draft of this paper after which it was thoroughly revised by all authors. The first author is the guarantor. All authors had full access to the study data. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

  • Funding The study was financed with a grant from the Research Foundation—Flanders (see www.fwo.be). All authors benefited from this grant as promotor or were employed as researchers with this grant. Payment was made to their institutions (Grant number: T000718N).

  • Disclaimer The funder had no role in study design, data collection, analysis, interpretation, decision to publish or preparation of the manuscript.

  • Competing interests All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: author SM is a general practitioner working in the surroundings of the study site, and as such, he performed on call shifts at the study site and treated some of the studied patients. Due to the anonymity of the studied data, the exact number of study patients seen by him cannot be determined, but it was definitely below ten. He is also a board member of the studied general practice cooperative receiving meeting fees. HP is coordinator of the iCAREdata project (database used for this study). She had an appointment at the University of Antwerp for this project until September 2020. The authors declare no other relationships or activities that could appear to have influenced the submitted work.

  • Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.