Article Text
PDF
Abstract
Objective To develop a prioritisation framework to support priority setting for elective surgeries after COVID-19 based on the impact on patient well-being and cost.
Design We developed decision analytical models to estimate the consequences of delayed elective surgical procedures (eg, total hip replacement, bariatric surgery or septoplasty).
Setting The framework was applied to a large hospital in the Netherlands.
Outcome measures Quality measures impacts on quality of life and costs were taken into account and combined to calculate net monetary losses per week delay, which quantifies the total loss for society expressed in monetary terms. Net monetary losses were weighted by operating times.
Results We studied 13 common elective procedures from four specialties. Highest loss in quality of life due to delayed surgery was found for total hip replacement (utility loss of 0.27, ie, 99 days lost in perfect health); the lowest for arthroscopic partial meniscectomy (utility loss of 0.05, ie, 18 days lost in perfect health). Costs of surgical delay per patient were highest for bariatric surgery (€31/pp per week) and lowest for arthroscopic partial meniscectomy (−€2/pp per week). Weighted by operating room (OR) time bariatric surgery provides most value (€1.19/pp per OR minute) and arthroscopic partial meniscectomy provides the least value (€0.34/pp per OR minute). In a large hospital the net monetary loss due to prolonged waiting times was €700 840 after the first COVID-19 wave, an increase of 506% compared with the year before.
Conclusions This surgical prioritisation framework can be tailored to specific centres and countries to support priority setting for delayed elective operations during and after the COVID-19 pandemic, both in and between surgical disciplines. In the long-term, the framework can contribute to the efficient distribution of OR time and will therefore add to the discussion on appropriate use of healthcare budgets. The online framework can be accessed via: https://stanwijn.shinyapps.io/priORitize/.
- COVID-19
- health economics
- organisation of health services
- surgery
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information. All data are available in the online tool.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
Decision analytical modelling appears to be an efficient tool to compare the impact of delays in elective surgery due to the COVID-19 pandemic on patient quality of life and healthcare costs.
The framework is available via an online tool that can easily be adapted according to local settings (eg, regarding operation times, currencies) and new available evidence.
Since high-quality data regarding the consequences of the delay of surgery on deterioration are lacking, this could not be included in our model.
We used average data from literature rather than patient-level data, which could impact the applicability of our results to the individual patient.
Introduction
The extent to which the ongoing COVID-19 pandemic is disrupting global health, social welfare and the economy is unparalleled in modern history.1 Due to this pandemic, hospitals continue to have to drastically reduce elective surgeries. Current estimates suggest that worldwide more than 2 million operations per week have been cancelled during the first wave of this pandemic, and most of them comprise elective surgeries.2 3 In the UK alone a reduction of 2.3 million performed elective surgeries is seen from March 2020 until February 2022, increasing the number of patients waiting for elective surgery to 6 million.4 5 It was also estimated that if countries increase their usual surgical volume by 20% after the pandemic, it would take about 45 weeks to clear the backlog due to the disruption.2 With the current second wave and third waves, the number of delayed elective operations will only increase further. This not only affects the surgical disciplines, but also other related disciplines like gastroenterology, internal medicine, oncology, cardiology, neurology and general practitioners as they see the rise in time for referral of patients for surgery.
The word ‘elective’ implies that the indication for surgery is not ‘acute and life-saving’ like in the case of life-threatening emergency. In most hospitals ‘acute’ cases have been scheduled without restriction during the pandemic. For the elective cases, it is likely that their suboptimal health status persisted during the extended waiting period, but there might also be patients where the delay to surgery may lead to deterioration of the disease and limit treatment options. However, it is also conceivable that their symptoms decrease during their extended waiting period, without affecting their personal life much, ultimately leading to cancelling of surgery.
The COVID-19 pandemic provides a unique opportunity to study these effects of delay of elective surgeries. Moreover, the discussion on healthcare interventions where scientific support for their added value is limited or even lacking, has also been reopened. That is, healthcare professionals also have a responsibility to contribute to the affordability and accessibility of the healthcare system as a whole.6 7 If healthcare can be made more sensible and qualitatively better, we can deliver more healthcare for less money. This requires not only a new mindset, but also reliable models and data to quantify the consequences of delay or even cancellation of surgery on patients and society. Models like ours will help to build an evidence-based framework which can be used to support priority setting for elective surgeries and subsequent optimisation of operating room (OR) capacity. Therefore, our aim was to develop a framework to support priority setting for elective surgeries based on the impact on patient well-being and cost.
Methods
Decision analytical models were developed to estimate the consequences of delaying multiple elective surgical procedures, taking into account health impact and cost. The final framework, including all individual models, provides information on relevant factors that should be taken into account when prioritising operations, that is, loss in health-related quality of life (HRQoL), healthcare costs due to delay and the duration of the operation. We used data from available literature to calculate expected health loss and costs due to delay of surgery. The decision analytical models were developed in accordance with the modelling good research practices and described according to the Consolidated Health Economic Evaluation Reporting Standards guidelines (online supplemental S1).8 Ethical approval was not required for this study as all data was obtained via literature searches.
Supplemental material
Selected elective procedures
All procedures that could wait for at least 2 months after diagnosis according to the urgency categories of the Dutch Healthcare Authority during the early phase of the COVID-19 pandemic were considered for our model.9 Clinical experts from multiple specialties were consulted to determine useful examples of clinical dilemmas in times of COVID-19. We decided to compare procedures within and between specialties to demonstrate how to prioritise within and between disciplines. The following elective procedures were included: general and gastrointestinal surgical procedures (inguinal hernia repair, laparoscopic sleeve gastrectomy (LSG), Roux-en-Y laparoscopic gastric bypass (LRYGB), partial colectomy for non-acute Crohn’s disease and ulcerative colitis, sphincteroplasty), urological/gynaecological procedures (male sling procedure, tension-free vaginal tape procedure), orthopaedic procedures (total hip replacement, total knee replacement, total shoulder replacement, arthroscopic partial meniscectomy) and one otorhinolaryngological procedure (septoplasty) (table 1).
The 13 surgical procedures that are currently included in the framework
Data acquisition and validation
For each case, input regarding cost and quality of life was derived from recent literature via semi-systematic literature searches in PubMed. Keywords included the disease of interest, the type of surgery, length of stay, costs (resource use/healthcare utilisation) and quality of life. The search strategy can be found in online supplemental S2. Ideally, a randomised controlled trial (RCT) or meta-analysis of multiple RCT’s comparing surgery to watchful waiting or non-surgical care, was used to inform the model. If these were not available, alternative high-quality data sources, such as observational cohort studies or equivalent alternatives, were retrieved. If studies comparing surgery to watchful waiting or non-surgical care were not available, studies that measured outcomes before and after surgery were used to estimate the effect of postponing surgery. The quality of the studies was assessed using a checklist in which we scored the validity of the operation times and utilities used. In addition, for each case study a clinical expert was consulted to ensure that all important aspects of the patient population, disease and surgery were captured. To validate our data, we also compared them with data from the Dutch National Institute for Public Health and the Environment (RIVM) that studied the consequences of delayed surgery for the Dutch government.10
Quality of life
Effectiveness was measured in terms of utility values, which reflects HRQoL on a 0–1 scale, with 0 representing death and 1 representing full health. Utility values were derived from the EQ-5D (EuroQol 5-dimensional) questionnaire.11 12 When available, differences in utilities between surgery and watchful waiting were extracted at 6–12 months intervals to calculate the gain in utility which can be reached by performing the surgery. If a watchful waiting cohort was not available, the baseline utility (measured before surgery) of surgical patients was taken to calculate the gain in utility which can be reached by performing the surgery.
We assumed that gain in utility that can be reached by performing a surgery represents the loss in utility in case surgery is delayed. That is, if an operation that increases a patients utility with 0.2 is postponed for 1 year, we assume a total loss of utility of 0.2 over that year. Figure 1A shows how we calculated the impact of delayed surgery on the loss of quality of life (in utility values).
Overview of the methods used. (A) Loss in quality of life (QoL) due to delayed or postponed surgery was calculated by extracting the QoL before surgery from the QoL after surgery and multiplying this with the duration of the delay (1 week in our analyses). (B) The costs (in €) associated with waiting for surgery were calculated by extracting the average costs after surgery from the average costs before surgery and multiplying this with the duration of the delay (1 week in our analyses). (C) The net monetary loss (NML) (monetary measure to calculate the total societal loss of delaying surgery) was calculated by multiplying the loss in QoL by the willingness to pay (€20 000) and adding the extra costs associated with waiting for surgery. The willingness to pay represents the amount of money society is willing to pay for 1 year in full health. (D) Surgery associated NML per week divided by operating time. Relevant when trying to optimise the operating schedule. During a 2-hour surgery, also two operations of 1 hour could be performed. In other words, the 2-hour surgery needs to be associated with twice as much NML as the 1 hour surgeries to be as worthwhile to perform.
Costs
The extra healthcare expenditure due to waiting for surgery was determined by calculating the difference in healthcare expenditure before and after surgery (figure 1B). Only costs from a healthcare perspective were included, for example, extra visits to the hospital, general practitioner, physiotherapist. Costs of surgery itself were not included, as we assumed that all patients would receive surgery. To enable a comparison between procedures we extracted the resource use (eg, number of extra hospital visits) rather than the actual cost from literature. The resource use was multiplied by standard unit prices for each procedure, ensuring a similar calculation of costs across operations. When available, unit prices were derived from the Dutch guideline for costing research.13 Otherwise, unit prices were obtained from hospital fees. We excluded medication costs since this was often not reported or the reporting lacked detailed information necessary for our model. Costs were calculated in Euros (€) and based on the 2019 price level.
Operating time
Operating time for all surgical procedures was extracted from literature to weigh the impact of surgery against the time needed to perform the surgery (see analysis). Operating time was considered to be the total time the patient was in the operating theatre, including anaesthesia and surgery (skin-to-skin) time, and was extracted from literature. To validate these data, we compared them with the empirical data provided by two hospitals. Furthermore, in the online available framework, the operating time can be adjusted to match operating times for a specific setting.
Analysis
We calculated the loss of quality of life (in utilities) and extra costs per week delay of surgery based on the obtained utility values and costs. Subsequently, we calculated the net monetary loss, which is defined as the total loss of waiting another week for surgery, expressed in monetary terms. The net monetary loss is calculated by multiplying the loss in quality of life due to waiting 1 week for surgery by a threshold value, and subsequently the extra costs of waiting another week for surgery are added. We used a threshold value of €20 000 per year of full health, as recommended for conditions with a relatively low burden of disease by the Dutch guidelines for cost-effectiveness (figure 1C).13 As an example, let’s assume a surgical procedure leads to a 0.2 gain in utility and a decrease in the patient’s healthcare expenses of €50 per week. Delaying this procedure for 1 week results in a net monetary loss of 
. The procedure with the highest net monetary loss therewith provides the most ‘value’ when prioritised. Subsequently, we also took into account the operating time since more patients can benefit from procedures with short operating times given a fixed OR capacity. For example, when a surgical procedure ‘X’ can be performed twice in the time frame of procedure ‘Y’, procedure ‘Y’ has to result in twice as much value to have a similar value in the same OR time (figure 1D). Therefore, the net monetary loss per week was weighted for the operating time, resulting in the net monetary loss per week per OR minute.
Last, we calculated the impact of postponing these elective surgeries during one of the COVID-19 waves, assuming 30% delay in these 13 elective surgeries over a 3-month period as compared with the year before. We calculated the impact of postponing elective surgeries in total costs and total net monetary loss.
Empirical example
To illustrate how our framework works and can be used in clinical practice, we applied it on real world data from a large regional hospital in The Netherlands. Data used from this hospital comprise the actual numbers of patients waiting for each of the 13 included procedures on 30 June in 2020, 2019 and 2018 and the average waiting time for each procedure in these years. Based on these data we calculated the total net monetary loss after the first COVID-19 wave (30 June 2020) as compared with 2019 and 2018. This was done by multiplying the number of patients that are waiting by the average waiting time and the net monetary loss for that procedure.
Interactive surgical prioritisation framework
The decision analytical models for the elective surgical procedures were wrapped in an interactive web-based framework developed to further stimulate engagement and discussion between the relevant stakeholders, that is, surgical disciplines, anaesthesiology, other referring medical disciplines and decision-makers. By default, the interactive framework shows the results presented in this paper, but users of the framework can alter some of the parameters (eg, the operation time) or select procedures relevant to their departments or strategy. In this way the framework can be used on different strategic levels, ie, department level or hospital level (for decisions across departments). Furthermore, cost prices of the different resources and currencies can be altered to make the framework applicable for other countries. The framework was built using R (V.4.0.2, The R Foundation for Statistical Computing) with shiny (V.1.5.0) and shinydashboard (V.0.7.1) packages.14 15 The interactive framework is available via https://stanwijn.shinyapps.io/priORitize/.
Patient and public involvement
There was no patient or public involvement in the study.
Results
Quality of life
The highest loss in quality of life due to delayed surgery was found for total hip replacement (utility of 0.27, ie, 99 days lost in perfect health when waiting for a year), followed by total shoulder and knee replacement (utilities of 0.22 and 0.22, respectively, ie, 80 days lost in perfect health when waiting for a year) (table 2). The lowest loss in quality of life was found for arthroscopic partial meniscectomy (utility of 0.05, ie, 18 days lost in perfect health when waiting for a year), see also figure 2A. For sphincteroplasty, the male sling procedure and the tension-free vaginal tape procedure utility values were not available in literature.
Overview of the data in the surgical prioritisation framework
Overview of results. (A) Loss in quality of life (QoL) due to delayed or postponed surgery expressed as a utility score. A utility reflects QoL on a 0–1 scale, with 0 representing death and 1 representing full health. (B) Extra healthcare expenditure due to waiting for surgery. (C) The net monetary loss combines QoL and costs due to waiting for surgery, it is therefore the total loss of waiting another week for surgery, expressed in monetary terms. (D) Surgery associated net monetary loss per week divided by operating time (ie, it reflects the total cost per week per operating room minute).
Cost
Delay of LSG or LRYGB bariatric surgery resulted in the highest costs (€31 pp per week), followed by partial colectomy for non-acute Crohn’s disease (€17 pp per week), and ulcerative colitis (€16 pp per week). Delay of arthroscopic partial meniscectomy was found to result in the lowest costs (−€2 pp per week), see also figure 2B. For sphincteroplasty, total shoulder replacement, male sling procedure and tension-free vaginal tape procedure, no literature was available to determine the extra resource use due to waiting for surgery.
Net monetary loss
Combining the loss in quality of life and extra costs resulted in a calculation of the net monetary loss per week. Total hip replacement was found to result in the highest loss per week of delay (€114 per week per procedure), followed by total knee replacement (€95 per week per procedure), and partial colectomy for non-acute Crohn’s disease (€94 per week per procedure). Arthroscopic partial meniscectomy appears to result in the lowest loss per week (€18 per week per procedure), see also figure 2C. It should be noted that the net monetary loss could only be calculated for procedures for which we could find information regarding the quality of life and costs in the literature.
Net monetary loss weighted by operating time
When the OR time per procedure is taken into account, the net monetary loss per week per OR minute shows that LSG provides the most value (€1.2 per week per OR minute), followed by LRYGB (€0.9 per week per OR minute), and total knee replacement (€0.9 per week per OR minute). Arthroscopic partial meniscectomy seems to provide the least value (€0.3 per week per OR minute), see also figure 2D.
Impact of surgical delay
For the 13 included elective surgeries, we conservatively estimated that 30% was delayed for 3 months as compared with the total number that was performed in the year before COVID-19 (ie, 27 500 elective surgeries for the 13 included procedures). In total, a 30% delay in the 13 selected elective surgical procedures resulted in €0.3 million extra costs for the healthcare system and a total impact on both cost and quality of life of €3.6 million. The impact of a 10% to 50% surgical delay for each procedure can be found in online supplemental S3.
Empirical example
The impact of the COVID-19 crisis was clearly visible in the surgical waiting times of a large regional hospital in The Netherlands (online supplemental S4). After the first COVID-19 wave (ie, on 30 June 2020), 624 patients were waiting for 1 of the 13 included procedures, while on the same day in 2019 and 2018, 291 and 257 patients were waiting, respectively. As a consequence, the total net monetary loss after the first wave was €873 504, while the total net monetary losses were €172 664 and €124 224 in 2019 and 2018, respectively. Compared with 30 June in 2019 and 2018, the total net monetary after the first wave increased with 506% (€700 840) and with 703% (€749 280), respectively.
Discussion
We developed a surgical prioritisation framework that provides information that can be used to set priorities in elective surgeries. For example, the highest loss in quality of life due to delayed surgery was found for total hip replacement (utility of 0.27, ie, 99 days lost in perfect health when waiting for a year); the lowest for arthroscopic partial meniscectomy (utility of 0.05, ie, 18 days lost in perfect health when waiting for a year). Costs of surgical delay were highest for LSG and LRYGB (€31/pp per week) and lowest for arthroscopic partial meniscectomy (−€2/pp per week). Total hip replacement and total knee replacement resulted in the highest net monetary losses per week (€114 and €95, respectively), while septoplasty and arthroscopic partial meniscectomy had a net monetary loss per week of €22 and €18, respectively. In case we assumed that 30% of the 13 included procedures were delayed over a 3-month period as compared with the total numbers of procedures performed a year earlier, the delay resulted in €0.3 million extra costs for the Dutch healthcare system and a total impact on both cost and quality of life (net monetary loss) of €3.6 million. Data from a large regional hospital in The Netherlands show that more than twice as many patients were waiting for 1 of the 13 modelled operative procedures after the first COVID-19 wave as compared with 2019 (624 vs 291 patients, respectively). Consequently, the extra net monetary loss caused by these waiting times was €700 840, which is an increase of 506% compared with 2019.
Several other models to study the effect of delayed surgery and to inform surgical recovery plans have been developed. Degeling et al, for example,16 developed a model to estimate the impact of delayed cancer diagnosis and treatment on survival outcomes and healthcare costs based on a shift in the cancer disease stage at treatment initiation. They showed that a conservative 3-month delay in cancer diagnosis and treatment due to the COVID-19 pandemic, results in an excess health cost of $12 million in Australia over 5 years for the 2020 diagnosed patients for four cancers. Gravesteijn et al17 also developed a model that supports prioritisation of care. They, however, focused on semi-elective surgeries, including cardiothoracic, oncological and transplantation surgery, whereas we focused on elective surgeries. Needless to say that patients with cancer and patients awaiting organ transplantation have a completely different profile as far as prognosis of their disease on the one hand and burden of awaiting treatment, on the other, is concerned. Furthermore, they used the global burden of disease by the WHO to estimate the quality-adjusted life year for one-third of the surgeries, and for the other two-thirds they used estimates by an expert panel. Our quality of life data are based on literature data from comparative studies using validated quality of life measures, which is in agreement with the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) recommendation to use health-utility data collected from patients.18 Wang et al19 developed a framework to model surgical backlog recovery. In contrast to our model, they did not include quality of life assessment to guide prioritisation of care. They used available resources and bed capacity that are adjustable to other contexts, aiding region-specific decision-making. The COVIDSurg Collaborative2 and Brandman et al20 separately developed models to predict the size of the backlog and time needed to restore this backlog. Although these models are different from the present framework, combining both perspectives might result in a comprehensive context specific policy to clear the surgical backlog.
The major strength of our approach is that the data used from literature are completely transparent in the online framework, and that it can easily be adapted according to local settings (eg, regarding operation times) and new available evidence. Our model was built with high-quality QoL and cost data that were derived from RCTs or comparative studies. We had the unique opportunity to cross validate our results to a national study by the Dutch Institute for Public Health and the Environment and empirical data from a large local hospital.10 The concordance appeared to be very high with more than 75% overlap. Furthermore, by calculating the net monetary losses per week weighed by OR minute we were able to make a comparison between procedures and surgical disciplines based on the surgery time. This provides new insights on how to allocate valuable surgery time when comparing these operations, to maximise value.
Some potential limitations should also be discussed. First, we used average data from literature rather than patient-level data, which could impact the applicability of our results to the individual patient. However, our goal was to develop a practical framework to support priority setting able to generalise and compare on department and surgery level instead. The model is therefore useful in general during the COVID-19 pandemic as well as for policymaking in striving for quality-driven healthcare.
Second, we did not yet take into account other related factors such as intensive care unit or personnel capacity, the number of beds available, the risk of exposing patients to perioperative COVID-19 infection or psychological consequences. This was outside the scope of this paper but can be added in a future model, and of course these factors can be taken into account in the individual trade-off.
Third, impact of waiting on medication costs (eg, pain medication that patients need while waiting for surgery), could not be taken into account because they were either not reported in literature or not described in enough detail to be suitable for inclusion in the model. In order to be able to take medication costs into account, better reporting of cost data, that is, categorisation of cost data, in clinical studies is needed. Furthermore, it could be expected that some patients need extra home care or had a prolonged stay in a nursing home because they are waiting for surgery. These costs were not reported in literature and were therefore not included in the model. Consequently, the total cost presented are an underestimation of the real cost.
Fourth, besides impact on quality of life, delayed surgery may have a variety of consequences regarding the deterioration of the disease ranging from ‘no harm’ (varices, inguinal hernia) to ‘complications’ (easy or difficult to treat, medically or surgically: Crohn’s disease, ulcerative colitis) to disease-related death. Currently, high-quality data regarding the consequences of the delay of surgery on deterioration are lacking and could therefore not be included in our model. Because of the elective nature of the included procedures, we believe that deterioration with high impact (like disease-related death) will be limited. However, if this COVID-19 pandemic will prove that delaying the included procedures do lead to high impact deterioration it is necessary to include the consequences of delaying surgery, the model can be adapted accordingly.
Fifth, so far, we only modelled 13 elective surgical procedures whereas there are many more. Since we developed an online framework, new data can easily be added to inform future decision-making, for example, additional high quality data comparing surgery to watchful waiting or non-surgical care. Others can also provide us with relevant information on other procedures, which we will check on consistency and validity, before adding them to the online framework.
Sixth, for some procedures no data on quality of life or costs were available in literature. The fact that no relevant data were retrieved from literature for sphincteroplasty, male sling procedure and tension-free vaginal tape procedure illustrates how difficult it is, and will be, to calculate the added value of these procedures. It renders this type of surgery ‘vulnerable’ in strategic discussions, but also stimulates groups active in this complex field to come up with data in support of continuing this type of operations. We are, however, aware of research projects that will follow the patients currently ‘waiting’ due to the backlog of the pandemic.21 Hopefully, these projects will provide us with more accurate data, which are critical to obtain reliable estimates.
The ongoing pandemic is having a collateral damage effect on healthcare and the delivery of surgical care to millions of patients worldwide. This is an effect that most certainly will persist for years to come. It is to be expected that cancer and other acute surgery, if cancelled during the pandemic, will be prioritised in most settings, whereas the impact on other elective surgeries for benign conditions will be cumulative, adding to the existing waiting times. Governments and other policymakers will be requested to fund substantial increases in surgical volume to clear backlogs, and this framework may help them to prioritise on evidence regarding QoL and cost savings rather than on a mixture of numbers and expert opinion.
When addressing the backlog of postponed elective surgeries, it is tempting to start with surgeries that cause a high net monetary loss when delayed on the one hand and have large volumes on the other. However, as we look at bariatric surgery, we see a discrepancy between population impact and net monetary loss per OR minute. Although bariatric surgery has one of the highest net monetary losses of all procedures described in this paper, it has the lowest impact on population level due to small volumes. Also, when resuming total knee and total hip replacement first, huge numbers of patients need to be operated taking a lot of valuable OR time, while for bariatric surgery only a small number of patients needs to be operated. Therefore, we would like to emphasise that objective measures are indispensable for fair and justifiable prioritisation of surgeries, and that these choices are preferably based on the net monetary loss per OR minute. Such medical care prioritisation data may add to future discussions on ‘appropriate use’ of healthcare budgets.
In conclusion, our online framework can be used in deciding how to address the postponed elective surgeries after the COVID-19 pandemic. Furthermore, the model will also be useful during possible future repeated waves of COVID-19 or in the long-term as it provides relevant information regarding an efficient distribution of OR time.
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information. All data are available in the online tool.
Ethics statements
Patient consent for publication
Acknowledgments
In adherence to the ICMJE guidelines, we would also like to acknowledge the contributions of our collaborators: CM, MJ, IS and NvdB helped with the modelling. Model input and feedback was received from CT, MS, CR, SvdG, TvT, RP, JR, and NS. All collaborators have read and approved the final version of the manuscript before submission.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @MaroeskaRovers, @Stan_Wijn
Collaborators EBS-group collaborators: Charlotte Michels, Milica Jevdjevic, Ilse Spenkelink, Niels van den Berkmortel, Casper Tax, Michiel Sedelaar, Camiel Rosman, Sebastiaan van der Goes, Tony van Tienen, Rudolph Poolman, Jelle Ruurda, Niek Stadhouders, Paul van Leest.
Contributors MR, SW, JG, HG, MS and TG have contributed to the conception and design of the study. Modelling was performed by SW, SM, RV, MS and TG. Model input and feedback was received from RvdP, BB, HG and our collaborators. MR, SW, MS and TG drafted the manuscript. The online framework was developed by SW. MR is guarantor for this paper. All authors have made contributions to the drafting and revising of the article. All authors have read, reviewed and approved the final version of the manuscript before submission.
Funding The study was funded by NWO (Dutch Research Council), VICI-project Rovers, project number 91818617. The funder had no role in the study design; in the collection, analysis and interpretation of data; in the writing of the report; and in the decision to submit the article for publication. All authors were independent from the funder. All authors had full access to all of the data and can take responsibility for the integrity of the data and the accuracy of the data analysis. There was no commercial involvement in the study.
Competing interests MMR had financial support by means of a VICI grant from NWO (Dutch Research Council) for the submitted work, the other authors had no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous 3 years; no other relationships or activities that could appear to have influenced the submitted work.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.