Introduction Higher mortality for patients admitted to intensive care units (ICUs) during the weekends has been occasionally reported with conflicting results that could be related to organisational factors. We investigated the effects of ICU organisational and staffing patterns on the potential association between weekend admission and outcomes in critically ill patients.
Methods We included 59 614 patients admitted to 78 ICUs participating during 2013. We defined ‘weekend admission’ as any ICU admission from Friday 19:00 until Monday 07:00. We assessed the association between weekend admission with hospital mortality using a mixed logistic regression model controlling for both patient-level (illness severity, age, comorbidities, performance status and admission type) and ICU-level (decrease in nurse/bed ratio on weekend, full-time intensivist coverage, use of checklists on weekends and number of institutional protocols) confounders. We performed secondary analyses in the subgroup of scheduled surgical admissions.
Results A total of 41 894 patients (70.3%) were admitted on weekdays and 17 720 patients (29.7%) on weekends. In univariable analysis, weekend admitted patients had higher ICU (10.9% vs 9.0%, P<0.001) and hospital (16.5% vs 13.5%, P<0.001) mortality. After adjusting for confounders, weekend admission was not associated with higher hospital mortality (OR 1.05, 95% CI 0.99 to 1.12, P=0.095). However, a ‘weekend effect’ was still observed in scheduled surgical admissions, as well as in ICUs not using checklists during the weekends. For unscheduled admissions, no ‘weekend effect’ was observed regardless of ICU’s characteristics. For scheduled surgical admissions, a ‘weekend effect’ was present only in ICUs with a low number of implemented protocols and those with a reduction in the nurse/bed ratio and not applying checklists during weekends.
Conclusions ICU organisational factors, such as decreased nurse-to-patient ratio, absence of checklists and fewer standardised protocols, may explain, in part, increases in mortality in patients admitted to the ICU mortality on weekends.
- weekend effect
- intensive care unit
- organizational factors
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
Large prospective cohort including several Brazilian intensive care units with different organisational features.
Analyses corrected for both relevant patient-level and centre-level confounders.
Large number of models built may have increased type 1 error.
Variable selection for main multivariable may have been arbitrary.
Higher mortality for patients admitted during the weekends has been repeatedly reported in studies carried out in both wards1 2 and intensive care units (ICUs).3 4 The so-called weekend effect is often ascribed to the imbalance of organisational and staffing features available on weekdays, which either do not occur or are suboptimal during the weekends.5 This notion has driven several healthcare policies, especially in England where junior doctors’ contracts were changed on the assumption that increased weekend working would mitigate the weekend effect.6 This was followed by a great deal of discussion on the veracity of a weekend effect due to uncertainties on the reliability of disease coding on admission during weekends6 7 and biases related to different profiles of illness severity and comorbidities in weekend-admitted patients.8 9
If the weekend effect is a real phenomenon among ICU admissions, it could be mediated by several organisational factors including staffing features such as full-time intensivist coverage and nurse staff levels10 and use of other ancillary tools designed to sustain the continuity of care such as checklists and number of institutional protocols available.11 However, the association between these organisational characteristics and the weekend effect in ICUs were not thoroughly evaluated. We have used a large national prospectively collected database to examine the hypothesis that the weekend effect would only be manifest in settings in which suboptimal organisation, staffing or clinical practices could impact on continuity of care.
This was a post hoc analysis of the ORganizational CHaractEeriSTics in cRitcal cAre (ORCHESTRA) study, a multicentre retrospective cohort study of critical care organisation and outcomes in critically ill patients admitted in 78 Brazilian ICUs during 2013 from 1 January to 31 December.11 All patients in the database were selected. From the initial database of 59 693 patients, we excluded 79 patients with missing admission date/hour, leaving 59 614 patients for analysis. In case of readmissions during the study period, we considered only the first ICU admission.
We defined weekend admission as any admission to the ICU occurring between Friday 19:00 and Monday 07:00.
We considered several organisational factors in the analysis including the use of checklists (structured evaluations using a digital or printed instrument with multiple components focused on prevention of common ICU complications and adherence to best practices) during weekends, the implementation of protocols in the ICU (among a predefined set of 10 protocols aiming at the adherence to best practices and prevention of acquired complications for frequent conditions in the ICU; see online supplementary file for details), the presence of full-time intensivist coverage 24/7 in the ICU, presence of a low nurse/bed ratio and ICU type. Full-time intensivist coverage was defined as the presence of a board-certified intensivist in the ICU 24 hours a day, 7 days a week. A low nurse/bed ratio was defined as a mean nurse/bed <0.20 inside the ICU (ie, the mean nurse/bed ratio considering all shifts in the ICU was lower or equal to 0.20). This 0.20 cut-off was established after inspecting the univariable association between nurse/bed ratio and inhospital mortality in a generalised additive model (online supplementary figure 1; see electronic supplementary material for details). For checklists, dummy coding was used to represent units with checklists 7 days a week versus those without weekend checklists (ie, units that had checklists on weekdays only or did not apply checklists at all).
Supplementary file 1
We estimated organ failure and the severity of illness using the Sequential Organ Failure Assessment (SOFA) score12 and the Simplified Acute Physiology Score (SAPS) 3.13 The Charlson Comorbidity Index (CCI) was calculated as previously reported.14 Baseline performance status (assessed 1 week before hospital admission) was defined according to the impairment degree as absent/minor, moderate and severe corresponding to performance classes of Eastern Cooperative Oncology Group of 0–1, 2 or 3–4, respectively, as previously described.15 Hospital length of stay (LOS) before ICU admission was collected and stratified in tertiles for the analysis.
No outcome was missing. Baseline performance status was missing in 3476 patients. Multiple imputation using random forest models was used to impute missing values for this variable, as previously described.15 There were no other missing values in the variables included in the analysis.
We used a mixed multivariable logistic regression to assess the independent association between each predictor and hospital mortality at the patient level. The ICU where the patient was admitted was added as a random effect (random intercept) in the model, while all the other variables were added as fixed effect. Continuous variables were scaled and centred before entering the mixed model. For patient-level variables, we initially considered those associated with hospital mortality with a P value lower than 0.1 for the multivariable model; from the initial set of variables, the decision to add then to the model was based on clinical relevance. In case of collinearity, the decision to keep the variable to enter in the multivariable analysis was also based on clinical grounds. For example, both SAPS 3 score and SOFA scores are highly correlated, but the SAPS 3 also includes information on age, reason for admission, organ failures, comorbidities and hospital LOS before ICU admission. In this sense, we have chosen to use SOFA score for the main multivariable models and added age, reason for admission, comorbidities (evaluated using the CCI) and LOS before ICU admission as individual covariates. The following patient-level variables were included in the main model: age, SOFA score, CCI, baseline performance status impairment, admission type (medical, elective surgery and non-elective surgery) and hospital LOS before ICU admission stratified in tertiles. For centre-level variables, we have chosen to add variables known from previous studies to be associated with worse ICU.10 11 These variables included use of checklists during the weekends, presence of full-time intensivist coverage, low baseline nurse/bed ratio and ICU type (medical/surgical or other). Weekend admission was forced in the model. No stepwise selection was performed. Type of funding (public vs private) was not added to the model due to large collinearity with most organisational features.
We defined that sensitivity analysis for the following subgroups would be performed: admission type, quartiles of baseline SAPS 3 score, patients admitted due to sepsis, patients admitted on mechanical ventilation, ICUs with or without decrease in nurse/bed ratio during weekends (regardless of baseline values), ICUs stratified per presence of full-time intensivist coverage, ICUs with or without checklist during the weekends and according to number of ancillary protocols in the ICU (above or below the median values of protocols per ICU). We repeated the subgroup analyses of organisational features (decrease in nurse staff, absence of full-time intensivist coverage, absence of checklist and number of ancillary protocols) after stratifying our sample according to scheduled surgical versus non-scheduled ICU admissions (both emergency surgery and clinical).16 For the number of ancillary protocols, we split the samples in patients admitted to units with less or at least eight protocols, since this value split the number of included ICUs in half. Scheduled surgery admissions are defined as any admission after a surgical procedure that was scheduled at least 24 hours before its start and for which an ICU bed was requested before the procedure started.
Propensity score analysis
We performed a propensity score 1:1 analysis pairing patients per their predicted probability of weekend admission based solely on the patients’ factors. The probability of weekend admission was obtained by creating a logistic regression with the following variable included as predictors, based on clinical grounds: age, SOFA score, SAPS 3 score, CCI, degree of performance status impairment, hospital LOS before ICU admission and admission type. Patients were matched using the nearest neighbour method considering the logit as the distance method. Maximum distance allowed was 0.40. At each matching, the unit with the closest logit still unmatched was used. After checking the balance of the propensity-matched groups, we compared hospital mortality for propensity-matched patients using χ2 test.
Effects of organisational changes during weekends in weekend versus weekday mortality
One additional approach to scrutinise the impact of organisational features in weekend admission mortality was applied by performing a linear regression at the ICU level considering the change in mortality (weekend minus weekday mortality for the ICU) as dependent variable and both SAPS 3 and organisational features as predictors. Due to the relatively small number of ICUs, we have chosen to minimise the number of predictors; in this sense, only SAPS 3 score was used to control for patient-centred variables since it encompasses information of reason for admission, age, LOS before admission and comorbidities. The organisational features selected were the same used for splitting subgroups in multivariable mixed model. After linear regression, we assessed the relative importance of each predictor using the method suggested by Lindeman, Merenda and Gold.
Temporal association between organisational features and outcome in patients admitted during weekends
The time-dependent effects of organisational factors in patients admitted to the ICU on weekends was assessed through an approach based on multiple sequential random forest models at the patient level from the day of admission (day 0) until 7 days after with hospital mortality as outcome of interest. The first regression included all patients admitted on weekends; the second regression included all patients except those who died in the first day, the third regression included all patients except those who died in the first 2 days, and so on until the first 7 days. Again, for simplicity, we used SAPS 3 score as a single predictor for patient-level confounders. Organisational features (presence/absence of weekend checklists, low nurse/bed ratio, number of protocols and presence/absence of full-time intensivist coverage) were added to the random forest model. Relative importance was assessed at each model based on mean decrease of Gini value and displayed as percentage over time.
All analyses were performed in R Project version 3.4.217 with packages ggplot2, lme4, dplyr, tidyr, relimpo and gridextra. A P value below 0.05 was considered significant. This report follows the Strengthening the Reporting of Observational Studies in Epidemiology guideline18 (shown in online supplementary file).
The need for informed consent was waived.
Patients and ICU features
Out of 59 693 patients admitted to the participating ICUs during the study period, 59 614 (99.9%) had available ICU admission date and hours and were selected for analysis. Main ICU and hospital characteristics are depicted in online supplementary table 1 (see electronic supplementary material). Most ICUs were private (online supplementary table 1). A total of 41 894 patients (70.3%) were admitted on weekdays and 17 720 patients (29.7%) were admitted on weekends. The number of admissions decreased during weekends especially due to a decrease in elective surgeries and also due to a decrease in medical admissions (figure 1). A comparison between patients admitted at weekend versus weekdays is shown in table 1 and in online supplementary figure 2. Patients admitted on weekends were more severely ill (with higher SOFA and SAPS 3 scores) and more frequently admitted due to medical reasons than patients admitted during weekdays (table 1). When compared with patients admitted during weekdays, a slightly lower percentage of weekend admissions occurred in ICUs with a low nurse/bed ratio (49.8% vs 48.6%, respectively; P=0.007) and/or fewer than eight protocols (46.9% vs 45.4%, respectively; P=0.001), as shown in online supplementary figure 2.
The overall ICU and hospital mortality rates were 9.5% and 14.4%, respectively. While hospital mortality for medical admissions per week day of ICU admission slightly fluctuated over the week, higher mortality was seen for elective surgical patients admitted on Sunday (figure 2). For non-elective surgeries, large fluctuations were observed with a peak mortality for those admitted on Tuesday (figure 2). In univariable analysis, hospital mortality was significantly higher for patients admitted at weekends (16.2 vs 13.6%, OR 1.22, 95% CI 1.17 to 1.29; P<0.001).
Mixed model logistic regression
After adjusting for relevant patient-level and ICU-level characteristics in the multivariable analysis, weekend admission was no longer associated with increased hospital mortality (OR 1.05, 95% CI 0.99 to 1.11; P=0.095; full model reported in online supplementary table 2); mixed effects shown in online supplementary figure 3); centre effect was markedly present). We present the results of the effect of weekend admission on hospital mortality in the several prespecified subgroups in figure 3 and in online supplementary tables 3–19. A ‘weekend effect’ was apparent for elective surgical admissions (OR 1.34; 95% CI 1.10 to 1.64; P=0.004; figure 3 and online supplementary table 4) and in patients admitted to ICUs without checklists during the weekends (OR 1.08; 95% CI 1.00 to 1.17; P=0.045; figure 3 and online supplementary table 17). We did not find a statistically significant association between other organisational characteristics (nurse/bed ratio, presence of full-time intensivist coverage or the number of protocols) and increased mortality in patients admitted during the weekends.
Scheduled surgical versus unscheduled ICU admissions
We repeated the analyses separately for scheduled surgical admissions and unscheduled admissions (42 977 and 16 637 admissions, respectively). Results are shown in figure 4 and in online supplementary tables 20–37 of the electronic supplementary material. A ‘weekend effect’ was observed in scheduled surgical admissions. However, weekend effect was only present on scheduled surgical admissions when there was a decrease in weekend nurse/bed ratio (OR 1.40; 95% CI 1.09 to 1.79, P=0.008), no weekend checklists (OR 1.41; 95% CI 1.09 to 1.83, P=0.009) or a lower number of protocols (OR 1.42; 95% CI 1.06 to 1.89, P=0.018) (figure 4). There was no ‘weekend effect’ in unscheduled admissions.
Propensity score results
Out of the study population, 35 440 patients (17 720 weekend admissions and 17 720 weekdays admissions) were matched in the propensity score analysis. All patients admitted on weekend could be paired. The distribution of variables in matched patients stratified per weekend effect is shown in online supplementary figure 4. Mortality was 15.5% for patients admitted on weekdays and 16.1% for those admitted during weekends (P=0.112).
Change in weekend–weekday mortality regression
After linear regression using the change of weekend to weekday mortality as dependent variable, only SAPS 3 score and low number of protocols (fewer than eight protocols) were associated with increases in mortality in weekends at the ICU level (both P<0.001). There was a trend for presence of weekend checklists and lower changes in weekend–weekday mortality (P=0.08). The model R2 was 74.03%, with SAPS 3 being the most important predictor (66% of all variance), followed by low number of protocols (29%) and presence/absence of weekend checklists (3%, which was not statistically significant).
Temporal association of organisational features and outcome
The relative contribution of SAPS 3 to hospital mortality decreased in the first 7 days after ICU admission, while the relative contribution of number of protocols increased. The other organisational features remained reasonably stable. The relative contribution of other organisational features remained stable (figure 5).
In the present study, we provide evidence that, after proper consideration patients’ characteristics along with those related to ICU organisation, the association between admissions during the weekend and patients’ outcomes is weak and probably not relevant in the scenario of several Brazilian ICUs. However, a residual ‘weekend effect’ might be present for scheduled surgical patients admitted to ICUs with a disrupted continuity of care and adherence to best practices. The weekend effect was therefore seen primarily in hospitals with suboptimal care processes. Nonetheless, we were not able to definitely determine whether this adverse impact is explained by processes of care themselves or a reflection of other hospital characteristics (eg, quality of surgeons, general ward care and available resources).
Although previous studies suggested an increase in mortality in patients admitted during the weekend,4 the ‘weekend effect’ concept has been recently challenged.6 Criticism of reports suggesting the presence of weekend effect identified three major pitfalls in analysis: (1) studies were performed using administrative databases and might suffer with inconsistent coding, (2) comorbidities may have not been properly considered and (3) Illness severity might not be adequately accounted for. Black6 cited three studies in which, after correction of these issues, the ‘weekend effect’ was no longer significant.
In the present analysis, we intended to overcome the limitations summarised by Black.6 We based our analysis on a quality improvement administrative database that considers International Classification of Diseases (ICD) coding for diagnosis and that also includes major predefined medical diagnosis.11 This database includes robust prospective clinical data collected at the bedside and not only administrative and/or procedures notes. We also considered the presence of comorbidities and baseline health status by adjusting for CCI and to performance status impairment.14 15 Finally, severity of illness admission was assessed using well validated and widely accepted scores of severity of illness and organ failure.12 19 20 Our work does not corroborate with the hypothesis that the ‘weekend effect’ is a universal feature in the ICUs. This was highlighted by both the main mixed regression model and by the propensity score analysis performed. In fact, when only unscheduled admissions were considered, no evidence of weekend effect was found regardless of the presence/absence of organisational features in the ICU; this is in accordance with recent reports the of absence of the weekend on unplanned ICU admissions.21 We could find evidence of a weekend effect only in secondary subgroup analyses, suggesting that it might be restricted to scenarios when there is a break in continuity of care during weekends, specifically the absence of patient-centred checklists11 16 or for schedule admissions.
The case for worse outcomes in elective surgical admissions on weekends is complex, with both ICU and hospital features being able to explain, at least partly, this association.22 23 Differences in surgical care on weekends may play an important role. For example, early recognition and treatment of surgical complications may be delayed on weekends and even the adopted surgical procedure may be different for similar scenarios during weekends.24 25 In this way, checklists may aid at prompt recognition of complications (both clinical26 and surgical)27 and improve adherence to daily goals of care.28 Higher nurse staffing may also aid at reducing postoperative complications.10 In our analysis, there was no statistically significant weekend effect in scheduled surgical admissions in units that did not have a decrease in nurse/bed ratio on weekends or that applied checklists on weekends, thereby corroborating to this concept.
Interestingly, several factors potentially related to worse care on weekends did not reach statistical significance in the whole studied population. Despite the rationale for increased mortality when there is a decrease in nurse/bed ratio during weekend29 or when there is an absence of full-time intensivist coverage,30 we did not find evidence of weekend effect in both scenarios when all admissions types were considered. It is conceivable that well-structured ICUs with weekend checklists and protocols would be less susceptible to the variation in care driven by the decrease in staff during weekends, thereby mitigating the effect of the latter on mortality. This was only apparent for scheduled surgical admissions in our analysis. However, when we evaluated changes in mortality in weekends versus weekday at the ICU level using a linear model, a low number of protocols was the stronger organisational predictor of increase in mortality on weekends. It should be acknowledged that multiple interactions between organisational features could be expected and that some of the absence of statistically significant findings could be due to Simpson’s paradox.
For patients admitted on weekends, we found that the LOS ICU could modulate the association between organisational features and outcome, as shown in figure 5. As expected, illness severity as measured by SAPS 3 score was the most important factor associated with mortality, but its relative contribution decreased during the first 7 days, while the relative contribution of number of protocols increased. The other organisational features remained reasonably stable. This suggests that the main determinant of mortality is a global individual marker of illness severity (as expected) and that the more a patient stays in the ICU, the higher the association between number of protocols and outcome. Interestingly, protocols were also the only variable associated with reduced change in mortality during weekends in the ICU-level linear model. Future studies that assess organisational features and outcome in the ICU should also consider that organisational features may be more important for patients with prolonged ICU LOS.
Our analysis has several constraints. First, the large number of secondary analyses and models built could increase type 1 error due to overtesting. Additionally, the large sample size may also facilitate observing statistical significant results that may not be clinically relevant. Therefore, our results and especially our subgroup analyses should be considered exploratory and interpreted in the context of multiple models where some groups had small number of patients. It is important to highlight, however, that trends were constant during all analysis performed. We nevertheless advice caution when interpreting our results, especially the effects size reported. Second, although the large number of ICUs involved, our results refer to a selected sample of units from a single country and caution is needed when generalising to other settings. Third, most patients were mostly admitted to private units and had a medium–low illness severity, thereby resulting in a low average mortality. However, no weekend effect was demonstrated in the subgroup analysis per the severity of illness and in septic or mechanically ventilated patients. Fourth, the low nurse/bed ratio reported is common in Brazil and are in accordance with local regulations that stipulate up to 10 beds per nurse in the ICUs but limits its generalisation to different European and North American settings. Fifth, we were unable to assess interactions between the several organisational features. Frequently several organisational features occur together and may have synergistic (or even antagonist) effects that should be properly explored in the future. Sixth, we only assessed ICU organisational factors; structure, organisational and staffing patterns at the emergency departments and wards may also play an important role in determining hospital outcome. Finally, we did not assess other relevant outcomes such as readmissions and longer follow-up mortality, and multiple imputation was used to correct for an important patient variable (performance status).
After considering an appropriate disease coding and accounting for comorbidities, illness severity and ICU organisation characteristics, we did not observe worse outcomes in patients admitted to ICUs during weekends. Nonetheless, a ‘weekend effect’ might still occur in scheduled surgical admissions, especially in ICUs with a decrease in nurse/bed ratio and absence of checklists during the weekends as well as in ICUs with a low number of protocols. The ICU LOS potentially modulates the association between organisational features and outcomes in critically ill patients.
Contributors Manuscript conception: FGZ, TCL, TDC, FAB, JIFS and MS. Statistical analyses: FGZ, TCL, FAB, TDC, JIFS and MS. Data collection: MF, HSF, AMJ, JCRV, ACPC, MFK, BFM, FC, JMV, WNV, RC, MMG, MOM and EBC. Wrote the manuscript: FGZ, TCL, TDC, FAB, JIFS and MS. Approved the final version: all authors.
Funding This study was supported by the National Council for Scientific and Technological Development (CNPq) (Grant No 304240/2014-1), Carlos Chagas Filho Foundation for Research Support of the State of Rio de Janeiro (FAPERJ) and by departmental funds from the D’Or Institute for Research and Education.
Competing interests MS and JIFS are founders and equity shareholders of Epimed Solutions, which commercialises the Epimed Monitor System, a cloud-based software for ICU management and benchmarking. The other authors declare that they have no conflict of interest.
Ethics approval The Brazilian National Ethics Committee (CAAE: 19687113.8.1001.5249) approved the study.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Data are not publicly available due to local ethical restrictions.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.