Article Text
PDF
Abstract
Objective SARS-CoV-2 has caused a pandemic claiming more than 4 million lives worldwide. Overwhelming COVID-19 respiratory failure placed tremendous demands on healthcare systems increasing the death toll. Cost-effective prognostic tools to characterise the likelihood of patients with COVID-19 to progress to severe hypoxemic respiratory failure are still needed.
Design We conducted a retrospective cohort study to develop a model using demographic and clinical data collected in the first 12 hours of admission to explore associations with severe hypoxemic respiratory failure in unvaccinated and hospitalised patients with COVID-19.
Setting University-based healthcare system including six hospitals located in the Galveston, Brazoria and Harris counties of Texas.
Participants Adult patients diagnosed with COVID-19 and admitted to one of six hospitals between 19 March and 30 June 2020.
Primary outcome The primary outcome was defined as reaching a WHO ordinal scale between 6 and 9 at any time during admission, which corresponded to severe hypoxemic respiratory failure requiring high-flow oxygen supplementation or mechanical ventilation.
Results We included 329 participants in the model cohort and 62 (18.8%) met the primary outcome. Our multivariable regression model found that lactate dehydrogenase (OR 2.36), Quick Sequential Organ Failure Assessment score (OR 2.26) and neutrophil to lymphocyte ratio (OR 1.15) were significant predictors of severe disease. The final model showed an area under the curve of 0.84. The sensitivity analysis and point of influence analysis did not reveal inconsistencies.
Conclusions Our study suggests that a combination of accessible demographic and clinical information collected on admission may predict the progression to severe COVID-19 among adult patients with mild and moderate disease. This model requires external validation prior to its use.
- COVID-19
- RESPIRATORY MEDICINE (see Thoracic Medicine)
- PREVENTIVE MEDICINE
- Public health
- Health informatics
Data availability statement
Data are available in a public, open access repository. SAS code and some de-identified data will be published at Dryad, an online repository. Prisoners will not be included. Any data published there is contingent on additional approval from TDCJ's IRB. Reasonable requests for additional data will be considered.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- COVID-19
- RESPIRATORY MEDICINE (see Thoracic Medicine)
- PREVENTIVE MEDICINE
- Public health
- Health informatics
Strengths and limitations of this study
Our study used objective and measurable demographic and clinical information regularly available in healthcare settings even among patients unable to communicate.
Our primary outcome corresponds to the WHO ordinal score which would allow us to compare our results with other studies and in other settings.
Our model could serve as an effective point of service tool during early admission to assist in clinical management and allocation of resources to unvaccinated patients.
Our study is a retrospective study of unvaccinated patients with COVID-19, and validation of our prediction model in the rest of our study population is still needed.
In addition, testing our model in a more recent cohort after emergence of new SARS-CoV-2 variants will be needed to assess its robustness.
Introduction
SARS-CoV-2 is a novel coronavirus discovered in 2019. It is the aetiological agent for the largest viral pandemic of the 21st century thus far, followed by H1N1 influenza A that emerged in 2009–2010.1 2 During the early pandemic, a case series from the Wuhan province showed that 81% of COVID-19 cases were mild, 14% progressed to severe disease, and 5% developed critical illness defined as respiratory failure, septic shock and/or multiple organ dysfunction.3 COVID-19-associated hospitalisation caused an overwhelming demand on the healthcare system of the USA. Shortage in ventilators and personal protection equipment posed significant challenges in management of cases in US hospitals early in the pandemic.4 During 2020, the Center of Disease Control and Prevention estimated 375 000 deaths attributed to COVID-19.5
Unfortunately, there has been a global lag in uptake of COVID-19 vaccines due to hesitancy and logistics. Unvaccinated individuals with COVID-19 remain up to 25 times more likely to be hospitalised or dead compared with vaccinated individuals. Rising hospitalisation and deaths among unvaccinated individuals are driving a new pandemic surge posing again a significant burden to the health system.6 7
Studies evaluating the risk of progression among infected subjects admitted to the hospital have used different outcomes to define severe diseases. These included criteria from the American Thoracic Society on severity of community-acquired pneumonia,8 the Berlin definition of acute respiratory distress syndrome,9 death or mechanical ventilation,10 11 and/or the WHO ordinal scale.12 13 The WHO ordinal scale to classify the clinical status of patients with COVID-19 has been widely adopted in randomised control trials such as ACTT-1 and ACTT-2.14–16 Harmonisation of the measures used to evaluate the severity of COVID-19 across different studies could ease the comparison of study results and application of evidence-based interventions. However, the heterogeneity in the definitions of severe illness and the limited availability of certain laboratory tests, especially in low-resource settings, have decreased the generalisability of these tools. Laboratory tests such as serum interleukin 6 or procalcitonin may not be accessible in small medical centres. Similarly, information on comorbidities may not be available in patients unable to provide a history. Simple, objective and accessible tools to predict progression to severe COVID-19 are still needed to guide clinicians during case surges and dwindling of resources.
To address this need, we conducted a retrospective cohort study in the University of Texas Medical Branch (UTMB) Health System to develop an exploratory model for severe hypoxemic respiratory failure in unvaccinated hospitalised patients with COVID-19.
Methods
Study design
We hypothesised that a combination of objective clinical and laboratory findings on admission can identify subjects with higher risk of progression to severe respiratory failure due to COVID-19 in our hospitals. To test this hypothesis, we performed a retrospective, multisite cohort study on adult patients admitted for COVID-19 to the UTMB Health System.
UTMB Health System includes six hospitals located in the Galveston, Brazoria and Harris counties of Texas. These hospitals are distributed across over 50 miles, though populations served are similar overall. We retrieved the medical record numbers of all patients ≥18 years old admitted to hospitals in any of the four campuses with a positive SARS-CoV-2 molecular test between 19 March and 30 June 2020. We used the WHO ordinal scale of disease severity for COVID-19 to define our outcomes.17 This is an 11-category ordinal scale ranging from a value of 0 for patients with no virological evidence of infection to 10 for patients who died due to COVID-19. Our primary outcome was defined as reaching a WHO ordinal scale between 6 and 9 during admission corresponding to severe respiratory failure requiring oxygen supplementation using high-flow nasal cannula (HFNC) or mechanical ventilation. Patients initially presenting with a WHO ordinal scale <6 who were discharged at the time of review of their medical record were enrolled. Patients who met ordinal scale 6–9 on the first vital signs obtained on admission were excluded. The maximum ordinal scale score met during admission was considered the subjects’ ordinal score.
Patient and public involvement
None.
Data collection
We collected data directly from the Epic (Verona, Wisconsin, USA) electronic medical records. The data were transcribed into a questionnaire created in the REDCap (Nashville, Tennessee, USA) data capture system. Data coders were trained using a dummy dataset before using medical records. All coders were trained until they could obtain 100% accuracy on dummy datasets before proceeding to data collection. Eighty-nine randomly selected charts underwent evaluation by the principal investigators and the data extraction personnel. These evaluations were compared to calculate the inter-rater reliability using kappa statistics. When the personnel had a kappa <0.8, they were retrained, and discrepancies were discussed with the principal investigators. Evaluations were repeated until a kappa >0.8 was reached. The data extraction personnel collected data on demographics, clinical history and course, vital signs, peak oxygen requirement and laboratory results (online supplemental eTable A). The maximum oxygen requirement at any given day after admission was used as the peak oxygen requirement, and the subject was deemed to have met the primary endpoint if the peak oxygen requirement was HFNC or more intensive. Data on admission laboratory results include absolute neutrophil and lymphocyte counts; serum lactate dehydrogenase (LDH), D-dimer, C reactive protein (CRP), procalcitonin and troponin I. Only the first laboratory tests obtained within 12 hours of admission were recorded. If these labs were not obtained during this window, they were registered as missing.
Supplemental material
Statistical analysis
The REDCap dataset was downloaded to a database on SAS (V.9.4) and R (V.4.0.2). Frequencies, means with SDs (±SD), and medians with IQRs were calculated to describe the distribution of the variables. Pearson correlations were performed for bivariate analysis; variance inflation factors (VIFs) were calculated for each variable prior to initial modelling, with a factor ≥5 being considered possibly collinear.18 Mean imputation was used to replace body mass index (BMI) when the value was missing. Multiple imputation was not performed because data were not missing at random relative to the primary outcome. To evaluate the effect of using the sample mean to replace BMI missing values, the analysis was also performed excluding those cases.
A multivariable logistic regression analysis was conducted to model variables with the highest predictive value for severe COVID-19. Variables for the model were selected based on review of the literature on COVID-19 and clinical relevance (eg, objectivity and availability). The composite variables age/BMI and age/sex were created and used in the models because of existing evidence of interactions between those variables individually. Stratified analysis was performed for interaction terms ultimately included in the model. Stepwise Akaike Information Criteria (AIC) reduction was used to optimise the model, reducing residual deviance while prioritising model simplicity. Cook’s Distance method was used for assessing points of influence, where a Cook’s D≥1 was considered highly influential. The Hosmer-Lemeshow goodness-of-fit (GOF) test statistic was used to evaluate the match between the predicted and observed risk of progression to WHO ordinal score 6–9. A receiver operating characteristic curve (ROC) and area under the curve (AUC) analysis was performed to assess overall model fidelity.
Several sensitivity analyses were performed. One was to assess the biasing effect of mean imputation on BMI. We excluded all cases where BMI was missing for this analysis. The second was to assess whether DNI status meaningfully affected results. Because some patients may have initiated DNI during the course of admission (which we could not verify), we excluded all patients who had a DNI in place by the time of discharge or death. The final was to examine whether a model that ordinally discriminated between HFNC and intubation was more robust (where <HFNC=3, HFNC=5 and intubation=6; values according to WHO scale). Because comparative AUC analysis between cumulative logit and binomial logistic regression is not possible, qualitative differences parameter selection, magnitude and per cent concordance were assessed. Proportional odds assumptions were tested using Χ2 methods.
Results
We identified 930 subjects admitted to the UTMB Health System with a positive SARS-CoV-2 test during 19 March–30 June 2020. The first 352 consecutive charts were reviewed to develop the predictive models. The demographics and clinical characteristics of the cohort prior to exclusion are shown in online supplemental figure 1. Twenty-three subjects were excluded because they met WHO ordinal scale scores between 6 and 9 on the first vital signs measured or because most values of interest were missing (figure 1).
Flow chart for cohort selection.
Three hundred twenty-nine subjects were included in the final cohort and 62 (18.8%) met the primary endpoint. The Texas Department of Criminal Justice population accounted for 27.6% of cohort population but there were no significant differences in the proportion of subjects meeting the primary endpoint according to inmate status (p=0.459, data not shown). Subjects reaching the ordinal scale 6–9 were significantly older than subjects who did not (table 1). More male subjects met the primary endpoint but the difference between groups was not statistically significant (table 1). The top three comorbidities for subjects with ordinal scales <6 were cardiovascular, 51.5%; diabetes mellitus, 32.4%; and pulmonary, 21.0%. For subjects with ordinal scale 6–9, the top three conditions were cardiovascular, 67.3%; diabetes mellitus, 38.5%; and liver disease, 15.4% (table 1). Twenty-four subjects died during admission (7.3%) and 20 of them met criteria for ordinal score 6–9. Seven per cent (6.7%) of subjects with ordinal score <6 and 9.7% of subjects with ordinal scale 6–9 had DNI order. Comfort care was implemented in 1.9% of those with ordinal score <6 and in 16.1% of those with ordinal score 6–9. The characteristics of subjects with ordinal scales 6–9 across all campuses are shown in table 1. Fourteen (4.3%) were missing BMI values.
Demographic and clinical characteristics of subjects included in the cohort
The variables included in the initial regression model were admission date, age/sex, age/BMI, oxygen saturation, neutrophil to lymphocyte ratio (NLR), procalcitonin, D-dimer, LDH, CRP, troponin I, duration of symptoms prior to admission and Quick Sequential Organ Failure Assessment (qSOFA) score (online supplemental figure 2).
The initial model was highly significant and identified several candidate predictive variables; none of these variables had a VIF >3. The candidate clinical and laboratory variables age, BMI, oxygen saturation, qSOFA score, CRP, procalcitonin, NLR, D-dimer and LDH were incorporated into prognostic model. All subjects with elevated troponin I levels (6 of 6) were intubated which precluded the evaluation of this variable as a predictor in the analysis. After stepwise AIC reduction, the final model included seven variables: oxygen saturation, NLR, D-dimer, qSOFA, LDH, AgexBMI and admission date (table 2). Because admission date did not meaningfully improve model performance (AUC=0.84 without vs 0.85 with) and complicates clinical use, we exclude that factor here.
Final multivariable regression model after stepwise AIC reduction
Stratified analysis (BMI ≥30 vs BMI <30) was limited due to small sample of patients with lower BMIs. The higher BMI strata, however, demonstrated similar effect sizes for age and BMI; qSOFA score was not retained (online supplemental figure 3). Running the model excluding subjects with missing BMI values (n=34) did not affect the general significance or GOF of the model (table 3).
Multiple regression model excluding cases with missing BMI
ROC/AUC analysis of the final model indicated an AUC of 0.84, indicating high efficacy of the overall model in predicting severe disease (figure 2).
ROC curve, final model. ROC, receiver operating characteristic.
ROC/AUC analysis of the BMI sensitivity analysis was nearly identical to the final model (AUC=0.83; figure 3).
ROC curve, BMI sensitivity analysis. BMI, body mass index; ROC, receiver operating characteristic.
Modelling with a cohort excluding patients with active DNI likewise did not result in meaningful change (AUC=0.82; online supplemental figure 4).
The ordinal logit model (table 4) reached similar parameter selection, but had slightly lower concordance (80%) than the binary model (85%) and did not include admission CRP or biological sex. The proportional odds assumption was not obviously violated by Χ2 testing (p=0.11).
Ordinal logistic regression analysis
Discussion
Our study evaluated demographic and clinical variables measured within 12 hours of hospital admission in patients with COVID-19 as potential predictors of progression to severe respiratory failure. We used the WHO ordinal scale 6–9 to define patients with severe respiratory failure requiring significant life-sustaining therapies. Our analysis demonstrated that a combination of routine accessible laboratory tests, vital signs and demographic variables may yield a useful clinical tool to assess risk of COVID-19 severe respiratory failure among patients admitted to our health system. This model used objective and measurable information available in acute care settings even if the patient is unable to communicate. The model could serve as an effective point of service tool during early admission to assist in clinical management and allocation of resources to unvaccinated persons. The significant factors, like admission LDH, were robust to numerous sensitivity analyses.
LDH was highly predictive of severe disease in our model and was robust to sensitivity analysis. Subjects with abnormal LDH on admission were 2.36 times more likely to progress to severe hypoxemic respiratory failure after controlling for other factors. This finding is supported by prior reports that LDH can predict severity of disease.8 9 However, our findings differ from those by Liang et al showing increased LDH only increases the likelihood of severe disease by 0.2%.8 A recent meta-analysis by Katzenschlager et al evaluated the association between LDH levels and admission to the intensive care unit (ICU) (12 studies) or death (23 studies) in patients with COVID-19. Although LDH levels were statistically higher in those critically ill (pooled difference of medians: 140 U/L (95% CI 81 to 199)) and those who died (pooled difference of medians: 189 U/L (95% CI 155 to 223)), the modest absolute increase in LDH levels was deemed clinically irrelevant by the authors.19 The differences between these studies and ours may be explained by the different outcome definitions used. In our system, the use of HFNC was not necessarily associated with ICU admission but was included as part of the endpoint. The ordinal logistic regression performed, however, supports that collapsing these two outcomes into a composite outcome yields comparable predictive utility.
NLR has been associated with adverse outcomes in patients with COVID-19.8 11 13 Adverse outcomes observed in these studies also included death, which likely accounted for the absolute risk difference in our study.8 13 Ioannou et al reported that a ratio higher than 12.7 was associated with a 2.5-fold increase in the odds for mechanical ventilation in patients with COVID-19.11 In our cohort, a higher NLR was associated with modest increases in the odds for reaching ordinal scale 6–9. The contrast between our results and those of Ioannou et al may be related to the inclusion of less severe disease categories in our primary endpoint such as receiving HFNC.
In our cohort, age and BMI were important predictors of COVID-19 respiratory failure. BMI was positively associated with progression to WHO ordinal score 6–9. Because the average BMI was >30 in our study, meaningful stratification analysis was precluded. While the mean BMI imputation biases towards significance, multiple imputation is not appropriate when data are not missing randomly relative to primary outcome. However, excluding patients who did not have valid BMI data did not meaningfully change our model findings. In addition, the overall AUC did not change in comparison with the original model. Although the age/BMI composite variable was retained by stepwise AIC reduction after excluding cases without valid BMI data, using a composite variable introduces unneeded complexity to the model. Thus, we ultimately decided to exclude the term to maintain simplicity. These data highlighted the association of BMI with severe COVID-19 and add to previous studies that support this association.11 20 21 A proposed mechanism for this association is the increased work of breathing in patients with high BMI that impairs their capacity to adjust to changes in lung function leading to earlier non-invasive ventilation or mechanical ventilation.21
We have chosen our predictors based on clinical practicality and mechanistic plausibility. Several factors—D-dimer, CRP and sex—were not significant predictors but augmented the AUC collectively. Thus, it is not surprising that some sensitivity analyses (ie, ordinal logistic regression) did not retain some or all of these factors. While D-dimer elevation was not found to be a significant predictor for respiratory failure or death in some studies,8 12 others found an association with adverse outcomes early in the pandemic.22 23 Pulmonary vasculature thrombosis was observed in autopsies of patients with COVID-19,24 and a recent study has suggested that heparin-based anticoagulation may protect non-critical patients with COVID-19 from inpatient death.25 In our model, D-dimer was not statistically associated with developing the primary endpoint in the multivariable analysis.
A smaller proportion of subjects in our cohort met the WHO ordinal scale 6–9 than subjects in other early 2020 cohorts. Only 19% of the subjects included in our modelling cohort met the primary endpoint compared with 22%–26% reported by other authors.10 12 Our cohort was enrolled during a phase of rapidly evolving COVID-19 therapies and management approaches. With improvements in early interventions against virus replication and associated inflammation, the number of patients requiring high-flow oxygen or mechanical ventilation is expected to change. We also found a lower inpatient death rate compared with reports published around the same period.12 All the subjects included in our cohort have been discharged at the time of data collection. It is possible that a subpopulation of these subjects was readmitted and expired after data collection was completed. Our study design limited data collection to the primary subject admission and may have missed mortality that occurred in subsequent encounters.
We included patients with DNI and comfort care orders in our cohort. Although this is a group of subjects that would have not been able to reach all the ordinal scale scores in our endpoint, they would have been eligible for high-flow oxygen and vasopressors. The sensitivity analysis that omitted subjects with DNI did not significantly change the predictive fidelity of the model. The inclusion of this subpopulation in our cohort likely provided a conservative estimate of the odds of meeting ordinal scales 6–9.
Our study has several limitations to acknowledge. Troponins were not included in our model because all subjects with abnormal troponin met the primary outcome. Elevated troponin suggested myocardial injury which can be due to a direct effect from SARS-CoV-2 infection and/or a complication from sepsis and the inflammatory response described in COVID-19. The role of troponin as a predictor of COVID-19-associated mortality has been suggested in other studies.26 27 However, larger studies are necessary to evaluate their role in predicting severe COVID-19 respiratory failure. Additionally, our cohort was constructed prior to introduction of COVID-19 vaccination and therapeutic interventions such as dexamethasone or remdesivir.28 Most importantly, the validation of our prediction model in the rest of our study population and in more recent cohorts after the emergence of new SARS-CoV-2 variants will be critical to assess its real-world clinical utility. Our prediction model could contribute by aiding clinicians who desire point-of-care decision support in early COVID-19.
Conclusion
This study provides a preliminary model for early identification of patients with COVID-19 at odds of progressing to severe COVID-19 within the first 12 hours of admission. This model will require further validation in larger datasets. Future studies will use this model as a tool for predicting severe COVID-19 in resource-limited settings where effective vaccines and therapies are still unavailable.
Data availability statement
Data are available in a public, open access repository. SAS code and some de-identified data will be published at Dryad, an online repository. Prisoners will not be included. Any data published there is contingent on additional approval from TDCJ's IRB. Reasonable requests for additional data will be considered.
Ethics statements
Patient consent for publication
Ethics approval
The study protocol was approved by the UTMB Institutional Review Board (20-0126) and the Texas Department of Criminal Justice (TDCJ) Institutional Review Board (#819-RM20).
Acknowledgments
We thank Daniel Z Bao, Ashley E Chen and Kyra Curtis from UTMB Medical School for their contribution to data collection. We thank Dr Ion Mitrache, Director of Clinical Analyst at UTMB, for assistance in clinical data management. We also thank Dr. Xiaoying Yu for statistical methods consulting.
References
Footnotes
JWD and BW are joint first authors.
JWD and BW contributed equally.
Contributors CC-F and WH both contributed to data collection and manuscript drafting. DR and PK both contributed to project development and data allocation. MMC has been the senior author since project inception and providing editorial revision to the manuscript. JD designed and performed all statistical analyses and drafted the methods, results and discussion sections in conjunction with ET and BW. He is responsible for the overall integrity and contents of the manuscript. ET was responsible for project conception, some data collection and some manuscript drafting, and BW was responsible for the project conception, data collection execution and drafting all parts of the manuscript.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.