Table 2

Performance characteristics for the trained ensembles and SWIFT. CUH, Cambridge University Hospitals NHS Foundation Trust; MIMIC III, Medical Information Mart for Intensive Care; SWIFT, Stability and Workload Index for Transfer.

Ensemble: transferEnsemble: MIMIC-III onlyEnsemble: CUH onlySWIFT
AUROC 0.7095 (0.0260) 0.6079 (0.0256)0.6092 (0.0320)0.6082 (0.0786)
Sensitivity0.59170.5792 0.5972 0.5682
Specificity 0.6640 0.57000.50300.6234
F1 0.1321 0.10740.09560.1156
DOR 2.8635 1.82431.50072.1780
Brier score 0.0402 (0.0009) 0.1830 (0.0048)0.0462 (0.0015)NA
  • AUROC and Brier score are presented as mean (SE) over 10 cross-validation folds. All characteristics other than AUROC and Brier score are computed from a particular operating point on the ROC curves (figure 2) and prevalence; this operating point is chosen such that sensitivity is the highest available value less than or equal to 0.60. The best value for each performance measure is in bold.

  • AUROC, area under the receiver operating characteristic curve; Brier score, mean square forecast error, where the forecast is the probabilistic output of the classifier; CUH, Cambridge University Hospitals NHS Foundation Trust; DOR, diagnostic OR, the ratio of true positives to positive test results, divided by the ratio of true negatives to negative test results; F1, two times the harmonic mean of precision and sensitivity (recall), where precision is the ratio of true positives to the sum of true positives and false positives; NA, not applicable; Sensitivity, the ratio of detected positive examples to all positive examples; Specificity, the ratio of true negatives to the sum of true negatives and false positives.