Detailed description of the externally validated DPMs.
| External validated DPM name | Citation type of validation | Delirium # (%) | Sens Spec PPV NPV (external) | AUROC (95% CI) | Model components | Cog. assess tool and cut-off |
| AWOL tool | Pendlebury et al (2016) Broad val. | 1st val: 14 (9) 2nd val: 95 (31) (any delirium) 67: prevalent 28: incident | Mod. AWOL Cut-off: 3 Any delirium
| 1st val: 0.69 (0.54 to 0.83) Incident delirium 2nd val: Cohort 1 (MMSE) 0.78 (0.68 to 0.88) Cohort 2 (AMTS) 0.73 (0.63 to 0.83) | Original AWOL Tool Age >80 years 1 pt Failure to spell WORLD backwards 1 pt Disorientation 1 pt Illness severity 1 pt Modified AWOL Tool Age >80 years 1 pt Diag of dementia 1 pt MMSE <24, AMTS <9 1 pt Illness severity 1 pt | MMSE <24 AMTS <9 |
| Clinical prediction rule: cardiac surgery | Rudolph et al (2009) Narrow val. | Dev: 63 (52) Val: 48 (44) (incident delirium) | Not reported | Dev: 0.74 Val: 0.75 Did not report CI | Weighted points-regression MMSE <23 2 pts MMSE 24–27 1 pt Hx of stroke/TIA 1 pt GDS >4 1 pt Abnormal albumin 1 pt Stratified into point categories 0 pt 1 pt 2 pts ≥3 pts – high-risk group RR in high-risk group: 4.9 (3.8–6.2) | MMSE -stratified score |
| DEAR | Freter et al
27 Narrow val. | Dev: (2005) 18 (14) Val: (2015) Preop= 163 (58) Postop= 118 (42) | Sens 0.68 Spec 0.73 PPV 0.65 NPV 0.76 Optimal cut-off score: 3 pts (incident postop delirium) | Dev: (2005) 0.77 (0.64 to 0.87) Val: (2015) AUROC not published | MMSE <23 1 pt Functional dependence 1 pt Sensory impairment 1 pt Substance use 1 pt Age >80 years 1 pt Not weighted. 0–5 score, cut-off of 3 indicating high risk. | MMSE Cut-off ≤23 |
| Delirium at discharge prediction model | Inouye et al
31 Narrow val. | Dev: 58 (12) Val: 28 (6) (incident delirium) | Not reported | Dev: 0.80 Val: 0.75 Did not report CI Calibration: χ2 trend: p<0.001 | Delirium at discharge prediction Dementia diagnosis or mBDRS >4 1 pt Vision impairment 1 pt ADL impairment 1 pt Charlson score 1 pt Restraint use during delirium 1 pt Not weighted. 0–1 pt=low risk 2–3 pts=intermediate risk 4–5 pt=high risk RR in high-risk group: 10.2 (3.2–32.7) | MMSE <24 mBDR ≥4 |
| Delirium Prediction Score (DPS) | Carrasco et al
23 Narrow val. | Dev: 25 (0.06) Val: 12 (12) (incident delirium) | Sens 0.88 Spec 0.74 PPV 0.22 NPV 0.99 | Dev: 0.86 (0.82 to 0.91) Val: 0.78 (0.66 to 0.90) | DPS=[5×BUN/Cr ratio]−(3× Barthel Index). Cut-off is: >−240=high risk for delirium In conventional units, cut-off is: >−160=high risk for delirium | None. Pfeffer Functional Activities Questionnaire as a proxy for prior dementia |
| Delphi score | Kim et al
35 Narrow val. | Dev: 112 (20) Val: 99 (18) (incident delirium) | Sens 0.81 Spec 0.93 PPV 0.70 NPV 0.96 Optimal cut-off score: 6.5 pts | Dev: 0.911 (0.88 to 0.94) Val: 0.938 (0.91 to 0.97) | Age (years) 60–69 0 70–79 1 >80 2 Low physical activity Self-sufficient 0 Need assist. 2 Heavy ETOH No 0 Yes 1 Hearing impairment No 0 Yes 1 History of delirium No 0 Yes 2 Emergency surgery No 0 Yes 1 Open surgery No 0 Yes 2 ICU admission No 0 Yes 3 Preop CRP (mg/dL) <10 0 >10 1 Max points: 15 Optimal cut-off: 6.5 High risk: >7 pts | No measure of cognition. Excluded participants if MMSE <24 |
| e-NICE rule | Rudolph et al
44 Broad val. Dev: 2343 (8) Val: 64 (26) (incident delirium) | Cohort AUROC CI TPR FPR Dev: 0.81 (0.80 to 0.82) Val: AUROCs* Original 0.69 (0.61 to 0.77) 64%–33% mRASS 0.72 (0.65 to 0.79) 69%–35% TMYB 0.73 (0.66 to 0.80) 78%–43% MoCA 0.74 (0.66 to 0.81) 75%–43% *Any delirium Original model: AUROC of 0.68 (95% CI 0.59 to 0.77) in incident delirium. Did not report sens, spec, PPV and NPV | Weighted points/OR Cog. impair Medications, diagnosis or both 4 pts Age >65 years 2 pts Age >80 years 3 pts Infection 2 pts Fracture 4 pts Vision 1 pt Severe illness 2 pts 0–2 pts=low risk 2–5 pts=intermediate risk 6–8 pts=high risk ≥9 pts=very high risk | e-NICE Tool Diagnosis of dementia, medications for dementia or both qualified as ‘cognitive impairment’ in model. Prospective cohort, additional: mRASS TMYB MoCA <18 | ||
| Inouye Prediction Rule (IPR) | Inouye et al
30 Narrow val. | Dev: 27 (25) Val: 29 (17) (incident delirum) | Did not report | Dev: 0.74 (0.63 to 0.85) Val: 0.66 (0.55 to 0.77) Calibration below: Dev: X2trend p<0.00 001 Val: X2trend p<0.002 | Baseline cognitive impairment 1 pt High BUN/Cr ratio 1 pt Severe illness (Composite score: APACHE II >16+RN rating) 1 pt Vision impairment 1 pt Not weighted. 0 pt=low risk 1–2 pts=intermediate risk 3–4 pts=high risk RR in high risk group: 9.5 (no CI) | MMSE cut-off <24 Family/caregiver bDRS Excluded those with history of severe dementia |
| IPR | Kalisvaart et al
34 Broad val. | Val: 74 (12) | Did not report | Val: 0.73 (0.65 to 0.78) Calibration: X2 p<0.05 X2trend p<0.002 | Externally validated IPR in surgical hip fracture population.
| MMSE cut-off <24 |
| IPR | Rudolph et al
45 Broad val. | Val: 23 (23) Any delirium 10: prevalent 13: incident | Did not report | Val: 0.56 (0.42 to 0.74) Incident delirium Calibration: X2 1.3, p=0.53 | Externally validated IPR in medical VA population, investigated feasibility of chart abstraction tool. | MMSE cut-off <24 |
| IPR | Pendlebury et al 33 Broad val. | Val: 95 (31) Any delirium 67: prevalent 28: incident | Cut-off 2 pts All delirium
| Val: Incident delirium Cohort 1 (MMSE) 0.73 (0.62 to 0.84) Cohort 2 (AMTS) 0.70 (0.60 to 0.81) | Baseline cognitive impairment 1 pt High BUN/Cr ratio 1 pt Severe illness (SIRS >2) 1 pt Vision impairment 1 pt 4 pts=incident delirium | Original model: MMSE <24 Modified model: MMSE <24 AMTS <9 |
| Isfandiaty model | Pendlebury et al
33 Broad val. | Dev: 87 (19) Val: 95 (31) Any delirium 67: prevalent 28: incident | Cut-off 4 pts All delirium
| Dev: 0.82 (0.77 to 0.88) Val: Incident delirium Cohort 1 (MMSE) 0.83 (0.74 to 0.91) Cohort 2 (AMTS) 0.77 (0.67 to 0.86) | Baseline cognitive impairment 3 pts Functional dependency 2 pts Infection with sepsis 2 pts Infection without sepsis 1 pt Weighted score Score=7 for incident delirium Cohort 1: MMSE Cohort 2: AMTS | Original model: Chart review Modified model: MMSE <24 AMTS <9 |
| Martinez et al 2012 model | Pendlebury et al
33 Broad val. | 1st Val: 76 (25) 2nd Val: 95 (31) Any delirium 67: prevalent 28: incident | Modified model Cut-off 2 pts All delirium
| 1st Val: 0.85 (0.80 to 0.88) Incident delirium 2nd Val: Cohort 1 (MMSE) 0.78 (0.68 to 0.88) Cohort 2 (AMTS) 0.75 (0.65 to 0.84) | Martinez et al 2012 original model Age >85 years 1 pt Dependent in >5 ADLs 1 pt Drugs on admit: 1 pt/drug 2 pts/antipsych
Score >1=high risk for delirium Modified model Age >85 years 1 pt Dependency in >5 ADLs 1 pt Diag of dementia MMSE <24 AMTS <9 1 pt | Original model: No cognitive measure Modified model: MMSE <24 AMTS <9 |
| Pompei et al 1994 model | Pompei et al
43 Broad val. | Dev: 64 (15) Val: 86 (26) (21=prevalent delirium) | Sens 0.83 Spec 0.50 PPV 0.38 NPV 0.89 *Pts stratified as low or moderate to high risk | Dev: 0.74 ±0.05 Val: 0.64 ±0.05 Calibration: X2trend p<0.0001 | Weighted points Baseline cognitive impairment 2 pts Depression 2 pts Alcoholism 3 pts >4 comorbidities 3 pts 0–3 pts=low risk 4–7 pts=moderate risk 8–10 pts=high risk | MMSE Less than high school <21 High school <23 College education <24 |
| Precipitating risk factors | Inouye and Charpentier29 Narrow val. | Dev: 35 (18) Val: 47 (15) (incident delirium) | Not reported | No AUROC reported Calibration: X2trend p<0.001 | Physical restraint use 1 pt Malnutrition 1 pt >3 medications added 1 pt Bladder catherisation 1 pt Any iatrogenic event 1 pt Not weighted. 0 pt=low risk 1–2 pts=intermediate ≥3 pts=high risk RR of high risk: 17.5 (8.1 to 37.4) | None used in model |
| Risk Model for Delirium (RD) | Moerman et al
41 Narrow val. | Val: 102 (27) (incident delirium) | Sens 0.81 Spec 0.56 PPV 0.41 NPV 0.89 Optimal cut-off score: 4 pts | Val: 0.73 (0.68 to 0.77) | Weighted points Delirium: previous hospitalisation 5 pts Dementia 5 pts Clock drawing
Impaired vision 1 pt Problems with ADL
Daily >4 alcohol 2 pts ≥5 pts=high risk | CDT −11:10 Two categories 1: small mistakes 2: big mistakes |
| Susceptibility score | Pendlebury et al
48 Broad val. | Val: 308 (28) (incidence delirium) | Sens 0.71 Spec 0.88 PPV 0.5 NPV 0.95 Cut-off score: 5 pts | Val: 0.81 (0.70 to 0.92) Improved with age eliminated to 0.84 (0.77 to 0.92) | Weighted points Dementia/cog impair 2 pts Age >80 years 2 pts Severe illness (SIRS+) 1pt Infection-working diagnosis 1 pt Vision impairment 1 pt >5 pts=high risk ORs for >5 risk score: 25.0 (3.0 to 208.9) RR for >5 risk score: 5.4 | Known diagnosis of dementia or MMSE <24 AMTS <9 |
ADL, activities of daily living; AMTS, Abbreviated Mental Test Score; AUROC, area under the receiver operating curve statistic; CI, Confidence Intervals; RR, Relative Risk; TPR, True Positive Rate; FPR, False Positive Rate; BUN/CR, Blood Urea Nitrogen/Creatinine ratio; CDT, Clock Drawing Test; CRP, C reactive protein; ETOH, alcohol use; Dev, development; DPM, delirium prediction model; GDS, Geriatric Depression Score; Hx, History; ICU, intensive care unit; IPR, Inouye Prediction Rule; mBDR, Modified Blessed Dementia Rating; bDRS, Blessed Dementia Rating Scale; MMSE, Mini-Mental Status Exam; MoCA, Montreal Cognitive Assessment; mRASS, Modified Richmond Agitation-Sedation Scale; NPV, negative predictive value; PPV, positive predictive value; RN, Registered Nurse; Sens, Sensitivity; Spec, Specificity; SIRS, Systemic Inflammatory Response Syndrome; TIA, Transient Ischemic Attack; TMTYB, the months of the year backwards; VA, Veterans Administration; val, validation.