NLP performance for abstracting Framingham HF phenotype criteria from EHRs. Validation dataset (N=406)
| HF criteria variables (n)* | Sensitivity, % (95% CI) | Specificity, % (95% CI) | PPV, % (95% CI) | Note types used |
| Weight loss ≥4.5 kg† (27) | 81.5 (61.9 to 93.7) | 96.0 (93.6 to 97.8) | 59.5 (43.7 to 75.3) | Structured data |
| Jugular venous distension (56) | 60.7 (46.8 to 73.5) | 91.7 (87.6 to 94.8) | 61.8 (49.0 to 74.6) | ED, AN |
| Hepatojugular reflux (0) | N/A | 99.7 (98.2 to 100.0) | 0.00 | ED, AN |
| PND (27) | 55.6 (35.3 to 74.5) | 89.4 (85.2 to 92.7) | 33.3 (19.2 to 46.7) | ED, AN, DC |
| Orthopnea (64) | 59.4 (46.4 to 71.5) | 92.7 (88.7 to 95.6) | 67.9 (55.7 to 80.1) | ED, AN, DC |
| Pulmonary basilar rales (93) | 61.3 (50.6 to 71.2) | 66.4 (59.7 to 72.6) | 43.8 (35.3 to 52.3) | ED, AN, DC |
| S3 gallop (5) | 40.0 (5.3 to 85.3) | 95.1 (92.0 to 97.2) | 11.8 (0.00 to 27.14) | ED, AN, DC |
| Pulmonary oedema (48) | 91.7 (80.0 to 97.7) | 51.0 (44.5 to 57.5) | 27.3 (20.4 to 34.2) | ED, AN, DC, IR |
| Cardiomegaly (162) | 54.3 (46.3 to 62.2) | 96.0 (90.9 to 98.7) | 96.7 (93.0 to 100.0) | ED, AN, DC, IR |
| Lower extremity oedema (163) | 74.8 (67.5 to 81.3) | 75.5 (67.7 to 82.2) | 77.2 (70.7 to 83.7) | ED, AN, DC |
| Hepatomegaly (3) | 33.3 (0.8 to 90.6) | 99.0 (97.2 to 99.8) | 33.3 (0.00 to 86.2) | ED, AN, IR |
| Dyspnoea on exertion (263) | 79.1 (73.7 to 83.8) | 74.5 (59.7 to 86.1) | 94.5 (91.5 to 97.5) | ED, AN, DC |
| Bilateral pleural effusion (79) | 75.9 (65.0 to 84.9) | 73.1 (66.5 to 79.0) | 51.7 (42.6 to 60.8) | ED, AN, DC, IR |
*Instances in total cohort that criteria were identified by manual ARIC abstractors (reference standard).
†Weight loss during hospitalisation based on structured daily patient weight data.
AN, admission note; ARIC, Atherosclerosis Risk in Communities; DC, discharge summary; ED, emergency department ; EHRs, electronic health records; HF, heart failure; IR, imaging report; NLP, natural language processing; PND, paroxysmal nocturnal dyspnoea; PPV, positive-predictive value.