Accuracy of case ascertainment algorithms in identifying individuals experiencing homelessness
| Reference standard definition: housing episode (n=144 148 overall, with 3443 homeless episodes) | |||||||||
| Algorithm definition | TP | FP | FN | TN | Sensitivity (%) (95% CI) | Specificity (%) (95% CI) | PPV (%)(95% CI) | NPV (%)(95% CI) | LR+ |
| 1 indicator+/−0 days | 372 | 528 | 3071 | 140 177 | 10.8 (9.8 to 11.9) | 99.6 (99.6 to 99.7) | 41.3 (38.2 to 44.6) | 97.9 (97.8 to 97.9) | 28.8 |
| 1 indicator+/−15 days | 482 | 591 | 2961 | 140 114 | 14.0 (12.9 to 15.2) | 99.6 (99.5 to 99.6) | 44.9 (42.0 to 47.9) | 97.9 (97.9 to 98.0) | 33.3 |
| 1 indicator+/−45 days | 619 | 665 | 2824 | 140 040 | 18.0 (16.7 to 19.3) | 99.5 (99.5 to 99.6) | 48.2 (45.5 to 50.9) | 98.0 (98.0 to 98.1) | 38.0 |
| 1 indicator+/−90 days | 718 | 765 | 2725 | 139 940 | 20.9 (19.5 to 22.2) | 99.5 (99.4 to 99.5) | 48.4 (45.9 to 51.0) | 98.1 (98.0 to 98.2) | 38.4 |
| 1 indicator+/−180 days | 861 | 897 | 2582 | 139 808 | 25.0 (23.6 to 26.5) | 99.4 (99.3 to 99.4) | 49.0 (46.6 to 51.3) | 98.2 (98.1 to 98.3) | 39.2 |
| 1 indicator OR postal code+/−0 days | 450 | 679 | 2993 | 140 026 | 13.1 (12.0 to 14.2) | 99.5 (99.5 to 99.6) | 39.9 (37.0 to 42.7) | 97.9 (97.8 to 98.0) | 27.1 |
| 1 indicator OR postal code+/−15 days | 572 | 758 | 2871 | 139 947 | 16.6 (15.4 to 17.9) | 99.5 (99.4 to 99.5) | 43.0 (40.4 to 45.7) | 98.0 (97.9 to 98.1) | 30.8 |
| 1 indicator OR postal code+/−45 days | 714 | 845 | 2729 | 139 860 | 20.7 (19.4 to 22.1) | 99.4 (99.4 to 99.4) | 45.8 (43.3 to 48.3) | 98.1 (98.0 to 98.2) | 34.5 |
| 1 indicator OR postal code+/−90 days | 824 | 967 | 2619 | 139 738 | 23.9 (22.5 to 25.4) | 99.3 (99.3 to 99.4) | 46.0 (43.7 to 48.3) | 98.2 (98.1 to 98.2) | 34.8 |
| 1 indicator OR postal code+/−180 days | 994 | 1135 | 2449 | 139 570 | 28.9 (27.4 to 30.4) | 99.2 (99.1 to 99.2) | 46.7 (44.6 to 48.8) | 98.3 (98.2 to 98.3) | 35.8 |
| 1 CIHI indicator+/−0 days | 368 | 466 | 3075 | 140 239 | 10.7 (9.7 to 11.8) | 99.7 (99.6 to 99.7) | 44.1 (40.8 to 47.5) | 97.9 (97.8 to 97.9) | 36.9 |
| 1 CIHI indicator+/−15 days | 477 | 528 | 2966 | 140 177 | 13.9 (12.7 to 15.0) | 99.6 (99.6 to 99.7) | 47.5 (44.4 to 50.6) | 97.9 (97.9 to 98.0) | 39.6 |
| 1 CIHI indicator+/−45 days | 613 | 595 | 2830 | 140 110 | 17.8 (16.6 to 19.1) | 99.6 (99.5 to 99.6) | 50.7 (47.9 to 53.6) | 98.0 (97.9 to 98.1) | 42.0 |
| 1 CIHI indicator+/−90 days | 710 | 693 | 2733 | 140 012 | 20.6 (19.3 to 22.0) | 99.5 (99.5 to 99.5) | 50.6 (48.0 to 53.2) | 98.1 (98.0 to 98.2) | 41.7 |
| 1 CIHI indicator+/−180 days | 852 | 822 | 2591 | 139 883 | 24.8 (23.3 to 26.2) | 99.4 (99.4 to 99.5) | 50.9 (48.5 to 53.3) | 98.2 (98.1 to 98.3) | 41.8 |
| 1 CIHI indicator OR postal code+/−0 days | 444 | 575 | 2999 | 140 130 | 12.9 (11.8 to 14.1) | 99.6 (99.6 to 99.6) | 43.6 (40.6 to 46.6) | 97.9 (97.8 to 98.0) | 32.3 |
| 1 CIHI indicator OR postal code+/−15 days | 566 | 652 | 2877 | 140 053 | 16.4 (15.2 to 17.7) | 99.5 (99.5 to 99.6) | 46.5 (43.7 to 49.3) | 98.0 (97.9 to 98.1) | 36.9 |
| 1 CIHI indicator OR postal code+/−45 days | 707 | 734 | 2736 | 139 971 | 20.5 (19.2 to 21.9) | 99.5 (99.4 to 99.5) | 49.1 (46.5 to 51.6) | 98.1 (98.0 to 98.2) | 42.1 |
| 1 CIHI indicator OR postal code+/−90 days | 817 | 852 | 2626 | 139 853 | 23.7 (22.3 to 25.2) | 99.4 (99.4 to 99.4) | 49.0 (46.6 to 51.3) | 98.2 (98.1 to 98.2) | 41.9 |
| 1 CIHI indicator OR postal code+/−180 days | 985 | 1017 | 2458 | 139 688 | 28.6 (27.1 to 30.1) | 99.3 (99.2 to 99.3) | 49.2 (47.0 to 51.4) | 98.3 (98.2 to 98.3) | 42.4 |
| Reference standard definition: annual housing experience (n=4 91 213 calendar years overall, with 2290 homeless years) | |||||||||
| Algorithm definition | TP | FP | FN | TN | Sensitivity (%) (95% CI) | Specificity (%) (95% CI) | PPV (%)(95% CI) | NPV (%)(95% CI) | LR+ |
| 1 indicator+/−0 days | 429 | 334 | 1861 | 488 589 | 18.7 (17.2 to 20.4) | 99.9 (99.9 to 99.9) | 56.2 (52.7 to 59.7) | 99.6 (99.6 to 99.6) | 274.2 |
| 1 indicator+/−15 days | 454 | 352 | 1836 | 488 571 | 19.8 (18.2 to 21.5) | 99.9 (99.9 to 99.9) | 56.3 (52.9 to 59.7) | 99.6 (99.6 to 99.6) | 275.4 |
| 1 indicator+/−45 days | 487 | 406 | 1803 | 488 517 | 21.3 (19.6 to 23.0) | 99.9 (99.9 to 99.9) | 54.5 (51.3 to 57.8) | 99.6 (99.6 to 99.6) | 256.1 |
| 1 indicator+/−90 days | 529 | 472 | 1761 | 488 451 | 23.1 (21.4 to 24.9) | 99.9 (99.9 to 99.9) | 52.8 (49.8 to 55.9) | 99.6 (99.6 to 99.7) | 239.3 |
| 1 indicator+/−180 days | 590 | 588 | 1700 | 488 335 | 25.8 (24.0 to 27.6) | 99.9 (99.9 to 99.9) | 50.1 (47.2 to 52.9) | 99.7 (99.6 to 99.7) | 214.2 |
| 1 indicator OR postal code+/−0 days | 512 | 433 | 1778 | 488 490 | 22.4 (20.7 to 24.1) | 99.9 (99.9 to 99.9) | 54.2 (51.0 to 57.3) | 99.6 (99.6 to 99.7) | 252.5 |
| 1 indicator OR postal code+/−15 days | 543 | 458 | 1747 | 488 465 | 23.7 (22.0 to 25.5) | 99.9 (99.9 to 99.9) | 54.2 (51.1 to 57.3) | 99.6 (99.6 to 99.7) | 253.1 |
| 1 indicator OR postal code+/−45 days | 581 | 525 | 1709 | 488 398 | 25.4 (23.6 to 27.2) | 99.9 (99.9 to 99.9) | 52.5 (49.6 to 55.5) | 99.7 (99.6 to 99.7) | 236.3 |
| 1 indicator OR postal code+/−90 days | 629 | 610 | 1661 | 488 313 | 27.5 (25.7 to 29.3) | 99.9 (99.9 to 99.9) | 50.8 (48.0 to 53.5) | 99.7 (99.6 to 99.7) | 220.2 |
| 1 indicator OR postal code+/−180 days | 707 | 754 | 1583 | 488 169 | 30.9 (29.0 to 32.8) | 99.9 (99.8 to 99.9) | 48.4 (45.8 to 51.0) | 99.7 (99.7 to 99.7) | 200.2 |
| 1 indicator+/−0 day OR PSTLYEAR postal code | 588 | 356 | 1702 | 488 567 | 25.7 (23.9 to 27.5) | 99.9 (99.9 to 99.9) | 62.3 (59.2 to 65.3) | 99.7 (99.6 to 99.7) | 352.6 |
| 1 indicator+/−15 days OR PSTLYEAR postal code | 706 | 402 | 1584 | 488 521 | 30.8 (29.0 to 32.8) | 99.9 (99.9 to 99.9) | 63.7 (60.8 to 66.5) | 99.7 (99.7 to 99.7) | 375.0 |
| 1 indicator+/−45 days OR PSTLYEAR postal code | 734 | 452 | 1556 | 488 471 | 32.1 (30.2 to 34.0) | 99.9 (99.9 to 99.9) | 61.9 (59.1 to 64.6) | 99.7 (99.7 to 99.7) | 346.7 |
| 1 indicator+/−90 days OR PSTLYEAR postal code | 766 | 518 | 1524 | 488 405 | 33.4 (31.5 to 35.4) | 99.9 (99.9 to 99.9) | 59.7 (56.9 to 62.3) | 99.7 (99.7 to 99.7) | 315.7 |
| 1 indicator+/−180 days OR PSTLYEAR postal code | 816 | 633 | 1474 | 488 290 | 35.6 (33.7 to 37.6) | 99.9 (99.9 to 99.9) | 56.3 (53.7 to 58.8) | 99.7 (99.7 to 99.7) | 275.2 |
| 1 CIHI indicator+/−0 days | 423 | 300 | 1867 | 488 623 | 18.5 (16.9 to 20.1) | 99.9 (99.9 to 99.9) | 58.5 (54.9 to 62.0) | 99.6 (99.6 to 99.6) | 301.0 |
| 1 CIHI indicator+/−15 days | 448 | 315 | 1842 | 488 608 | 19.6 (18.0 to 21.2) | 99.9 (99.9 to 99.9) | 58.7 (55.2 to 62.2) | 99.6 (99.6 to 99.6) | 303.6 |
| 1 CIHI indicator+/−45 days | 480 | 358 | 1810 | 488 565 | 21.0 (19.3 to 22.7) | 99.9 (99.9 to 99.9) | 57.3 (53.9 to 60.6) | 99.6 (99.699.6) | 286.3 |
| 1 CIHI indicator+/−90 days | 521 | 405 | 1769 | 488 518 | 22.8 (21.1 to 24.5) | 99.9 (99.9 to 99.9) | 56.3 (53.0 to 59.4) | 99.6 (99.6 to 99.7) | 274.7 |
| 1 CIHI indicator+/−180 days | 581 | 519 | 1709 | 488 404 | 25.4 (23.6 to 27.2) | 99.9 (99.9 to 99.9) | 52.8 (49.9 to 55.8) | 99.7 (99.6 to 99.7) | 239.0 |
| 1 CIHI indicator OR postal code+/−0 days | 508 | 370 | 1782 | 488 553 | 22.2 (20.5 to 23.9) | 99.9 (99.9 to 99.9) | 57.9 (54.6 to 61.1) | 99.6 (99.6 to 99.7) | 293.1 |
| 1 CIHI indicator OR postal code+/−15 days | 539 | 390 | 1751 | 488 533 | 23.5 (21.8 to 25.3) | 99.9 (99.9 to 99.9) | 58.0 (54.8 to 61.2) | 99.6 (99.6 to 99.7) | 295.1 |
| 1 CIHI indicator OR postal code+/−45 days | 576 | 442 | 1714 | 488 481 | 25.2 (23.4 to 27.0) | 99.9 (99.9 to 99.9) | 56.6 (53.5 to 59.6) | 99.7 (99.6 to 99.7) | 278.2 |
| 1 CIHI indicator OR postal code+/−90 days | 622 | 502 | 1668 | 488 421 | 27.2 (25.4 to 29.0) | 99.9 (99.9 to 99.9) | 55.3 (52.4 to 58.2) | 99.7 (99.6 to 99.7) | 264.5 |
| 1 CIHI indicator OR postal code+/−180 days | 699 | 634 | 1591 | 488 289 | 30.5 (28.7 to 32.4) | 99.9 (99.9 to 99.9) | 52.4 (49.8 to 55.1) | 99.7 (99.7 to 99.7) | 235.4 |
| 1 CIHI indicator+/-0 day OR PSTLYEAR postal code | 583 | 322 | 1707 | 488 601 | 25.5 (23.7 to 27.3) | 99.9 (99.9 to 99.9) | 64.4 (61.2 to 67.5) | 99.7 (99.6 to 99.7) | 386.6 |
| 1 CIHI indicator+/−15 days OR PSTLYEAR postal code | 701 | 365 | 1589 | 488 558 | 30.6 (28.8 to 32.5) | 99.9 (99.9 to 99.9) | 65.8 (62.9 to 68.5) | 99.7 (99.7 to 99.7) | 410.0 |
| 1 CIHI indicator+/−45 days OR PSTLYEAR postal code | 728 | 404 | 1562 | 488 519 | 31.8 (29.9 to 33.7) | 99.9 (99.9 to 99.9) | 64.3 (61.5 to 67.0) | 99.7 (99.7 to 99.7) | 384.7 |
| 1 CIHI indicator+/−90 days OR PSTLYEAR postal code | 760 | 451 | 1530 | 488 472 | 33.2 (31.3 to 35.1) | 99.9 (99.9 to 99.9) | 62.8 (60.0 to 65.4) | 99.7 (99.7 to 99.7) | 359.8 |
| 1 CIHI indicator+/−180 days OR PSTLYEAR postal code | 809 | 564 | 1481 | 488 359 | 35.3 (33.4 to 37.3) | 99.9 (99.9 to 99.9) | 58.9 (56.3 to 61.5) | 99.7 (99.7 to 99.7) | 306.2 |
Bold lines indicate optimal case algorithm definitions.
CIHI, Discharge Abstract Database, National Ambulatory Care Reporting System or Ontario Mental Health Reporting System; FN, false negative (flagged as housed but truly homeless); FP, false positive (flagged as homeless but not truly homeless); LR+, positive likelihood ratio; NPV, negative predictive value; PPV, positive predictive value; PSTLYEAR, ICES PSTLYEAR postal code, indicating the best estimate of an individual’s postal code for the year using ICES databases; TN, true negative (flagged as housed and truly housed); TP, true positive (flagged as homeless and truly homeless).