Comparison of different Rx-risk scoring and modelling methods to predict 1-year mortality in the Department of Veterans’ Affairs (DVA) population and external validation using the Pharmaceutical Benefits Scheme (PBS) population
| Models | DVA | PBS | ||||||
| AIC* | Difference in AIC† | C-statistic‡ (95% CI) | Relative IDI, P values | AIC* | Difference in AIC† | C-statistic‡ (95% CI) | Relative IDI, P values | |
| Base model (BM): age and sex | 80 538.5 | – | 0.738 (0.734 to 0.742) | – | 79 527.9 | – | 0.761 (0.756 to 0.766) | – |
| Rx-risk measures | ||||||||
| BM+unweighted Rx-risk | 79 420.1 | 1118.4 | 0.751 (0.747 to 0.754) | 14.0%, <0.0001 | 77 029.9 | 2498.0 | 0.796 (0.791 to 0.800) | 25.5%, <0.0001 |
| BM+DVA-weighted Rx-risk | 76 102.4 | 4436.1 | 0.786 (0.782 to 0.789) | 65.6%, <0.0001 | 73 143.8 | 6384.1 | 0.833 (0.829 to 0.837) | 92.0%, <0.0001 |
| BM+43 comorbidity indicators | 75 692.2 | 4846.3 | 0.791 (0.788 to 0.795) | 73.9%, <0.0001 | 71 689.1 | 7838.8 | 0.845 (0.842 to 0.849) | 114.8%, <0.0001 |
| Crude measures | ||||||||
| BM+prescription count | 79 105.9 | 1432.6 | 0.755 (0.751 to 0.759) | 18.6%, <0.0001 | 76 762.8 | 2765.1 | 0.799 (0.795 to 0.804) | 31.4%, <0.0001 |
| BM+unique ATC§ medicine count | 78 374.5 | 2164.0 | 0.762 (0.758 to 0.766) | 29.4%, <0.0001 | 75 369.1 | 4158.8 | 0.814 (0.810 to 0.818) | 50.0%, <0.0001 |
| BM+unique PBS item code¶ medicine count | 78 210.2 | 2328.3 | 0.764 (0.760 to 0.768) | 32.1%, <0.0001 | 75 108.8 | 4419.1 | 0.816 (0.812 to 0.820) | 55.8%, <0.0001 |
*The model with the lowest AIC value is considered the best fit.
†AIC score compared with the AIC score of the base model. A model with a lower score of 10 (or more) is considered superior.
‡Possible range 0–1, with 1 indicating perfect prediction and 0.5 indicating chance prediction. A c-statistic between 0.8 and 0.9 is generally considered as excellent, and between 0.7 and 0.8 acceptable.
§Anatomical Chemical Therapeutic (ATC) Classification System, count based on the number of unique ATC codes dispensed.
¶PBS, count based on the number of unique PBS item codes dispensed.
AIC, Akaike information criterion model; IDI, integrated discrimination improvement.