Article Text

PDF

The validity of the Rx-Risk Comorbidity Index using medicines mapped to the Anatomical Therapeutic Chemical (ATC) Classification System
  1. Nicole L Pratt,
  2. Mhairi Kerr,
  3. John D Barratt,
  4. Anna Kemp-Casey,
  5. Lisa M Kalisch Ellett,
  6. Emmae Ramsay,
  7. Elizabeth Ellen Roughead
  1. Quality Use of Medicines and Pharmacy Research Centre, University of South Australia, Adelaide, South Australia, Australia
  1. Correspondence to Associate Professor Nicole L Pratt; nicole.pratt{at}unisa.edu.au

Abstract

Objectives To provide a map of Anatomical Therapeutic Chemical (ATC) Classification System codes to individual Rx-Risk comorbidities and to validate the Rx-Risk Comorbidity Index.

Design The 46 comorbidities in the Rx-Risk Index were mapped to dispensing’s indicative of each condition using ATC codes. Prescription dispensing claims in 2014 were used to calculate the Rx-Risk. A baseline logistic regression model was fitted using age and gender as covariates. Rx-Risk was added to the base model as an (1) unweighted score, (2) weighted score and as (3) individual comorbidity categories indicating the presence or absence of each condition. The Akaike information criterion and c-statistic were used to compare the models.

Setting Models were developed in the Australian Government Department of Veterans’ Affairs health claims data, and external validation was undertaken in a 10% sample of the Australian Pharmaceutical Benefits Scheme Data.

Participants Subjects aged 65 years or older.

Outcome measures Death within 1 year (eg, 2015).

Results Compared with the base model (c-statistic 0.738, 95% CI 0.734 to 0.742), including Rx-Risk improved prediction of mortality; unweighted score 0.751, 95% CI 0.747 to 0.754, weighted score 0.786, 95% CI 0.782 to 0.789 and individual comorbidities 0.791, 95% CI 0.788 to 0.795. External validation confirmed the utility of the weighted index (c-statistic=0.833).

Conclusions The updated Rx-Risk Comorbidity Score was predictive of 1-year mortality and may be useful in practice to adjust for confounding in observational studies using medication claims data.

  • chronic disease burden
  • claims data
  • comorbidity
  • weighting
  • mortality prediction
  • model validation

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

Statistics from Altmetric.com

Strength and limitations of this study

  • This study provides an up-to-date list of medicines identified by Anatomical Therapeutic Chemical (ATC) codes mapped to individual Rx-Risk categories.

  • Rx-Risk mapped to ATC codes can be easily adapted for use in other health systems making this index a useful resource for researchers worldwide.

  • The Rx-Risk Index has been updated and mapped to ATC codes based on medicine availability in Australia; hence, modifications may be required for use in other health systems.

  • This study was limited to patients over 65 years of age, so Rx-Risk category weights derived in this study may not be applicable to younger populations.

Introduction 

The prevalence of multimorbidity in the population is increasing,1 2 and patients with multiple conditions are at greater risk of adverse outcomes.3 In observational studies, in which the aim is to determine the association between medicine use and adverse events, adjustment for multimorbidity is required to avoid biased results. Reliable methods for identifying and controlling for multiple comorbidities are required in order to make valid comparisons between treatments. The Rx-risk is a measure for determining an individual’s current comorbidities based on their prescription medicine dispensing.4 5 It was initially developed for predicting costs of healthcare6 and was subsequently adapted to predict mortality in outpatient populations.7 8 Rx-risk has been found to be a better predictor of 1-year and 3-year mortality (1-year mortality: weighted Rx-risk c-statistic=0.728, 95% CI 0.723 to 0.733, 3-year mortality: 0.731, 95% CI 0.728 to 0.734) compared with simple prescription counts in the same time periods (1-year mortality: 0.715, 95% CI 0.710 to 0.720, 3-year mortality: 0.718, 95% CI 0.715 to 0.721).9

The first pharmacy-based measure of comorbidity, developed in 1992, was the Chronic Disease Score (CDS),4 consisting of 17 comorbidity categories. The CDS was subsequently updated and renamed in 2003 as the Rx-Risk-V Index, consisting of 45 categories of comorbidity.10 The Rx-Risk-V Index was then adapted to only include comorbidities for which a medicine could be prescribed and therefore could be applied to prescription claims data resulting in an index based on 42 categories of comorbidity.11 Due to continual advances in pharmaceutical disease management and as new medicines are used to treat particular diseases, for example, treatment for hepatitis B and C, the Rx-Risk requires periodical updating and revalidation. Additionally, the original published weights for the Rx-Risk Score were calculated by predicting cost of treatment rather than risk of death which is a more clinically relevant outcome.6 A list of Rx-Risk comorbidities and their corresponding medicines mapped to a standardised international coding system has not been published previously and would facilitate use of the index across health systems. Other comorbidity scores such as the Elixhauser and Charlson Index12 13 require diagnostic information in their construction. The advantage of the Rx-Risk is that it requires prescription data only and provides researchers with the ability to measure comorbidity even in a predominately outpatient setting.

The aim of this study was to facilitate the use of the Rx-Risk in practice by providing a list of the individual Rx-isk categories mapped, using clinical expertise, to WHO’s Anatomical Therapeutic Chemical (ATC) Classification System14 and to determine the validity of the Rx-Risk Index, in predicting 1-year mortality in an outpatient population.

Method

Rx-Risk Index mapping

The Rx-Risk Index consists of 46 comorbidity categories. For each Rx-risk category, medicines indicative of that condition were mapped (see table 1). This mapping was performed at the ATC classification level and performed by consensus between two pharmacists. If an individual had ≥1 dispensing for a medicine in a given category, then they were considered to have been treated (using medicines) for that comorbidity. Medications in both the Department of Veterans’ Affairs (DVA) and Pharmaceutical Benefits Scheme (PBS) datasets are coded using WHO ATC Classification System14 and PBS item codes.15

Table 1

List of Rx-risk comorbidity categories, corresponding Anatomical Therapeutic Chemical (ATC) codes, and score weightings in relation to 1-year mortality risk in Department of Veterans’ Affairs (DVA) and Pharmaceutical Benefits Scheme (PBS) data.

Data sources

The primary data source was the Australian Government DVA’s administrative claims database. This database contains details of all prescription medicines, medical and allied health services, and hospitalisations subsidised by DVA. The current treatment population consists of 223 181 members of the Australian veteran community, which includes veterans, war widows and widowers. The median age of the DVA treatment population is 75 years and 62% are men. DVA also maintains a client file containing gender, date of birth and date of death information.

External validation of the Rx-Risk Index was conducted using the PBS 10% sample of the Australian population. This dataset contains information on the dispensing of prescription medicines, and includes basic demographic information regarding gender, year of birth and year of death. It is maintained by the Australian Government Department of Human Services. The current treatment population consists of 1 346 340 members of the Australian community. The median age of the PBS treatment population is 43 years and 48% are men.

Patient and public involvement

Patients and public were not involved in this research.

Study population

The DVA study population included individuals with at least one healthcare encounter in the 6-month period from 1 July 2013 to 31 December 2013. A healthcare encounter included any of the following: a medication dispensing, a doctor’s visit or hospitalisation. Analysis was restricted to veterans who were DVA Gold Card holders prior to 1 July 2013 (ensuring they were eligible for all DVA subsidised services; thus, the dataset captured all their health claims), and were between the ages of 65 and 100 years at 1 January 2015 (n=135 406). Inclusion criteria for the PBS cohort were people with a healthcare encounter (defined as at least one medicine dispensing) in the 6-month period from 1 July 2013 to 31 December 2013, who were aged between 65 and 100 years at 1 January 2015 (n=303 135).

The primary outcome for this study was death recorded in 2015; hence, patients were only included if they were alive as at 1 January 2015. Rx-Risk Scores were calculated separately in each dataset using prescription claims for supplied medicines over a 1-year baseline period between 1 January 2014 and 31 December 2014.

Calculating Rx-Risk Scores and prescription counts

The full Rx-Risk Index has 46 comorbidity categories; however, tuberculosis and hepatitis B and C were removed from the Rx-Risk Index for this study as the number of individuals with these conditions in the DVA cohort was less than 10 so weights could not be generated. This resulted in 43 categories in the validation study. The following forms of the Rx-Risk Score were generated. The unweighted Rx-Risk Score was calculated as the count of the number of different comorbidity categories for which an individual was treated with a possible score ranging from 0 to 43. The weighted Rx-Risk Score was calculated by adding the 43 indicator variables to a logistic regression model with mortality as the outcome including age and gender as covariates. From this model, each comorbidity category was assigned a weight according to the statistical significance and magnitude of the OR generated from the logistic regression model (table 2).7 The weighted Rx-Risk Score for an individual was then the sum of the weighted indicator variables. As an example, the unweighted Rx-Risk Score for a patient who has two comorbidities ‘pain’ and ‘renal disease’ is 2, while their weighted Rx-Risk Score is 9 that is the sum of the weight for pain (3) and renal disease (6).

Table 2

Weighting algorithm used to score the Rx-Risk Index*

Three crude prescription counts were also calculated: (1) total number of prescriptions dispensed in the baseline period (1 January 2014–31 December 2014), (2) total number of unique medicines dispensed in the baseline period based on ATC codes, and (3) total number of unique medicines dispensed during the baseline period based on PBS item codes.15 The distinction between ATC and PBS codes was made because different strengths or formulations of the same medicine have the same ATC code but different PBS codes; for example, if a patient is dispensed two different strengths of the same medicine, this will be counted once in the total number of ATC medicines dispensed but twice in the total number of PBS medicines dispensed.

Statistical analysis

Primary analysis was performed in the DVA database. A baseline logistic regression model was calculated for mortality using age and gender as predictors. The comorbidity scores, individual comorbidities and crude prescription counts were added to the baseline model separately. Age, comorbidity scores and crude prescription measures were included in the models as continuous variables assuming linear associations. Models using the individual comorbidities were developed with an indicator variable included for the presence (1) or absence (0) of each individual Rx-isk category. The overall goodness of fit of each model was compared with the baseline model using the Akaike information criterion (AIC).16 The model with the lowest AIC value is considered the best fit. The difference between the AIC values of two models must be greater than 10 for one model to be considered superior to the other. Model discrimination was compared based on the c-statistic and the relative integrated discrimination improvement (IDI).17 The value of the c-statistic can range from 0 to 1, with 1 indicating perfect prediction and 0.5 indicating chance predictions. A c-statistic between 0.8 and 0.9 is generally considered as excellent, and between 0.7 and 0.8 acceptable.18 Using 1000 bootstrap samples, 95% CIs were calculated for the c-statistics.9

Internal validation of weighted scores

We used 10-fold cross validation to internally validate the binary logistic regression model used to calculate the Rx-risk category weights. This subsets the DVA cohort, using random sampling without replacement, into 10 equal folds. Each fold is a 10% subset of the DVA cohort. The training set was ninefold and was used to calculate Rx-risk category weights. The calculated weights were then applied to the testing set (ie, fold left out of training set). A binary logistic regression model of 1-year mortality including age, gender and weighted Rx-risk was built separately for the training set and testing set and the c-statistics recorded. This process was repeated 10 times until each fold was used as the testing set once. This resulted in 20 c-statistics being calculated, 10 each on the training and testing set. The average c-statistic was recorded for each set.

Sensitivity analyses

Two sensitivity analyses were carried out. In the first sensitivity analysis, we used the lower confidence limit of the OR to determine the Rx-risk category weights rather than the estimated OR itself. We did this as some comorbidities are uncommon in this population resulting in large CIs. In the second sensitivity analysis, we calculated the Rx-risk category weights (based on the OR) generated from 5000 bootstrap samples. For each Rx-risk category, the weight was calculated as the median of the 5000 weights for that category generated in each bootstrap sample. We did this sensitivity analysis to test the stability of the weights.

External validation

To determine the external validity of the weights, the calculated Rx-risk category weights derived from the DVA dataset were applied to the PBS cohort. The AIC model fit and c-statistics were calculated to determine the validity of the model.

Results

Table 1 presents the ATC-mapped Rx-risk categories, derived empirical weights and number of treated individuals in the DVA populations. The most frequent comorbidities identified by the Rx-Risk Index in the DVA cohort were hypertension (53%), gastro-oesophageal reflux disease (GORD) (51%), hyperlipidaemia (50%) and conditions treated with antiplatelets (39%). In the DVA cohort of 135 406 people, with a mean age of 83 years (SD 9.5), 47% were men. The median unweighted Rx-Risk Score was 5 (IQR 3–7) and the median weighted Rx-Risk Score was 3 (0–6).

The baseline model, comprising only age and gender, predicted 1-year mortality moderately well in the DVA cohort (c-statistic=0.738, 95% CI 0.734 to 0.742). The addition of Rx-risk to the model increased the performance of the model: unweighted Rx-risk c-statistic=0.751 (95% CI 0.747 to 0.754), IDI=14.0%, p<0.0001; weighted Rx-risk c-statistic=0.786 (95% CI 0.782 to 0.789), IDI=65.6%, p<0.0001; 43 individual comorbidities c-statistic=0.791, (95% CI 0.788 to 0.795), IDI=73.9%, p<0.0001. The model including the 43 comorbidity indicator variables had the lowest AIC (75 692), highest c-statistic (0.791) and the highest discrimination improvement (73.9%). The models including the weighted Rx-Risk Score or 43 comorbidity measures were better predictors of 1-year mortality than any of the crude prescription measures (table 3). Results of the internal 10-fold cross validation were consistent with the main analysis with an average c-statistic of 0.785 achieved over the 10 testing datasets compared with a c-statistic of 0.786 in the primary analysis.

Table 3

Comparison of different Rx-risk scoring and modelling methods to predict 1-year mortality in the Department of Veterans’ Affairs (DVA) population and external validation using the Pharmaceutical Benefits Scheme (PBS) population

In the sensitivity analysis using the lower 95% CI limit in the weighting algorithm, we found a very similar performance with a c-statistic of 0.787 compared with the weighting algorithm using the estimated OR (c-statistic 0.786). In the second sensitivity analysis in which the weights were calculated from 5000 bootstrap samples, we found a similar c-statistic (0.786) compared with the primary analysis.

The PBS cohort consisted of 303 135 people, with a mean age of 75 years (SD 7.4) and 45% were men. Similar to the DVA cohort, the most frequent comorbidities identified by the Rx-Risk Index in the PBS cohort were hypertension (54%), hyperlipidaemia (52%) and GORD (41%). When the DVA-derived Rx-isk category weights were applied to the PBS population, we found a c-statistic of 0.833 (table 3) showing good external validity.

Discussion

This paper presents the Rx-Risk Index with each comorbidity category mapped to medicines at the ATC classification level using clinical expertise. The mapped index provides a resource for researchers working with health claims data using ATC codes or that have mappings to the ATC codes to calculate comorbidity, based on prescription medicine dispensing’s in an outpatient population. We have shown that the updated Rx-risk was highly predictive of 1-year mortality in both the populations examined and is a valid measure of comorbidity in an outpatient population and is therefore likely to be useful in a range of observational data settings.

All forms of Rx-risk predicted mortality better than just age and sex alone. The best results for predicting 1-year mortality were achieved when modelling Rx-risk as individual comorbidities or as a weighted score. The unweighted Rx-Risk Score had similar performance to simple prescription medicine counts. Internal and external validation showed that the weighted index was predictive of 1-year mortality within the veteran population and show good external validity when applied to a general population setting. These results suggest that the weighted Rx-Risk Score is likely to be generalisable to other populations. Making the ATC map available to researchers will facilitate the use of the Rx-risk in place of comorbidity estimated simply by prescription counts.

The Rx-Risk Index has been updated accounting for the introduction of new medicines to the market, making this index a useful resource for researchers. The updated Rx-risk has 46 comorbidities; however, three (tuberculosis, and hepatitis B and C) were removed in the analysis stage as there were insufficient cases in the DVA cohort. A younger or larger sample may have allowed these comorbidities to be assessed. For consistency, these comorbidities were also excluded from the analysis in the PBS cohort despite there being few but sufficient cases. The Rx-Risk Index has been updated and mapped to ATC codes based on medicine availability in Australia; hence, modifications may be required for use in other health systems.

The healthcare encounter inclusion criteria were not the same for both cohorts. The DVA cohort included people with a hospitalisation or general practitioner visit, while the PBS cohort was limited to those with a prescription dispensing only. However, the difference across populations is small, as 96.7% of the DVA cohort had a medication dispensed in the 6-month selection period.

Comorbidity scores are often used in observational studies to reduce the potential for confounding. The advantage of these summary scores is that they simplify the inclusion of individual covariates for each comorbidity into a single summary score. This is a particular advantage when sample sizes are small or the outcome under study is rare. Our analysis demonstrated the performance of the weighted Rx-Risk Score as well as the model which included each individual comorbidity category. Additionally, including all individual comorbidities as indicator variables may not be appropriate in some studies, such as those looking at the effect of a particular treatment on an adverse event when the treatment itself is included in the construction of the comorbidity score. For example, when determining whether the risk of non-steroidal anti-inflammatory drugs (NSAIDs) is associated with gastrointestinal bleeds, it would not be correct to adjust for inflammation/pain as an indicator as the medicines mapped to this comorbidity include NSAIDs. In this scenario, it would be advisable to remove inflammation/pain as an indicator or use the weighted Rx-Risk Score. Lastly, although the weighted Rx-Risk Score performed well in the PBS dataset, which suggests that the weights have a good external validity, the Rx-risk category weights derived in this study may not be applicable to all external populations. We limited our cohorts to patients over 65 years of age so factors that are predictors of mortality in this age group may not be predictors in a younger group.

Conclusion

The updated Rx-Risk Comorbidity Score is a valid measure of comorbidity and strongly predicted 1-year mortality in an outpatient population. The weighted Rx-Risk Score was found to be valid in an external population and may be useful in practice to adjust for confounding in observational studies using medication claims data.

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
  18. 18.
View Abstract

Footnotes

  • Contributors Research area and study design: NLP, MK, ER and EER. Data acquisition: NLP, MK and JDB. Data analysis and interpretation: NLP, MK, ER, AK-C and LMKE. Statistical analysis: NLP and MK. Mapping of the ATC codes to the Rx-risk categories: JDB, LMKE and EER. All authors drafted, edited and approved the final manuscript.

  • Funding This work was funded by the Australian Government Department of Veterans’ Affairs (DVA) as part of the Veterans’ Medicines Advice and Therapeutics Education Services (Veterans’ MATES) programme. EER is supported by NHMRC GNT 1110139. DVA reviewed this manuscript before submission but had no role in the design or conduct of this research.

  • Competing interests None declared.

  • Patient consent Not required.

  • Ethics approval This research was approved by the Australian Government Department of Veterans’ Affairs Human Research Ethics Committee and the University of South Australia Human Research Ethics Committee.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data sharing statement Data are available through the Australian Government Department of Veterans’ Affairs.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.