Objectives Multimorbidity, defined as the presence of 2 or more chronic medical conditions in an individual, is associated with poorer health outcomes. Several multimorbidity measures exist, and the challenge is to decide which to use preferentially in predicting health outcomes. The study objective was to compare the performance of 5 count-based multimorbidity measures in predicting emergency hospital admission and functional decline in older community-dwelling adults attending primary care.
Setting 15 general practices (GPs) in Ireland.
Participants n=862, ≥70 years, community-dwellers followed-up for 2 years (2010–2012). Exposure at baseline: Five multimorbidity measures (disease counts, selected conditions counts, Charlson comorbidity index, RxRisk-V, medication counts) calculated using GP medical record and linked national pharmacy claims data.
Primary outcomes (1) Emergency admission and ambulatory care sensitive (ACS) admission (GP medical record) and (2) functional decline (postal questionnaire).
Statistical analysis Descriptive statistics and measure discrimination (c-statistic, 95% CIs), adjusted for confounders.
Results Median age was 77 years and 53% were women. Prevalent rates ranged from 37% to 91% depending on which measure was used to define multimorbidity. All measures demonstrated poor discrimination for the outcome of emergency admission (c-statistic range: 0.62, 0.65), ACS admission (c-statistic range: 0.63, 0.68) and functional decline (c-statistic range: 0.55, 0.61). Medication-based measures were equivalent to diagnosis-based measures.
Conclusions The choice of measure may have a significant impact on prevalent rates. Five multimorbidity measures demonstrated poor discrimination in predicting emergency admission and functional decline, with medication-based measures equivalent to diagnosis-based measures. Consideration of multimorbidity in isolation is insufficient for predicting these outcomes in community settings.
- frail elderly
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
Comparison of five count-based measures of multimorbidity in predicting emergency admission and functional decline.
The prospective study design and calculation of the medication exposure using linked pharmacy claims data are strengths of this study. Some previous studies have been limited by study design (cross-sectional) and data available (eg, self-report).
This study is set in primary care, which is important as many studies to date have been conducted in secondary care.
A small number (n=21, 3%) of study participants had some missing data for the outcome measure of functional decline and were excluded.
Characterising the impact of multimorbidity, defined as the presence of two or more chronic medical conditions in an individual, in predicting poorer health outcomes for community-dwelling older people has emerged as an important concept in the last decade.1 A challenge is deciding which multimorbidity measure to use in research and clinical practice. Several measures have been developed and tested , including, most commonly, simple disease counts, the Charlson comorbidity index and the Adjusted Clinical Groups (ACG) system.2 However, primary care research comparing different measures for relevant patient outcomes is relatively limited. To date, studies have largely focused on mortality and cost outcomes with methodological limitations (eg, cross-sectional study design) and using a narrow definition of multimorbidity (eg, index chronic condition plus another condition), therefore reducing generalisability.3–8
A systematic review concluded that diagnosis-based measures (eg, disease counts) perform best in predicting mortality outcomes, whereas medication-based indices (eg, RxRisk-V) demonstrated better predictive accuracy for healthcare usage.9 However, these findings were largely based on the application of individual measures in different populations rather than direct comparison of measures in the same population.9 In addition, there is a paucity of research examining the performance of different multimorbidity measures in predicting other relevant outcomes, such as emergency admission and functional decline.
Using a count-based approach, such as simple disease or medication counts, in measuring multimorbidity has several advantages in that it is reasonably simple to apply and replication is more straightforward, important for achieving consistent definitions of multimorbidity across research studies. The aim of this study was to compare the performance of different count-based measures of multimorbidity in predicting emergency hospital admission and functional decline in older community-dwelling adults attending primary care.
The STrengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines were adhered to in the conduct and reporting of this cohort study.10 This study was a secondary analysis of prospectively collected data examining the association between potentially inappropriate prescribing and adverse health outcomes in older people.
Study design and study population
This is a 2-year prospective cohort study of older community-dwelling patients recruited from 15 general practices (GP) in Ireland (2010–2012). At baseline in 2010, a proportionate stratified random sampling approach was used to recruit patients for study participation. Study inclusion criteria were: (1) age ≥70 years on 1 January 2010 and (2) in receipt of a valid general medical services (GMS) card. Approximately 96% of people aged ≥70 years are in receipt of a GMS card which provides free access to public health services and prescribed medications, subject to a maximum copayment of €25 monthly.11
The following exclusion criteria were applied: (1) receiving palliative care; (2) cognitive impairment at the level that would affect their ability to complete the outcome measure (defined as Mini Mental State Examination ≤20); (3) significant hearing/speech/visual impairment; (4) currently experiencing a psychotic episode; (5) hospitalised long-term, in a nursing home, homeless or in sheltered accommodation; and (6) recent bereavement (within 4 weeks). Each participant's GP applied the exclusion criteria and determined eligibility for participation at baseline and at follow-up.
Exposure of interest: measures of multimorbidity
All exposures of interest were measured at baseline (2010) using the GP medical record to identify medical diagnoses and linked data from the national pharmacy claims database to determine prescribed medications.
Total disease counts
This measure involves a simple count of chronic medical conditions. Chronic medical conditions were recorded from the GP electronic medical record over a prespecified 6-month period. The International Classification of Primary Care (ICPC-2) definition of a chronic disease was used.12 Multimorbidity was defined as the presence of two or more chronic medical conditions in an individual.13
Selected conditions disease count
Barnett et al14 developed a set of 40 chronic medical conditions for inclusion when measuring multimorbidity using a large Scottish primary care cohort (n=1.75 million adults). Chronic conditions were selected based on health impact and prevalence. For morbidities with lifelong implications (eg, congestive heart failure), the presence of the condition was on the basis of it ever being recorded in the GP record. However, for other conditions where lifelong remission/cure is possible, the morbidity had to be recorded in a defined period (eg, cancer in the previous 5 years) or in terms of relevant prescribing (eg, epilepsy currently treated).14 Multimorbidity was defined as the presence of two or more of these specified conditions.
Charlson comorbidity index
This measure was developed in an inpatient US population to predict mortality and includes 19 conditions that have been selected and weighted in relation to their association with mortality risk.15 Medical conditions recorded from the GP record were reviewed and each study participant was assigned a score based on the components of the index.
Number of dispensed medication classes
This measure was a count of the number of dispensed medication classes and was calculated using linked pharmacy claims data from the national Health Services Executive (HSE)-Primary Care Reimbursement Scheme (PCRS) pharmacy claims database. This database has complete coverage for medications dispensed to patients with a GMS card. Patients gave consent for their pharmacy claims data to be linked to their unique GMS number. In the HSE-PCRS, dispensed prescriptions are coded using the WHO Anatomical Therapeutic Chemical (WHO-ATC) classification system and defined daily doses, strength, quantity of medication and mode of administration is available. Polypharmacy was defined as the concurrent prescription of ≥4 medications and high-risk polypharmacy as ≥10 medications.9
The RxRisk-V was developed specifically for older people and classifies patients' chronic medical conditions based on the WHO-ATC medication classification system of their dispensed medications.16 In validation studies, the RxRisk-V has demonstrated criterion validity and reliability when compared to patients' medical diagnoses.2 The RxRisk-V was calculated using the linked HSE-PCRS pharmacy claims data.
Emergency admission was defined as ‘unplanned overnight stay in hospital’ and was recorded from review of each participant's GP electronic medical record.17 The number of emergency admissions, reason for admission and length of hospital stay were recorded over 2 years of follow-up (2010–2012). In addition, ambulatory care sensitive (ACS) admissions were identified. These are a subset of all emergency admissions that occur due to select medical conditions (eg, asthma, congestive heart failure and cellulitis) that are considered more amenable to prevention through primary care management.18 A list of included ACS conditions is provided in online supplementary appendix 1.
The short functional survey (SFS), a subset of the Vulnerable Elders Survey (VES-13), was used as the measure of functional status.19 The VES-13 was developed in 1993 in an older Medicare population in the USA to predict functional decline and death over 2 years.19 It includes items relating to age, self-rated health, physical function and the SFS. The SFS comprises five questions relating to personal care, mobility and activities of daily living and has been validated to predict functional decline with similar accuracy as the longer 12-item Activities of Daily Living scale.20
Disability in the activity was defined as ‘having difficulty and receiving help to perform the activity’ or ‘not doing the activity due to their health’. Functional ability was assumed for all other possible responses to the questions (‘no difficulty’, ‘difficulty but does not receive help’ or ‘not doing the activity but for reasons other than health’). Disability in each item was awarded a score of 1, so a person unable to complete all five tasks was scored 5. Functional decline was defined as an increase in the SFS score by ≥1 point between baseline and follow-up. In addition, any study participant who entered a nursing home during the follow-up was also considered to have experienced functional decline.
Age, gender and deprivation were included in adjusted analyses, in keeping with previous studies.8 ,21 Age and gender were determined from the GP medical record. Each patient's address was geocoded according to electoral division and patient deprivation was estimated from the deprivation score of the patient's address. This approach is based on the Small Area Health Research Unit (SAHRU), which shares similarities with the Townsend and Carstairs deprivation indices widely used in the UK.22
Baseline descriptive statistics of the cohort are described. The performance of each measure was assessed by investigating the discrimination (equivalent to the area under the receiving operating characteristic (ROC) curve). This score ranges from 0 to 1, where a value of 0.5 represents the same performance as chance, 0.5–0.7 represents poor model discrimination, 0.7–0.9 represents reasonable discrimination and ≥0.9 represents excellent discrimination.23 Discrimination was assessed using the non-parametric method by calculating a c-statistic with 95% CIs for each measure considered as continuous variables.
Different cut-points within the same measure were then examined to determine which offered optimal discrimination for the outcome of interest (eg, 0 conditions vs 1 condition, 0–1 conditions vs 2 conditions etc). Once the optimal cut-point was established for each measure, all five measures were then compared to examine which offered the best discrimination for each of the outcomes of interest. These measures were then adjusted for age, gender and deprivation to see how this impacted overall predictive accuracy. A series of ROC plots were generated to examine visually the differences between the measures in predicting the outcomes of interest. Model goodness of fit was assessed using the Hosmer-Lemeshow statistic. All analyses were conducted using Stata V.13. (StataCorp, Texas, USA)
Of 904 baseline study participants, 862 (95%) were included in this 2-year follow-up study (see table 1).
Median age was 77 years and 53% were women. Participants were excluded if there was incomplete hospital admission data for the follow-up period as follows: 19 moved GP practice; 14 moved to a nursing home; and 9 GP medical record reviews were missing. A sensitivity analysis was also conducted, excluding participants who had died during the follow-up (n=53), which made no appreciable difference to the overall results.
Exposure: multimorbidity measures
The prevalence of patients with multimorbidity as defined by each of the five multimorbidity measures is presented in table 2.
A total of 626 (73%) patients met the definition for multimorbidity according to the total disease counts measure and 484 (56%) according to the selected condition count measure. Three hundred and fourteen study participants (37%) had ≥2 of the Charlson index conditions, 789 (91%) ≥2 RxRisk-V conditions and 73% (n=632) prescribed ≥4 medications with 13% (n=119) prescribed ≥10 medications.
Outcome: (1) Emergency admission
A total of 246 study participants (29%) were admitted as an emergency at least once during the 2-year follow-up. Of these, 159 (18%) were admitted once, 56 (7%) were admitted twice and 31 (4%) were admitted ≥3 times. A total of 110 (13%) had an ACS emergency admission. Fifty-three study participants (6%) died during the follow-up.
Overall unadjusted and adjusted c-statistics (95% CIs) are presented in table 3.
All measures had similar adjusted predictive accuracy (c-statistic range: 0.62 to 0.65) for emergency admission. The selected condition count (c-statistic: 0.63 (95% CI 0.57 to 0.69)) and the RxRisk-V (c-statistic: 0.63 (95% CI 0.56 to 0.69)) demonstrated the greatest predictive accuracy for this outcome. For ACS admission, overall measure performance was marginally better (c-statistic range: 0.63 to 0.68). The Charlson index (c statistic: 0.67 (95% CI 0.58 to 0.75), RxRisk-V (c-statistic: 0.67 (95% CI 0.61 to 0.73) and medication count (c-statistic: 0.67 (95% CI 0.59 to 0.75) all demonstrated similar poor predictive accuracy for this outcome.
Determining an optimal measure cut-point for emergency admission
c-Statistics at different cut-points for emergency admission are presented in online supplementary appendix 2. The optimal count cut-points were: (1) RxRisk-V ≥6; (2) number of medications ≥6; (3) disease count ≥3; (4) selected condition count ≥2; and (5) Charlson index ≥1. These five measures at their optimal cut-points and adjusted for age, gender and deprivation were then compared for the outcome of emergency admission. Overall, there was no statistically significant difference between any of the five measures for predicting emergency admission (p=0.24) (see online supplementary appendix 3).
Identical statistical analysis was conducted for the outcome of ≥1 ACS emergency admissions and c-statistics for different cut-points. Overall, there was no statistically significant difference between any of the measures for predicting ACS admission (p=0.95) (see online supplementary appendix 4). The optimal count cut-points were: (1) RxRisk-V ≥5; (2) number of medications ≥6; (3) disease count ≥3; (4) selected condition count ≥2; and (5) Charlson index ≥1. The five multimorbidity measures at their optimal cut-points, adjusted for age, gender and deprivation, were then compared for the outcome of ACS admission (see online supplementary appendix 5).
Outcome: (2) Functional decline
Of 791 patients eligible, a total of 673 (85%) patients completed a postal questionnaire at the follow-up which included the SFS. There were 21 (3%) with missing data for ≥1 items of the SFS and these patients were excluded from the analysis. A further 14 study participants had been admitted to a nursing home and were considered to have experienced functional decline. Therefore, the study sample was n=666. Online supplementary appendix 6 presents the differences between postal questionnaire respondents at follow-up compared to non-respondents. Non-respondents were older, were taking a greater number of medications and were more socioeconomically deprived.
Of questionnaire respondents, a total of 56 (8.4%) reported a decline in ≥1 of the SFS items at follow-up compared to baseline. An additional 14 participants' were admitted to a nursing home resulting in a total of 70 participants' (10.5%) with functional decline. Overall unadjusted and adjusted analyses for this outcome are presented in table 4.
In unadjusted analysis, all measures demonstrated poor discrimination (c-statistic range: 0.57 to 0.62). Discrimination performance was poor and following adjustment for age, gender and deprivation the best-performing measure for this outcome was the RxRisk-V (c-statistic: 0.61 (95% CI 0.55 to 0.67)).
Determining an optimal measure cut-point for functional decline
c-Statistics at different cut-points for the outcome of functional decline are presented in online supplementary appendix 7. The optimal count cut-points were as follows: (1) RxRisk-V ≥5, (2) number of medications ≥5, (3) disease count ≥3, (4) selected condition count ≥3 and (5) Charlson index ≥3. The performance of measures at their optimal cut-points, adjusted for age, gender and deprivation, were then compared for the outcome of functional decline. There was no statistically significant difference between any of the measures for predicting functional decline (p=0.40) (see online supplementary appendix 8).
Different count measures of multimorbidity demonstrated similar poor discrimination for the outcome of emergency admission in this study with the Rx-Risk-V performing best (c-statistic: 0.63). Predictive accuracy for ACS admission was marginally better and three measures (RxRisk-V, number of medications and Charlson index) demonstrated equivalent predictive accuracy (c-statistic 0.67). All measures demonstrated poor discrimination for functional decline over 2-year follow-up (c-statistic range: 0.55–0.61), indicating that measuring multimorbidity in isolation is insufficient if aiming to predict these outcomes.
Regarding multimorbidity prevalence, a total of 626 participants (73%) met the definition for multimorbidity using the total disease counts measure and 484 (56%) according to the selected condition count measure. Three hundred and fourteen (37%) had ≥2 of the Charlson comorbidity index conditions. For medication count measures, a total of 789 participants (91%) had ≥2 chronic RxRisk-V conditions and 73% (n=632) met the definition for polypharmacy (≥4 medications) with 13% (n=119) prescribed ≥10 medications (high-risk polypharmacy). These findings indicate that multimorbidity prevalence in this older community-based cohort varies considerably depending on the measure selected to define it. Considering more than one drug will often be needed to treat a single condition, it seems clinically intuitive that a higher medication count is required to define multimorbidity.
Comparison with existing literature
Few primary care studies have compared the performance of different multimorbidity measures for the outcome of emergency admission. One US study (n=14, 192 adult veterans) reported similar performance of the RxRisk, Charlson index and ACG system in predicting hospital admission (c-statistic range: 0.61–0.64).24 Additional US population-based studies found that the RxRisk-V predicted future healthcare use better than diagnosis-based measures.16 ,25 A Canadian study, which used administrative data (n=137, 700, aged ≥65 years) and examined different measures of multimorbidity in predicting hospital admission, reported only modest discrimination for the best-performing measure of simple disease counts (c-statistic 0.67).26 In the UK, a large-scale cross-sectional study (n=95,372 adults) reported that medication count was the more powerful predictor of future primary care consultations when compared to diagnosis-based measures.27
The current study adds to the literature in this area and suggests that while all measures demonstrate poor discrimination, medication-based measures, such as the RxRisk-V, performed marginally better than diagnosis-based measures in predicting emergency and ACS admissions. For research purposes, these measures can be applied to pharmacy claims databases rather than requiring medical record review and simple medication counts are easy to conduct in clinical practice. However, it is important to highlight that emergency admission is an inherently difficult outcome to predict accurately.28 Existing admission risk prediction models, which include a variety of clinical, socioeconomic and prior healthcare usage variables, rarely achieve model discrimination of ≥0.8, and the performance of the various multimorbidity measures should be judged in this context.29
There has been very limited research comparing the performance of different multimorbidity measures in predicting functional decline. One Italian study (n=633 aged ≥65 years) measured incident basic activities of daily living (BADL) disability over 5-year follow-up.8 The functional status was assessed using a modified version of the Guralnik's lower physical performance battery and 9.6% developed incident disability. Disease counts had the highest predictive value (c-statistic 0.85).8 However, all study participants had an index condition of congestive heart failure which limits the generalisability of this study to typical primary care populations.
Interestingly, a recent analysis argues that increasing levels of multimorbidity in older people may be related to increased diagnosis through screening and more vigilant recording of diagnoses rather than worsening health.30 This is supported by the fact that older people's self-rated health has not declined in tandem with increasing morbidity levels.30 If this is the case, then perhaps it is not surprising that count-based multimorbidity measures in isolation are not enough to predict emergency admission and functional decline. What is needed is a way of incorporating the burden and severity of conditions into multimorbidity measures to allow better capture of disease combinations that predict poorer health outcomes.31 Research efforts to conceptualise and capture the burden of multiple diseases for patients are ongoing.32 ,33
Clinical and research implications
The current study indicates that the choice of measure for multimorbidity will have implications in terms of prevalence estimates. Using the cut-point of ≥2 to define multimorbidity for the RxRisk-V measure would result in almost the entire study population categorised as multimorbid (91%), while using the selected condition list would identify a smaller proportion (56%). Existing multimorbidity intervention studies have largely used diagnosis-based inclusion criteria, with a smaller number of studies using this approach in combination with medication count.34 An example is the OPTIMAL primary care randomised controlled trial that recruited patients with two or more chronic conditions prescribed four or more medications for a self-management intervention that succeeded in improving activity participation.35 Using this combined approach seems pragmatic and should ensure patients with more complex multimorbidity are included. Future research studies need to carefully consider participant selection, choice of intervention and relevant outcomes in community-based multimorbidity trials.36
A systematic review examining multimorbidity measures included in risk prediction models designed to predict future hospital admission and readmission reported that of 21 studies which included a multimorbidity measure, a total of 15 (71%) used the Charlson comorbidity index (or adapted version).37 Only two studies used a medication-based measure of multimorbidity.37 The current study indicates that medication-based measures may be considered for use in risk prediction models with this outcome of interest. However, as mentioned, higher cut-points are needed to achieve optimal discrimination.
Strengths and limitations
This prospective primary care study adds to the limited literature in comparing different multimorbidity measures to predict health outcomes for older community-dwelling people. The study population was not selected based on the presence of any one index condition, which improves the generalisability of the findings. The robustness of the multimorbidity measures calculation is enhanced by diagnosis measurement using GP electronic medical record data for and use of linked pharmacy claims data for the medication-based measures. The outcomes of emergency admission and ACS admission were recorded from review of the GP electronic medical record. ACS emergency admissions are increasingly of interest due to their perceived preventability through primary care interventions. However, only a limited number of risk prediction models have been developed specifically to identify ACS admissions.29 Understanding which multimorbidity measure to include preferentially in predicting ACS admission is important and topical.
Regarding study limitations, the study involves 15 general practices from Ireland and as such may not be generalisable to other settings. As one of the outcome measurements depended on participants' filling in a postal questionnaire those with cognitive impairment at the level that would impact their ability to complete the outcome measure (defined as Mini Mental State Examination ≤20) were excluded from this study which may also affect on generalisability of the findings. Accuracy of diagnosis information was dependent on accurate recording of information taken from the GP electronic medical record. All GP medical record data was recorded by the same researcher and a random sample of 10% of all data was double checked by an independent reviewer with extensive data cleaning undertaken to maximise accuracy and reducing the likelihood of misclassification bias. A small number (n=21, 3%) of study participants had some missing data for the outcome measure of functional decline and were excluded.8
This study indicates that though all multimorbidity measures demonstrate poor discrimination, medication-based measures are equivalent to diagnosis-based measures in predicting emergency admission. The choice of multimorbidity measure used may have a significant impact on prevalence. Consideration of multimorbidity in isolation is insufficient in predicting which primary care patients are most likely to experience emergency admission or functional decline.
Thank you to Dr Caitriona Cahir who led the baseline study data collection. Thank you to the Health Services Executive (HSE)-Primary Care Reimbursement Scheme (PCRS) for the use of pharmacy claims database.
Contributors EW led the study, collected the follow-up data, did the statistical analysis and wrote the paper. RMD offered methodological guidance, conducted the statistical analysis and contributed to the paper. KB offered significant methodological guidance to the planning of the study, conducted the data analysis and contributed to the final paper. TF provided clinical and methodological guidance and contributed to the final paper. SMS provided clinical and methodological guidance and contributed to the final paper.
Funding This work was supported by the Health Research Board (HRB) of Ireland under the Research Training Fellowship for Healthcare Professionals award, grant number HPF/2012/20 for EW. This research was conducted as part of the HRB Scholar's programme in Health Services Research under grant number PhD/2007/16 at the HRB Centre for Primary Care Research, grant HRC/2007/1.
Competing interests None declared.
Patient consent Obtained.
Ethics approval Ethical approval for this study was granted by the Royal College of Surgeons in Ireland (RCSI) Human Research Ethics committee and all participants gave informed consent prior to participating.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement The data set supporting the conclusion of this article is not available for sharing as it is subject to ongoing analysis by the authors.