Article Text

Original research
Systematic review on the instruments used for measuring the association of the level of multimorbidity and clinically important outcomes
  1. Eng Sing Lee1,2,
  2. Hui Li Koh1,
  3. Elaine Qiao-Ying Ho3,
  4. Sok Huang Teo1,
  5. Fang Yan Wong1,
  6. Bridget L Ryan4,5,
  7. Martin Fortin6,
  8. Moira Stewart5
  1. 1Clinical Research Unit, National Healthcare Group Polyclinics, Singapore
  2. 2Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore
  3. 3Saw Swee Hock School of Public Health, National University of Singapore, Singapore
  4. 4Department of Epidemiology and Biostatistics, Western University Schulich School of Medicine and Dentistry, London, Ontario, Canada
  5. 5Centre for Studies in Family Medicine, Department of Family Medicine, Western University Schulich School of Medicine and Dentistry, London, Ontario, Canada
  6. 6Department of Family Medicine and Emergency Medicine, Université de Sherbrooke, Sherbrooke, Quebec, Canada
  1. Correspondence to Dr Eng Sing Lee; Eng_Sing_LEE{at}nhgp.com.sg

Abstract

Objectives There are multiple instruments for measuring multimorbidity. The main objective of this systematic review was to provide a list of instruments that are suitable for use in studies aiming to measure the association of a specific outcome with different levels of multimorbidity as the main independent variable in community-dwelling individuals. The secondary objective was to provide details of the requirements, strengths and limitations of these instruments, and the chosen outcomes.

Methods We conducted the review according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines (PROSPERO registration number: CRD42018105297). We searched MEDLINE, Embase and CINAHL electronic databases published in English and manually searched the Journal of Comorbidity between 1 January 2010 and 23 October 2020 inclusive. Studies also had to select adult patients from primary care or general population and had at least one specified outcome variable. Two authors screened the titles, abstracts and full texts independently. Disagreements were resolved with a third author. The modified Newcastle-Ottawa Scale was used for quality assessment.

Results Ninety-six studies were identified, with 69 of them rated to have a low risk of bias. In total, 33 unique instruments were described. Disease Count and weighted indices like Charlson Comorbidity Index were commonly used. Other approaches included pharmaceutical-based instruments. Disease Count was the common instrument used for measuring all three essential core outcomes of multimorbidity research: mortality, mental health and quality of life. There was a rise in the development of novel weighted indices by using prognostic models. The data obtained for measuring multimorbidity were from sources including medical records, patient self-reports and large administrative databases.

Conclusions We listed the details of 33 instruments for measuring the level of multimorbidity as a resource for investigators interested in the measurement of multimorbidity for its association with or prediction of a specific outcome.

  • primary care
  • protocols & guidelines
  • general medicine (see Internal Medicine)

Data availability statement

Data sharing not applicable as no datasets generated and/or analysed for this study. All data relevant to the study are included in the article or uploaded as supplemental information. There are no additional, unpublished data arising from this research.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Strengths and limitations of this study

  • This review builds on Huntley et al’s 2012 review article and provides an updated, comprehensive list of instruments that measure levels of multimorbidity in community-dwelling individuals.

  • A thorough literature search of three major electronic databases was conducted with the involvement of a health science librarian.

  • The review is reported based on Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.

  • This review excluded non-English language articles and grey literature.

Background

Multimorbidity is defined as the co-occurrence of two or more chronic medical conditions in an individual.1 It is a growing public health challenge and accounts for most of the expenditures in the healthcare system.2 The complex interactions of several coexisting diseases have profound implications on individuals3 4 and their healthcare providers.5 6

There are multiple instruments for measuring multimorbidity and many of them do not usually specify the severity of individual conditions.7 No gold standard multimorbidity measurement instrument exists and there is also no agreed categorisation of the available instruments. Sarfati8 9 classified the various measurement instruments into four broad approaches. They are as follows: (1) by simple counts of individual conditions (ie, Disease Count), (2) by organ or system-based approaches, (3) by weighting conditions and combining them into indices and (4) by other miscellaneous approaches. Most of these measurements are used to measure the prevalence or patterns of multimorbidity. However, they can also be used to predict an outcome or to evaluate an intervention for a desired outcome. A set of core outcomes of multimorbidity (COSmm) was proposed after consulting a panel of international experts in multimorbidity intervention studies using a Delphi process.10 Core outcome sets represent the minimum that should be measured and reported in all clinical trials of multimorbidity.11

Huntley et al12 published a systematic review in 2012 describing the instruments used to measure the morbidity burden in primary care and the general population. They found 17 different instruments from 194 articles. The most widely used instruments and those with the most significant evidence of validity were the Charlson Comorbidity Index (CCI), Disease Count and the Adjusted Clinical Groups (ACG) system.12 However, this review was conducted in 2009 and multimorbidity research has increased exponentially since then.

The present review was to build on the review article by Huntley et al12 in order to provide a current and comprehensive list of instruments that measure levels of multimorbidity for community-dwelling individuals. We used the term ‘level of multimorbidity’ to refer to the combined effects of multiple conditions on an individual. The main objective of this review was to list instruments for measuring the levels of multimorbidity. We specifically look for studies that measure the association of a clinically important outcome with different levels of multimorbidity as the main independent variable in community-dwelling individuals. Our second objective was to provide details of the requirements, strengths and limitations of these instruments, and the chosen outcomes in the studies so that clinicians and researchers can select or develop instruments that match their needs for predicting a specific outcome.

Methods

A protocol for this systematic review (CRD42018105297) was published online on PROSPERO.13 We searched MEDLINE, EMBASE, CINAHL and also manually searched the Journal of Comorbidity for potential studies. The medical subject headings and keywords used for the search are shown in online supplemental appendix 1.

We selected studies that included (1) adult patients from primary care or the general population as the majority of patients with multimorbidity are managed by primary care physicians14; (2) at least one specified outcome variable; and (3) published full-text articles from 1 January 2010 to 23 October 2020. Studies were excluded if they (1) selected patients from the hospital or nursing home only or patient data were drawn solely from the hospital or the nursing home; or (2) selected patients with an index condition; or (3) used level of multimorbidity as a covariate and not the main independent variable; or (4) were not written in English. We did not include a specific definition of multimorbidity instrument because, given a lack of consensus in the literature on the use of this term, we wanted to include a diverse range of studies on the above topic.

One reviewer (ESL) conducted a preliminary screen of titles and abstracts to exclude articles that were irrelevant. Abstracts of the remaining articles were screened independently by two reviewers (ESL and EQ-YH) according to the eligibility criteria. Disagreements were resolved through discussion until a consensus was reached. The full-text articles were then retrieved for the agreed list and independently assessed according to the eligibility criteria by the same reviewers. Disagreements were resolved through discussion with a third reviewer (TSH) until a consensus was reached. After agreement on the list of articles, the reference lists of included articles were hand-searched for additional eligible articles. We reported multimorbidity instruments that were described in all selected articles.

The risk of bias of the study design of selected articles was next appraised independently by three reviewers (ESL, EH and TSH) using the modified Newcastle-Ottawa Scale (NOS).15 16 Each article was assessed under the three broad categories: (1) selection, (2) comparability and (3) outcome (online supplemental appendices 2 and 3).

We contacted the authors, as needed, for additional information or clarification up to three times spaced 1 week apart. We contacted 25 authors and 19 of them replied. Any disagreements on the risk of bias were resolved among the three reviewers through regular meetings. HLK and FYW were responsible for tracking and updating the final outcome of the risk of bias assessment.

Patient and public involvement

This research was done without patient involvement. Patients were not invited to comment on the study design and were not consulted to develop patient relevant outcomes or to interpret the results. Patients were not invited to contribute to the writing or editing of this document for readability or accuracy.

Results

The number of included studies was 96, of which 69 were assessed to have low risk of bias. A summary of the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow chart is depicted in figure 1. Forty-eight studies selected participants from the general population and the other 48 studies selected participants from primary care. Most of the studies in this systematic review were from Europe and North America with very few Asian studies. There were 44 cohort studies, of which 36 were assessed to have low risk of bias, and 52 cross-sectional studies, of which 33 were assessed to have low risk of bias. We found 33 unique instruments from the 96 studies. The instruments were categorised according to Sarfati8 9 into (1) simple counts of individual conditions; (2) organ or system-based approaches; (3) conditions that have been weighted and combined into indices; and (4) other approaches. A total of 150 outcomes were reported from all the studies. No studies were excluded for an outcome that was not deemed to be clinically important. Online supplemental appendices 4 and 5 summarise the risk of bias assessment of each study. Table 1 provides a summary of the study design, population source, age group, multimorbidity measurements, outcome measures and risk of bias assessment of all the studies.

Table 1

Summary of included studies

Figure 1

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram.

Table 2 summarises the 33 instruments that were identified from all the studies. Table 3 provides a summary of multimorbidity instruments and their associations with the outcomes measured from all the included studies.

Table 2

Description of instruments used for measurement of multimorbidity and the data sources and resources required

Table 3

Summary of multimorbidity instruments and their associations with outcomes measured from all the included studies

Simple counts of individual conditions

Disease Count was based on the total number of all the conditions an individual had, usually from a prespecified list of chronic conditions. It was used in 59 out of the 96 studies (61.5%). Disease Count was reported to be associated with activity limitations, continuity of care, disability, healthcare cost, healthcare utilisation, medications, mental disorders, mortality, general health, physical function, quality of life and self-rated health (table 3).

Organ or system-based approaches

There were two instruments in this category. They were Cumulative Illness Rating Scale (CIRS)17 18 and Organ Systems with Chronic Disease Count (Organ-CDC).19

Weighted indices

There were 17 unique weighted instruments found in the included studies. The original CCI with its different modifications was the most frequently used instrument and was used in 29 studies. The CCI was based on Disease Count, but the 17 conditions were weighted originally based on their impact on 1-year mortality.20 The final score was derived by the summation of all the weighted conditions. There were many variations and modifications of the score including the addition of psychosocial factors. The CCI instrument was found to be associated with multiple outcomes other than 1-year mortality.

Most of the other weighted index instruments were novel, like the Multimorbidity-Weighted Index (MWI), in which the investigators built multivariable prognostic models from a set of potential predictor conditions and weighted the conditions based on an outcome of clinical interest. The most common outcomes chosen were mortality and physical function. Other outcomes included health expenditure,21 health utility index22 and severity of the most severe condition.23 The Multidimensional Multimorbidity Score (MDMS)24 was unique as it was weighted based on health behaviours and patient symptoms and not based on any specific outcome.

Other approaches to measuring multimorbidity

Other approaches included case-mix and pharmaceutical-based instruments. For case-mix approach, the ACG and Resource Utilisation Band were the most commonly used instruments.25 Most of the case-mix instruments required proprietary software licenses from the USA and obtained data from electronic medical records or administrative data. The Clinical Risk Groups instrument was similar but took into account the severity of individual conditions.26

The second group of instruments in this category was related to pharmaceutical data. The most frequent type was the unweighted Drug Count. The other three (Chronic Disease Score, A Veterans Association adapted pharmacy-based case-mix instrument like RxRisk-V and modified Probability of repeated admission risk prediction tool using RxRisk-V) were all weighted indices. Except for the Drug Count that was based on a self-report questionnaire, the rest required a prescription drug database to obtain the data.

Outcomes

We classified the 150 outcomes into 17 categories as reported in the core outcomes set of multimorbidity research (COSmm).10 The most commonly reported outcomes were healthcare use (n=45), mortality (n=18), health-related quality of life (n=18) and physical function (n=13). The different studies unanimously showed that higher levels of multimorbidity was associated with higher healthcare use and mortality, lower health-related quality of life and poorer physical function. Seven outcomes in the COSmm were not found in all the 96 studies. These were treatment burden, self-management behaviour, self-efficacy, adherence, communications, shared decision-making and prioritisation. There were 19 outcomes that were not described in the COSmm. These included cognitive function, risk of suicide, frailty and falls. The outcomes not found to have any association with the instruments for measuring the level of multimorbidity were preventive care,27 sickness absence episodes (female)24 and gait speed.28

Discussion

Summary of findings

Thirty-three unique instruments for measuring the level of multimorbidity were identified and categorised according to the classification by Sarfati.8 The most commonly used instrument was ‘Disease Count’. It was also the only instrument that was associated with the three essential outcomes from the core outcomes set of multimorbidity research (COSmm),10 that is, quality of life, mental health and mortality.

Comparison with previous research

Although the most common instrument identified in this systematic review was similar to that of Huntley et al,12 several instruments including Duke Severity of Illness Checklist (DUSOI) and Functional Comorbidity Index identified in their article were not found in this systematic review. The possible reasons for not finding these instruments in this review could be due to the lack of interest in the instrument by the research community in recent years (to our knowledge, the last publication using DUSOI was in 2004),29 or the exclusion of studies specifying an index condition.

Advantages and disadvantages of selected instruments

Disease Count

The advantage of using ‘Disease Count’ is its simplicity and the ease of data ascertainment with minimal resources required. However, using ‘Disease Count’ does not consider the severity of each condition where the complexity of multimorbidity may not be properly addressed.30 The other disadvantage noted was the lack of transparency in the operational definition of multimorbidity, especially regarding the list of conditions considered for multimorbidity and the cut-points used. Despite its simplicity, the level of multimorbidity measured using ‘Disease Count’ was the only instrument that was found to be associated with the three essential core outcomes (quality of life, mental health and mortality).

Weighted indices

The common weighted indices identified in this systematic review were CCI, Elders Risk Assessment (ERA), Elixhauser Index (EI) and MWI. These weighted indices were often used in prognostic models to build complex multivariable regression models in which the weights were calculated from hazard ratios, odds ratios or regression coefficients.31

The advantage of these weighted indices is that the weights allow the adaptation of an index to a specific outcome. An investigator could recalibrate the correct weight by creating a prognostic model to produce a contextualised instrument for a different setting. Prognostic models can provide clinically relevant risk stratification and help to allocate resources.32 The disadvantage of such indices is that calculated weights are greatly influenced by the population, outcomes used, and the instrument’s original conception and purpose, hampering the ability to compare across studies.

Case-mix

The ACG system has a good track record in the USA and several other countries, especially for measuring the outcomes of healthcare utilisation. However, the instrument is proprietary, and the exact algorithm of the instrument is not open to the public and may not be suitable in certain settings. The Clinical Risk Group (CRG) system has a good track record in Spain. It measures the severity of each condition and its algorithm is fully transparent. The common disadvantage of both systems is the financial costs involved in obtaining the license.

Pharmaceutical-based instruments

Medication-based indices include versions of the Chronic Disease Score,33 which later became known as the RxRisk,34 and its adaptation for use in the veteran population, the RxRisk-V.35 Like the Disease Count, its main advantage is the ease of use with minimal resources required. However, many studies were not transparent regarding which type of drugs were included.

Data sources

Data sources used by these instruments included medical record information, patient self-report, clinical judgement and large administrative databases. Each data source has its inherent advantages and disadvantages. For patient self-report, patients with cognitive impairment may under-report symptoms and may be seen less frequently by their physicians, resulting in an under-recognition or undertreatment of conditions.36 It has also been shown that health administrative data based on billing system underestimated the prevalence of many chronic conditions.37

The available data in a particular setting may strongly influence the ultimate instrument chosen for multimorbidity research. As there is currently no consensus on the gold standard for sources of data, it is difficult to assess which data source was superior from this review.

Outcomes

There were 17 multimorbidity outcomes identified by a Delphi process involving a panel of international experts in multimorbidity intervention studies.10 However, only 10 out of the 17 outcomes were reported in the 96 studies identified in this systematic review. The most common outcome that was investigated was healthcare use. The seven missing outcomes belong to ‘patient-reported impact and behaviours’ and ‘consultation-related’ outcome groups, most likely indicating that there is a dearth of multimorbidity studies looking at these two groups of outcomes measures.

Clinical implications

Ideally, a single instrument measuring the level of multimorbidity should be able to predict a variety of relevant outcomes. However, Byles et al38 reported that a single instrument could not be used to predict different outcomes, in different patient groups and settings, unless different weights were assigned to these factors in calculating a score. Such multiple-scoring instruments may be the way forward for validation of prognostic models for different outcomes and different populations with established multimorbidity instruments. For example, depending on the outcome, study population and setting, the choice of conditions included in the multiple-scoring instrument should include those with a high prevalence in that study population and the weights should be determined by their significant impact (ie, outcome) on the affected population.

For pragmatic reasons, the final selection of the conditions to be included in such a multiple-scoring instrument may still have to take into account the availability of relevant and reliable data. A certain degree of reductionism will also have to be accepted because a single instrument will not be able to encompass all the nuances of the different interactions of chronic conditions on an individual living in his/her unique milieu. We recommend that researchers perform validation studies using the instruments listed in this systematic review to adjust the weights according to the specific outcome of interest for the study population relevant to their setting.

Strengths and limitations of the study

The main strengths of this systematic review were the involvement of a health science librarian in our search strategy, a published protocol, adherence to the protocol without major changes during the systematic review process,39 and the critical appraisal of all the primary studies with a risk of bias assessment tool.

The systematic review had several limitations. We excluded grey literature and included only studies that were published in the English language. We also did not contact authors directly for a suggestion of studies, nor identified a list of instruments from the preliminary search and then performed an additional search using the same databases.40 Additionally, this systematic review did not review the validity and reliability of all the instruments as it was beyond the scope of the intended work. We have, however, included the references of the original articles or validation studies in table 2 for each of the instrument where available. Finally, this review specifically aimed to look at the association of the level of multimorbidity as the main independent variable and excluded the level of multimorbidity as a mediating, confounding or effect-modifying variable. This strict criterion excluded 17 studies (figure 1) as a result. Excluding these 17 studies did not alter the findings as the instruments used in all the 17 studies were Disease Count (n=9), CIRS (n=3), CCI (n=3), EI (n=1) and Aggregated Diagnosis Groups (n=1) where no new instruments were identified.

Conclusions

In this systematic review, we found 33 instruments for measuring the level of multimorbidity in community-dwelling individuals that predict or explore the association of multimorbidity with at least one specified outcome. Disease Count and weighted indices like the CCI, the ERA and EI were commonly used for measuring the level of multimorbidity. Other approaches to measuring the level of multimorbidity included case-mix or pharmaceutical-based instruments.

We found continuing interest in measuring the level of multimorbidity with Disease Count and Drug Count. There has also been a rise in the development of novel weighted indices using prognostic models or validation of existing well-established instruments like the CCI over the last few years. There is currently an absence of a gold standard for where to obtain chronic disease information. The most suitable instrument will depend on the specified outcome of interest, the study population and the type of data and resources available.

Finally, there is still much work to improve on the body of knowledge of multimorbidity when most investigators in the last decade measured multimorbidity without including some of the important outcome measures of multimorbidity. We also suggest that a clear description of the instruments is required in the publication of multimorbidity studies to counter the frequent lack of information currently seen so as to contribute to robust multimorbidity research in future.

Data availability statement

Data sharing not applicable as no datasets generated and/or analysed for this study. All data relevant to the study are included in the article or uploaded as supplemental information. There are no additional, unpublished data arising from this research.

Acknowledgments

We would like to thank Mr John Costella, Collections and Content Strategies Librarian, Western University for his assistance and guidance in the search strategy. We would also like to thank Dr Mohammed Zulhakim, Family Medicine Resident Doctor, Ministry of Health Holdings, for his help in testing out the various risk assessment tools and his suggestions on the modified Newcastle-Ottawa Scale.

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
  18. 18.
  19. 19.
  20. 20.
  21. 21.
  22. 22.
  23. 23.
  24. 24.
  25. 25.
  26. 26.
  27. 27.
  28. 28.
  29. 29.
  30. 30.
  31. 31.
  32. 32.
  33. 33.
  34. 34.
  35. 35.
  36. 36.
  37. 37.
  38. 38.
  39. 39.
  40. 40.
  41. 41.
  42. 42.
  43. 43.
  44. 44.
  45. 45.
  46. 46.
  47. 47.
  48. 48.
  49. 49.
  50. 50.
  51. 51.
  52. 52.
  53. 53.
  54. 54.
  55. 55.
  56. 56.
  57. 57.
  58. 58.
  59. 59.
  60. 60.
  61. 61.
  62. 62.
  63. 63.
  64. 64.
  65. 65.
  66. 66.
  67. 67.
  68. 68.
  69. 69.
  70. 70.
  71. 71.
  72. 72.
  73. 73.
  74. 74.
  75. 75.
  76. 76.
  77. 77.
  78. 78.
  79. 79.
  80. 80.
  81. 81.
  82. 82.
  83. 83.
  84. 84.
  85. 85.
  86. 86.
  87. 87.
  88. 88.
  89. 89.
  90. 90.
  91. 91.
  92. 92.
  93. 93.
  94. 94.
  95. 95.
  96. 96.
  97. 97.
  98. 98.
  99. 99.
  100. 100.
  101. 101.
  102. 102.
  103. 103.
  104. 104.
  105. 105.
  106. 106.
  107. 107.
  108. 108.
  109. 109.
  110. 110.
  111. 111.
  112. 112.
  113. 113.
  114. 114.
  115. 115.
  116. 116.
  117. 117.
  118. 118.
  119. 119.
  120. 120.
  121. 121.
  122. 122.
  123. 123.
  124. 124.
  125. 125.
  126. 126.
  127. 127.
  128. 128.
  129. 129.
  130. 130130.
  131. 131.
  132. 132.
  133. 133.
  134. 134.
  135. 135.
  136. 136.
  137. 137.
  138. 138.
  139. 139.
  140. 140.

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Related Data

Footnotes

  • Contributors All authors made substantial contribution to the design of the work. BLR, MF and MS mentored and provided guidance to the review. ESL, together with a health science librarian, created the search strategy and conducted the search. ESL, EH, TSH, HLK and FYW screened articles, extracted and analysed data. ESL drafted the manuscript with valuable inputs from BLR, MF and MS and all authors reviewed drafts and approved the final version.

  • Funding This study was supported by the Singapore Ministry of Health’s National Medical Research Council under the Centre Grant Programme (Ref No: NMRC/CG/C019/2017).

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.