Article Text

Download PDFPDF

Evaluation of published assessment tools for comorbidity in liver transplantation: a systematic review protocol
  1. Zhi Qu1,2,
  2. Jill Gwiasda2,
  3. Harald Schrem2,3,
  4. Alexander Kaltenborn2,
  5. Lena Harries1,2,
  6. Jan Beneke2,
  7. Volker Amelung1,2,
  8. Christian Krauth1,2
  1. 1 Institute for Epidemiology, Social Medicine and Health Systems Research, Hannover Medical School, Hannover, Germany
  2. 2 Core Facility Quality Management and Health Technology Assessment in Transplantation, Integrated Research and Treatment Facility Transplantation (IFB-Tx), Hannover Medical School, Hannover, Germany
  3. 3 General, Visceral and Transplant Surgery, Hannover Medical School, Hannover, Germany
  1. Correspondence to Dr Zhi Qu; qu.zhi{at}


Introduction Liver transplantation is considered the best therapy option for end-stage liver disease. Different factors including recipient comorbidity at time of transplantation are supposed to have substantial impact on outcomes. Although several studies have focused on comorbidity assessment indices for liver transplant recipients, there is no systematic review available on the methodological details and prognostic accuracy of these instruments. The aim of this study is to systematically review recipient comorbidity assessment indices in the context of liver transplantation.

Methods and analysis PubMed, Embase, Web of Science and PsyINFO databases will be searched. Studies describing, using or evaluating specific assessment tools to predict the effect of comorbidity on clinical outcomes after liver transplantation will be included. The selection will be conducted independently by two reviewers. The study characteristics and methodological information on published comorbidity assessment tools will be extracted into a predefined structural table. This approach will be deployed to systematically extract information on the validity, reliability and practical feasibility of investigated comorbidity assessment tools for comparative evaluation. Narrative information synthesis will be conducted, and additional meta-analytical comparison will be performed, if appropriate.

Ethics and dissemination All data are collected from published literature. Thus, formal ethics review for the research is not required. The findings of this systematic review will be published in a peer-reviewed journal and presented at relevant conferences. The results of this systematic review will be highly relevant for further research on prognostic models, clinical decision making and optimisation of donor organ allocation.

PROSPERO registration number CRD42017074609.

  • liver transplantation
  • comorbidity
  • risk adjustment
  • systematic review

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See:

Statistics from

Strengths and limitations of this study

  • In the liver transplantation context, there is still no systematic review on the methodological details and use of comorbidity indices available. The current protocol outlines an approach to comprehensively understand how published comorbidity indices have been used to measure the effect of comorbidity on various outcomes.

  • This study provides a systematical review on the profile of published comorbidity indices and quantifies the impact, accuracy and validity of comorbidity assessment tools in liver transplantation.

  • A major limitation is that the relationships between different comorbidity-related constructs are complex, which may result in comorbidity measurements from included publications that may be affected by other coexisting chronic conditions.


Liver transplantation is widely accepted as the standard treatment for end-stage liver disease and the treatment of hepatocellular carcinoma.1 Recipient comorbidities at the time of transplantation may substantially affect both short-term and long-term recipient outcome.2–4 For example, cardiovascular disease5 and congestive cardiac failure6 have been shown to increase short-term mortality, while diabetes mellitus7 8 and renal insufficiency9 have been shown to increase long-term mortality.2 Quantifying the impact, accuracy and validity of comorbidity assessment tools in liver transplant will be beneficial for the systematic improvement of meaningful prognostic models for the prediction of clinical outcomes, which also may improve organ allocation rules.

Appropriate ways of quantifying relevant comorbidities form the core of meaningful comorbidity assessment. The simple counting of diseases or medical conditions, organ or organ system function-based approaches and weighted indices have their particular advantages and shortcomings.10 The use of each recipient’s individual number of conditions is the most explicit way to evaluate the individual comorbidity status and simplify the analysis of the impact of coexistent comorbidities on outcome such as patient survival. Organ or organ system-based measuring methods such as the Kaplan-Feinstein Index11 and the Adult Comprehensive Evaluation-2712 could be used to evaluate their impact on outcome and thus also the severity of individual comorbidity. Both methods have been applied in different populations, for example, patients with cancer.10 Weighted comorbidity indices have special advantages, such as their feasibility to define the profile of patients’ multiple disease burden. These indices are simple and clear and can thus be easily applied by healthcare professionals. Comorbidity measuring schemes like the Charlson Comorbidity Index13 and the Elixhauser Index14 are widely applied and have been validated in many studies.2

Several studies focused on the specified comorbidity indices for liver transplant patients,15–17 while others reviewed the influence of comorbidity in other clinical fields, such as non-traumatic brain rehabilitation,18 cardiovascular disease19 or cancer.10 20 Some of these instruments have been used as special assessment tools for the prediction of the effects of comorbidities on outcomes. To the best of our knowledge, there is currently no systematic review available on the use and methodological details of comorbidity measurements in the liver transplantation context.

Therefore, the purpose of the study is to:

  1. Provide a systematic review on the profile of published comorbidity measurements in the context of liver transplantation and to investigate how they have been used to measure the effect of comorbidity on various outcomes such as mortality, graft loss and healthcare resource cost (ie, cost of treatment procedure and length of stay in hospital).

  2. Assess the validity, reliability and practical feasibility of published comorbidity assessment tools.

Method and analysis


The protocol is consistent with the requirements of the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P).21 In accordance with PRISMA-P guidelines, this protocol was registered with the International Perspective Register of Systematic Reviews database online (; the registration number is: CRD42017074609. After registration, any important protocol amendments will be documented and included in dissemination.

Patient and public involvement

In the intended systematic review study, no patient or member of public involved in development of research questions, design of study protocol, its future execution or advocacy of results.

Eligibility criteria

We included studies describing, using or evaluating specific assessment tools (indices, scores and so on) for comorbidity in the context of adult liver transplantation and whether these instruments have been used to predict the effect of comorbidity on clinical outcome including patient and graft survival as well as healthcare resource cost.

The target population of included studies includes adults (age ≥17 years) who have either been listed for liver transplantation or who have been liver transplanted or who are on long-term care or treatment after liver transplantation.

Reports on randomised controlled trials, non-randomised interventional and observational studies, as well as retrospective studies and secondary data analyses will be included.

Qualitative research, case reports, editorials, letters to the editor, abstracts, conference materials, systematic reviews or meta-analyses will be excluded.

Definition of comorbidities

Feinstein has defined the concept of comorbidity as ‘any distinct additional clinical entity that has existed or may occur during the clinical course of a patient who has the index disease under study’.22 In the current study, we are interested in the comorbidities that potentially influence the prognosis and clinical management of liver transplant patients, rather than the disease co-occurring with the indication for liver transplantation or results and complications of transplant. Reports of diagnosed comorbid disease at least 3 months prior to transplant including physical and psychological disease would be investigated in this systematic review. The comorbidities will be grouped in accordance with the latest version of International Classification of Diseases, 10th Revision diagnosis codes.

Search strategy

The search strategy is developed in collaboration with clinical transplant and epidemiology experts, following the guideline of the Center for Reviews and Dissemination Guidance.23 Supplementary appendix 1 provides the search strategy for literature review, which includes the Medical Subject Headings (MeSH)terms ‘liver transplantation’ and ‘comorbidity’. ‘Diagnosis-related groups’ and ‘case-mix’ may collaborate to the information on comorbidity and thus increase the sensitivity of the search on the broad term of comorbidity as has been suggested before.19 Published studies will be searched from the following databases: PubMed, Embase, Institute for Scientific Information Web of Science and PsychINFO. Results from all databases will be limited to the English language and no restriction on publication date. Citavi software (V.5.4, Swiss Academic Software GmbH) will be used for reference management.

Supplementary file 1

Study selection

Study selection will be conducted independently by two reviewers (ZQ and JG) in two steps. First, titles and abstracts will be screened to exclude literature that does not fulfil the inclusion criteria or that fulfils the exclusion criteria. Second, full texts will be evaluated to check the fulfilment of the prespecified eligibility criteria. Disagreements will be resolved by consensus or by discussion with a third reviewer (HS or CK).

Data extraction

The data extraction includes the following items:

Publication title, the published year, all authors, the geographic location of the investigated population, investigated sample size, basic patient demographic characteristics including median age and age distribution as well as gender distribution, primary transplant indications grouped according to European Liver Transplant Registry guidelines for grouping,24 type of liver transplantation (eg, split liver transplantation, living donor liver transplantation, deceased donor liver transplantation with and without donation after cardiac death and liver transplantation after machine perfusion of donated organs), model for end-stage liver disease scores at listing and transplantation, investigated outcome measures, narrative summary of main findings, type of deployed comorbidity assessment tool (single comorbidity such as diabetes, comorbidity counts, comorbidity index with or without weighing) and identified role of comorbidity (descriptor, covariate, predictor or outcome).

The specification and justification of identified comorbidity measurement tools will be extracted by one of the authors (ZQ) using a predefined evaluation table that will be checked independently by another author (JG). Supplementary appendix 2 contains an example of the predefined structural table proposed for data extraction. This data extraction form was formulated on the basis of a pilot study by the authors of this paper using several predefined studies.

Supplementary file 2

Quality assessment

The validity and feasibility of each included assessment tool will be evaluated following the process introduced by Safarti et al, and the items from the quality assessment tool reported by Jacob et al will complement the assessment table.10 25

  1. Validity:

    1. Content and face validity: both these measures relate to the degree to which a measure actually evaluates the construct that it purports to measure.26

    2. Concurrent validity refers to the degree to which the measure correlates with another measure taken at the same time.10

    3. Predictive validity is the extent to which the measure is able to predict future outcomes of interest, such as patient or graft survival. A main criterion is the precision of the comorbidity measurement predictions. Additional criteria may include model calibration, sensitivity and specificity of prediction, areas under the receiver operating characteristic curve, description of external model validation depending on the type of study endpoint and usage of a comorbidity measurement. The Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) guideline and the TRIPOD statement will be used for the assessment of predictive validity.25 27

  2. Reliability is ‘the extent to which repeated measurements of a stable phenomenon by different people at different times and places get similar results’.28

  3. Feasibility relates to the simplicity, cost, time and effort required to use the measure.10

Risk of bias assessment

Risk of bias will be assessed with the Prediction study Risk Of Bias Assessment Tool for risk of bias and applicability in prognostic model studies.29 This is justified based on the results of our preliminary pilot study, because the included studies would mainly be prognostic studies.

The domain for evidence quality assessment in Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group guideline, which includes consistency, directness, precision and publication bias, will also be applied. The quality of evidence will be graded as high, moderate, low and very low in accordance with the GRADE guideline. Two initial reviewers will independently assess the risk of bias on each included study, and the third reviewer will mediate in situations of disagreements. The consistency of agreement will be assessed with Cohen’s kappa.30


Depending on the included studies and their results, qualitative or quantitative information synthesis will be conducted. Qualitative analysis will be performed following the Guidance for Narrative Synthesis in Systematic Review.31 Basic study characteristics will be tabulated and summarised to highlight their similarities and differences. The extracted information on comorbidity measurement from included studies will also be tabulated and grouped by empirically important variables such as population subgroup (eg, donor, recipient, waiting list transplant candidate and so on), data source (eg, administrative data, clinical medical record, self-reported data or doctor/nurse report) and comorbidity assessment tools type (eg, single comorbidity, count of comorbidity number, comorbidity severity and comorbidity index and so on). Textual description of different comorbidity assessment tools will explain and state the key feature in the context of liver transplantation. The result of the systematic quality assessment of published studies will be summarised as described above.

Meta-analytical comparison will be conducted, if appropriate. Given different comorbidity measuring tools are applied to describe patient characteristics, risk adjustment and outcome prediction in this field, there is yet no gold standard to assess the comorbidity measurement tools, and meta-analysis is unlikely to be performed on the scope of current research. However, the meta-analytical approach recommended by Sharabiani et al 32 will be helpful to summarise the result of different assessment tools in comparative studies if two or more included studies assessed the predictive validity of comorbidity tools. This approach uses the hypergeometric test to identify the comparators with significantly superior/inferior performance for outcome prediction providing a further profile to describe the predictive capability of comorbidity indices.


As far as we know, this is the first protocol for the systematic investigation of comorbidity assessment tools in the context of liver transplantation. Previous studies on comorbidity measurement methods mainly focus on patients with cancer10 20 or cardiovascular disease19 and a non-traumatic brain injury population.18 Between these cohorts with very different diseases, both the frequency of relevant comorbidities and their impact on specific outcomes are apparent. While there may be similarities, a distinct consideration of main diagnoses and outcome is mandatory. When comparing the influence of comorbidities on clinical outcome between surgical and non-surgical patients, the differences may present even more obviously.33 Still, a general approach to analyse comorbidities can guide the focus to groups of medical conditions of general importance, though adaptation and extended analyses have to be considered.

From our pilot study, we learnt that the most widely accepted comorbidity indices in the field of liver transplantation were the Charlson Comorbidity Index and its adaptations.13 When applying this index to the population of liver transplantation, some of the comorbidity groups, such as metastatic carcinoma or dementia according to Charlson and Quan,13 34 do not seem useful, since they represent contraindications for transplantation. Furthermore, the comorbidity group ‘liver disease’ has to be applied to every patient in the liver transplant setting and therefore does not quantify the severity of the disease sufficiently in the context of liver transplantation. The Charlson Comorbidity  Index has its own strengths including its simplicity and feasibility of application when using multiple patient data sources and its favourable validity and reliability on mortality prediction. However, its predictive power has been shown to be poor to moderate when the outcome of interest is healthcare resource consumption.35 Healthcare resource cost is of high interest in the resource intensive context of liver transplantation. The Elixhauser Comorbidity  Index is another main comorbidity measurement system used in many studies that found this tool to be slightly superior in the prediction of mortality when compared with the Charlson Comorbidity  Index. The disadvantage of the Elixhauser system lies in its complexity as it measures comorbidity with 30 binary variables that may lead to a overfitting when patient groups are small.36 Furthermore, the applications of the Elixhauser Comorbidity  Index results in potential risks of misclassifying complications as comorbidities.37

Volk and colleagues recalibrated the Charlson Comorbidity  Index, so that disease groups according to Charlson, which did not have a significant influence on survival following liver transplantation, were eliminated from this new index, whereas the remaining disease groups were weighted differently.2 13 This recalibration was a useful step towards the application of comorbidity indices in the context of liver transplantation. However, it remains unclear whether all relevant comorbidity groups that have a significant influence on outcomes following liver transplantation have been identified and regarded so far.

The intended review will focus on comorbidity, but many related constructs such as coexisting disease, other coexisting chronic medical conditions and the functional status also contribute to the comorbidity thus potentially interact with the association between comorbidity and outcomes. Although the relationships between these concepts are complex, we will try to increase the sensitivity of the search strategy. Since the definitions of comorbidity-related constructs among different studies are not identical, the extension of the term comorbidity, for example, by additional usage of ‘multimorbidity’, ‘coexistent condition’, ‘co-occurring condition’ can be used in the literature search to find more potentially relevant publications. Comorbidities in the possibly included studies will be reviewed in depth on the basis of exposure and defined study endpoints, and the review of the coexisting diseases or conditions will likely improve the understanding of the role of comorbidities for outcomes after liver transplantation.

Ethics and dissemination

All data are collected from published literature. Thus, a formal ethics review for the intended research is not required. The authors will publish the findings of this systematic review in a peer-reviewed journal and present it at relevant national and international conferences. This is the first protocol for a systematic review of comorbidity assessment tools in the context of liver transplantation. The expected results of such a systematic review will be highly relevant and helpful for further research on prognostic models in liver transplantation and will thus likely provide better tools for clinical decision making as well as for the optimisation of donor organ and healthcare resource allocation .


  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
  18. 18.
  19. 19.
  20. 20.
  21. 21.
  22. 22.
  23. 23.
  24. 24.
  25. 25.
  26. 26.
  27. 27.
  28. 28.
  29. 29.
  30. 30.
  31. 31.
  32. 32.
  33. 33.
  34. 34.
  35. 35.
  36. 36.
  37. 37.


  • Contributors ZQ is the guarantor of the review protocol. ZQ, HS and JB conceptualised the study. ZQ registered the study on PROSPERO. JG and ZQ drafted the research protocol. JG, HS, LH, AK, VA and CK critically reviewed the manuscript and contributed important intellectual contents.

  • Funding This work was supported by a grant from the German Federal Ministry of Education and Research (reference number: 01EO1302).

  • Disclaimer The funding source had no role in the design of this study and will not have any role during its execution, data analyses, results interpretation or decision on submission.

  • Competing interests None declared.

  • Patient consent Not required.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.