Article Text
PDF
Abstract
Introduction Acute kidney injury (AKI) is common and is associated with negative long-term outcomes. Given the heterogeneity of the syndrome, the ability to predict outcomes of AKI may be beneficial towards effectively using resources and personalising AKI care. This systematic review will identify, describe and assess current models in the literature for the prediction of outcomes in hospitalised patients with AKI.
Methods and analysis Relevant literature from a comprehensive search across six databases will be imported into Covidence. Abstract screening and full-text review will be conducted independently by two team members, and any conflicts will be resolved by a third member. Studies to be included are cohort studies and randomised controlled trials with at least 100 subjects, adult hospitalised patients, with AKI. Only those studies evaluating multivariable predictive models reporting a statistical measure of accuracy (area under the receiver operating curve or C-statistic) and predicting resolution of AKI, progression of AKI, subsequent dialysis and mortality will be included. Data extraction will be performed independently by two team members, with a third reviewer available to resolve conflicts. Results will be reported using Preferred Reporting Items for Systematic Reviews and Meta-Analysis guidelines. Risk of bias will be assessed using Prediction model Risk Of Bias ASsessment Tool.
Ethics and dissemination We are committed to open dissemination of our results through the registration of our systematic review on PROSPERO and future publication. We hope that our review provides a platform for future work in realm of using artificial intelligence to predict outcomes of common diseases.
PROSPERO registration number CRD42019137274.
- adult nephrology
- dialysis
- acute renal failure
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
Novel presentation of data summarising the use of predictive modeling to determine outcomes after acute kidney injury (AKI), which has not been conducted previously.
Use of a rigorous and exhaustive search strategy that will capture all articles from inception to present with clearly defined parameters for inclusion and exclusion.
Use of well-validated tools, such as Preferred Reporting Items for Systematic Reviews and Meta-Analysis, Covidence, PROSPERO and Prediction model Risk Of Bias ASsessment Tool to capture, assess and report our findings.
Limiting to only hospitalised adult patients, which will reduce generalisability of our results.
Variability in definitions of AKI or stages of AKI in which the prediction occurs may exist across the literature.
Introduction
Acute kidney injury (AKI) is associated with poor long-term outcomes
AKI is a complex syndrome caused by multiple aetiologies and is characterised by a sudden decrease in kidney function. There are different stages of the syndrome, often defined by the Kidney Disease: Improving Global Outcomes (KDIGO) criteria based on increases in serum creatinine or decreases in urine output, and are outlined in figure 1.1 Despite advances in healthcare, AKI is an increasingly common complication, estimated to occur in up to 15% of hospitalised patients and up to 60% of critically ill patients.2 3 Moreover, it is often associated with adverse short-term and long-term patient outcomes,4 including increased acute renal replacement therapy, chronic kidney disease, end-stage kidney disease and mortality.5 6 The mortality rate associated with AKI can be as high as 50%–80%, with little improvement over the past several decades.7 Considering the impact and consequences of AKI, it has been recognised that early identification of AKI and its outcomes is desirable in hospital settings, as even a small increase in serum creatinine level leads to a fourfold greater increase in mortality.8
Proposed Kidney Disease: Improving Global Outcomes staging for acute kidney injury. eGFR, estimated glomerular filtration rate; RRT, renal replacement therapy.
The rising use and limitations of machine-learning (ML) models in the medical field
The past decade has seen significant development in the use of artificial intelligence (AI) in medicine. AI is a branch of engineering, defined as the ability of a machine to reason, communicate and function with minimal human intervention.9 In medicine, the virtual branch of AI, which encompasses ML, includes algorithms and statistical models that learn from data and deduce patterns.9 Such ML algorithms can also be used to predict outcomes after disease. A good prognostic ML model should have good discrimination, good calibration and good performance in different subsets of patients. Calibration is defined as the ability of the model to correctly determine the probability of an outcome to occur, whereas discrimination refers to how well the model differentiates those at higher risk of having an event from those at lower risk.10 The performance of ML models is evaluated on the basis of predictive accuracy, commonly presented as a C-statistic or area under the receiver operating curve (AUC). The development of risk prediction models should also include internal validation within the original sample to quantify the predictive ability of the model and external validation to evaluate the predictive ability of the model in other participant data.11
Over the years, there has been a greater focus on risk assessment and the use of novel clinical prediction models to predict both risk of disease and clinical outcomes. Studies have shown encouraging results from the implementation of clinical prediction models to assess outcomes after diseases such as hepatocellular carcinoma and fragility fractures, to name a few.12 13 Recently, novel time-updated predictive models have been developed to assess patient risk and predict onset of AKI in real time in hospitalised patients and have shown promising results.14 However, despite the increased demand and development of risk prediction models in the recent years, inefficient statistical methods, small sample size, missing data and lack of validation are some common faults that have limited their use.11
Clinical implications of an accurate prediction model for outcomes after AKI
Models that predict outcomes are particularly useful when advising patients’ families regarding continued life support, improving quality of care and tailoring interventions towards individual patients.7 The ability to accurately predict clinical outcomes in patients with AKI is of particular importance as AKI is a heterogenous syndrome that requires individualised care and management. The benefits of clinical prediction models for the prediction of outcomes among patients with AKI are twofold; they will allow for individualised care to reduce unfavourable outcomes while maximising the efficient use of resources for AKI management.
Need for a systematic review of the literature to fill the current gap in knowledge
Numerous studies have been published that use human-based, score-based and machine-based models to predict outcomes after AKI, however it has yet to be determined which model is the most efficacious in predicting outcomes, and what common features are shared among these models. Understanding the strengths and limitations of these models will aid in future efforts to create efficacious prediction methods that can be implemented into clinical practice to improve patient outcomes.15 The multifactorial origin of AKI and its significant impact on limited hospital resources, such as need and availability of renal replacement therapy (RRT) machines and staff equipped and educated in dialysis, highlight the need for a standardised outcome-based prediction model. To this end, we propose to conduct a systematic review of the literature to identify and describe the currently published models for predicting outcomes after AKI in hospitalised patients and assess their reliability and use in the current healthcare system.
Objectives
The aims of our systematic review are to:
Identify and describe the various clinical models used in the prediction of outcomes after AKI in hospitalised patients.
Assess the performance of different methods of prediction for outcomes after AKI.
Describe the variables used in the development of various prediction models.
Methods and analysis
Study design
A systematic review will be performed to identify and describe clinical models for the prediction of outcomes in hospitalised patients with AKI, using the guidelines described by Preferred Reporting Items for Systematic Reviews and Meta-Analysis.
Search methods for identification of studies
A medical librarian will develop a strategy using keywords and controlled vocabulary for AKI, prediction models and statistical measurements for prediction of outcomes. The Yale MeSH Analyzer (http://mesh.med.yale.edu) will be used in the initial strategies of the search strategy formulation to harvest controlled vocabulary and keywords from high relevant, known articles. This search will be peer reviewed by a second medical librarian using the Peer Review of Electronic Search Strategies checklist.16 The search will then be reviewed by the entire research team in a consultation to assess the sensitivity and specificity of the search terms in regards to the research question. The systematic search will be conducted in Cochrane Library, Ovid Embase, Ovid Medline, PubMed, Scopus and Web of Science Core Collection. The search will be limited to English language, human subjects and publication type and will have no limitation in publication date, searching from the database’s inception to time of the search. Please see the attached search online supplemental appendix for the full search strategy.
Supplemental material
Citations from all databases will be imported into a Covidence X9 library. Duplicate citations will be removed within Endnote. The individual citations will then be uploaded into Covidence for title/abstract review by two independent screeners with a third reviewer available to resolve ties.
Keywords in search
The search includes controlled vocabulary and keywords for acute kidney injury, prediction or artificial intelligence and death or dialysis (please refer to AKI search online supplemental appendix for more details).
Criteria for considering studies in this review
Study design: Original data from retrospective and prospective randomised controlled trials and observational cohort studies with at least 100 subjects (to minimise the risk of model overfitting and increase generalisability of the study).
Population: Studies that enrol adult (age greater than or equal to 18 years) hospitalised patients who have a diagnosis of AKI (based on KDIGO criteria or others).
Intervention: Studies that examine a multivariate model for predicting outcomes after AKI. Models can be human based, score based and/or machine based. Validation studies of an existing model will also be considered.
Outcome: Studies that report on the prediction of one of the three main outcomes of interest will be reviewed: resolution of AKI (defined as return to prehospitalisation baseline serum creatinine (Cr) and/or urine output); progression of AKI with subsequent use of renal replacement therapy (dialysis) and mortality. Studies must report discrimination statistics of their model.
Exclusion criteria
Studies to be excluded will include: studies predicting AKI development rather than outcomes of AKI; those that use AKI as a predictor of an outcome; those with descriptive outcomes of AKI; those highlighting associations of risk factors with AKI; those that describe a model with no discrimination statistics (ie, AUC or C-statistic); non-human studies; paediatric studies; studies with a sample size <100; studies involving community-acquired AKI; studies describing treatment-related outcomes; studies describing models with a single predictor and studies describing models using novel clinical biomarkers as predictors, as this is out of the scope of our review.
Study selection
Covidence will be used for initial abstract screening and full-text review. The titles and abstracts of all potential eligible articles that have been added to Covidence will be reviewed by two authors independently, whereby each author chooses to include or exclude the article in the study based on their relevancy to our review. Articles will be included or excluded only when there is an agreement between the two reviewers. Conflicts will be resolved by an independent third-party reviewer.
Articles that have been included based on abstract screening will then undergo full-text review by two independent reviewers who will determine whether each article is eligible for full-text inclusion based on the predefined eligibility criteria outlined above. Similarly, any conflicts will be resolved by an independent third-party reviewer.
Data extraction
Once all articles have been reviewed for inclusion, data extraction will commence. Two authors will perform extraction on each article using a standardised Excel sheet with the variables to be collected. Data to be extracted will include: manuscript title, country of publication, date published, dates of study, type of study, inclusion criteria and stage of AKI, exclusion criteria, location of patients (intensive care unit (ICU) vs non-ICU), number in cohort or in randomised control trial, outcome being predicted (resolution, dialysis or death), model used, variables used to create model, method of variable selection, validation of the study and value of the statistical measure (AUC/C-statistic). Any disagreements that arise in the process of data extraction will be resolved through discussion, and, if needed, referral to a third party. We will strive for an inter-rater reliability of at least 85%.
Data statement
We plan to establish a data repository after the completion of the review. This data repository will contain all relevant data and will be available for dissemination to the public. We also plan to present and incorporate the data we collect as abstracts, manuscripts and presentations at scientific meetings.
Risk of bias and quality assessment
Assessment of risk of bias will be done using Prediction model Risk Of Bias ASsessment Tool which assesses the risk of bias in diagnostic and prognostic prediction model studies taking into consideration four domains (participants, predictors, outcome, analysis). It assesses both risk of bias and concerns regarding applicability of studies that developed or validated multivariable prediction models for diagnosis or prediction.17
Data analysis
This review will provide a summary of the collected data using descriptive statistics, graphical plots and a narrative synthesis. Each study will be described according to its study design, prediction model, predictors, outcome(s) predicted and model robustness.
Some of the questions we will present data on include (but are not limited to):
What models exist to predict outcomes after AKI and how well do they perform? A data report will be constructed showcasing the different models that have been used to predict outcomes of interest after AKI, alongside the strength of the model.
What are the most commonly used features in models predicting outcomes after AKI? Data will be collected and reported to highlight the main features used to establish predictive models.
What are some of the major limitations and challenges in predictive modelling for outcomes after AKI? Limitations will focus on strength of models, and whether discrimination and calibration has been referenced to and/or described and a report will be made on the AUC for each model. Limitations will also focus on validation of studies and report whether the individual study was internally validated, externally validated or not validated.
Patient and public involvement
No patient involved.
Ethics and dissemination
This systematic review will capture and present data on the current predictive models for predicting outcomes after AKI present in the literature to direct future studies in the field, with the goal of more patient-centred care and the more effective use of limited healthcare resources. Our group aims to broaden the understanding of predictive modelling and its significance in healthcare by providing information on the current paradigm and trends over time. This review will be presented at national and international conferences as oral and poster presentations and the final manuscript with results and conclusions will be published in a peer-reviewed journal. The details of our systematic review are also currently registered with PROSPERO.
Acknowledgments
We acknowledge the efforts of all authors and resources provided by Yale University to conduct the review. We would also like to acknowledge members of the Clinical and Translational Research Accelerator lab at Yale University for their constant support.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @methodsmanmd
Contributors TA, MM and FPW contributed evenly towards the conceptual design, protocol construction, registration and manuscript writing. AG contributed towards design and description of methodology and provided all keywords of her search (in the main manuscript and supplementary body). We acknowledge and thank SM for her contributions to this review protocol.
Funding Supported by R01DK113191 and P30DK079310 (FPW).
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.