A systematic review finds methodological improvements necessary for prognostic models in determining traumatic brain injury outcomes

Abstract

Objectives

To describe the modeling techniques used for early prediction of outcome in traumatic brain injury (TBI) and to identify aspects for potential improvements.

Study Design and Setting

We reviewed key methodological aspects of studies published between 1970 and 2005 that proposed a prognostic model for the Glasgow Outcome Scale of TBI based on admission data.

Results

We included 31 papers. Twenty-four were single-center studies, and 22 reported on fewer than 500 patients. The median of the number of initially considered predictors was eight, and on average five of these were selected for the prognostic model, generally including age, Glasgow Coma Score (or only motor score), and pupillary reactivity. The most common statistical technique was logistic regression with stepwise selection of predictors. Model performance was often quantified by accuracy rate rather than by more appropriate measures such as the area under the receiver-operating characteristic curve. Model validity was addressed in 15 studies, but mostly used a simple split-sample approach, and external validation was performed in only four studies.

Conclusion

Although most models agree on the three most important predictors, many were developed on small sample sizes within single centers and hence lack generalizability. Modeling strategies have to be improved, and include external validation.

Introduction

Traumatic brain injury (TBI) remains the main cause of death and disability in young people. Worldwide, injury is the cause of the largest number of disability-adjusted life years lost, which includes years lost to death and to varying degrees of disability [1]. TBI is by definition a heterogeneous disease, in terms of cause, pathology, severity, and prognosis. The wide variation in prognosis is already captured in the Hippocratic aphorism: “no head injury is too severe to despair of, nor too trivial to ignore.” Even today considerable uncertainty may exist on the expected outcome in individual patients. Yet, confident predictions are essential toward provision of accurate information on expectations to relatives and care givers. Physicians frequently—and often unwittingly—utilize prognostic estimates toward therapeutic decision making and allocation of resources. For accurate outcome prediction, multiple risk factors need to be considered jointly in a prognostic model because single factors have insufficient predictive value to distinguish patients who will do well from those who will do poorly. Prognostic models are generally created by multivariable analysis including predictors such as age and Glasgow Coma Scale (GCS) [2].

Following the pioneering work by Jennett et al. [3], many studies have been published on prognosis in TBI, and the Glasgow Outcome Scale (GOS) has become the generally accepted standard for assessing patient outcome [4]. Although this work moved the field toward a more realistic statistical base and away from clinical guesswork, prognostic models in TBI have not had a widespread impact on clinical practice. Partly, this may be because clinicians are not familiar with prognostic models. However, probably more important is that the validity and usefulness of prognostic models in TBI has not been demonstrated with sufficient clarity and certainty to convince clinicians of their added value. We aimed to identify how previous studies have dealt with four important modeling issues: the study population, choice of predictors and outcome, model development, and model validation. We propose recommendations for future prognostic modeling studies.