Background and objectives: To develop prediction models for outcomes following trauma that met prespecified performance criteria. To compare three methods of developing prediction models: logistic regression, classification trees, and artificial neural networks.
Methods: Models were developed using a 1996-2001 dataset from a major trauma center in Victoria, Australia. Developed models were subjected to external validation using the first year of data collection, 2001-2002, from a state-wide trauma registry for Victoria. Different authors developed models for each method. All authors were blinded to the validation dataset when developing models.
Results: Prediction models were developed for an intensive care unit stay following trauma (prevalence 23%) using information collected at the scene of the injury. None of the three methods gave a model that satisfied the performance criteria of sensitivity >80%, positive predictive value >50% in the validation dataset. Prediction models were also developed for death (prevalence 2.9%) using hospital-collected information. The performance criteria of sensitivity >95%, specificity >20% in the validation dataset were not satisfied by any model.
Conclusion: No statistical method of model development was optimal. Prespecified performance criteria provide useful guides to interpreting the performance of developed models.