Validation in prediction research: the waste by data splitting

Ewout W Steyerberg

doi:10.1016/j.jclinepi.2018.07.010

Validation in prediction research: the waste by data splitting

J Clin Epidemiol. 2018 Nov:103:131-133. doi: 10.1016/j.jclinepi.2018.07.010. Epub 2018 Jul 29.

Author

Ewout W Steyerberg¹

Affiliation

¹ Professor of Clinical Biostatistics and Medical Decision Making, Chair, Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, The Netherlands; Professor of Medical Decision Making, Department of Public Health, Erasmus MC, Rotterdam, The Netherlands. Electronic address: e.steyerberg@erasmusmc.nl.

PMID: 30063954
DOI: 10.1016/j.jclinepi.2018.07.010

Abstract

Accurate prediction of medical outcomes is important for diagnosis and prognosis. The standard requirement in major medical journals is nowadays that validity outside the development sample needs to be shown. Is such data splitting an example of a waste of resources? In large samples, interest should shift to assessment of heterogeneity in model performance across settings. In small samples, cross-validation and bootstrapping are more efficient approaches. In conclusion, random data splitting should be abolished for validation of prediction models.

MeSH terms

Data Accuracy
Diagnosis*
Diagnostic Techniques and Procedures
Humans
Outcome Assessment, Health Care* / methods
Outcome Assessment, Health Care* / standards
Prognosis*
Reproducibility of Results*
Research Design
Sample Size*
Treatment Outcome