The different designs compared in this study.
| Design | CV | Test set? | Hyperparameter selection | Model development | Internal validation |
| No test/validation set | 0 | No | Using all data | Using all data | Using all data |
| Test/validation set | 0 | Yes | Using 10% validation data | Using 80% training data | Using 10% test data |
| Threefold CV | 3 | No | Using threefold CV on all data | Using all data | Using threefold CV on all data |
| Threefold CV with test set | 3 | Yes | Using threefold CV on 80% training data | Using 80% training data | Using 20% test data |
| Fivefold CV | 5 | No | Using fivefold CV on all data | Using all data | Using fivefold CV on all data |
| Fivefold CV with test set | 5 | Yes | Using fivefold CV on 80% training data | Using 80% training data | Using 20% test data |
| Ten-fold CV | 10 | No | Using 10-fold CV on all data | Using all data | Using 10-fold CV on all data |
| Ten-fold CV with test set | 10 | Yes | Using 10-fold CV on 80% training data | Using 80% training data | Using 20% test data |
CV, cross-validation.