Table 1

Overview of the main criteria for evaluating statistical methods in the four considered examples

ExampleEvaluation criterionTarget value
A: testing and CIType 1 errorClose to and not greater than nominal value α
Type 2 errorLow
Coverage of (1–α) CIClose to and not lower than nominal value 1–α
B: explainingMean coefficient valuesClose to true values (low bias)
Precision of coefficient estimationHigh (low variance)
Coverage of CIClose to and not lower than nominal value 1–α
Sensitivity of variable selectionHigh
Specificity of variable selectionHigh
C: predictingPrediction error on independent dataLow
Accuracy measuresHigh
D: clusteringAgreement with true cluster structureHigh
All settingsStabilityHigh
Computational costLow
Success of the computation (eg, ‘convergence’)Yes
  • The last column indicates which values the considered evaluation criterion takes if the investigated method is good.