Table 3

Pairwise agreement between treatment hierarchies obtained from the different ranking metrics measured by Spearman ρ, Kendall τ, Yilmaz Embedded Image and Average Overlap

Embedded ImagevsEmbedded ImageEmbedded Imageversus relative treatment effectEmbedded Imageversus relative treatment effectEmbedded ImageversusEmbedded Image
Spearmanρ0.9 (0.8 to 0.96)1 (0.99 to 1)0.9 (0.8 to 0.97)1 (0.98 to 1)
Kendallτ0.8 (0.67 to 0.91)1 (0.95 to 1)0.8 (0.69 to 0.91)1 (0.93 to 1)
YilmazEmbedded Image0.78 (0.6 to 0.9)1 (0.93 to 1)0.79 (0.65 to 0.9)1 (0.93 to 1)
Average Overlap0.85 (0.72 to 0.96)1 (0.91 to 1)0.88 (0.79 to 1)1 (0.94 to 1)
  • Medians, first and third quartiles are reported.

  • Relative treatment effect stands for the relative treatment effect against fictional treatment of average performance.

  • PBV, probability of producing the best value; SUCRAB, surface under the cumulative ranking curve (calculated in Bayesian setting); SUCRAF, surface under the cumulative ranking curve (calculated in frequentist setting).