Table 2

Crude and adjusted reliability for evaluating consultations for different numbers of raters using GCRS (transformed 0–10 data)

Number of ratersCrude reliability* (95% CI)Reliability adjusted for evaluator bias* (95% CI)
10.58 (0.28 to 0.66)0.65 (0.36 to 0.71)
20.73 (0.44 to 0.79)0.78 (0.53 to 0.83)
30.80 (0.54 to 0.85)0.85 (0.63 to 0.88)
40.85 (0.61 to 0.88)0.88 (0.69 to 0.91)
50.87 (0.66 to 0.91)0.90 (0.74 to 0.93)
60.89 (0.70 to 0.92)0.92 (0.77 to 0.94)
70.91 (0.73 to 0.93)0.93 (0.80 to 0.95)
80.92 (0.76 to 0.94)0.94 (0.82 to 0.95)
90.93 (0.78 to 0.95)0.94 (0.84 to 0.96)
100.93 (0.80 to 0.95)0.95 (0.85 to 0.96)
  • *Calculated from the estimated SDs shown in table 1.

  • GCRS, Global Consultation Rating Scale.