Number of raters | Crude reliability* (95% CI) | Reliability adjusted for evaluator bias* (95% CI) |
---|---|---|
1 | 0.58 (0.28 to 0.66) | 0.65 (0.36 to 0.71) |
2 | 0.73 (0.44 to 0.79) | 0.78 (0.53 to 0.83) |
3 | 0.80 (0.54 to 0.85) | 0.85 (0.63 to 0.88) |
4 | 0.85 (0.61 to 0.88) | 0.88 (0.69 to 0.91) |
5 | 0.87 (0.66 to 0.91) | 0.90 (0.74 to 0.93) |
6 | 0.89 (0.70 to 0.92) | 0.92 (0.77 to 0.94) |
7 | 0.91 (0.73 to 0.93) | 0.93 (0.80 to 0.95) |
8 | 0.92 (0.76 to 0.94) | 0.94 (0.82 to 0.95) |
9 | 0.93 (0.78 to 0.95) | 0.94 (0.84 to 0.96) |
10 | 0.93 (0.80 to 0.95) | 0.95 (0.85 to 0.96) |
*Calculated from the estimated SDs shown in table 1.
GCRS, Global Consultation Rating Scale.