Table 3

Updated criteria for good measurement properties21 22

Measurement propertyRatingCriteria
Structural validity+CTT:
CFA: CFI or TLI or comparable measure>0.95 OR RMSEA
<0.06 OR SRMR<0.08
IRT/Rasch:
No violation of unidimensionality: CFI or TLI or comparable measure>0.95 OR RMSEA<0.06 OR SRMR<0.08
AND no violation of local independence: residual correlations among the items after controlling for the dominant factor<0.20 OR Q3’s<0.37
AND no violation of monotonicity: adequate looking graphs OR item scalability>0.30
AND adequate model fit:
IRT: χ2>0.01
Rasch: infit and outfit mean squares≥0.5 and ≤ 1.5 OR Z‐standardised values > −2 and<2
?CTT: not all information for '+' reported IRT/Rasch: model fit not reported
Criteria for '+' not met
Internal consistency+At least low evidence for sufficient structural validity AND Cronbach’s alpha(s) ≥ 0.70 for each unidimensional scale or subscale 6
?Criteria for ‘at least low evidence for sufficient structural validity’ not met
At least low evidence for sufficient structural validity AND Cronbach’s alpha(s) <0.70 for each unidimensional scale or subscale
Reliability+ICC or weighted Kappa≥0.70
?
ICC or weighted Kappa not reported
ICC or weighted Kappa<0.70
Measurement error+
?
SDC or LoA<MIC
MIC not defined SDC or LoA>MIC
Hypotheses testing for construct validity+The result is in accordance with the hypothesis
?No hypothesis defined (by the review team)
The result is not in accordance with the hypothesis
Cross‐cultural validity/measurement invariance+No important differences found between group factors (such as age, gender and language) in multiple group factor analysis OR no important DIF for group factors (McFadden’s R2<0.02)
?No multiple group factor analysis OR DIF analysis performed
Important differences between group factors OR DIF were found
Criterion validity+Correlation with gold standard≥0.70 OR AUC≥0.70
?Not all information for '+' reported
Correlation with gold standard<0.70 OR AUC<0.70
Responsiveness+The result is in accordance with the hypothesis OR AUC≥0.70
?No hypothesis defined (by the review team)
The result is not in accordance with the hypothesis OR AUC<0.70
  • ACU, area under the curve; CFA, confirmatory factor analysis; CFI, Comparative Fit Index; CTT, classical test theory; DIF, differential item functioning; ICC, intraclass correlation coefficient; IRT, item response theory; LoA, limits of agreement; MIC, minimal important change; RMSEA, root mean square error of approximation; SDC, smallest detectable change; SRMR, standardised root mean residuals; TLI, Tucker‐Lewis Index.