Measurement property | Rating | Criteria |
---|---|---|
Structural validity The degree to which the scores of an instrument are an adequate reflection of the dimensionality of the construct to be measure | + | CTT: CFA: CFI or TLI or comparable measure > 0.95 OR RMSEA < 0.06 OR SRMR < 0.08a EFA/PCA: Rotation method specified (e.g. varimax, promax, oblimin, etc.)b AND Variance explained (total and/or per factor/component) reportedb IRT/Rasch: No violation of unidimensionality: CFI or TLI or comparable measure > 0.95 OR RMSEA < 0.06 OR SRMR < 0.08 AND no violation of local independence: residual correlations among the items after controlling for the dominant factor < 0.20 OR Q3's < 0.37 AND no violation of monotonicity: adequate looking graphs OR item scalability > 0.30 AND adequate model fit: IRT: χ2 > 0.01 Rasch: infit and outfit mean squares ≥ 0.5 and ≤ 1.5 OR Z-standardized values > ‐2 and < 2 |
? | CTT: Not all information for ‘ + ’ reported IRT/Rasch: Model fit not reported | |
– | Criteria for ‘ + ’ not met | |
Internal consistency The degree of interrelatedness among items | + | At least low evidence for sufficient structural validity AND Cronbach's alpha(s) ≥ 0.70 for each unidimensional scale or subscale |
? | Criteria for “At least low evidence for sufficient structural validity” not met | |
– | At least low evidence for sufficient structural validity AND Cronbach’s alpha(s) < 0.70 for each unidimensional scale or subscale | |
Reliability The extent to which scores for patients who have not changed are the same for repeated measurement under the following conditions: over time (test–retest); by different persons on the same occasion (inter-rater) | + | ICC or weighted Kappa ≥ 0.70 |
? | ICC or weighted Kappa not reported | |
– | ICC or weighted Kappa < 0.70 | |
Measurement error The systematic and random error of a patient’s score that is not attributed to true changes in the construct to be measured | + | SDC or LoA < MIC |
? | MIC not defined | |
– | SDC or LoA > MIC | |
Hypothesis testing for construct validity The degree to which the scores of an instrument are consistent with hypotheses, based on the assumption that the instrument validly measures the construct to be measured | + | The result is in accordance with the hypothesis |
? | No hypothesis defined (by the review team) | |
– | The result is not in accordance with the hypothesis | |
Responsiveness The ability of an instrument to detect change over time in the construct to be measured | + | The result is in accordance with the hypothesis OR AUC ≥ 0.70 |
? | No hypothesis defined (by the review team) | |
– | The result is not in accordance with the hypothesis OR AUC < 0.70 | |
Interpretability The degree to which one can assign qualitative meaning to an instrument’s quantitative scores or changes in scores | Not applicable |