Scale development activity | Procedures |
Phase 1: item development | |
1.1 Domain specification | Literature review. |
1.2 Item generation | Literature review and Delphi methodology. |
1.3 Content validity | By target population: two styles of cognitive interviews were used in the first eight sites, building on Delphi methodology. |
1.4 Face validity | Pretesting and debriefing with enumerators at each site. |
Phase 2: scale development | |
2.1 Data collection | Enumerator training and survey implementation. |
2.2 Item reduction | We will drop items with cumulative missing cases >30% (ie, ‘don’t know’, ‘non-applicable’ or true missing responses) in any one site. |
We will assess the performance of each item’s variation with other items in the scale using a correlation matrix; items with very low (<0.30) interitem correlation coefficients and very low (<0.30) item-total correlation coefficients across multiple sites will be considered for deletion, as will items that misfit the model, that is, with residual correlations >0.20. | |
Item reduction in Rasch paradigm: item severity and item discrimination test. | |
2.3 Identify factor structure | We will use factor analysis across multiple sites to test for factor structure; items with very low factor loadings (<0.30), split factor loadings (high factor loadings (>0.50) in two domains) and high residual variances will be considered for deletion. |
2.4 Assess measurement equivalence | We will use multigroup confirmatory analysis (a form of measurement invariance) on data from multiple sites to test for exact invariance in the hypothetical scale; invariance will be assessed in terms of factor structure (configural model), factor loadings (matric model), mean intercepts (scalar model) and factor means (strict model). |
We will use confirmatory factor analysis alignment optimisation to estimate the group-specific factor means and variances of scale items across all sites; it assesses approximate invariance of scale items across multiple sites. | |
Phase 3: scale evaluation | |
3.1 Score scale items | Finalised scale items will be used in their unweighted form as sum scores or in weighted form as factor scores. |
3.2 Assess reliability (internal consistency) of scale items | We will use Cronbach’s alpha and the Rasch reliability statistic to test the internal consistency of the scale items within each site and aggregated across sites. |
3.3 Assess scale validity | We will measure predictive, convergent and discriminant validity of the final scale items using criteria that were selected based on their strong theoretical relevance in the water insecurity literature; tests of water insecurity differences between ‘known groups’ will also be performed. |
*Adapted from ref 34.