Domain specification

The domains and subdomains of the scales are based on the conceptual model of women and girls’ empowerment developed by Van Eerdewijk et al.10 While many conceptual frameworks of empowerment have been developed, this framework has been adopted by the Bill & Melinda Gates Foundation (BMGF) to inform their programming, including within and beyond WaSH. Van Eerdewijk et al define empowerment as ‘the expansion of choice and strengthening of voice through the transformation of power relations, so women and girls have more control over their lives and futures.’10 The conceptual model includes three domains of empowerment—resources, agency and institutional structures—each with multiple subdomains.10

Using this conceptual model, we conducted a systematic review of peer-reviewed literature related to empowerment in WaSH. Details of the methods and results of the systematic review are described elsewhere.11 We developed sanitation-specific definitions for each subdomain of empowerment, drawing on the conceptual model’s definitions with adaptations informed by the literature and team members’ expertise (table 2). Adaptations included removing the mention of ‘girls’ and excluding the market and state subdomains of institutional structures, as well as laws and policies, to focus on adult women’s empowerment at individual, household and community levels. Of note, we identified and defined two additional subdomains of empowerment relevant to sanitation from the systematic review: privacy and freedom of movement (within resources and agency, respectively).11

Item generation

Through the systematic review and a parallel landscape analysis, we searched for existing instruments to measure each domain and subdomain of empowerment. As we identified instruments, we collated and assessed them for potential use and/or adaptation in our scales. While the systematic review focused on peer-reviewed publications in WaSH, the landscape analysis included grey literature and was not restricted to the WaSH sector. The landscape analysis identified 62 instruments that had not been identified through the systematic review (online supplemental table 1).

Candidate items were identified or created deductively, through a multistep process.12 First, we identified broad topics that had emerged through the systematic review for each subdomain of empowerment. We simultaneously compiled items identified in the systematic review and landscape analysis, organising them by empowerment subdomain. Where possible, we revised or adapted existing items to align with identified subdomain-specific topics; most newly developed items were developed to fill gaps in emergent topics that existing items did not meet. Across all subdomains, we allowed for some redundancy, formulating items to capture the same latent construct in different ways. Finally, we reviewed the item sets by subdomain of empowerment, focusing on comprehensiveness and alignment with our operational definitions and made revisions as needed. This resulted in 15 draft scales, one for each subdomain of empowerment. All scales had ordinal response options of ‘strongly disagree’, ‘disagree’, ‘agree’ and ‘strongly agree’; frequency response options of ‘never’, ‘sometimes’, ‘often’ and ‘always’ or, in the case of Freedom of Movement, ‘not at all’, ‘only with accompaniment’, ‘alone with permission’, ‘alone if I tell someone’ or ‘alone without telling anyone’

In addition, we developed six draft indices. An index, in contrast to a scale, does not represent a latent construct. Rather than having a shared cause in the form of an underlying latent construct, the items in an index together share an effect on a latent construct.9 We developed five indices to measure women’s actual experiences in relation to the four subdomains of agency (household-level decision-making, community-level decision-making, leadership, collective action and freedom of movement). The sixth index was for safety and security, to measure women’s awareness of actual experiences of sanitation-related violence among women they know. In addition to providing valuable information on women’s actual experiences, the indices also contribute to scale validation. We hypothesised that associations would exist between women’s self-reported experiences (or awareness of others’ experiences) and the corresponding sub-domains of agency, which would enable the assessment of construct validity of the scales.

Face validity and content validity assessment

We assessed face validity and content validity (ie, if items adequately measure the domain of interest) of the initial scales through two methods: cognitive interviews (CIs) and expert review.13 Due to the length of the scales, CI guides were prepared separately for each of the three domains of empowerment (Agency; Resources; and Institutional Structures).

CIs took place in two cities, Tiruchirappalli, India and Kampala, Uganda, in July and August 2019. These cities were selected by BMGF based on the presence of existing BMGF-funded citywide inclusive sanitation (CWIS) programmes and the strength of local partners. In each city, 12 interviewers were recruited, all with research experience and local language skills. Interviewers received a 5-day training on the purpose of the research, key concepts and definitions of empowerment, CI methods, ethics and all aspects of the CI guides, with time to practice and provide feedback on translations and phrasing.

Neighbourhoods were purposively selected in each city with the goal of sampling from different wealth strata. We defined the study population a priori to be adult women (age 18 and older). With the support of local leaders (Uganda) and staff who had previously carried out data collection in these areas (India), the research team purposively sampled participants with the aim of having an equal distribution across three subgroups to represent varied life stages: between ages 18 and 25 and preferably unmarried; between ages 25 and 40 and preferably married or living with a partner; and over age 40. A city coordinator and/or field supervisor was present throughout data collection to guide participant selection, monitor quality control and provide feedback.

In each city, data collection teams—comprised of one interviewer and one note-taker—conducted CIs with 9–16 individuals per domain of empowerment. Interviews were conducted in the local language (Tamil in Tiruchirappalli and Luganda or English in Kampala). Participants were asked to ‘think aloud’ about each survey item, and interviewers probed participants to assess understanding and relevance of each item and to identify opportunities for improving wording and translation. Responses were audio recorded and note-takers took detailed field notes. In addition to the women’s empowerment items, data collection included questions on sociodemographic characteristics of the respondent and her household as well as on access to and behaviours related to sanitation.

Members of our team led daily debriefings with data collection teams and took detailed field notes on data collectors’ reports of scale items that were difficult to administer or understand, confusing or missing response options and other problems. Team members also kept detailed notes throughout the training, data collection and debriefing processes to capture emerging issues and additional topics that should be included or excluded during scale revision.

Key informant interviews (KIIs) were also conducted with WaSH and gender experts in Uganda (n=13) and India (n=20) to provide additional contextual information and insight to inform the inclusion, exclusion or addition of items. Key informants were selected with the assistance of local teams in each site and included representatives from government, non-governmental organisations, the private sector and academia.

Finally, a panel of four expert reviewers evaluated candidate items. Each expert reviewer received the draft scales, our sanitation-specific definitions for each subdomain of empowerment and instructions and background information. They provided item-specific and overall comments and feedback.

Item refinement and survey preparation for Phase 2

We revised the scales based on field notes from the CIs, KIIs, debriefings and feedback from the expert reviewers. Revisions included eliminating, combining and rephrasing items. We also added new items where gaps had been noted, especially sanitation-specific menstruation items and items about interactions between individuals and sanitation service providers. In preparation for phase 2, we added measures to assess the criterion and construct validity of each scale (online supplemental table 2).

Content validity assessment for new and modified items

Following refinement and revision of the scale, beginning in November 2019, trained data collectors (selected from among those who worked on phase 1 of the study) conducted a second round of CIs with 12 individuals each in Tiruchirappalli and Kampala to evaluate newly added items and translations prior to broader survey implementation. Team members carried out rapid analysis of data from the CIs and revised the survey instruments as needed.

Sample size and participant selection in Tiruchirappalli and Kampala

Following revisions from phase 2 CIs, cross-sectional surveys were administered to a sample of respondents in Tiruchirappalli and Kampala.

Consensus is lacking on optimal sample size for scale development. However, the literature suggests a range of 5–15 respondents per item.14–16 We aimed for each scale to have at least 15 responses per item. At the time of the calculations, our longest scale consisted of 61 items, requiring a sample size of 915 respondents. We rounded this number up to 1000 respondents per city to allow for approximately 10% non-response and missing data. We aimed to readminister the survey with 5%–10% of participants (N=50–100) per city to assess test–retest reliability.

As in phase 1, we purposively selected neighbourhoods in each city from low and moderate wealth strata, in coordination with CWIS partner organisations and local government. Specifically, partners provided lists of neighbourhoods in each city and identified several priority areas for data collection. In Tiruchirappalli, our team worked with the CWIS partners to select priority slum neighbourhoods and non-priority middle-income neighbourhoods. In Kampala, our team matched priority neighbourhoods to other neighbourhoods of similar population size and income level, using census data.

Participant selection followed a simple random sampling strategy, specifically a random-walk sampling method. Enumerators were instructed to walk in pairs through selected neighbourhoods, with one enumerator working on each side of the street and to knock on every third door. To be eligible, the respondent needed to be a woman age 18 or older who spoke Tamil (in India) or English or Luganda (in Uganda), who was mentally competent (demonstrated through understanding of the study description and consent) and had no hearing or speech impediments that would prevent comprehension or participation. If the selected household had an individual present who met the inclusion criteria and consented to participate, the enumerator would administer the survey.

Respondents were asked whether they would be willing to participate in the same survey a second time. Those who completed the full survey and agreed to participate again were revisited within 1 month. We selected a 1-month time frame because several of the scales refer to ‘the past month,’ and we aimed to capture responses within an overlapping period to reduce the likelihood of a meaningful change in any domains of women’s empowerment leading to changes in responses to survey items.

Survey implementation and management in two sites

In each site, 12–14 female enumerators were recruited. As in phase 1, enumerators completed 5 days of training covering key concepts of the study, research ethics and logistics, with 2 days for interactive practice administering the survey to one another and talking through any challenges.

Enumerators first obtained oral (India) or written (Uganda) consent, then administered the full survey instrument. The survey instrument included all 15 scales and six indices, plus modules on demographics, water and sanitation access and behaviours, menstruation and additional items to assess validity. Each site had at least one city coordinator and/or field supervisor who provided day-to-day supervision and quality control. Data collection lasted 24–30 days in each site in December 2019 and January 2020.

Data were collected using tablets with Ona software (ona.io). In each site, the survey was programmed in both English and either Tamil (India) or Luganda (Uganda). Field supervisors checked surveys for completeness at the end of each day. Data were then uploaded to a secure data storage platform.

Statistical analysis

Following initial data cleaning and management, we will randomly split the data into equal subsamples for exploratory and confirmatory factor analyses. We will examine summary statistics for items, then follow an a priori analysis plan that includes elements of classical test theory and item response theory (IRT). The analysis plan involves the following steps: item reduction based on missingness, factor extraction, item reduction based on IRT, dimensionality confirmation, measurement invariance assessment, scale scoring, reliability testing and validity testing. Factor analysis will be conducted using Mplus software; all other analyses will be conducted using SAS and R.

Content validity assessment for new and modified items

As in phase 2, trained data collectors will conduct CIs to evaluate newly added items and translations prior to broader survey implementation. CIs will be done in two cities, to be chosen based on data collection schedules. Team members will do a rapid analysis of data from the CIs and revise the survey instruments as needed.

Survey participant selection in three sites

We will administer cross-sectional surveys to a sample of respondents in three cities: Narsapur and Warangal (India) and Lusaka (Zambia). As in prior phases, these cities were selected based on the presence of existing CWIS programmes. Data collection is scheduled to take place first in Narsapur and Warangal (August–October 2021), followed by Lusaka (October–December 2021).

For phase 3, we anticipate having fewer items in the scales and, correspondingly, a smaller sample size. In general, a minimum sample size of 300 for EFA and 300 for CFA is considered good.33 Thus, we will aim for a sample size of 700 respondents per city in phase 3 to ensure a sufficient sample size for EFA and CFA analyses. The strategy for neighbourhood and participant selection, as well as inclusion criteria, will remain the same as in phase 2. Surveys will be implemented in the predominant local language(s).

Data collection and management

As in previous phases, enumerators will be recruited and given a 5-day training. Data collection will follow the same procedures as in phase 2, with enumerators obtaining consent, administering the full survey instrument in the local language(s) most appropriate to each location and entering responses electronically on a tablet. As in phase 2, the survey instrument will include all 15 scales and six indices, plus modules on demographics, water and sanitation access and behaviours, menstruation and additional items to assess validity. In addition, we will assess mental health using the Center for Epidemiologic Studies Depressions Scale-10 and the Patient Health Questionnaire-4, well-being using the WHO Well-Being Index (WHO-5) and life satisfaction using the Personal Wellbeing Index. Finally, we will add survey items related to COVID-19 to assess how sanitation-related behaviours and experiences may have been impacted by the pandemic.

Statistical analysis

For any scales that have been substantially revised based on results from phase 2, we will conduct EFA to assess the factor structure, followed by CFA. Following the same procedures as phase 2, we will use GRM IRT approaches to assess psychometric properties of the new and revised items within these scales. For all scales, we will then conduct CFAs to test the optimal factor structures identified in phase 2. We will use the same fit indices and criteria described above to assess model fit. As in phase 2, we will test for measurement invariance at both the group (city) and item level using multiple-group CFA and IRT methods; calculate the reliability coefficient, coefficient omega and test for convergent, discriminant and external criterion validity.