Article Text

Original research
Assessing the reliability of phone surveys to measure reproductive, maternal and child health knowledge among pregnant women in rural India: a feasibility study
  1. Angela Ng1,
  2. Diwakar Mohan1,
  3. Neha Shah1,
  4. Kerry Scott1,
  5. Osama Ummer2,
  6. Sara Chamberlain3,4,
  7. Aarushi Bhatnagar5,
  8. Diva Dhar6,
  9. Smisha Agarwal1,
  10. Rajani Ved7,8,
  11. Amnesty Elizabeth LeFevre9
  12. Kilkari Impact Evaluation Team
    1. 1Department of International Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, USA
    2. 2Oxford Policy Management, New Delhi, Delhi, India
    3. 3BBC Media Action, New Delhi, Delhi, India
    4. 4BBC Media Action, London, UK
    5. 5Health, Nutrition and Population, World Bank New Delhi Office, New Delhi, India
    6. 6The Bill and Melinda Gates Foundation, Seattle, Washington, USA
    7. 7National Health Systems Resource Centre, New Delhi, Delhi, India
    8. 8The Bill and Melinda Gates Foundation, Delhi, India
    9. 9Division of Public Health and Family Medicine, University of Cape Town, School of Public Health and Family Medicine, Cape Town, South Africa
    1. Correspondence to Angela Ng; angelang1201{at}gmail.com

    Abstract

    Objectives Efforts to understand the factors influencing the uptake of reproductive, maternal, newborn, child health and nutrition (RMNCH&N) services in high disease burden low-resource settings have often focused on face-to-face surveys or direct observations of service delivery. Increasing access to mobile phones has led to growing interest in phone surveys as a rapid, low-cost alternatives to face-to-face surveys. We assess determinants of RMNCH&N knowledge among pregnant women with access to phones and examine the reliability of alternative modalities of survey delivery.

    Participants Women 5–7 months pregnant with access to a phone.

    Setting Four districts of Madhya Pradesh, India.

    Design Cross-sectional surveys administered face-to-face and within 2 weeks, the same surveys were repeated among two random subsamples of the original sample: face-to-face (n=205) and caller-attended telephone interviews (n=375). Bivariate analyses, multivariable linear regression, and prevalence and bias-adjusted kappa scores are presented.

    Results Knowledge scores were low across domains: 52% for maternal nutrition and pregnancy danger signs, 58% for family planning, 47% for essential newborn care, 56% infant and young child feeding, and 58% for infant and young child care. Higher knowledge (≥1 composite score) was associated with older age; higher levels of education and literacy; living in a nuclear family; primary health decision-making; greater attendance in antenatal care and satisfaction with accredited social health activist services. Survey questions had low inter-rater and intermodal reliability (kappa<0.70) with a few exceptions. Questions with the lowest reliability included true/false questions and those with unprompted, multiple response options. Reliability may have been hampered by the sensitivity of the content, lack of privacy, enumerators’ and respondents’ profile differences, rapport, social desirability bias, and/or enumerator’s ability to adequately convey concepts or probe.

    Conclusions Phone surveys are a reliable modality for generating population-level estimates data about pregnant women’s knowledge, however, should not be used for individual-level tracking.

    Trial registration number NCT03576157.

    • community child health
    • maternal medicine
    • neonatal intensive & critical care

    Data availability statement

    Extra data can be found by emailing Amnesty LeFevre at aelefevre@gmail.com Not applicable.

    https://creativecommons.org/licenses/by/4.0/

    This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.

    Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • COVID-19 has accelerated the use of remote data capture strategies, however, few studies have undertaken a rigorous process of tool development inclusive of reliability testing.

    • Study findings suggest that phone surveys can be a reliable modality for measuring knowledge among pregnant women in rural India at a population level, but have low reliability at an individual level.

    • To optimise reliability, limit the use of questions which are culturally sensitive, or have open-ended or multiresponse options.

    • The sample included pregnant women with reported access to a phone during the day; a population arguably more advantaged than women without access to a phone which could limit the generalisability of findings.

    Introduction

    COVID-19 has accelerated the demand for remote data capture solutions, including phone surveys. There is a growing importance and reliance on remote data collection including phone surveys (in India and elsewhere) as COVID-19 has impacted the ease and budgets for data collection.1 Recommendations on questions to consider in determining the suitability of phone surveys to measure varied outcomes are emerging.2 However, despite an increasing body of literature on methods,3 4 including sampling,5 6 ethics and obtaining consent,5 7 8 improving response rates and other performance metrics,9 significant gaps persist in the standards for developing phone survey tools.

    Efforts to measure health practice, prevention and careseeking in India have most commonly been done via face-to-face survey methods, including the population-based National Family Health Surveys (NFHS) carried out approximately every 5–7 years.10 11 In addition to being costly and time-intensive, these population-based surveys are limited by lengthy recall bias.12 Phone surveys have the potential to be administered more proximally to event of focus, at lower cost, with more minimal resources, and with potentially less burden to the beneficiary.

    In 2019–2020, NFHS findings stated that 54% of women have a mobile phone that they themselves use.13 Growing access to mobile phones at a household level, and more specifically among women, provides an opportunity to develop alternative modalities for the routine collection of data on health behaviours and knowledge. Despite their immense potential, few applications of phone surveys have been reported for measuring health knowledge, behaviours and outcomes in India11 and even less information is available on the methods influencing the development of these tools. As phone ownership and access increase in India,14 a reliable phone survey tool would enable timely, routine and low-cost measurement of the population health status, including knowledge, careseeking and practices.15

    In this manuscript, we outline the process for developing and refining a phone survey tool for the measurement of reproductive, maternal, newborn, child health and nutrition (RMNCH&N) knowledge among pregnant women in four districts of Madhya Pradesh. We start by developing overall and domain-specific RMNCH&N knowledge scores and then identify the determinants of higher knowledge scores. We then assess differences in the reliability of the knowledge questions over different modalities including face-to-face surveys at two time points (hereafter called test–retest) and caller-attended telephone interviews (CATIs; hereafter called phone surveys). This study illuminates gaps in RMNCH&N knowledge among pregnant women, determinants of RMNCH&N knowledge levels and the reliability of phone survey tools compared with face-to-face tools.

    Methods

    Study setting

    Data collection activities were carried out from August to October 2018 in four districts (Hoshangabad, Mandsaur, Rewa and Rajgarh) of Madhya Pradesh. Madhya Pradesh is a state in central India with a population of over 75 million, predominantly Hindu, and based in rural areas.16 Significant gaps between men and women persist with regard to literacy (82% of men are literate as compared with 59% of women), and only 29% of women report having access to mobile phones that they themselves can use.17 For nearly all health indicators, Madhya Pradesh falls below national averages. Despite near universal attendance of at least one antenatal care (ANC) visit during pregnancy, only 53% of women receive services within the first trimester, and 36% receive the recommended four visits.17 While institutional delivery rates have increased over time, 20% of women deliver outside the formal health sector, and only 18% receive a health check after birth from a trained provider within 2 days of birth.17 Among children, 51 per 1000 live births will die prior to their first birthday and 65 per 1000 live births will die prior to their fifth birthday.17 High mortality rates are affected by undernutrition (43% of children are underweight, 42% are stunted), low rates of exclusive breast feeding (58%) and gaps in access to basic health services including immunisation. Where the latter is concerned, only 73% of children 12–23 months have received the recommended three doses of the diphtheria, pertussis (whooping cough), and tetanus (DPT) vaccine.17

    Study design and sampling

    The target population was women 5–7 months pregnant at randomisation, ≥18 years of age, speak and understand Hindi, and own or have access to a mobile phone during the morning and afternoon. Survey data and sampling technique were captured as part of baseline survey activities conducted as part of the impact evaluation of the Kilkari mobile health intervention described in detail elsewhere.18 A face-to-face cross-sectional survey was conducted among (n=5095) women with reported access to a mobile phone during the day (Survey Activity A in figure 1). The survey sample size was powered to detect a 5% change in the practice of exclusive breast feeding from baseline to endline (12 months later), assuming an alpha of 0.05, power of 0.80, 20% loss to follow-up and 35% loss of women due to changes in mobile phone numbers (SIM change). Women excluded included BSNL mobile subscribers due to poor network coverage and those who did not provide consent to participate in the trial. Frontline health workers in the study villages and census of villages produced the list of eligible women. The sample was randomised and stratified by gestational age, parity, age of woman and ownership of phone in Stata using the sample command.19

    Figure 1

    Completed interviews for face-to-face and CATI surveys. CATI, caller-attended telephone interview.

    Within 2 weeks of the original survey, two separate surveys were conducted on random samples of these women: test–retest (n=205) and phone surveys (n=375). To assess reproducibility in the surveys, stability of the instrument and inter-rater reliability, a subsample of women who completed the main survey was reinterviewed face-to-face to assess the inter-rater reliability of the responses from the main survey (Survey Activity B in figure 1). In order to develop a validated phone survey tool, pregnant women who previously completed a face-to-face interview and were not interviewed in the test–retest phase were interviewed a second time over the phone to assess the intermodal reliability of the responses from the main survey (Survey Activity C in figure 1).

    Each sample size was calculated assuming a kappa of 0.80, margin of error 0.05, and alpha of 0.5, proportion of positive responses of 0.35 for rater 1 and 0.40 for rater 2: 146 participants were needed. To accommodate a 15% loss to follow-up or refusal for the test–retest, a minimum of 168 women were targeted and ultimately 205 interviewed. Assuming a 20% response rate for the phone survey, a random sample of 880 women were contacted within 1–2 weeks of the initial face-to-face interview, and ultimately 375 completed the interviews.

    Data collection

    The survey included seven modules on: (1) mobile phone ownership, use and digital literacy; (2) pregnancy, delivery, child health and family planning knowledge; (3) birth history; (4) experiences during pregnancy care; (5) content of pregnancy care; (6) delivery and postpartum care intentions; and (7) client satisfaction with accredited social health activist (ASHA) services. Survey questions were drawn from standardised national survey tools, the literature and expert judgement and enhanced through cognitive testing and piloting.18 20 This manuscript focuses on knowledge; findings from other modules are reported elsewhere.18 21

    Female enumerators conducted face-to-face and phone surveys. The phone surveys were conducted out of the National Health Systems Resource Centre in New Delhi. Surveyors had a minimum of high school education and received training on tool content and survey implementation, including informed consent. Supervisors provided day-to-day support and oversight.

    Data analysis

    Data were analysed using Stata V.15 (Stata Corporation, College Station, Texas, USA). First, a conceptual framework was developed comprised of factors believed to influence pregnant women’s knowledge from previous literature (figure 2).18–35 At the individual level, characteristics such as phone ownership, healthcare characteristics, socioeconomic and sociodemographic factors were considered. A wealth index was created by dividing the women into five quintiles using principal component analysis. At the community level, social gender norms were analysed such as participation in self-help groups, women being the primary health decision-maker and family living structure as well as geographical characteristics of the community.

    Figure 2

    Conceptual framework for identifying the determinants of pregnant women’s RMNCH&N knowledge. ANC, antenatal care; ASHA, accredited social health activist; RMNCH&N, reproductive, maternal, newborn, child health and nutrition.

    In order to best categorise knowledge among pregnant women, survey questions were consolidated into five thematic domains: (1) maternal nutrition and pregnancy danger signs, (2) family planning, (3) essential newborn care, (4) infant and young child feeding, and (5) infant and young child care (online supplemental table 1). For each question, if there was only one correct answer, a binary score was given: 1 if the women knew the correct answer, 0 if not. If there were multiple correct answer options, each correct option was weighted so if all correct options were selected, the final score for that question would be 1. Each question then had equal weight in its category. All the knowledge domains were averaged together at the end to create a final composite score.

    A simple linear regression was performed on the final composite score with the factors thought to have an impact on them. A multiple linear regression was performed to see all together which factors would have an impact on overall women’s knowledge. In order to find the best model, only variables with a p value of 0.20 or less during bivariate analyses were used. A multiple linear regression was also performed on each individual knowledge domain to see if particular factors influenced one category more than another.

    If the difference in prevalence between survey questions was <15%, kappa scores were used to assess the reliability of the question. Questions were deemed unreliable if the difference in prevalence between survey questions was >15% or if the Cronbach’s alpha was 0.7 or higher.22 Kappa coefficients were adjusted for the difference in prevalence levels using the Prevalence Adjusted Bias Adjusted Kappa score. This was to determine if the questions themselves were reliable, and if so, if the question was reliable over the phone. If the threshold of 0.7 was met for just the test–retest, the question was deemed to be reliable for face-to-face surveys in this context. If the threshold of 0.7 was met for both test–retest and phone survey, the question was deemed reliable for phone surveys in this context.

    Patient and public involvement

    Pregnant women were first engaged upon identification in their households as part of a household listing carried out in mid/late 2018. A small number of pregnant women were involved in the refinement of survey tools through qualitative interviews, including cognitive interviews, which were carried out to optimise survey questions, including the language and translation used. COVID-19 and associated travel restrictions prevented further engagement with women interviewed to disseminate study findings.

    Results

    Pregnant women characteristics

    Among 5095 interviewed through the face-to-face survey, 95% were Hindu, 58% were between 18 and 24 years of age, 68% had at least one child, 47% belonged to Other Backward Castes, 8% belong to a self-help group, 41% had completed high school or higher, and 57% were able to read the whole sentence (table 1). A majority of the women interviewed resided in Rewa (57%), followed by Mandsaur, Hoshangabad, and Rajgarh falling at 16%, 8%, and 19%, respectively.

    Table 1

    Characteristics of pregnant women drawn from a face-to-face survey in four districts of Madhya Pradesh

    Survey reliability

    For each survey question, inter-rater reliability was assessed by comparing reported results from face-to-face surveys conducted at two time points (test–retest; n=205), while intermodal reliability was assessed by comparing findings from the face-to-face survey with phone surveys (n=375). Table 2 presents prevalence and reliability estimates by question.

    Table 2

    Prevalence and reliability estimates for knowledge questions among pregnant women

    Overall results suggest that population-level estimates of knowledge over time and across modalities were similar, but reliability among individual women in the sample was poor. Prevalence estimates illustrate the latter, demonstrating that women’s knowledge across and within domains was low. Questions with low reliability included true/false questions and those with unprompted, multiple response options.

    Figure 3 graphs the correlation between the prevalence and bias-adjusted kappa statistics for face-to-face surveys against the in-person retest and the phone survey. The correlation forms a positive linear line, representing a strong positive reliability between the face-to-face and phone survey options.

    Figure 3

    Measurement of degree in which repeated measurements in pregnant women interviewed (test–retest) provide similar answers.

    Determinants of RMNCH&N knowledge

    Figure 4 shows average knowledge scores for the five composite knowledge domains: maternal nutrition and pregnancy danger signs (52%), family planning score (58%), immediate post-birth care for the neonate (47%), infant and young child feeding (56%), and infant and young child care (58%). The average of all domains was used to create a final composite score (54%).

    Figure 4

    Average knowledge scores for composite knowledge domains.

    Table 3 presents results from bivariate and multivariate regressions on factors influencing pregnant women’s knowledge (n=5095). In the multivariable model, women’s knowledge was significantly higher among older women (25–34 years, p<0.001; >34 years, p=0.02); with higher education (p=0.009); able to read a full sentence (p<0.001); living in a nuclear family (p<0.001); who reported being the primary decision-maker in health decisions (p<0.001); had attended a greater number of ANC visits (p<0.001) and reported being satisfied with ASHA services (p<0.001). Women from the rural districts of Mandsaur and Rajgarh had significantly lower knowledge scores than women in the reference periurban district of Hoshangabad. Women’s phone ownership, socioeconomic status and caste were not associated with significant differences in knowledge.

    Table 3

    Factors associated with pregnant women’s RMNCH&N knowledge in four districts of Madhya Pradesh, India

    Online supplemental table 2 explores determinants of knowledge for each of the five domains. Across all domains, knowledge was significantly higher among older women (25–34 years of age), with a least one child, able to read a whole sentence and who reported being satisfied with services provided by the ASHA. Knowledge of maternal nutrition and pregnancy danger signs was significantly higher among women living in a nuclear family, with higher education and who had attended a greater number of ANC visits. Family planning knowledge was significantly higher among women in the highest socioeconomic strata, with primary school or higher education. Knowledge of immediate post-birth care for the newborns was similarly higher among women in the highest socioeconomic strata and with higher education.

    Discussion

    The study sought to develop and refine a phone survey tool for the measurement of RMNCH&N knowledge among pregnant women in four districts of Madhya Pradesh. Findings suggest that there is a correlation between face-to-face and retest reliability and face-to-face and phone reliability. Instances were not observed where there was retest reliability for face-to-face and poor reliability for face-to-face versus phone. Questions with unprompted and more possible response options (eg, foods to eat during pregnancy) had similarly poor inter-rater and intermodal reliability as questions that sought to pin respondents down to one answer (eg, How many tetanus injections should you have during pregnancy?). This suggests that the questions themselves may not be well suited to repeat questioning. Overall knowledge levels were low for most questions asked. This underscores the need to bolster awareness among women and improve access to health information. It too may indicate that some questions and response options used were not culturally appropriate or relevant for this context (eg, alcohol, coffee and cigarettes are uncommonly consumed in rural India by women which may explain the low awareness of these items when asked ‘What should you not eat or drink during your pregnancy?’). The collective body of findings suggests that survey estimates may be appropriate for ascertaining population-level estimates rapidly, however, contraindicated in instances where longitudinal tracking or individual-level analyses are required. This may have critical implications for those looking to use remote surveys for measuring intermediate programme outcomes, including knowledge.

    Study findings contribute to the limited body of literature on how to optimise the development of phone surveys for use in the measurement of population-level knowledge and health outcomes. Few studies to date have sought to refine phone survey tools through a rigourous process of reliability testing.23 Findings from a 2017 systematic review identified only 10 studies which have reported using reliability testing to develop phone survey tools; five of these included face-to-face and CATI survey comparisons.23 In Honduras and Peru,24 comparisons were made across independent samples, while in Lebanon25 and two separate surveys in Brazil,26 comparisons were made using the same sample across two modalities (as in our study).18 22 Overall findings broadly suggest that there was concordance in the results across CATI and face-to-face modalities.21 However, for some outcomes, prevalence estimates did differ across modalities.27 Studies in Honduras and Peru compared additional phone survey modalities, including text messages and interactive voice response (IVR), and concluded that CATI surveys had the lowest discordance with the face-to-face surveys.24 Elsewhere, more recent efforts have sought to compare the reliability of alternative phone survey delivery modalities (IVR vs CATI) for measuring non-communicable disease risk factors.28 In our context, low literacy rates precluded testing phone survey modalities other than CATI surveys.

    Underpinning efforts to rigorously develop the phone survey tool was a design to measure knowledge among pregnant women. Overall knowledge levels among pregnant women in these four districts of Madhya Pradesh ranged from 47% for essential newborn care to 58% for family planning. Areas of lower knowledge included what should be put on the cord after delivery, how to keep the baby warm after delivery, what foods to eat during pregnancy, what food and drinks to avoid during pregnancy, etc. Knowledge scores were significantly higher among older women; those with higher levels of education and literacy; living in a nuclear family; who reported being the primary decision-maker in health decisions; who had attended a greater number of ANC visits and reported being satisfied with ASHA services.

    Other studies have measured women’s knowledge levels on topics such as neonatal danger signs, immediate post-birth care for the neonate, infant and young child feeding, breast feeding and family planning. Compared with 18% of women in Ethiopia and 15% of women in Uganda who have good knowledge of neonatal danger signs, our women have a higher relative knowledge at 58%.29 30 In a survey among hospital-delivered mothers in Sri Lanka, while 90% of mothers knew about advantages of breast feeding compared with our survey of 56%, only 22% correctly answered what should be applied to an umbilical cord compared with our survey at 47%.31 In India, universal knowledge existed among married women in Uttar Pradesh about at least one family planning method and 90% of women in Lucknow were aware of contraceptive methods in family planning existed comparatively to our study’s lower family planning knowledge at 58%.32 33 This is all subject to each study’s own verified survey methods and composite score generation. Developing one standardised survey questionnaire across studies to measure women’s RMNCH&N knowledge would lend more insight into how comparative these data are.

    There is limited empirical evidence of factors influencing pregnant women’s knowledge. A study conducted in Delhi assessed knowledge components that included immunisation against tetanus, number of antenatal visits, folic acid tablets, danger signs during pregnancy or labour, and discussions with elders, mothers-in-law, doctors, or friends.34 Like our study, it found that women in joint families were also significantly more likely to have higher knowledge.34 Another study in Rewa district also found that women whose husbands and in-laws played a dominant role in decision-making had lower knowledge of key danger signs during pregnancy.35 This is consistent with our study, which included Rewa district among our four study districts, as we found if the women played a role in the decision-making process, she had significantly higher knowledge of maternal danger signs. In a Uganda study, no association was found with attending ANC visits and knowledge of danger signs, which is inconsistent with our findings.30 Surprisingly, while NFHS found that more family planning methods were seen in women of higher wealth index, wealth was not found to be a determinant in this study.11 Taken together, our findings suggest that to increase women’s knowledge in the future, one should empower women to play a larger role in the health decision-making process, strengthen ASHA services, promote education, and increase access and uptake of ANC services. Further research to understand sources and quality of sources for knowledge should be undertaken as well.

    Limitations

    This survey was limited to RMNCH&N knowledge in women 5–7 months pregnant with access to a mobile phone during the day, in four predominately rural districts of Madhya Pradesh. This may have led to a pro-rich bias in the sampling given the profile of beneficiaries with access to mobile phones.13 Survey questions were derived from expert review and large national survey tools, including the demographic and health survey for India and the Multiple Indicator Cluster Survey. Cognitive interviews were used to refine questions on respectful maternity care, a process described elsewhere.19 20 36 Ideally, the same process would have been followed for refining knowledge questions. However, budget and time constraints made this added step infeasible. Phone surveys were administered by Delhi-based enumerators whose demographic profile differed from respondents. Alternative phone survey modalities, including text messages, IVR and Unstructured Supplementary Service Data (USSD), were considered but deemed less desirable to using CATI surveys, which could allow beneficiaries to speak with and ask for clarification questions.

    Conclusions

    Study findings offer insight into the challenges associated with reliably measuring RMNCH&N knowledge among pregnant women using face-to-face and remote survey tools. Population-level estimates of knowledge over time and across modalities were similar, but reliability among individual women in our sample was poor. This suggests that the use of phone surveys to measure knowledge is contraindicated in instances where longitudinal tracking or individual-level analyses are required; however, it may be appropriate as a means of gaining population-level insights. Among women assessed, overall knowledge levels were low; however, knowledge scores were significantly higher among older women; those with higher levels of education and literacy; living in a nuclear family; who reported being the primary decision-maker in health decisions; who had attended a greater number of ANC visits and reported being satisfied with ASHA services.

    Data availability statement

    Extra data can be found by emailing Amnesty LeFevre at aelefevre@gmail.com Not applicable.

    Ethics statements

    Patient consent for publication

    Ethics approval

    Institutional Review Boards from the Johns Hopkins Bloomberg School of Public Health in Baltimore, Maryland, USA and Sigma Research and Consulting in Delhi, India provided ethical clearance for study activities. Verbal informed consent was obtained from all study participants.

    Acknowledgments

    We thank the women and families of Madhya Pradesh who generously gave their time to support this work. We are humbled by the opportunity to convey their perspectives and experiences. We additionally are grateful to Neha Dumke and the team at the National Health Systems Resource Centre (NHSRC) for their support. This work was made possible by the Bill and Melinda Gates Foundation. We thank Diva Dhar, Suhel Bidani, Rahul Mullick, Dr Suneeta Krishnan, Dr Neeta Goel and Dr Priya Nanda for believing in us and giving us this opportunity. We additionally wish to thank the larger team of enumerators from OPM-India and NHSRC who worked tirelessly over many months to implement the surveys that form the backbone of our analyses. We additionally thank Vinit Pattnaik at OPM and Erica Crawford at Johns Hopkins University for the financial management support of our work.

    References

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • Twitter @kerfully, @OsamaUmmer?t=ac8vollkoRHLSuYvW_n5mQ&s=08, @Amnesty_LeFevre

    • Collaborators Kilkari Impact Evaluation Team: Smisha Agarwal, Salil Arora, Jean JH Bashingwa, Aarushi Bhatanagar, Sara Chamberlain, Rakesh Chandra, Arpita Chakraborty, Neha Dumke, Priyanka Dutt, Anna Godfrey, Suresh Gopalakrishnan, Nayan Kumar, Simone Honikman, Alain Labrique, Amnesty LeFevre, Jai Mendiratta, Molly Miller, Radharani Mitra, Diwakar Mohan, Deshen Moodley, Nicola Mulder, Angela Ng, Dilip Parida, Nehru Penugonda, Sai Rahul, Shiv Rajput, Neha Shah, Kerry Scott, Aashaka Shinde, Aaditya Singh, Nicki Tiffin, Osama Ummer, Rajani Ved, Falyn Weiss, Sonia Whitehead.

    • Contributors AEL is the overall study PI and conceived the idea for this study along with DM, AB, RV, KS and SC. AEL is responsibile for the overall content and serves as the guarantor. Data collection tools were designed by AEL, NS, DM, KS, AB, SA, OU and RV with inputs from DD. Phone survey data collection was led and supervised by NS and RV. Face-to-face data collection was supervised by AB, OU, AEL, DM and AEL. AN and NS led the analyses with oversight from DM and inputs from AEL. AN and AEL wrote the manuscripts with edits and inputs from all coauthors. AN generated all final tables and figures.

    • Funding Bill and Melinda Gates Foundation.

    • Disclaimer The study sponsors had no role in the design, implementation or interpretation of study findings. The study team were independent from the funders.

    • Competing interests All authors have completed the Unified Competing Interest form (available on request from the corresponding author) and declare that the research reported was funded by the Bill and Melinda Gates Foundation. AG, SC, PD are employed by BBC Media Action, one of the entities supporting programme implementation. The authors do not have other relationships and are not engaged in activities that could appear to have influenced the submitted work.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.