Article Text

Original research
WHO Standards-based questionnaire to measure health workers’ perspective on the quality of care around the time of childbirth in the WHO European region: development and mixed-methods validation in six countries
  1. Emanuelle Pessa Valente1,
  2. Benedetta Covi1,
  3. Ilaria Mariani1,
  4. Sandra Morano2,
  5. Marina Otelea3,4,
  6. Ioana Nanu5,
  7. Micaela Iuliana Nanu5,
  8. Helen Elden6,7,
  9. Karolina Linden6,
  10. Mehreen Zaigham8,
  11. Eline Skirnisdottir Vik9,
  12. Sigrun Kongslien10,
  13. Ingvild Nedberg10,
  14. Raquel Costa11,12,13,
  15. Carina Rodrigues11,12,
  16. Heloísa Dias14,
  17. Daniela Drandić15,
  18. Magdalena Kurbanović16,
  19. Emma Sacks17,
  20. Moise Muzigaba18,
  21. Ornella Lincetto18,
  22. Marzia Lazzerini1
  23. IMAgiNE EURO Study Group
    1. 1 WHO Collaborating Centre for Maternal and Child Health, Institute for Maternal and Child Health - IRCCS “Burlo Garofolo”, Trieste, Italy
    2. 2 Medical School and Midwifery School, Genoa University, Genoa, Italy
    3. 3 University of Medicine and Pharmacy Carol Davila, Bucharest, Romania
    4. 4 SAMAS Association, Bucharest, Romania
    5. 5 National Institute for Mother and Child Health "Alessandrescu - Rusescu", Bucharest, Romania
    6. 6 Institute of Health and Care Sciences, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden
    7. 7 Region Västra Götaland, Department of Obstetrics and Gynecology, Sahlgrenska University Hospital, Gothenburg, Sweden
    8. 8 Department of Obstetrics and Gynecology - Institution of Clinical Sciences Lund, Lund University, Lund and Skåne University Hospital, Malmö, Sweden
    9. 9 Department of Health and Caring Sciences, Western Norway University of Applied Sciences, Bergen, Norway
    10. 10 Department of Health and Care Sciences, Faculty of Health Sciences, UiT The Arctic University of Norway, Tromsø, Norway
    11. 11 EPIUnit - Institute of Public Health, University of Porto, Porto, Portugal
    12. 12 Laboratory for Integrative and Translational Research in Population Health (ITR), Porto, Portugal
    13. 13 Human-Environment Interaction Lab, Universidade Lusófona, Porto, Portugal
    14. 14 Regional Health Administration of the Algarve, IP (ARS - Algarve), Albufeira, Portugal
    15. 15 Roda – Parents in Action, Zagreb, Croatia
    16. 16 Faculty of Health Studies, University of Rijeka, Rijeka, Croatia
    17. 17 Department of International Health, Johns Hopkins University, Baltimore, Maryland, USA
    18. 18 Department of Maternal, Newborn, Child and Adolescent Health and Ageing, World Health Organization, Geneva, Switzerland
    1. Correspondence to Dr Emanuelle Pessa Valente; emanuelle.pessavalente{at}burlo.trieste.it

    Abstract

    Objectives Develop and validate a WHO Standards-based online questionnaire to measure the quality of maternal and newborn care (QMNC) around the time of childbirth from the health workers’ perspective.

    Design Mixed-methods study.

    Setting Six countries of the WHO European Region.

    Participants and methods The questionnaire is based on lessons learnt in previous studies, and was developed in three sequential phases: (1) WHO Quality Measures were prioritised and content, construct and face validity were assessed through a Delphi involving a multidisciplinary board of experts from 11 countries of the WHO European Region; (2) translation/back translation of the English version was conducted following The Professional Society for Health Economics and Outcomes Research guidelines; (3) internal consistency, intrarater reliability and acceptability were assessed among 600 health workers in six countries.

    Results The questionnaire included 40 items based on WHO Standards Quality Measures, equally divided into four domains: provision of care, experience of care, availability of human and physical resources, organisational changes due to COVID-19; and its organised in six sections. It was translated/back translated in 12 languages: Bosnian, Croatian, French, German, Italian, Norwegian, Portuguese, Romanian, Russian, Slovenian, Spanish and Swedish. The Cronbach’s alpha values were ≥0.70 for each questionnaire section where questions were hypothesised to be interrelated, indicating good internal consistence. Cohen K or Gwet’s AC1 values were ≥0.60, suggesting good intrarater reliability, except for one question. Acceptability was good with only 1.70% of health workers requesting minimal changes in question wording.

    Conclusions Findings suggest that the questionnaire has good content, construct, face validity, internal consistency, intrarater reliability and acceptability in six countries of the WHO European Region. Future studies may further explore the questionnaire’s use in other countries, and how to translate evidence generated by this tool into policies to improve the QMNC.

    Trail registration number NCT04847336

    • Quality in health care
    • OBSTETRICS
    • NEONATOLOGY

    Data availability statement

    Data are available on reasonable request. All data relevant to the study are included in the article or uploaded as online supplemental information. All relevant data are provided in the paper. Additional details can be provided by contacting the corresponding author with a reasonable request.

    http://creativecommons.org/licenses/by-nc/4.0/

    This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

    Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • This is a multicountry study on the development and validation of a WHO Standards-based questionnaire for conducting online surveys on quality of maternal and newborn care, from the perspective of health workers in WHO European region.

    • The major strength of the questionnaire is the multiphase approach used for its development: the questionnaire was based on lessons learnt and adapted from a pilot study; content, construct and face validity were assessed throughout a Delphi study among a multidisciplinary group of experts; the tool was then translated and back translated following the Professional Society for Health Economics and Outcomes Research guidance.

    • Internal consistency, intrarater reliability and acceptability were assessed among a large sample of health workers from six countries of WHO European Region.

    • One disadvantage is that the process of validation can be quite lengthy, and, as such especially in the context of a pandemic such as COVID-19.

    Introduction

    High-quality respectful care around the time of childbirth is a fundamental aspect of human rights and, according to recent global estimates, could prevent more than 100 000 maternal deaths and 1.3 million neonatal deaths annually.1–3 Despite some maternal and newborn health indicators in high-income countries being better in comparison to low-income and middle-income countries, existing evidence shows that improvements are needed in the quality of care provided to women and newborns in every country.4–9

    The COVID-19 pandemic has challenged health systems worldwide and increased pre-existing fragilities such as the shortage of skilled professionals and equipment, potentially exacerbating health inequities and increasing social and economic disparities, both among and within countries.10–13 Rapid changes in the workplace and in the procedures of delivering care have constrained the quality of maternal and newborn care (QMNC) and has increased stress among health workers.14 15 Global maternal and fetal outcomes have worsened during the COVID-19 pandemic, with an increase in maternal deaths, stillbirths, and maternal depression.16–18 The pandemic has also amplified the need to improve data collection systems, to enhance the monitoring of key indicators, and to better manage the public health response to current and future emergencies.11 12 19–21

    In 2016, the WHO developed a framework22 and a list of Standards23 for improving the QMNC. The WHO Standards23 define a set of 318 Quality Measures, divided into three key domains—experience of care, provision of care and availability of resources—which can be used by hospital managers to assess the QMNC. Many of these WHO Quality Measures—such as those related to the availability of equipment, training opportunities and quality improvement initiatives—should be assessed by utilising health workers as one of the sources of data. Both service providers’ and services users’ perspectives are critical to assess QMNC and get important suggestions for health system improvement.24 Health workers are the cornerstone of any health system, having a key role in contributing to health services preparedness and response to emergencies, but often they are poorly involved in designing the quality improvement mechanisms.24–27 Exploring health workers’ perspectives on key aspects of provision of care, experience of care, availability of resources and the reorganisation of the health services will provide critical information on the QMNC, but also has the potential, if properly implemented through a participatory quality improvement approach, to increase staff ownership on critical aspects of QMNC, to improve working conditions and to increase motivation of workforces.24–26

    There are a lack of WHO Standards-based validated tools for collecting data on health workers perspectives of the QMNC.13 28–31 A unified comprehensive approach to measure QMNC as defined by the WHO Standards, through validated tools, would allow for comparisons of data across settings and over time, and allow for efficient monitoring.22–25 In particular, in the context of the COVID-19 pandemic, innovative methods enabling rapid data collection from multiple countries is essential.

    Since 2016, the WHO Collaborating Centre for Maternal and Child Health Burlo Garofolo, Trieste, Italy, has worked on developing and validating tools to collect data on priority WHO Quality Measures. Pilot studies were conducted in Italy between 2016 and 201929–33 and were scaled up in a multicountry project in the WHO European Region, called IMAgiNE (Improving Maternal Newborn CarE).34

    Through the IMAgiNE study research network, two complementary questionnaires were conceived: one questionnaire to collect key WHO Quality Measures from the perspective of key service users (ie, mothers) and one from the perspective of health workers, each including 40 priority WHO Quality Measures. Results on the development, validation and use of the questionnaire from the perspective of service users has been reported elsewhere.30 This paper describes the process of development and validation of the second tool, the questionnaire for health workers.

    Methods

    Pilot studies

    The IMAgiNE health workers’ questionnaire was developed based on pilot studies conducted between 2016 and 2019; these studies have been described elsewhere.29 Briefly, development of the tools in the pilot studies included an extensive literature review (online supplemental table 1) and a Delphi study with a multidisciplinary group of international experts to assess content validity, construct coherence and face validity.29 Thereafter, the questionnaire was field tested to further assess face validity and evaluate acceptability, and the tool was improved at each stage.29 Finally, it was used in one tertiary facility in Italy, showing good acceptability. In all, 105/136 (77.2%) of the health workers answered the questionnaire, with good utility: data were reported to be used by 35 decision-makers for developing written recommendations for improving QMNC in their facilities.29 This version of the questionnaire included 117 multiple choice questions, plus three questions on sociodemographic variables and one open-ended question.

    Supplemental material

    Development of the questionnaire for the WHO European Region

    During July 2020–June 2021, the questionnaire was further optimised and adapted for the IMAgiNE project among countries of the WHO European Region, and updated to collect data on QMNC during the COVID-19 pandemic. The process included three phases (figure 1), based on recommendations for development of health-related questionnaires35–40 and previous pilot studies.29–33 The characteristics of the questionnaire are synthesised in table 1. Considering logistic limitations of printed questionnaires during the COVID-19 pandemic and the possibility of using diverse recruitment strategies (ie, institutional emails and websites) according to the professional profile of the study population41–43 an online format was chosen. Online questionnaires can be fielded quicker, and less expensively, than traditional mail questionnaires.43 It was predefined that the questionnaire had to collect a set of 40 prioritised WHO Quality Measures,23 ten for each of the four domains of the tool: provision of care, experience of care, availability of human and physical resources, and organisational changes related to the COVID-19 pandemic response.

    Figure 1

    Phases of questionnaire development and multicountry validation.

    Table 1

    Characteristics of the IMAgiNE questionnaire for health workers

    Since the process was carried forward over 12 months, and the research network of the IMAgiNE study was growing in the meanwhile, during subsequent phases an increasing number of partners from different countries was involved.

    Phase 1: content, construct and face validity

    First, the questionnaire developed for the pilot study in Italy29 was translated and back translated into English, by native speakers, and was then adapted for an online survey.

    Second, additional items related to the implementation of appropriate procedures and resources for the COVID-19 pandemic were developed in order to assess the health facilities’ preparedness and response during the COVID-19 pandemic. During March to July 2020, relevant WHO guidelines, professional association statements, protocols released by European countries, and studies were searched and selected for review by three experts (EPV, BC and ML). Additionally, reference lists of selected studies were handsearched, grey literature related to respectful care and health services preparedness to COVID-19 pandemic was reviewed using Google, and experts in the field (from WHO and from other networks) were consulted.13 44–64

    Third, 40 of the existing WHO Quality Measures were prioritised through a Delphi process, involving two rounds. The Delphi process35–40 was carried out among a multidisciplinary group of 26 experts (psychologists, physicians, midwives, lactation consultants, and reproductive rights advocates) from 11 countries of the WHO European Region. Experts were asked to: (1) prioritise the 40 Quality Measures, ten for each of the four domains of the questionnaire; (2) assess comprehensiveness and clarity of the questions and suggest rewording; (3) suggest additional relevant questions and (4) provide any other comments on content, construct, structure, face validity and expected acceptability of the questionnaire. An ad hoc template was used for this purpose and criteria to reach consensus were predefined.38 Questions were developed in two different and parallel pathways: one for health workers providing care in the maternal area and one for health workers providing care in neonatal area. The questionnaire structure was developed based on the principle of increasing comprehension and acceptability from respondents: interrelated questions were numbered in a logical sequence,35 and were organised in six sections (eg, sections A, B, C…).

    In addition, a QMNC index, to be used as a complementary measure of QMNC, in line with other studies,30 31 65–67 was developed through the Delphi process. Further testing of this index is ongoing.

    Phase 2: translation and back translation

    Translation and back translations from English to other languages were performed by native speakers that are experts in healthcare and/or health research, and project partners, following the steps of Professional Society for Health Economics and Outcomes Research Task Force for Translation and Cultural Adaptation Principles of Good Practice, which include: (1) preparation; (2) forward translation; (3) reconciliation; (4) back translation; (5) back translation review; (6) harmonisation; (7) cognitive debriefing; (8) review of cognitive debriefing results; (9) proofreading and (10) final report.40 An ad hoc template was used, allowing the translation and back translation of each question separately (online supplemental table 2).

    Phase 3: internal consistency, intrarater reliability and acceptability in six countries

    This phase was performed using data from the following European regions and countries: South Europe (Italy and Portugal), Scandinavia (Norway and Sweden) and East Europe (Croatia and Romania). Data were recorded using REDCap V.8.5.21 —2021 Vanderbilt University, via a centralised platform.

    Internal consistency was analysed using Cronbach’s alpha correlation (alpha) for all sections where questions were meant to be interrelated (ie, sections C, D and E). For the three European regions, a required sample size of 104 respondents for analysis of work organisation, data management and communication (section C), 106 for Quality of care and practices performed (section D), and 104 for COVID-19 preparedness and response (section E) was calculated. In the null hypothesis an alpha of 0.55, in the alternative hypothesis an alpha of at least 0.70, 80% power was assumed, several items equal to 8, 12, and 16 for sections C, D and E, respectively, and a significance level of 2.5% with a one-tailed test. Internal consistency was considered good whenever Cronbach’s alpha ≥0.70.68

    Intrarater reliability was analysed on all questions on Quality Measures, using the Cohen’s kappa (K) statistic69 by inviting volunteer health workers to answer the questionnaire twice (test–retest responses with a maximum time gap of 7 days between the two responses). The estimated minimum sample size was 89 health workers, assuming in the null hypothesis a K value of 0.35, in the alternative hypothesis a K of at least 0.60, 80% power, a significance level of 2.5% with a one-tailed test and an anticipated proportion of the three possible answers of 0.10, 0.30 and 0.60. As additional parameter of intra-rater reliability, the Gwet AC1 was calculated to consider the possibility of Cohen’s Kappa paradox (ie, low kappa values in presence of a high degree of agreement due to substantial imbalance in the table’s marginal totals).70 71 For values of K or Gwet AC1 >0.60, the intra-rater reliability was considered good.72 Due to the limited sample enrolment, for this analysis data for all countries were considered together. Data were analysed using SAS (Statistical Analysis Software V.9.4, SAS Institute) and R V.3.6.1.

    Acceptability was evaluated by analysing responses to one open-ended question in the questionnaire, which explicitly asked health workers to comment on the quality of the questionnaire and provide practical suggestions on how to improve it. All comments were analysed in their national language by native speakers that are experts in healthcare and/or health research and project partners.

    Findings of all steps above were used for the final questionnaire optimisation, following consensus agreement among all partners of the IMAgiNE project, which at this stage included a multidisciplinary group of 58 partners from 19 countries in the WHO European Region.

    Before participating, consent was requested and all participants were informed about the objectives and methods of the study, including their rights in declining participation (a complete privacy policy was available for download). Anonymity was ensured by not collecting any information that could disclose participants’ identity.

    Patient and public involvement statement

    Health workers from several countries participated in the development, content and construct validation, assessment of face validity, internal consistency, intrarater reliability and acceptability of the questionnaire. Inputs received were used to optimise the questionnaire.

    Results

    Phase 1: content, construct and face validity

    As a result of the first step, the online English questionnaire was made available.

    Second, 22 Quality Measures were generated for the COVID-19 preparedness and response section of the questionnaire.

    The Delphi process with international experts prioritised 40 Quality Measures and defined a core set of 13 sociodemographic variables. It also optimised both the wording and the structure of the questionnaire, and added additional open-ended questions. The final questionnaire structure included six sections (online supplemental table 3). Table 2 shows the list of 40 key Quality Measures by domain.

    Table 2

    Quality measures of the IMAgiNE questionnaire for health workers

    In addition, a QMNC Index was developed. A predefined score (eg, 0-5-10 points) was attributed to each possible answer of each one of the 40 questions on Quality Measures of the IMAgiNE questionnaire for health workers. Higher scores indicating higher adherence to WHO Standards. The sum of all points in one specific domain could range from 0 to 100, while the total QMNC Index could range from 0 top 400 considering all domains (online supplemental table 4).

    Phase 2: translation and cultural adaptation

    The IMAgiNE questionnaire for health workers was translated and back translated into the following 12 languages: (1) Bosnian, (2) Croatian, (3) French, (4) German, (5) Italian, (6) Norwegian, (7) Portuguese, (8) Romanian, (9) Russian, (10) Slovenian, (11) Spanish and (12) Swedish.

    Phase 3: internal consistency, intrarater reliability and acceptability in six countries

    A total of 600 health workers participated in this phase; the sample included a heterogeneous group of professionals with different ages, genders, professional roles and experience. More than half of health workers had more than 10 years of experience in maternal and neonatal health (54.3%) with midwives representing 48.5% of the total sample. Detailed characteristics are presented in table 3.

    Table 3

    Health workers’ characteristics

    The Cronbach’s alpha values were ≥0.70, showing good internal consistency for all sections analysed. It is presented in online supplemental table 5.

    Findings on intrarater reliability are reported in online supplemental table 6. Overall, 164 health workers answered the questionnaire twice (test–retest), thus resulting in a power of 0.97. All K values or Gwet’s AC1 (in case of Kappa paradox) were equal or above the required value of 0.60, except for the question D5.1, that was edited to increase clarity.

    Regarding acceptability, only 10 (1.7%) respondents suggested improvement to the questionnaire wording, with all languages of the questionnaire available for validation receiving only one comment each, except for Swedish (four comments) and Norwegian (three comments).

    Online supplemental tables 7–13 present the final English version of IMAgiNE questionnaire for health workers in English, Italian, Portuguese, Norwegian, Swedish, Croatian and Romanian.

    Discussion

    Collecting the perspectives of health workers providing care to mothers and newborns during facility-based childbirth is essential for improving several aspects of the quality of care, in particular during challenging situations like the COVID-19 pandemic. This paper presents the results of the development and validation of a WHO Standards23-based online questionnaire on the QMNC in the WHO European Region, from the perspective of health workers. To our knowledge, no other similar online tool, explicitly based on the WHO Maternal and Newborn Quality of Care Standards,23 has been developed for health workers. This questionnaire complements an existing WHO Standards23-based questionnaire dedicated to collect service users’ (mothers’) perspectives on the QMNC.34 The availability of unified comprehensive approaches to measure QMNC as defined by the WHO Quality Measures, through validated tools, allows comparisons of data across settings and over time, allows triangulation with routinely collected official data and may support decision makers on designing and implementing future quality improvement initiatives that might improve health outcomes.

    Findings suggest that the questionnaire has good content and construct validity, face validity, internal consistency, intrarater reliability and acceptability in several countries of the WHO European Region. These relevant psychometric properties of the tool allow its utilisation in similar settings. Even though the small sample size by country/language did not allow to perform the exploratory and confirmatory factor analysis, useful to evaluate the underlying structure among variables,73 the cross-cultural careful planning and comprehensive methodological approaches74 used for this study ensure the strength of the validation process. Further results will be reported separately in coming publications.

    The process of developing this questionnaire was based on existing guidance32–37 and had several strengths. The questionnaire was based on previous pilot studies.29–33 75 The characteristics of the questionnaire were defined in advance, based on previous experience developing measurement tools.29–33 74 Both international experts and health workers of different nationalities and with different backgrounds were involved in the development process at different phases, including the assessment of content and construct validity, face validity, internal consistency, intrarater reliability and acceptability.74 Other questionnaires recently used for collecting multicountry health workers’ perspectives during COVID-19 pandemic did not go through a similar formal validation process.13 76 77 As a lesson learnt from this experience, we acknowledge that the process of validation can be quite lengthy, and, and may not be the most rapid in a pandemic.

    The number of Quality Measures collected by the tool (40 Quality Measures) may be seen as a limitation; however, this questionnaire should be seen as complementary to an already existing tool investigating maternal perspectives on the QMNC, also including 40 WHO Standards-based Quality Measures.30 When developing questionnaires, consideration has to be given to the length of the tool, not to decrease acceptability and to assure feasibility.35 41–43 During the COVID19-pandemic, health workers have seen an increase in their workload and an increase in requests to participate in many different surveys, thus critical attention should be given to avoid lengthy surveys, which may result in a low response rate.

    Another potential limitation of the questionnaire is that it only collects data on the QMNC from the health worker’s perspectives. Health workers may not fully be aware of their institutions’ policies and/or personal attitudes might have influenced answers. However, the fact that only health workers directly involved in maternal or neonatal care for at least 1 year should participate in the validation process should have minimised this risk. Thus, we suggest to collect data from health professionals with a minimum experience of 1 year of clinical work.

    In projects aiming at changing behaviours and improving quality of care, gathering information about opinions and view of key actors is essential.78 79 Opinions of both service users and service providers should not be dismissed. To get a fuller picture, data should ideally be collected, if feasible, from multiple data sources, including service users, service providers, from official data sources and from direct observation.31 75 78 79

    The QMNC index is intended as a complementary (not substitutive) way to quantitatively measure QMNC in a synthetic format and should always be interpreted looking at detailed results of the whole list of Quality Measures collected. Responsiveness and other properties of the QMNC index shall be further evaluated and published in future studies.

    Both the maternal and the health workers’ questionnaires will be used among partners of the IMAgiNE study networks, and research findings from individual countries or specific subgroup analysis (eg, data health professionals in the maternal area) will be reported in future publications. With this multicountry survey we have the possibility to explore a variety of local practices during the different phases of the COVID-19 pandemic, and to identify relevant influencing factors on the quality of care provided around childbirth (ie, healthcare policies, etc). This data may allow for domains relevant to QMNC over time and across countries comparison.

    The ultimate objective of the tool described in this paper is to help stakeholders, department directors and policy-makers understand at a glance what works well and what needs to be changed or improved in the health facilities where women give birth, and babies are born, to ensure the QMNC. Future research shall further explore how better use the findings from this questionnaire across different settings and which can be the most effective strategies for translating quality of care evidence into policies in the best interest of mothers, newborns and health workers.

    Conclusions

    Findings suggest that the online health workers’ IMAgiNE questionnaire, based on WHO Standards, has good content, construct validity, face validity, internal consistency, intrarater reliability and acceptability in several countries of the WHO European Region. Further research may explore in depth the use of this questionnaire in other countries, documenting the responsiveness of the QMNC index, and test approaches for translating data generated into quality improvement policies across settings.

    Data availability statement

    Data are available on reasonable request. All data relevant to the study are included in the article or uploaded as online supplemental information. All relevant data are provided in the paper. Additional details can be provided by contacting the corresponding author with a reasonable request.

    Ethics statements

    Patient consent for publication

    Ethics approval

    Ethical approval was obtained from the Institutional Review Board of the coordinating centre, the IRCCS Burlo Garofolo Italy (IRB-BURLO protocol number: 617/2016 and 05/2020) and from Ethical committees from Portugal (Instituto de Saúde Pública da Universidade do Porto, CE 20159, and Centro Hospitalar Universitário do Algarve, UAIF 101/2021) and Norway (Norwegian Regional Committee for Medical Research Ethics, ref n. 2020/213047). As no personal information was collected, no further ethical approval from the Croatian, Swedish, and Romanian ethics review authority was required beyond the approval of the ethical committee of the coordinating centre.

    Acknowledgments

    We gratefully acknowledge all health workers from all countries who have answered the invitation to contribute to all phases of the validation process.

    Norway: Tone Engen, Department of health and caring sciences, Western Norway University of Applied Sciences for back translation of Norwegian questionnaire.

    References

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • Twitter @Linden_Ka, @DanielaDrandic

    • Correction notice The article has been corrected since it was published online. The co-author Marina Otelea's surname was incorrectly published as Otalea; this has been amended.

    • Collaborators IMAgiNE EURO Study Group: Bosnia-Herzegovina: Amira Ćerimagic (NGO Baby Steps, Sarajevo, Bosnia-Herzegovina). France: Rozée Virginie, Elise de La Rochebrochard,(Sexual and Reproductive Health and Rights Research Unit, Institut National d’Études Démographiques [INED], Paris, France), Kristina Löfgren (Baby-friendly Hospital Initiative [IHAB], France). Germany: Céline Miani, Stephanie Batram-Zantvoort, Lisa Wandschneider (Department of Epidemiology and International Public Health, School of Public Health, Bielefeld University, Bielefeld, Germany). Italy: Giuseppa Verardi, Beatrice Zanin (Institute for Maternal and Child Health IRCCS Burlo Garofolo, Trieste, Italy). Israel: Ilana Chertok,(Ohio University, School of Nursing, Athens, Ohio, USA), (Ruppin Academic Center, Department of Nursing, Emek Hefer, Israel), Rada Artzi-MedvediK (Department of Nursing, The Recanati School for Community Health Professions, Faculty of Health Sciences at Ben-Gurion University [BGU] of the Negev, Israel). Latvia: Elizabete Pumpure, Dace Rezeberga, Agnija Vaska (Riga Stradins University Department of Obstetrics and Gynaecology, Rīga, Latvia), Dārta Jakovicka, Paula Rudzīte (Riga Stradins University Faculty of Medicine, Rīga, Latvia), Elīna Ērmane, Katrīna Paula Vilcāne (Riga Stradins University, Rīga, Latvia). Luxembourg: Maryse Arendt (Beruffsverband vun de Laktatiounsberoderinnen zu Lëtzebuerg asbl [Professional association of the Lactation Consultants in Luxembourg], Luxembourg, Luxembourg), Barbara Tasch (Beruffsverband vun de Laktatiounsberoderinnen zu Lëtzebuerg asbl [Professional association of the Lactation Consultants in Luxembourg], Luxembourg, Luxembourg), (Neonatal intensive care unit, KannerKlinik, Centre Hospitalier de Luxembourg, Luxembourg, Luxembourg). Poland: Barbara Baranowska, Urszula Tataj-Puzyna, Maria Węgrzynowska (Department of Midwifery, Centre of Postgraduate Medical Education, Warsaw, Poland). Portugal: Catarina Barata (Instituto de Ciências Sociais, Universidade de Lisboa, Lisboa, Portugal), Teresa Santos (Universidade Europeia, Lisboa, Portugal), (Centro de Investigação Interdisciplinar em Saúde [CIIS] da Universidade Católica Portuguesa, Lisbon, Portugal). Russia: Ekaterina Yarotskaya (Department of International Cooperation National Medical Research Center for Obs., Gyn. & Perinatology, Moscow, Russia). Serbia: Jelena Radetić, Jovana Ružičić (Centar za mame, Belgrade, Serbia). Slovenia: Zalka Drglin, Barbara Mihevc Ponikvar, Anja Bohinec (National Institute of Public Health, Ljubljana, Slovenia). Spain: Serena Brigidi (Department of Anthropology, Philosophy and Social Work. Medical Anthropology Research Center [MARC]. Rovira i Virgili University [URV], Tarragona, Spain), Lara Martín Castañeda (Institut Català de la Salut, Generalitat de Catalunya, Spain), Ana Canales Viver (Institut Català d'Antropologia [ICA], Barcelona, Spain). Sweeden: Verena Sengpiel (Region Västra Götaland, Department of Obstetrics and Gynecology, Sahlgrenska University Hospital, Gothenburg, Sweden). Switzerland: Claire De Labrusse, Alessia Abderhalden, Anouck Pfund, Harriet Thorn (School of Health Sciences [HESAV], HES-SO University of Applied Sciences and Arts Western Switzerland, Lausanne, Switzerland).

    • Contributors ML conceived the study, with major inputs from EPV and BC. EPV, BC, IM, SM, MO, IN, MN, HE, KL, MZ, ESV, SK, IN, RC, CR, HD, DD, MK, ES, MM, OL and ML contributed to the tool validation. IM analysed data, with major inputs from EPV, BC and ML. EPV and ML wrote the first draft, which major inputs from BC, IM, SM, MO, IN, MN, HE, KL, MZ, ESV, SK, IN, RC, CR, HD, DD, MK, ES, MM and OL. EPV and ML are guarantors for this study. All authors approved the final version of the manuscript for submission.

    • Funding This study was funded by the Institute for Maternal and Child Health IRCCS Burlo Garofolo, Trieste, Italy (number N/A).

    • Disclaimer The authors alone are responsible for the views expressed in this article and they do not necessarily represent the views, decisions, or policies of the institutions with which they are affiliated.

    • Competing interests None declared.

    • Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.