Article Text
PDF
Abstract
Introduction Woman-centred care is the right of every woman receiving maternity care, irrespective of where care is being received and who is providing care. This protocol describes a planned systematic review that will identify, describe and critically appraise the psychometric properties of maternity patient-reported outcome measures (PROMs) and patient-reported experience measures (PREMs). The woman-centricity of PROM and PREM development and content validation (ie, the extent to which women were involved in these processes) will also be assessed. This information will be used to develop a maternity PROMs and PREMs database to support service and system performance measurement, and value-based maternity care initiatives.
Methods and analysis This study will be guided by the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) guideline for systematic reviews of outcome measurement instruments. Studies identified via MEDLINE, CINAHL Plus, PsycINFO and EMBASE describing the development, content validation and/or psychometric evaluation of PROMs and PREMs specifically designed for maternity populations throughout pregnancy, childbirth and postnatal periods will be considered if published from 2010 onward, in English, and available in full text. The COSMIN risk of bias checklist will be used to evaluate the quality of studies reporting on the development, content validation and/or psychometric evaluation of PROMs and PREMs. COSMIN criteria for good content validity will be used to assess the woman-centricity of PROM and PREM development and content validation studies. COSMIN standards of good psychometric properties will be used to evaluate the validity and reliability of the identified instruments.
Ethics and dissemination Ethical permission for this research is not required. The findings of this research will be submitted for publication in an international, peer-reviewed journal. Abstracts for national and international conference presentations will also be submitted. The proposed maternity PROMs and PREMs database will be freely accessible online, and developed with consumer input to ensure its usefulness to a range of maternity care stakeholders.
PROSPERO registration number CRD42021288854.
- health services administration & management
- quality in health care
- obstetrics
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
Employing the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) guidance at all stages provides a transparent, uniform and robust approach to the conduct of this systematic review.
Compiling evidence on the woman-centricity (ie, the involvement of women) in instrument development and content validation is yet to be evidenced in the peer-reviewed literature, and aims to support performance measurement and value-based assessment that is meaningful to women.
Developing a publicly available database of maternity patient-reported outcome measures (PROMs) and patient-reported experience measures (PREMs) aims to promote best practice instrument selection and implementation to support the measurement of services and systems, and contribute to operationalising value-based healthcare.
A potential limitation of this review is using COSMIN guidance (developed for PROMs) to evaluate the development, content validation and psychometric evaluation of PREMs.
Additionally, the review will only include PROMs and PREMs published after 2010, and studies published in English.
Introduction
The concept of woman-centred care (WCC) is underpinned by the principles of choice, control, continuity of carer and a woman’s right to self-determination.1–3 WCC is typically associated with midwifery practice,4 but this misrepresents the reality that receiving WCC is the right of every woman, irrespective of where or by whom she receives care. Coupled with a ‘risk avoidant’ obstetric culture and increasing rates of intervention at birth (particularly in high-income countries),3 5 6 women’s values and preferences for aspects of care beyond a successful live birth (eg, desire for a natural birth) are often a secondary consideration. This has subsequently challenged the implementation of value-based maternity care, where consumer perspectives are at the centre of outcome measurement.
Value-based healthcare (VBHC) is the purported goal of every health system. At its core, VBHC aims to improve patient health outcomes relative to the cost of achieving those improvements.7 However, VBHC frameworks that exist on this principle alone have been called into question as they oversimplify the complex construct of ‘value’,8 particularly what value means to patients in different circumstances.9 Indeed, in the context of maternity care, women value a diverse array of factors, including care continuity, equitability, promoting normal reproductive processes, choosing where they give birth, being treated respectfully, emotional support and transparent communication.10–13 Consequently, value-based maternity care represents far more than a successful live birth.
One means of capturing the experiences and outcomes of maternity care that women value is using patient-reported outcome measures (PROMs) and patient-reported experience measures (PREMs). Despite capturing different elements of healthcare encounters, both types of instrument are designed to measure and evaluate service and system performance from the consumer’s perspective.14 By responding to the outcomes and experiences reported by consumers, health services and systems are better able to support VBHC. However, this is only achieved if the content of PROMs and PREMs aligns with what is viewed as important and relevant to care consumers (ie, women). Thus, woman-centric instrument development and content validation—that is, the involvement of women in defining what is relevant, comprehensive and comprehendible instrument content—is crucial to supporting meaningful, value-based measurement in maternity care.15
Specifically, PROMs measure an individual’s health and well-being.16 17 They can capture a wide range of outcomes, largely related to physical, social and/or psychological functioning.16 18 19 Recent reviews of condition-specific PROMs used during pregnancy and childbirth have revealed instruments capturing recovery after childbirth,20 outpatient postpartum recovery,21 sleep in postpartum women,22 postpartum pain23 and functional recovery following caesarean section.24 However, PROMs capturing outcomes relevant to all women across the pregnancy, childbirth and postpartum continuum are missing.25
PREMs differ in that they are designed to capture an individual’s experience of receiving care, namely, their perception of what happened during a care encounter and how it happened.26 There are no reviews of maternity PREMs; however, a recent concept analysis identified several constructs commonly captured in relation to women’s experiences of maternity care. These include organisational aspects of care such as access and referral to maternity services, continuity of care, privacy and care costs; and interpersonal aspects of care such as information sharing, informed choice, emotional support, being treated with respect and dignity, and having confidence in the knowledge and ability of maternity care providers.27
We intend to develop a database hosting a repository of PROMS and PREMs to support the use of these instruments in health service and system performance measurement and evaluation as a part of achieving value-based maternity care. Specifically, we aim to identify and appraise PROMs and PREMs that capture outcomes and experiences (respectively) relevant to maternity care that is accessed by all women across the pregnancy, childbirth and postpartum continuum. This protocol describes the systematic process that will be undertaken to, first, identify and describe maternity PROMs and PREMs published in the peer-reviewed literature, and, second, critically appraise and summarise the psychometric properties of the identified instruments. Particular emphasis will be placed on the woman-centredness of PROM and PREM development. The database will subsequently summarise this information in a user-friendly format suitable for a range of maternity care stakeholders.
Methods and analysis
This study will be guided by the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) guideline for systematic reviews of outcome measurement instruments.28 COSMIN stipulates a 10-step process for performing a systematic review of PROMs (which will be extended to PREMs for the purposes of this research). Steps 1–4 pertain to conducting the literature search; steps 5–7 pertain to evaluating an instruments’ psychometric properties; step 8 pertains to evaluating the interpretability and feasibility of implementing instruments; and steps 9–10 pertain to writing the review discussion. This protocol will detail the processes we intend to undertake for steps 1–8.
Step 1: Formulate the aim of the review
The aim of this review is twofold. First, to identify and describe maternity PROMs and PREMs relevant to all women across the pregnancy, childbirth and postpartum continuum, published in the peer-reviewed literature. Second, to critically appraise and summarise the psychometric properties of the identified instruments, with particular emphasis on assessing the woman-centricity of instrument development and content validation.
Step 2: Formulate the eligibility criteria
The eligibility criteria stipulated in table 1 will be applied.
Systematic review eligibility criteria for studies reporting on maternity PROMs and PREMs
We specifically delineate PROMs from quality of life/utility measures. Quality of life/utility measures (eg, EQ-5D, Health Utility Index and SF-6D) are preference-based instruments despite often being referred to interchangeably as PROMs.29–32 While PROMs and quality of life/utility measures may capture similar constructs,33 they differ in how they are used and scored. PROMs were originally developed with the intent to assess health outcomes as a consequence of receiving healthcare, and are scored on an item-by-item or domain/dimension basis. Conversely, quality of life/utility measures were originally developed for the purposes of quantifying a person’s health state (without any reference to having received healthcare or not) and their present level of quality of life. Furthermore, quality of life/utility measures can then be used for determining quality-adjusted life years, where an individuals’ quality of life as it relates to their health state is scored as one of a finite number of health states relative to a utility index.33–36 As such, they will not be included in this review.
We also specifically delineate PREMs from patient satisfaction measures, despite being referred to synonymously throughout the literature.37 PREMs ask individuals to report on their experiences of care, where satisfaction measures ask individuals to evaluate their experiences. While report-style questions aim to be objective,16 evaluative questions are more likely to reflect an individuals’ expectations, attitudes and desire to appear socially desirable, and are thus influenced by attributes peripheral to their care experience.38–40 Additionally, where PREMs typically use frequency-based response scales (eg, on a scale of never to always),41–43 patient satisfaction measures tend to use agreement-based response scales (eg, on a scale of strongly disagree to strongly agree),44–46 which are more prone to acquiescence bias and straightlining.47 48 Thus, given the differences between these instruments, patient satisfaction measures will not be included in this review.
Step 3: Perform a literature search
We will search the following electronic databases: MEDLINE (via Ovid), CINAHL Plus (via EBSCOhost), PsycINFO (via Ovid) and EMBASE (via Elsevier). Our search terms will include the following concepts: (1) maternity care and maternal health services, (2) PROMs, (3) PREMs and (4) measurement properties. We will employ the search terms developed by COSMIN relevant to studies on measurement properties.49 These are available for each of the abovementioned databases. An example of our proposed MEDLINE search strategy is available in online supplemental file 1. Searches will be limited to only studies published in English and available in full text.
Supplemental material
Step 4: Select abstracts and full-text articles
After being exported from electronic databases, all search results will be imported into Covidence.50 Two reviewers will independently review all titles and abstracts to determine which articles warrant full-text retrieval and review. Full-text review will also be undertaken by two independent reviewers. Discrepancies at all stages will be addressed through reviewer consultation and consensus, and if needed, engagement of a third reviewer. The reference lists of all included papers will be hand searched for other potentially relevant studies.28
Steps 5–7: Evaluating the measurement properties of the included PROMs and PREMs
Data extraction: Characteristics of the included PROMs and PREMs
The following data will be extracted from included studies: (1) PROM/PREM name; (2) construct(s)/domain(s) captured; (3) target population and setting; (4) mode of administration (eg, online or postal) and administration time during perinatal care (eg, antenatal or postnatal); (5) recall period; (6) number of items; (7) response options and (8) original language. This information will be used to describe the included PROMs and PREMs. Information will be extracted per study and grouped where there have been multiple studies conducted for one instrument. One reviewer will extract all data.
Evaluating the methodological quality of studies
Methodological quality will be evaluated in relation to maternity PROM and PREM development, content validation and psychometric evaluation using the COSMIN risk of bias checklist.28 This checklist details specific study design elements that are important when assessing the measurement properties of an instrument. Only study design elements relevant to the measurement properties presented in table 2 and reported in studies will be assessed for risk of bias. Criteria for study design elements are rated using a scale of ‘very good’, ‘adequate’, ‘doubtful’, ‘inadequate’ or ‘n/a’. The lowest rating for any criteria will be used to describe the quality of the study underpinning that specific measurement property (ie, worst score counts).28 If multiple studies have been conducted to evidence a specific measurement property (eg, three studies report on an instruments’ internal consistency) and have provided variable results, the overall quality of the measurement property will be labelled ‘unclear’.
Elements of the COSMIN risk of bias checklist28 for assessing study design relative to PROM and PREM development, content validation and psychometric evaluation studies
One reviewer will first consult the COSMIN database of systematic reviews of outcome measurement instruments51 to determine whether other researchers have already evaluated the risk of bias of the included studies. If available, existing ratings will be used (as is recommended by COSMIN28). If not, or if additional evidence supporting an instrument has been published, one reviewer will determine the measurement property(ies) that need to be assessed per study, and two reviewers will independently complete the risk of bias checklist for each individual study. We will use the risk of bias Microsoft Excel template developed by COSMIN to document each rater’s scores.
Evaluating the content validity (woman-centricity) of PROM and PREM development
Content validity has been described as the most important measurement property of PROMs (and arguably, PREMs).52 It represents the degree to which the content of an instrument is an adequate reflection of the phenomena being measured.53 PROM and PREM items need to demonstrate appropriate relevance, comprehensiveness and comprehensibility to qualify as content valid.52 This assessment should be made by ‘experts’ of the target phenomena. In the context of maternity care, the women receiving and experiencing care are the experts. COSMIN also provide criteria to support studies that have asked health professionals about the relevance and comprehensiveness of items.52 Instruments that fail to demonstrate appropriate involvement of women in their development and content validation will be labelled as demonstrating ‘inadequate’ content validity.
COSMIN has developed a set of instructions specifically for evaluating the content validity of PROMs, which will be used in this study (for both PROMs and PREMs). The first two steps involve evaluating the quality of studies reporting on instrument development and content validation; this forms part of the COSMIN risk of bias checklist described above. The third step involves rating each development and content validation study against nine criteria for good content validity (box 1).52 For each of relevance, comprehensiveness and comprehensibility, if ≥85% of an instruments’ items fulfil the criteria, the study is deemed to have sufficient (+) evidence; if <85% of items fulfil the criteria, the study is deemed to have insufficient (−) evidence; and if there is inadequate information available or the study quality was inadequate (as identified through risk of bias assessment), the study is deemed to have indeterminate (?) evidence.52 From this, we will assign an overall content validation score (+, − and ?), which will represent the woman-centricity of PROM and PREM development. Two reviewers will undertake the content validation assessment.
Relevance, comprehensiveness and comprehensibility criteria for evaluating the content validity of maternity care instruments52
Relevance criteria
Are the included items relevant to maternity care?
Are the included items relevant to women?
Are the response options appropriate?
Is the recall period appropriate?
Comprehensiveness criteria
Are all key concepts included?
Comprehensibility criteria
Are the instrument instructions understood by women as intended?
Are items and response options understood by women as intended?
Are items appropriately worded?
Do the response options match the question?
Evaluating the sufficiency of measurement properties
Instruments will next be evaluated according to how well the reported measurement properties (eg, structural validity) comply with standards of good psychometric properties (table 3).28 This will indicate whether a PROM or PREM can be considered valid and reliable. Validity is the extent to which an instrument measures what it purports to measure.54 55 Reliability is the extent to which participant responses to an instrument can be replicated in unchanging circumstances (consistency).56 Reliability is also the extent to which an instrument is devoid of a measurement error.57 58
COSMIN updated criteria for good measurement properties28
Using the COSMIN updated criteria for good measurement properties, psychometric properties will be rated as + (provides sufficient evidence), – (provides insufficient evidence) and ? (provides indeterminate evidence) (table 3). Red text denotes criteria added based on prominence in the literature relative to instrument development and psychometric evaluation. COSMIN’s criteria of ‘Hypothesis testing for construct validity’ has been excluded from table 3 as the context of maternity care in this study is too broad for the review team to appropriately generate hypotheses suitable for all potential instruments. If a PROM or PREM has several studies reporting on its psychometric properties, each study will be evaluated individually (according to the reported psychometric properties), and an overall conclusion regarding the quality of the instrument will be provided for each psychometric quality. Any psychometric properties not assessed will be labelled as having ‘no evidence’. Two reviewers will undertake the good psychometric properties’ assessment.
Summarise and grade the quality of evidence
By summarising and grading the evidence available for an individual instrument, we can provide an overall conclusion as to the quality of that instrument. Thus, this will involve combining the results of each instruments’ risk of bias, content validity and psychometric property assessments into a single metric of ‘high’, ‘moderate’, ‘low’ or ‘very low’ evidence using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach.28 If the results across multiple studies pertaining to a single instrument are consistent, then results will be quantitatively pooled and a GRADE score will be reported. If results are inconsistent, they will not be pooled, no GRADE score will be reported and areas of inconsistency will be discussed (eg, if an instrument demonstrates differing levels of quality depending on the country in which it is used).
Step 8: Describe the interpretability and feasibility of instrument implementation
Interpretability is the extent to which meaning can be derived from participant responses to an instrument or changes in responses.59 This may include distinct patterns of responses among subgroups of the population, trends in responses over time and floor or ceiling effects. For the purposes of this review, we will extract and describe the following features of PROM and PREM interpretability: (1) distribution of responses in the study population and relevant subgroups; (2) proportion of missing data for items; (3) methods of handling missing data; (4) evidence of floor and ceiling effects and (5) minimally important changes or minimally important differences in responses. Interpretability, while not considered a measurement property in and of itself, is important for understanding the real-world application and biases associated with implementing PROMs and PREMs.
Feasibility refers to the ease and convenience with which a PROM or PREM can be implemented and administered in a real-world context.28 For the purposes of this review, we will extract and describe the following features of PROM and PREM feasibility: (1) available modes of administration; (2) length of the instrument; (3) estimated completion time; (4) level of readability; (5) ease of response calculation; (6) copyright; (7) cost of using an instrument; (8) equipment required for instrument administration; (9) availability of instrument for application in different settings and languages and (10) approvals required before instrument use. For the development of a maternity PROMs and PREMs database, this information will be critical for informing the real-world implementation of maternity PROMs and PREMs across health services and systems.
Patient and public involvement statement
The research team comprises members of Maternity Choices Australia, a national consumer advocacy organisation committed to the advancement of best-practice maternity care.60 These women are consumer representatives and have been involved in the conceptualisation of the research and protocol development, recognising the importance of operationalising WCC, and ensuring that maternity services are consumer informed. Importantly, they will aid the development of the maternity PROMs and PREMs database, supporting its usability by a range of maternity care stakeholders. They will also help disseminate the maternity PROMs and PREMs database through formal and informal engagement with key collaborative parties.
Ethics and dissemination
Ethical permission for this research is not required as the review will only use information from previously published research. The findings of this research will be submitted for publication in an international, peer-reviewed journal. Abstracts will also be submitted for national and international conference presentations.
Maternity PROMs and PREMs database
We intend for the maternity PROMs and PREMs database to be freely accessible online, and useful to all individuals involved in maternity health services and systems performance measurement, and value-based maternity care. The design of the database will be consumer informed to ensure that it is easy to understand, and provides information relevant to a range of maternity care stakeholders. The psychometric results (structural validity, internal consistency, reliability, measurement error, cross-cultural validity/measurement invariance, criterion validity and responsiveness) for each instrument will be summarised according to whether criteria were met (+), indeterminate (?) or not met (−) when all evidence for a specific instrument in considered collectively. The woman-centricity of instrument development will be similarly summarised according to the COSMIN criteria for good content validity. In addition, the database will summarise descriptive information for each instrument (eg, number of items, domains captured and country of development), summarise information regarding each instruments’ feasibility of use (eg, copyright and reuse considerations, available modes of administration, costs, etc) and provide links to the studies describing instruments. For PROMs or PREMs not freely available, we will also provide the appropriate contact information for the instruments’ original author or licensing agent.
We anticipate that the database will be updated annually. A member of the research team will re-run the search strategies (updating search terms as needed) and undertake the processes described in this protocol. This will support the identification of new instruments or additional evidence of PROM and PREM psychometric evaluation over time, ensuring that the database is up to date and aligns with advancements in PROM and PREM methodologies and results.
Ethics statements
Patient consent for publication
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @ClaudiaBull3, @EmilyCallander
Contributors CB contributed to the conception of the research and wrote the protocol manuscript. HJT and LC provided important intellectual content in revisions of the protocol manuscript. AR and SC provided important consumer representation and content relative to woman-centred care. EJC contributed to the conception of the research and provided important intellectual content in revisions of the protocol manuscript. All authors approved of the version submitted for publication and agree to be accountable for all aspects of the research.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.