Article Text
PDF
Abstract
Introduction In the UK, it is estimated that a disabling hearing loss (HL) affects 1 in 6 people. HL has functional, economic and social-emotional consequences for affected individuals. Intervention for HL focuses on improving access to the auditory signal using hearing aids or cochlear implants. However, even if sounds are audible and speech is understood, individuals with HL often report increased effort when listening.
Listening effort (LE) may be measured using self-reported measures such as patient-reported outcome measures (PROMs). PROMs are validated questionnaires completed by patients to measure their perceptions of their own functional status and well-being. When selecting a PROM for use in research or clinical practice, it is necessary to appraise the evidence of a PROM’s acceptability to patients, validity, responsiveness and reliability.
Methods and analysis A systematic review of studies evaluating the measurement properties of PROMs available to measure LE in HL will be undertaken. MEDLINE, EMBASE, CINAHL, PsychINFO and Web of Science will be searched electronically. Reference lists of included studies, key journals and the grey literature will be hand-searched to identify further studies for inclusion. Two reviewers will independently complete title, abstract and full-text screening to determine study eligibility. Data on the characteristics of each study and each PROM will be extracted. Methodological quality of the included studies will be appraised using the COnsensus-based Standards for the selection of health Measurement INstruments, the quality of included PROMs appraised and the credibility of the evidence assessed. A narrative synthesis will summarise extracted data.
Ethics and dissemination Ethical permission is not required, as this study uses data from published research. Dissemination will be through publication in peer-reviewed journals, conference presentations and the lead author’s doctoral dissertation. Findings may inform the selection of PROMs used to measure LE in HL.
- audiology
- patient-reported outcome measures
- hearing loss
- listening effort
- systematic review
- protocol
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
This systematic review protocol makes an original contribution by locating and synthesising evidence from primary research studies to ascertain the quality and credibility of patient-reported outcome measures (PROMs) used to measure perceived listening effort. This information may be used by researchers and clinicians to inform their choice of outcome measure and may provide evidence to support the development of new PROMs to assess listening effort.
A strength of this review is its use of the internationally recognised, validated COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist to assess the methodological quality of the included studies.
The protocol complies with a well-established systematic review methodology that is transparent and replicable. Data collection, data extraction and data synthesis methods are based on internationally recognised, published standards.
Limitations include an acknowledged language bias, whereby only studies and instruments in English will be included. However, studies published in English that describe cross-cultural validation of instruments from English into other languages will be included.
Introduction
Rationale
Individuals report that listening with a hearing loss is effortful, particularly in adverse listening conditions with consequences for an individual’s well-being and quality of life.1 The emphasis of audiological assessment is on ensuring adequate audibility of the speech signal. However, ensuring audibility has not been found to remove fully the listening difficulties experienced in hearing loss. The result is a continued need by individuals to invest high levels of listening effort. Over time, the requirement for consistently high expenditure of effort can negatively impact quality of life.2–5
Listening effort has been defined as the attentional and cognitive resources needed to understand an auditory message and is recognised as a specific form of mental effort that occurs when a task involves listening.1 Recently, there has been an increase in the number of published studies on listening effort commensurate with a growing appreciation of the role of cognition in listening.1 2 6–8 Importantly, listening effort as a theoretical construct is continuing to evolve with an increasing awareness of the importance of social and psychological factors (such as motivation and determination) to the experience of effortful listening.9
In the published literature to date, listening effort has been measured using physiological measures, behavioural measures and self-report measures. Self-report measures have included: condition-specific questionnaires to assess listening effort particularly; generic questionnaires of workload performance and scaling techniques such as visual analogue scales to rate magnitude of effort associated with a specific listening task. Validated self-reported questionnaires are also known as patient-reported outcome measures (PROMs). PROMs are used widely in clinical research as well as clinical practice to capture the individual’s perspective regarding their disease symptoms, functional status, quality of life, satisfaction or experience. They are used in clinical trials as outcome measures and PROM data can influence the delivery of care, facilitate clinical audit and inform policy decisions in healthcare.10
Within routine audiological practice, self-reported questionnaires such as PROMs are well accepted and used widely to measure an individual’s perception of benefit of and satisfaction with hearing aids or cochlear implants as well as the impact of their hearing device(s) on quality of life. PROMs have also been designed to elicit patient views on the specific difficulties associated with listening (eg, listening in noise, spatial listening, listening effort).11 Given the large number of questionnaires already in existence within the field of audiology, the selection of an appropriate PROM to measure a particular construct requires careful review of its measurement properties to determine suitability and should be supported by the best evidence available. Systematic reviews of the measurement properties of PROMs are considered the ‘gold standard’ of evidence review.
A scoping search revealed only a limited number of self-report instruments associated with the measurement of perceived listening effort and no relevant systematic reviews. McGarrigle et al 8 and Klink et al 12 have reviewed more generally the various methods used to measure listening effort including the use of self-report instruments (ie, PROMs), but to the authors' knowledge a systematic appraisal of the evidence regarding the quality and credibility of these instruments has not been undertaken. Therefore, the capability of existing questionnaires to measure listening effort is as yet unknown. This systematic review aims to identify PROMs currently available to measure perceived listening effort in individuals with hearing loss and to evaluate their measurement properties. The most up-to-date methodology and guidance in the conduct of systematic reviews of PROMs will be applied to ensure the highest level of evidence is obtained. Conducting this review will present a summary of available instruments and, if indicated, provide evidence of need for a new PROM to measure perceived listening effort in adults with hearing loss.
Objectives
This protocol will include the review of disease-specific and generic PROM instruments used to measure listening effort in adults with any degree of hearing loss. The specific objectives the review will address are as follows:
Identify PROMs used to measure perceived listening effort in hearing loss and collate independent evidence from studies describing any aspect of the development and validation of these measures.
Assess the methodological quality of the studies included in the review.
Assess the quality of the identified PROMs’ measurement properties.
Determine the credibility of the evidence regarding the quality of the included PROMs.
Methods and analysis
This protocol was developed using the Preferred Reporting for Items for Systematic Reviews and Meta-Analyses-Protocols (PRISMA-P) checklist13 and is registered with PROSPERO, the international prospective register of systematic reviews (PROSPERO Registration Number: CRD42016048808). The conduct and reporting of this review will be in accordance with the PRISMA statement.14
Eligibility criteria
A study will be included for review if it is a full-text, original article which reports on some aspect of the development and/or assessment of the measurement properties of an original version of a PROM used to measure listening effort. The following inclusion and exclusion criteria will be applied:
Inclusion criteria
Articles that report on perceived listening effort in the population of individuals with hearing loss.
Articles reporting the assessment of the psychometric measurement properties of PROMs (either generic or condition-specific) used to measure perceived listening effort.
All study types will be included if the article describes some aspect of PROM development or validation.
Articles in English.
Exclusion criteria
Studies using a PROM questionnaire as an outcome (with no evaluation of the PROM’s measurement properties).
Studies reporting on the development and validation of instruments designed to be completed by a proxy (eg, clinician-reported or partner/family reported outcome measures).
Articles written in a language other than English will be excluded, as the research team does not have the resources to support translation. However, articles in English reporting on the translation and/or cross-cultural validation of instruments into other languages will be included.
Information sources and search strategy
The systematic review will involve a broad search guided by the Cochrane Review Group’s checklist for electronic search strategies.15 Search strategies will be developed with an emphasis on sensitivity, not specificity, to ensure comprehensiveness and the identification of all relevant instruments. The search process will be documented in detail so it is repeatable. Any changes to the search strategy will be recorded together with a rationale.
Searches will be undertaken for three constructs: (1) terms relating to ‘listening effort’, (2) terms relating to ‘hearing loss’ and (3) terms relating to PROMs including a modified version of the Oxford PROM filter16 and names of questionnaires known to be used in the measurement of perceived listening effort. The terms within each group will be combined with a Boolean ‘OR’ command and then searched in combination using a Boolean ‘AND’ command. Relevant construct filters will be applied to specific databases where applicable.
A variety of information sources will be searched to ensure the search is comprehensive. The following databases will be searched without limitation by publication period: MEDLINE (Ovid), EMBASE (Ovid), CINAHL (EBSCO), PsychINFO (Ovid) and Web of Science (Thomson Reuters). In addition to electronic bibliographic databases, the reference lists of included studies and study registries will be hand-searched and documented on the study database. A hand-search of the ‘grey’ literature will be undertaken to include conference proceedings, theses, and informal communications (eg, emails, blogs).
The search strategy for MEDLINE (Ovid) will be developed iteratively with expert support from medical sciences librarians at Swansea University and Abertawe Bro Morgannwg University Health Board and tailored specifically for each database to be searched (see online supplementary appendix for the full MEDLINE (Ovid) search strategy). Test searches will be conducted to ensure the viability of the final strategies. Before commencing the searches, the strategies will be reviewed and checked against the Peer Review of Electronic Search Strategies (PRESS) Guidelines.17
Supplementary Material
[161101_SysRev1_BMJOpen_Hughesetal_Appendix1_SearchStrategy_vFINAL.pdf]
Study records
Literature search results will be saved and the titles and abstracts of identified studies will be downloaded to Endnote Reference Management software (V.7). Two reviewers will independently screen the search results against the review’s eligibility criteria using the online systematic review management software Covidence. Following initial screening of titles and abstracts, full-text reports will be retrieved for the remaining records. These full-text records will be subject to further screening to identify eligible studies and thereby arrive at a final set of records for inclusion in the review. Excluded records along with a reason for their exclusion will be logged in a table at the full-text stage of the screening process. Where queries exist regarding the eligibility of a study, additional information will be sought from study authors. If discrepancies persist, a third reviewer will be consulted and the decision whether to include a study reached through discussion. The reviewers will not be blinded to the journal titles, to the study, or to the institutions. A PRISMA flow diagram14 will be used to document the flow of information through the review phases.
Data items
Two reviewers will independently extract data from the included studies. Disagreements will be resolved through discussion and, if necessary, a third reviewer will be consulted. A data extraction form and an accompanying guidance document will be developed and pilot tested to standardise the information recorded and to aid analyses. Extracted information will be presented in tabular format. The following data will be extracted:
Summary data of included studies: the data extraction form will record summary data from the included studies as recommended by Patrick and Guyatt18 including:
Author
Year of publication
Study characteristics (ie, design). Studies will be categorised as either (1) those studies, which specifically set out to evaluate psychometric properties or (2) studies in which a candidate PROM has been used in a trial or observational study and evaluation of the instrument’s psychometric measurement properties is reported incidentally.
Name of the PROM including purpose/use of instrument
Population characteristics (eg, age, sex, description of hearing loss)
Setting
Location
Characteristics of included instruments: information from the included articles on PROMs used to measure perceived listening effort in hearing loss will be compiled including:
Name of PROM
Associated papers
Country of origin
Purpose of instrument
Scale design
No. of items
Domains covered
Patient involvement in outcomes selection
Mode and timing of administration
3. Evidence of the psychometric measurement properties of the studied PROMs.
Data synthesis
A systematic narrative synthesis of best evidence19 will be presented. Information will be displayed in tables and text format to summarise the characteristics of the included PROMs and the findings of selected studies including methodological quality, quality of psychometric properties of instruments and instrument credibility.
Assessment of methodological quality of included studies: the COnsensus-based Standards for the Selection of Health Measurement INstruments (COSMIN) checklist will be used to assess the methodological quality of studies included in the review.20 Mokkink et al.20–22 assert that it is important to assess the methodological quality of primary studies included in a review to minimise risk of bias. The COSMIN checklist was devised by an international panel of experts and represents the ‘gold standard’ for appraising and reporting on the methodological rigour of studies of instrument development and validation for inclusion in a systematic review.21 The COSMIN checklist comprises nine boxes, each with 5–18 items, concerning measurement standards for how each measurement property should be assessed. It will be used to ascertain whether a study meets the standards for good methodological quality. Each item will be given a rating using a 4-point rating scale (ie, ‘poor’, ‘fair’, ‘good’ and ‘excellent’).23 An overall score for each measurement property will be assigned by taking the lowest rating of any item within a box (ie, ‘worst score counts’). The methodological quality will, therefore, be assessed on a property-by-property basis, rather than as a cumulative score, and displayed in tabular format showing the methodological quality of each study per measurement property and per questionnaire. The measurement properties to be evaluated as defined by the COSMIN checklist include:
Internal consistency—‘The degree of interrelatedness among items’.20
Reliability—‘The proportion of the total variance in the measurements which is due to "true" differences between patients’.20
Measurement error—‘The systematic and random error of a patient’s score that is not attributed to true changes in the construct to be measured’.20
Content validity (including face validity)—‘The degree to which the content of an health-related patient-reported outcome (HR-PRO) instrument is an adequate reflection of the construct to be measured’.20
Construct validity (including structural validity, hypotheses testing and cross-cultural validity)—‘The degree to which the score of an HR-PRO instrument is consistent with the hypotheses (for instance, with regard to internal relationships, relationships to the scores of other instruments or difference between relevant groups) based on the assumption that the HR-PRO instrument validly measures the construct to be measured’.20
Cross-cultural validity—‘The degree to which the performance of the items on a translated or culturally adapted HR-PRO instrument are an adequate reflection of the performance of the items of the original version of the HR-PRO instrument’.20
Criterion validity—‘The degree to which the score of an HR-PRO instrument is an adequate reflection of a ‘gold standard’.20
Responsiveness—‘The ability of an HR-PRO instrument to detect change over time in the construct to be measured’.20
Quality of the psychometric measurement properties of PROMs: the quality of each PROM will be assessed using the criteria developed by Terwee et al. 24 A rating of ‘positive’, ‘indeterminate’, ‘negative’ or ‘unknown’ will be assigned to each measurement property per instrument based on the number of studies, the consistency of results across studies and their methodological quality.
Assessment of credibility: criteria proposed by Cohen et al 25 will be applied to appraise the degree to which the credibility of the included PROMs has been established. A rating of ‘well-established assessment’, ‘approaching well-established assessment’ or ‘promising assessment’ will be assigned based on the number of peer-reviewed articles that assess an instrument’s measurement properties, the information available, and the detailed statistics of an instrument's measurement properties.
Ethics and dissemination
No ethical issues are foreseen. The findings of this review will be submitted for publication in a peer-reviewed journal, reported at national and international ear, nose and throat and audiology conferences, and included in the lead author’s doctoral dissertation.
Conclusion
Ameliorating the burden of high levels of perceived listening effort experienced in hearing loss is one of the current challenges encountered by researchers and audiology professionals. This challenge is, in part, due to the multidimensional nature of the listening effort construct and debate concerning which methodologies are suitable for its measurement. Pichora-Fuller et al 1 have identified a lack of validated outcome measures of perceived listening effort suitable for use in the audiology clinic. This systematic review will summarise and critically appraise the psychometric measurement properties of existing PROMs used to assess listening effort that have been reported on in the published literature. It will enable researchers and clinicians to understand the quality of existing measures and confirm whether there is a need for a new PROM of perceived listening effort.
Acknowledgments
The authors thank Clare Boucher, Medical Sciences Librarian, Swansea University, and Philip Rawle, Librarian, Abertawe Bro Morgannwg University Health Board, for their assistance in the development of the search strategies.
References
Footnotes
Contributors SEH is the guarantor. SEH and HAH contributed to the development of the research question, selection criteria, search strategy, risk of bias assessment strategy and data extraction strategy. SEH piloted the search strategy and data extraction forms. SEH drafted the protocol manuscript. HAH, FLR, IB and CMM provided critical revisions. All authors reviewed and approved the final manuscript.
Competing interests None declared.
Patient consent Not required. Systematic review, no patient involvement.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement The full Medline (Ovid) search strategy has been submitted as an online supplementary file.