Article Text
PDF
Abstract
Objectives To identify patient-reported outcome measures (PROMs) and patient-reported experience measures (PREMs) in clinical quality registries, for people with cutaneous melanoma, to inform a new Australian Melanoma Clinical Outcomes Registry; and describe opportunities and challenges of routine PROM/PREM collection, especially in primary care.
Design Systematic review.
Primary and secondary outcome measures Which PROMs and PREMs are used in clinical quality registries for people with cutaneous melanoma, how they are collected, frequency of collection, participant recruitment methods and funding models for each registry.
Results 1134 studies were identified from MEDLINE, PreMEDLINE, Embase, PsychInfo, Cochrane Database of Abstracts of Reviews of Effects databases and TUFTS Cost-Effectiveness Analysis Registry, alongside grey literature, from database inception to 5th February 2020. Following screening, 14 studies were included, identifying four relevant registries: Dutch Melanoma Registry, Adelphi Real-World Disease-Specific Programme (Melanoma), Patient-Reported Outcomes Following Initial treatment and Long-term Evaluation of Survivorship Registry, and Cancer Experience Registry. These used seven PROMs: EuroQol-5 Dimensions, Functional Assessment of Cancer-General (FACT-G) and FACT-Melanoma (FACT-M), European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire-Cancer 30 (EORTC QLQ-C30), Fatigue Assessment Scale Hospital Anxiety and Depression Scale, Patient-Reported Outcome Measures Information System-29 and one PREM; EORTC QLQ-Information Module 26. PROMs/PREMs in registries were reported to improve transparency of care; facilitate clinical auditing for quality assessment; enable cost-effectiveness analyses and create large-scale research platforms. Challenges included resource burden for data entry and potential collection bias toward younger, more affluent respondents. Feedback from patients with melanoma highlighted the relevance of PROMs/PREMs in assessing patient outcomes and patient experiences.
Conclusions Clinical registries indicate PROMs/PREMs for melanoma care can be incorporated and address important gaps, however cost and collection bias may limit generalisability.
PROSPERO registration number CRD42018086737.
- clinical audit
- dermatological tumours
- public health
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
Patient-reported outcome measures (PROMs) and patient-reported experience measures (PREMs) have been highlighted by melanoma consumers (patients and carers) as important measures of quality that should be incorporated into clinical quality registries. Patients with melanoma provided feedback on the importance of this review and its relevance to patient outcomes and patient experiences.
This study captures publications of PROM and PREM collection across a range of database types.
Provides insight into PROM and PREM collection as described by authors directly involved in their respective registry, where published.
Only clinical quality registries in the public domain with melanoma-specific search terms were identifiable and included.
Evaluation of the validity of identified PROMs/PREMs from primary studies was not undertaken.
Introduction
Patient-reported outcome measures (PROMs) are described as ‘measurements of any aspect of a patient’s health status that come directly from the patient’.1 These tools are commonly used to assess patients’ self-perception of their functional status, health or quality of life. Patient-reported experience measures (PREMs) focus on patients’ satisfaction with care and cover topics such as dignity and respect; consistency and coordination of care; adequate involvement in, and explanation of care; trust and communication with nurses and doctors; and satisfactory discharge information.2 While these measures were originally developed for use in research, increasingly they are being used for broader purposes, such as inclusion in routine healthcare or clinical quality registries for quality assurance and benchmarking.3
PROMs and PREMs can be applied in several stages, such as initial patient evaluations, during treatment and in follow-up.4 These tools can complement the existing roles of clinical quality registries as platforms for quality assessment and benchmarking, as well as for large-scale research projects. In the case of cancer, the Prostate Cancer Outcomes Registry–Australia and New Zealand uses the 26-item Expanded Prostate Cancer Index Composite survey to assess the impact of diagnosis and treatment in men with prostate cancer towards improving patient well-being.5 Data collected by this registry have resulted in tailored strategies and targeted programmes to address important gaps in patient care.6 7
With the support from key partner organisations, funding was raised through the Melanoma March event to develop the Australian Melanoma Clinical Outcomes Registry (MelCOR). Collection of PROMs and PREMs is an important goal of MelCOR, however challenges in this are apparent. PROMs and PREMs are not routinely collected as a part of melanoma care (including primary care) in Australia, which tends to focus mostly on traditional outcomes such as prevention of recurrence, diagnosing subsequent primary melanomas8 and maximising life expectancy after diagnosis. Surgical morbidity parameters such as wound complications, pain and numbness are often added to these traditional measures.9 However, patient reports of diagnosis and treatment experiences are typically not evaluated, despite these being an important indicator of quality of care. For example, patient’s self-perception of keloid scarring following diagnostic excision or other surgery on the face or hands may be associated with significant psychological morbidity including anxiety, poor self-esteem and a decreased quality of life.10
As the value and usefulness of MelCOR will be enhanced by inclusion of PROMs and PREMs, we sought to understand which tools capture the most relevant generic and disease-specific information for people with cutaneous melanoma, and the opportunities and challenges associated with routine PROMs and PREMs collection. The aim of this systematic review was to identify PROMs and PREMs that are used in clinical quality registries and that might be applied to patients with early-stage cutaneous melanoma, including in primary care and specialist settings where much of the initial evaluation and management of this cancer occurs.
Methods
A systematic review of relevant literature was conducted. The search strategy included patients treated for cutaneous melanoma of any stage, to maximise the number of potential identifiable registries in this disease context, along with articles discussing the advantages and challenges of PROMs or PREMs implementation in registries. Five databases were searched from inception to December 2017: MEDLINE (including PreMEDLINE), Embase, PsychInfo, Cochrane Database of Abstracts of Reviews of Effects, and TUFTS Cost-Effectiveness Analysis Registry, without language restriction. Reference lists of relevant articles were hand searched for additional publications. Grey literature including digital theses, government reports (eg, the Australian Quality and Safety Commission), and industry reports (eg, International Consortium for Health Outcomes Measurement) were also searched. A secondary search of MEDLINE (including PreMEDLINE), Embase, PsychInfo and the Cochrane Database was conducted from December 2017 until 3rd February 2020. Guidelines for the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)11 were followed, where applicable.
Patient and public involvement
Two patients with melanoma provided feedback on the importance of this review and its relevance to patient outcomes. The PROSPERO protocol for the systematic review was circulated to melanoma consumer representatives, then face-to-face feedback about PROMs/PREMs in the new MelCOR Registry was sought. This occurred alongside the MelCOR Steering Committee meeting in 2018.
Search strategy
‘Patient reported outcome measures’ and ‘patient reported experience measures’ (including abbreviations PROMs or PREMs), along with related keywords for patients (consumers, participants) and outcomes (satisfaction, experience, assessment, symptom-measure), or quality of life, were combined with Boolean logic (AND/OR) with terms for melanoma (including lentigo maligna), to identify relevant studies (online supplemental appendix 1). Medical Subject Headings were adapted to each database as appropriate (figure 1—PRISMA flow chart).
Supplemental material
Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow chart.
Eligibility criteria
Publications involving the use of PROMs and/or PREMs in clinical registries reporting adults with cutaneous melanoma of any disease stage (ie, stage 0–IV) were included. PROMs and PREMs completed by proxy (eg, caregivers) were also included. Populations with mixed cancer that included cutaneous melanoma were eligible, however populations with non-cutaneous melanoma (eg, uveal melanoma), and patients with non-melanoma skin cancers only, were excluded. Studies that only used clinician-reported outcomes were excluded, as were PROMs/PREMs collected solely for research purposes (eg, within a clinical trial). All primary study types were eligible, and grey literature was also searched, with government reports and digital theses included. Cost-effectiveness analyses were included up to the full-text screening stage, and their reference lists hand searched for relevant articles. Review articles were excluded.
Study screening and data extraction
Citations and abstracts were imported into Covidence (Cochrane systematic review management program) for screening purposes. Title and abstracts were screened by two independent reviewers (ZB, AT), with discrepancies resolved through discussion, and consultation with a third reviewer (RLM) where necessary. Full-text articles were assessed if the paper was deemed to meet the inclusion criteria, was a cost-effectiveness study or the abstract did not contain sufficient information. Clinical registries from included studies which used PROMs and/or PREMs data collection from people with melanoma of any stage were identified for data extraction. For each clinical registry identified through this review, the PROMs/PREMs used, patient demographics, method of collection, collection time points and purpose of the registry were identified. Any descriptive text relating to advantages or challenges of registry collection was also extracted. The COnsensus-based Standards for the selection of health status Measurement Instruments four-point checklist and International Society for Quality of Life Research user’s guide to implementing patient-reported outcome assessment in clinical practice were not used to assess instrument quality, as a critique of PROMs/PREMs used in registries was not an aim of this systematic review. Likewise, quality assessment of individual studies was not performed, as the role of this review was to identify relevant clinical registries present in published works, and the PROMs/PREMs they collected. Risk of bias in individual studies was described; studies were not excluded due to bias.
Results
Database and hand searching yielded 1134 articles, with 988 articles remaining after removal of duplicates (figure 1). Following abstract and full-text screening, 14 relevant articles remained for analysis.12–25 These studies used either data collected from these registries to answer a research question, or were articles commenting on the advantages and challenges of PROM/PREM collection and implementation via a registry. A summary is provided in table 1. Four relevant clinical registries were identified, each from published literature: the Dutch Melanoma Treatment Registry, Adelphi Real-World Disease-Specific Programme (Melanoma), the Cancer Experience Registry, and the Patient-Reported Outcomes Following Initial treatment and Long-term Evaluation of Survivorship (PROFILES) registry. One registry, the Eindhoven Cancer Registry, was cited in the context of its utility for recruiting participants to the PROFILES registry.23 This registry was not included in our analysis, however, as it did not meet our inclusion criteria of collecting PROM/PREM data from people with melanoma. Across the 4 included registries, 10 PROMs and 1 PREM were used. A summary of findings including PROM/PREM collection method, frequency, participant recruitment and funding for each registry is presented in table 2.
Summary of included studies
Summary of registries using PROM and/or PREM data, including people with melanoma
The Dutch Melanoma Treatment Registry
The Dutch Melanoma Treatment Registry is a registry of patients based in the Netherlands with unresectable stage IIIc or IV melanoma. It is used for clinical auditing, improving the transparency of quality of care in the treatment of melanoma, to detail the effects and costs associated with treatment, and serves as a database for melanoma research. This registry collects three PROMs: EuroQol-5 Dimensions (EQ-5D), Functional Assessment of Cancer Therapy-General (FACT-G)26 and FACT-Melanoma (FACT-M).27 It also collects treatment activity through a Resource Use Questionnaire-Melanoma (RUQ-M). The Dutch Melanoma Treatment Registry aims to collect PROMs every 3 months and the RUQ-M every 6 months. Questions about employment and time lost from work due to melanoma and its treatment are used to quantify productivity gains and losses, and are often used in economic evaluations undertaken from a societal perspective. Data are prospectively collected using a web-based programme from patients diagnosed with metastatic melanoma who attend participating treatment centres.
The Adelphi Real-World Disease-Specific Programme (Melanoma)
The Adelphi Real-World Disease-Specific Programme (Melanoma) is a privately owned international healthcare database based in the UK, used for research purposes. Adelphi collates disease-specific programmes on over 60 diseases, including melanoma, with data used for both medical and market research. The Melanoma programme collects three PROMs; the EQ-5D, FACT-G and FACT-M from patients with all stages of melanoma under the care of participating clinicians, from multiple countries. Here, participating clinicians are asked to collect data on the next 10 patients being consulted for the disease of interest, with these patients asked to self-complete the relevant PROMs.28
PROFILES Registry
The PROFILES Registry is a Netherlands-based registry of cancer survivors and collects information from patients will all stages of melanoma. It exists to identify outcome and psychosocial risk assessment, analyse behavioural and biological factors associated with the outcomes of cancer treatment, and evaluate psychosocial and physical care needs in cancer survivors. It collects three PROMs: the European Organisation for Research and Treatment of Cancer Core Quality of Life Questionnaire-Cancer 30 questions (EORTC QLQ-C30),29 Fatigue Assessment Scale (FAS)30 and Hospital Anxiety and Depression Scale (HADS).31 In addition, it collects one PREM, the EORTC QLQ-Information Module 26 (EORTC QLQ-INFO26)32; a measure of patient satisfaction regarding the information they receive during treatment.
The Cancer Experience Registry
Finally, the Cancer Experience Registry is a US-based registry which all patients with cancer are eligible to register for, regardless of cancer type or stage. For melanoma it collects PROMs through the Patient-Reported Outcome Measures Information System-29 (PROMIS-29).33 It also has a registry-specific survey with specific patient experience items including treatment decision-making, body image and financial impact of cancer. Run by the Cancer Support Community, a not-for-profit organisation, patients self-elect to register to the Cancer Experience Registry through their online website, where they are provided with an informed consent waiver, and are invited to complete the survey. This information is used primarily for research purposes.
Discussion
The four registries identified in this review varied considerably with regard to purpose of data collection and methods for data collection and dissemination. Registries specific for melanoma, namely the Dutch Melanoma Treatment Registry and Adelphi Real-World Disease-Specific Programme (Melanoma) both favoured the use of health-related quality of life (HR-QoL) PROMs: EQ-5D, FACT-G and FACT-M in their data collection. The EQ-5D is a generic questionnaire, aimed at general populations, while the FACT-G and FACT-M are both disease-specific questionnaires aimed toward people undergoing cancer therapy and melanoma-specific treatment, respectively. As such, the data collected provide a comprehensive overview of participant HR-QoL and facilitate comparisons between registries.
PROMs collected in registries of people with mixed cancer types, such as PROFILES, used the EORTC QLQ-C30 which is a cancer-specific measure of HR-QoL. Specific comorbidities, such as fatigue and depression, are measured by separate instruments, namely the FAS and HADS. Conversely, the Cancer Experience Registry used PROMIS-29, a generalised PROM, and paired this with registry-specific items relating to the experience of melanoma treatment. PROFILES was the only identified registry that used a validated PREM, the EORTC QLQ-INFO26, assessing satisfaction with information received during treatment.
We noted registries tended to use PROMs developed in their geographical regions. European union (EU)-based registries used measures developed in the EU, such as those from the EORTC and EuroQol Group, whereas the US-based Cancer Experience Registry used the US-developed PROMIS-29. Given calibrated PROMs assessing similar constructs can be cross-walked, it is not necessary for registries to select identical measures to be considered comparable with one another. However, restricting patient-reported outcomes to validated PROMs or PREMs, as opposed to non-standardised questionnaires, is important for any findings to be generalisable.
Each registry applied a different method of participant recruitment and data collection, with associated advantages and disadvantages. The majority of registries favoured an online approach (Dutch Melanoma Treatment Registry, PROFILES and Cancer Experience Registry). The Cancer Experience Registry was unique in that it functions independently of treatment centres and existing cancer registries, with all participants volunteering to join through their online website. This presents an efficient means of collecting data, and minimises missing values and costs associated with active participant recruitment, printing/mailing of surveys and data entry. However, while the internet has a high penetrance in many countries including Australia, internet use (and smart phone apps) are associated with younger age groups and a higher socioeconomic status.34 As such, this method may run the risk of sample bias.
One potential solution to this is to supplement online tools with paper-based collection, a strategy employed by the PROFILES Registry. Here, patients meeting the criteria of a ‘cancer survivor’ within the existing Eindhoven Cancer Registry were actively recruited via mail, through their treating clinician.23 This mail package included a link to a secure website for response to the survey items, and also included a paper copy which could be mailed back, should the participant not have internet access. Another alternative to online collection could involve a call-based interactive voice-response system, although this method was not observed in the identified registries.
Similarly, the Dutch Melanoma Treatment Registry mailed eligible participants an invitation to join the registry, however data collection was conducted either through a web-based ‘patient portal’ system or collected through a method at the discretion of the participating institution, before forwarding the results to the Dutch Institute for Clinical Auditing. Here, patients were contacted regularly to provide up-to-date patient-reported outcome data. One issue inherent to this collection method is that while patients were free to complete their feedback in their own time, the timing and environment in which it was completed would have been variable, and thus poorly controlled. Conversely, the Adelphi Real-World Disease-Specific Programme (Melanoma) Registry had participants complete their feedback form directly following consultation with their clinician, therefore minimising bias relating to the timing of collection.
The use of large-scale clinical quality registries introduces a broad set of valuable applications. For registries involving patients undergoing active treatment, the use of PROMs and PREMs allow for benchmarked feedback on quality of care and medical practice variation provided by participating centres (including primary care providers), or cancer registries, as well as providing cost-effectiveness data of new treatment options to industry (eg, medical and pharmaceutical companies) and health technology agencies such as Medical Services Advisory Committee in Australia. It can improve accountability of melanoma treatment, by providing feedback to participating centres/clinicians of the health outcomes of their patients. For example, the Dutch Melanoma Treatment Registry provided fortnightly feedback on clinician’s quality of care.20 This can translate into a change in clinical care for patients with poor health outcomes. These registries also provide a robust platform for research and can be used in annual reports on the quality of life of people with melanoma.
A key challenge of implementing PROMs and PREMs in a melanoma registry on a national scale is their cost. For example, the costs associated with the Dutch Melanoma Treatment Registry are data entry, analysis, validation and training staff in data management, estimated to amount to €500 per patient registered.35 These costs might be reduced by developing strict criteria regarding which items are relevant to all registered patients, and which items are relevant only to a subset of those registered. Efficiency of data collection via integration with electronic health records and data linkage is another key consideration in reducing costs.35 As discussed previously, web-based collection provides a cost-efficient method of data acquisition, though carries with it the risk of introducing sample bias. Importantly, self-assessment tools have been shown to have similar reliability whether administered online or paper-based.36 37 As such, combining online and paper-based collection methods is a feasible compromise and may result in an increased volume of completed questionnaires, as illustrated by the PROFILES Registry. With respect to funding, the registries identified here were primarily funded by private companies, non-profit organisations or directly from melanoma treatment/research centres.
While this review suggests registry collection of PROMs/PREMs is feasible and describes various methods by which large-scale melanoma registries can integrate PROMs and PREMs, our analysis is limited to clinical quality registries identified from published and grey literature. Although the study search was comprehensive and we did not apply a language restriction, we are aware that other clinical quality registries may exist (eg, in hospital or health service databases), however any such registries outside the public domain were unable to be included. Reporting bias is also possible, if primary study authors did not use common search terms. A further limitation of our study was the inability to report on the validation of each PROM/PREM for use in melanoma care, due to incomplete reporting and assessment of instrument validation in the primary studies.
We found there was little evidence of how PROM and PREM data are currently being used in daily clinical practice. This is largely due to the fact that local (non-registry) datasets are more likely to be used in clinical care, and not identified by systematic review, for example, the ePROMs-MEL Study that is currently underway in Australia.38 Developing future registries within electronic health records could overcome this and provide greater integration with existing online health systems. Nonetheless, insights into the use of PROMs and PREMs in clinical practice with respect to melanoma diagnosis and treatment may provide useful strategies for data collection and feedback at a registry level. Our future plan for MelCOR is to pilot the feasibility and acceptability of the above-mentioned PROMs and PREMs with the addition of a new validated instrument, Melanoma Concerns Questionnaire,39 40 among patients, clinicians, researchers and registry stakeholders.
In summary, current examples provide evidence that clinical quality registries can feasibly incorporate PROMs and PREMs for melanoma care and will inform how best to implement these tools in our new MelCOR. Specific instruments such as the EQ-5D, FACT-M and QLQC–INFO26 may be useful to integrate into MelCOR, although further research as outlined above, is required to evaluate the acceptability of these, particularly in primary care. The integration of a general PROM, a disease-specific PROM and a PREM in future registries would provide a wide breadth of data for quality assurance, to minimise clinical practice variation and as a research platform. Cost and sample bias may limit generalisability. Importantly, routine assessment and identification of patients who have poor PROMs and PREMs provide an opportunity to deliver a clinical intervention that may improve HR-QoL, improving overall melanoma care for patients.
Acknowledgments
We thank Melina Willson from the Cochrane Breast Cancer Methods Group at the NHMRC Clinical Trials Centre, University of Sydney, for her guidance with Covidence software.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @RachaelMorton12
Contributors ZB—search strategy design, article screening, data extraction/analysis and manuscript preparation. AT—article screening, interpretation of results and reviewing the final manuscript. LC—interpretation of results and reviewing the final manuscript. RS—study design, interpretation of results and reviewing the final manuscript. MD—study design, interpretation of results, drafting and reviewing the final manuscript. MS—study design, interpretation of results and reviewing the final manuscript. HPS—study design, interpretation of results and reviewing the final manuscript. CA—study conception, interpretation of results and reviewing the final manuscript. GJM—study design, interpretation of results and reviewing the final manuscript. RLM—study conception and design, student supervision, interpretation of results and reviewing the final manuscript.
Funding This study was supported by an NHMRC Centre of Research Excellence in Melanoma Grant APP1135285, and a University of Sydney Summer Scholarship to ZB. RLM is supported by an NHMRC TRIP Fellowship APP1150989 ‘Incorporating patient-reported outcome measures (PROMs) into clinical registries’ and a University of Sydney Robinson Fellowship. HPS holds an NHMRC MRFF Next Generation Clinical Researchers Program Practitioner Fellowship APP1137127. RS is supported by Melanoma Institute Australia.
Competing interests RS has received honoraria for advisory board participation from MSD, Novartis and Qbiotics, and speaking honoraria from BMS. HPS is a shareholder of MoleMap NZ and e-derm consult GmbH, and undertakes regular teledermatological reporting for both companies. HPS is a Medical Consultant for Canfield Scientific, Revenio Research Oy and also a Medical Advisor for First Derm. CA is a Director of BioGrid Australia.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement All data relevant to the study are included in the article or uploaded as supplemental information. All data extracted from included studies are contained in tables 1 and 2 of this manuscript.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.