What are patient-reported outcome measures?

The Federal Drug Administration defines patient reported outcome measures (PROMs) as ‘any report of the status of a patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else’.1 Revicki et al 2 describe PROMs as validated self-reporting assessment tools that capture the patient experience. PROMs have been extensively evaluated for their sensitivity, specificity, overall accuracy and predictive value. They are now regarded to have excellent precision, similar to many other widely used clinical assessment tools including pathological tests or medical imaging reports.3 PROMs can provide an overview of a patient’s physical, emotional, functional or overall health status, or can be used to assess specific treatment outcomes or symptoms.4

PROMs in clinical practice

PROMs are commonly used as outcome measures in research. However, more recently, there is evidence that their real-time application in clinical practice can enhance clinical interactions and improve patient experience. Several studies have shown that using PROMs in routine care leads to improved quality of life (QOL)3 5 as well as improved communication, decision-making, care planning and patient satisfaction.6–8 Two recent studies demonstrated improvements in patient mortality and morbidity when technology-facilitated PROMs data collection was incorporated in oncology care.5 9 10

Given these evidence-based benefits, translating these findings into practice by integrating PROMs into routine clinical care is the next required step in the implementation cycle.

The complexities of implementing PROMs into the clinical setting

A number of systematic reviews3 11 12 reported that multiple organisational, technical and clinical factors need to be overcome before introducing PROMs. In particular, a lack of engagement from healthcare professionals, concerns about the workflow of generating and filing of PROM reports and lack of clearly defined approaches in how to respond to the PROM data that indicate a patient need (eg, elevated pain or depression) have been identified as barriers to successful implementation. The International Society of Quality of Life (ISOQOL) advocates a stepwise approach to implementing PROMs and provides a User’s Guide,13 which was updated in 2018. Klinkhammer-Schalke (2014) identified that a stepwise approach was most useful when integrating a PROM intervention into routine care, as it allows cycles of iterative learning during the implementation.7

Incorporating PROMs into clinical practice should be considered a complex intervention, with many elements impacting on the intervention, and vice versa14 Given these complexities, it has been recommended to use an implementation framework to increase the likelihood of success when aiming to integrate PROMs into routine care.15 Use of a framework approach can help to consider both the processes and intended outcomes of implementation. The Promoting Action Research in Health Services (i-PARIHS) framework appears well suited, as it highlights elements for consideration within the context (eg, the features of the particular clinic in which PROMs are to be integrated), the stakeholders (eg, patients, clinicians, administrative staff) impacted by the intervention and the evidence surrounding the intervention (eg, how much do stakeholders value the new PROM information presented to them).16 A unique feature of iPARIHS is that it stresses the central importance of a facilitator, who works with the local stakeholders to adapt the evidence-based intervention for the local context. Antunes et al’s 3 systematic review provided evidence for the important role of a facilitator of the implementation process, with enhanced successful uptake if one was present.17 18 For example, Baskerville et al 17 showed that medical practices were 2.76 more likely to adopt evidence-based guidelines when a facilitator was working in the local context.

Besides the implementation framework, the Medical Research Council (MRC) Guidelines for Implementation of Complex Interventions can provide guidance on how to best incorporate prespecified process measure. The Guidelines ‘can be used to assess fidelity and quality of implementation, clarify causal mechanisms and identify contextual factors associated with variation in outcomes’.18 The MRC approach ensures active evaluation throughout the implementation and highlights how to mitigate the impact that the introduction of new workflows has on the context, participants and the intervention.

In summary, the aim of this implementation study is to investigate implementation of symptom-reporting PROMs system into the outpatient oncology setting. The objective of the intervention will be to increase detection of symptoms by clinicians using the PROMs data. The implementation objectives include the successful engagement of clinicians to use PROMs in clinical practice, the successful use of technology to obtain PROMs data from patients and present reports to clinicians, and the identification of appropriate local strategies to respond to PROM information.

Study design

This mixed-methods study will use a stepped-wedge cluster design. PROMs will be introduced sequentially into two independent clinics, and all intervention and implementation outcomes will be prospectively evaluated. The stepped-wedge approach has been chosen as it is a pragmatic solution for the systematic introduction of a complex intervention19 and has been successfully used in a number of studies related to service delivery improvements.20 21 Another advantage of this study design is that it limits bias by randomly assigning the clinics to the intervention in sequential order. There are key elements that require attention with this study design including the consideration of timing of study time points, cluster equivalence within the setting and intervention uptake assessed by process measures.22 23

The first clinic will be observed during a current standard practice lead-in period for 4 weeks, then introduced into the Integrating Patient Reported Outcomes in a Medical Oncology Setting (iPROMOS) intervention, whereas the other clinic will continue with current standard practice and await implementation of iPROMOS. Data collection and intervention time points are presented in table 1.

This protocol was co-designed with clinicians, academics and patient representatives. The iPROMOS intervention was informed by pre-implementation data collected from health professionals and relevant local stakeholders (table 2). Reporting will follow Standards for Reporting Implementation Studies.24

Patient and public involvement

The process of consumer engagement through protocol development informed the research question and study protocol. Consumer representatives within the health services, and on a research advisory group, were approached to discuss the project. They confirmed a need for patient self-reporting of symptoms that are integrated into routine care. Their reports would need to be available to staff so that their concerns could be actioned. During the development of the protocol, consumer representatives were involved in the development of patient resources and collection of pre-implementation data. They also assessed the anticipated burden of the intervention on patients, and this will continue to be evaluated with consumer input through the study. This will be done through Plan, Do, Study, Act (PDSA) cycle evaluation from qualitative data collected and ongoing consumer representative input.

Results will be disseminated on information boards in the health service, and reported back to Consumer Representative Forums.

Key features of the intervention

Based on the published evidence5 and data from local clinicians as summarised in table 2, the PRO-Common Terminology Criteria for Adverse Events (CTCAE) was selected as the PROM to be implemented, as it was developed to extend an assessment by clinicians using the CTCAE25 and has been demonstrated to provide significant benefits for patient care and outcomes.10 PRO-CTCAE is a validated (119 of 124 items met at least one construct validity criterion) symptom-reporting PROM that has been demonstrated to be reliable (test–retest was 0.7 or greater for 39 of 49 prespecified terms) and responsive (item changes corresponded to the Quality of Life Questionnaire-Core 30 scale).26 There are a number of studies that have demonstrated that the PRO-CTCAE is acceptable to patients from differing cancer populations internationally.27 28 This PROM allows patients to report how much they experience each symptom, and the impact on their daily activities, on a five-point Likert scale (ranging from ‘none’ to ‘very much’). The core set of questions includes anorexia, constipation, dyspnoea, diarrhoea, fatigue, nausea, pain, sensory neuropathy, vomiting, cough, low mood and anxiety. Basch et al’s 5 study used a weekly completion schedule on an app with alerts sent to clinicians in real time. However, use of apps for patient reporting was not compatible with the health service’s patient confidentiality policy. The intervention was adapted to include PROM reporting only during scheduled attendances for outpatient clinic appointments. Thus, reporting to clinicians will occur in line with existing clinic visits, which may be weekly or less frequently depending on cancer diagnosis, stage and treatment regimen. PROMs reports will be made available for health professionals to view and respond to. This could include referring the patient to allied health or supportive care, counselling, or additional pharmacological support (eg, adjusting pain medications). PROMs will be added in paper format to the patient chart, and in keeping with local practice, and then will be scanned into the electronic medical record at a later date.

In summary, the iPROMOS intervention consists of (a) patients self-reporting symptoms (PRO-CTCAE PROM) using a touchscreen computer with data captured on a custom-built REDCap database; (b) reports of this information are generated in real time; (c) these reports are available to all healthcare team members and filed in the patients’ medical record and (d) a copy of the report is also provided to the patient. Usual care is clinician assessment of symptoms without the additional use of a PROM.

In the co-design process, using the broader research evidence, investigated to support clinician’s recommendations, a reported symptom of grade 2 or higher for nausea, vomiting or anorexia, and grade 3 for all other symptoms is considered significant.5 If there is an increase in symptoms greater than 2 points from the previous visit, this will also trigger a referral by established pathways to the relevant allied health professional.

Setting of the implementation

This project will be conducted in a tertiary teaching/quaternary referral hospital located in South-East Queensland, Australia. The health service for this centre is the largest in Australia, with the oncology outpatients’ department running up to 14 clinics in 1 day. Each of these clinics is oncologist specific, providing service for treatment, surveillance and follow-up for the patients in their care.

Contextual pre-implementation information has revealed key factors for successful integration of the intervention (table 2). Most importantly, the intervention needs to engage all members of the multidisciplinary team and the staff who will have access to the PROM information to address symptoms, disease management and treatment. To make this likely, the facilitator will aim to integrate the PROM collection and reporting as much as possible into the existing workflow processes already in place at the clinic. Evidence shows that workflows differ greatly between hospitals and even within clinics in a hospital, and that staff are reluctant to change anything that interrupts established practice, given the very complex environment they are managing.29 They are only willing to take on a new intervention when the benefits and processes for patient care are tangible and clear. For successful implementation, it has been identified that it is necessary to integrate with existing patient care pathways and technological infrastructure, rather than impose another layer, which would likely be met with resistance.29

Participants

This study will collect data from two main groups of participants: (a) patients and (b) the clinicians caring for them.

1. Patients who attend the randomised medical oncology outpatients’ clinics for treatment, medical review, active surveillance or routine follow-up, with sufficient English knowledge to read the questionnaires. Patients with significant cognitive impairment, visual difficulties or from a non-English-speaking background who might have difficulty in completing the forms will be excluded from the study.

   Patient Screening and Recruitment: patients attending selected clinics will be invited to use touchscreen computer to complete PROM information. The first page of the PROM collection form provides a patient information sheet and consent form. Potential participants will need to read the information and accept to enter PROM-reporting platform. If they do not wish to, they can choose to decline. Patient information will also be visible on a poster displayed in the clinical waiting area.

2. Staff who care for these patients including nursing and medical staff, pharmacists, dietitians, welfare workers, social workers, psychologists, speech therapists, physiotherapists and other allied health workers are eligible.

   Staff participation: an opt-out approach to consent staff has been approved by the ethics committee. Multidisciplinary staff will be contacted using various communication channels, directly by the facilitator–researcher to collect pre-implementation information, as well as through distribution of information brochures and posters developed in collaboration with the clinical teams.

Methods of evaluation

Sample size

Berry et al 32 conducted a randomised controlled trial that compared symptom reports between clinics using an electronic reporting tool. They assessed both processes and outcomes of care, comparing the impact of PROM reports between the control and intervention clinics. It was used to guide the sample size calculations because this study measured the identification of symptoms in usual care versus a symptom-PROMs intervention. To obtain an estimate of a minimal number of observations that should be included in each cluster in this study, Berry et al’s32 results were used. These researchers identified that a PROMs intervention increased symptom detection by 10%. Using these findings, and 80% power, given a baseline detection level of 0.75, 500 participant encounters would be needed to show improvement by 10% or more.

Quantitative analyses

Quantitative measures have been designed for the process measures of implementation evaluation, the outcome measures of the implementation and the outcome measures of the intervention. Descriptive statistics including counts, frequencies and proportions will be used to summarise data collected. Other statistical analyses to be used will include χ²analysis for comparing proportions, linear mixed models for longitudinal analyses and statistical control process analysis to identify trends over time.

Data from both clusters will be analysed using inverse variance weighting so that the difference can be estimated for all patient encounters. This analysis can be used to adjust for cancer types, or clustering by clinicians.33 This analysis will provide a measure of the intracluster effect, which can then be used for power calculations in future larger studies.34

Qualitative data

The facilitator site journal will be used to record observations, and will be content analysed to identify key themes, as a part of each PDSA cycle every 21 days.

The analysis of the facilitator site field notes will be used to triangulate other research findings highlighting aspects in need of further investigation. The function of field notes is to identify processes in a given situation and describe how participants contribute to, and impact, these.35 Extracted data will be interpreted in keeping with Miles et al’s36 approach using field notes who propose an analysis of systematic coding, word by word, presenting the data visually to identify patterns.

Data monitoring

Data monitoring will ascertain high data quality, ensure rigour and mitigate biases.

Data monitoring will be done through the following three processes:

1. Quantitative data will be double entered for a random sample of 10% records, and all records will be double entered should the error rate be greater than 5%.

2. Monthly meetings with expert facilitators who are not involved with the project to reflect on the implementation and evaluation of the project.

3. Supervision and oversight by the study team not directly involved in the process of implementation.

Safety considerations

The main purpose of the secondary outcome measures of the intervention is to measure the safety of using this implementation approach. A potential safety issue is that when patients complete the PROMs, they expect that staff will act on that information. If the implementation is not successful, staff may not do this in a timely fashion or at all, and patients who report symptoms may not receive suitable treatment. Any such issues where a PROMs report was not acted on will be noted and described using the data collection tools for the project. The facilitator will raise any issues where patient safety is at risk.

Data deposition and curation

All de-identified data will be stored on a REDCap database, on a secure university server. Patient information will be stored on their medical record and hospital-based servers that are password protected. Data will be stored for 5 years. A formal data management plan has been developed and approved by the Queensland University of Technology Research Unit.

Dissemination of results

Results will be disseminated in peer-reviewed publications and presented at national and international scientific meetings.