Article Text
PDF
Abstract
Objectives To design an electronic questionnaire for symptom monitoring and to evaluate the feasibility, usability and acceptability when applied to patients with metastatic lung cancer.
Setting Single-centre feasibility study.
Participants Patients with stage IV lung cancer in antineoplastic treatment.
Interventions This study describes the first three phases of a complex intervention design: phase 1, development of the intervention; phase 2, feasibility testing and phase 3, evaluation of the intervention. In phase 1, items were selected for the questionnaire and adjusted following patient interviews. In phase 2, patients completed the electronic questionnaire weekly during a 3-week feasibility test. In case of symptom deterioration, a nurse was notified with the aim to contact the patient. In phase 3, patients evaluated phase 2 by paper questionnaires, and interviews were conducted with the participating nurses.
Primary outcome measures The study outcomes: phase 1, usability and relevance; phase 2, recruitment rate, compliance and threshold functionality and phase 3, usability, acceptability and relevance.
Results In phase 1, a questionnaire was designed and reviewed by patients (n=8). The interviews revealed high usability and relevance of the intervention.
For phases 2 and 3, 20 of 29 approached patients (69%) responded to the questionnaire on a weekly basis. Two patients did not complete any questionnaires (compliance 90%). The remaining 18 patients completed 65 of a total of 72 possible questionnaires (7 missed, 93% completed). Reported symptoms led to a phone call from a nurse in 30% of the responses.
The patients reported high usability and acceptability of questionnaire and software. The substance of the telephonic conversations was relevant, and the study set-up was logistically acceptable.
Conclusions An electronic questionnaire designed for symptom monitoring revealed high usability, acceptability and relevance in the target population. In conclusion, the study set-up was considered feasible for a randomised controlled trial.
Trial registration number NCT03529851.
- oncology
- telemedicine
- interstitial lung disease
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
Electronic symptom monitoring of patients with lung cancer was tested for feasibility before conducting a randomised controlled trial.
A mixed-methods design was used to refine the electronic questionnaire.
Feasibility was tested by a three-phase complex intervention approach.
Patients were involved in evaluation and adjustment of the intervention.
Limitations include a short study period and a limited number of participating patients.
Introduction
Lung cancer is one of the most common cancers and the leading cause of cancer-related mortality globally.1 In Denmark, its annual incidence is approximately 4600 and more than 3700 persons die from the disease every year; thus, lung cancer accounts for 24% of all cancer-related deaths and 7% of the total mortality rate.2 3
Patients diagnosed with metastatic lung cancer frequently suffer from multiple and severe symptoms adversely affecting their health-related quality of life and causing psychological distress.4 5 These symptoms may impair their overall health condition, potentially reducing antineoplastic treatment efficacy.6 However, studies showed that symptom management during early palliative care may reduce the symptom burden and increase survival in patients with metastatic lung cancer.7 However, symptom deterioration between scheduled outpatient visits may go unnoticed which could delay a timely management. Furthermore, clinicians are not always consistent in their assessment of symptoms and often estimate them to be less severe than do patients themselves.8–10 These discrepancies may be remedied by the use of systematic communication tools.11 Patient-reported outcomes (PROs) used in clinical practice for symptom monitoring have been shown to improve patient satisfaction, patient–caregiver care and communication, and to lead to earlier symptom management.12 13 Currently, software solutions exist where patients can report symptoms from home to the department via the internet.14 15 In such set-ups, weekly electronic PRO (ePRO) monitoring has been found to improve overall survival and health-related quality of life in patients with lung cancer16 17 and in a broad cancer population during chemotherapy.18 Both studies used a threshold mechanism to notify clinicians in case of concerning symptoms. These results may be attributed to a combination of early detection of progressive disease, enhanced management of adverse events and improved palliative care.
Successful implementation of ePROs into clinical practice is a complex task involving several stakeholders. This task must be adapted to local logistic set-ups and it requires a clinically relevant ePRO system.19 20 However, no consensus has been established on which specific ePRO questionnaires should be used for patients with lung cancer or when and how clinicians should be notified of symptom deterioration in patients with lung cancer.16 17
The aims of this study were to design an electronic questionnaire for symptom monitoring and to evaluate its feasibility, usability and acceptability in patients with metastatic lung cancer.
Methods
Study population
The study was conducted in May–July 2018 at the Department of Oncology at Hospital Unit West Jutland, Herning, Denmark. Outpatients diagnosed with stage IV lung cancer with an available internet connection at home were eligible. Patients treated for lung cancer with a curative intent are not treated in our department, therefore patients with lower stages of disease were excluded. Patients were required to read and speak Danish, and they were receiving first-line or second-line medical antineoplastic treatment at the time of enrolment.
The electronic PRO software
The AmbuFlex system is a generic Danish PRO software system integrated into the electronic medical records at Hospital West Jutland.21 22 AmbuFlex has been used for follow-up on cancer and other chronic diseases since 2014, and it is used both in clinical practice and for research. Patients fill in health-related questionnaires via a home page and clinicians can access their responses in real-time on-screen. Mirroring longitudinal symptom development, consecutive answers are presented visually with colour bars, numbers and text. An automated threshold mechanism can be activated to identify patients needing clinical attention based on individual responses and symptom severity.23–25 The acceptability and usability of AmbuFlex in the clinical setting is deemed high by both nurses and physician.22 25 26
Study design
This feasibility study covers the first three phases of a complex intervention designed according to the Medical Research Council’s (MRC) guidelines27: phase 1, development of the intervention; phase 2, feasibility testing and phase 3, evaluation. After each phase, we adjusted the system before entering the next phase.27 28 ePRO monitoring in the clinic was implemented in accordance with the guidelines recommended by the International Society for Quality of Life Research.29 30 The purpose was to design a symptom monitoring system added to standard of care. Thus, the number of scheduled CT scans was not reduced for any patient in the current study or subsequent randomised controlled trial (RCT).
Definitions
The following definitions were used:
Usability: design factors affecting users’ experience of operating the questionnaire software and navigating it for the intended purpose.
Acceptability by patients: factors affecting users’ willingness to participate in weekly symptom self-reporting.
Acceptability by clinicians: factors affecting users’ willingness to use the system.
Relevance: a subjective perception of whether the questionnaire addressed issues deemed relevant both for patients to report to the hospital and for clinicians to be notified of.
The EORTC Item Library
The European Organisation for Research and Treatment of Cancer (EORTC) Item Library is an online database of hundreds of individual items from previously validated and translated EORTC questionnaires which allows for single-item combinations in construction of item lists for clinical and research purposes.31
Phase 1: development of the intervention
The development phase comprised an initial review of studies reporting improved survival by ePRO-based symptom monitoring.16–18 32 The authors who are also clinicians initially selected specific items for the study and integrated them into the AmbuFlex software. Patient interviews were finally conducted to appraise the need for further adjustment before phase 2.
Adjustment of the electronic questionnaire
To assess the usability and clinical relevance of the electronic questionnaire, we conducted semistructured interviews with individual patients. We kept recruiting interviewees until data saturation was reached. The interviewees were provided with a portable computer and a written instruction describing the login procedure. They were encouraged to fill in the questionnaire and comment on the procedure, while the investigator observed the process and conducted the interview.33 The interviews were supported by an interview guide to explore and address potential issues regarding usability and relevance perceived by patients or observer. The interviews were recorded, transcribed and analysed using thematic text analysis.34
To the best of our knowledge, there is no consensus on the use of threshold definitions for symptom monitoring. The authors then defined individual symptom thresholds through consensus discussion. During phase 2, the department was notified when symptoms reached these thresholds.
Phase 2: feasibility test
The second phase was a 3-week prospective feasibility test of the AmbuFlex PRO software for which a sample size of minimum 15 patients was considered sufficient based on general pilot and feasibility study recommendations.35 36 To compensate for potential compliance problems, we enrolled 20 patients.
The patients were provided with written instructions and asked to answer the ePRO questionnaire from a home internet connection a total of four times with 1-week intervals.
Two clinical oncology nurses with prior experience with the AmbuFlex system were trained in the study procedures. They were provided with a guide describing the threshold functionality of the software and how to review and manage symptom charts. The AmbuFlex software was programmed to automatically include a given patient on a notification list whenever the response exceeded the predefined symptom severity threshold. As a part of their daily work routine, the nurses were instructed to access the notification list, review responses and contact patients (figure 1). If a written comment in the comments field had triggered notification, the nurse could choose not to contact the patient if it was not deemed necessary. If a phone conversation indicated progressive disease, the planned CT scan should be brought forward and performed as soon as possible (usually done within a week); otherwise, the patient’s symptoms were treated according to best supportive care practice. The nurse recorded time spent on all study-related procedures and the number of phone calls on a daily basis.
The logistic set-up. Symptoms are reported weekly via the internet. Patients who have reported symptoms that require attention are placed on a notification list. The symptom chart is reviewed daily by a nurse who contacts the patients. ePRO, electronic patient-reported outcome.
The recruitment rate was defined by the number of enrolled patient/approached patients. Compliance was the proportion of enrolled patients responding to at least one questionnaire. The threshold algorithm was evaluated as the fraction of responses leading to notification and a subsequent phone call from the nurse.
Phase 3: evaluation of the intervention and study set-up
All patients participating in phase 3 filled in a paper questionnaire by the end of the study period, evaluating the electronic questionnaire and the software as a unified entity. Evaluation themes covered usability, acceptability and relevance. Later, we conducted semistructured interviews with the involved nurses to evaluate the logistic set-up and the clinical relevance of the chosen thresholds. The authors finally evaluated the results at a consensus meeting, agreeing on adjustments before initiation of the RCT.
Patient and public involvement
Patients participating in this study were involved in the design of the ePRO intervention for the subsequent RCT. The intervention was evaluated and adjusted based on questionnaires and interviews with the patients.
Results
Phase 1: development of the intervention
Initial item selection and threshold definitions for the electronic questionnaire
Our literature review identified two previous studies, both suggesting improved overall survival following weekly internet-based monitoring of patients with cancer.16–18 Studies of patients with lung cancer have suggested other tools for symptom monitoring but reported no improved clinical intervention outcomes.37 38 The first study16 17 included patients with stages II–IV lung cancer and focused solely on symptoms relevant to patients with lung cancer, and the second study18 focused on adverse events caused by antineoplastic treatment among patients receiving active treatment of whom 25% were diagnosed with lung cancer.
Before patient interviews in phase 1, we selected 12 symptoms for the ePRO questionnaire based on previous studies.16 18 32 39 Eleven of these symptoms were identical with the symptoms reported by Denis et al.16 Self-rated overall health was, due to known prognostic properties, included in the questionnaire instead of depression.40 41 The 12 symptoms selected for the initial version of the questionnaire are shown in box 1.
The initial symptoms selected for the electronic questionnaire
Symptoms graded by severity*
Overall health.
Dyspnoea.
Pain.
Fatigue.
Appetite loss.
Coughing.
Haemoptysis.
Alarm symptoms
Fever.
Voice.
Facial swelling.
Sense of a growing tumour.
Weight.
Other
Comments field.
*Items selected from the EORTC Item Library,(https://www.eortc.be/itemlibrary/).
Seven of the 12 selected symptoms were available as EORTC items and all graded by a Likert scale.31 For all EORTC items, the recall period was ‘the past week’. Three supplementary symptoms (facial swelling, hoarse voice and sense of a growing tumour) were considered alarm symptoms needing specific attention and were not scored. The wording of the initial versions of these items was produced by four study group members (RBF, CTM, NHH and HS). Current weight and temperature were entered in additional boxes. The items were intended to be used as a screening tool to identify patients with deterioration of specific symptoms requiring clinicians’ attention.
Semistructured interviews
Semistructured interviews were conducted in phase 1 with eight patients while they were filling in the questionnaire in the AmbuFlex system. They were interviewed about the ePRO software and the questionnaire design. Thematic text analysis was used, identifying the following five subthemes: usability, acceptability, inaccurate phrasing, insufficient number of items and lack of response options.
The ePRO system
The usability of the software was high. Patients had very few issues with login and response procedures. Two of them mentioned that they had difficulties using a computer, but both could complete the questionnaire by following the instructions. One patient stated that ‘it’s quick to answer’, and the other stated that ‘it was easy to log in’. However, the interviewer observed that a few clarifications of the login instructions were provided to the patients.
The acceptability of the AmbuFlex software was high. Patients wanted to participate even if it took some effort. In the words of one patient; ‘using a computer is difficult, but if it would help, I would do it’. No patients expressed reluctance using the electronic questionnaire.
Design of the questionnaire
All patients were pleased with the short length of the questionnaire. The majority of the symptoms were selected from the EORTC item bank, and no misconceptions were perceived in relation to these questions.
Several patients found two of the alarm questions referring to the time frame ‘since last time’ to be confusing since these questionnaire had not been answered earlier. The wording of the time frame was then changed to ‘during the past week’. There was one misunderstanding concerning the alarm symptoms. One patient felt that she was unable to evaluate whether she had a sense of a growing tumour and consequently could not answer this question. The issue was solved by adding the response option ‘I don’t know’ to the questionnaire.
Another patient wanted to be able ‘to describe the psychological burden of lung cancer’. We acknowledge that this issue is very important to many patients. However, due to the complexity of this theme and given the purpose of this study, we decided to confine remarks on psychological issues to the comments field. The patients made no further suggestions concerning other relevant symptoms.
Threshold definitions
We then defined the initial symptom severity thresholds for each item by consensus decision. For symptoms graded by severity (symptoms 2–6, box 1), the threshold was cut between ‘none’/‘a little’ and ‘quite a bit’/‘very much’. For self-assessed health, a score ≤3 was the threshold for notifying the department. Clinicians were notified only when a symptom had become worse since the previous week. The authors decided that any presence of haemoptysis should trigger notification.
Conditional branching was used for three of the alarm symptoms. If facial swelling or hoarse voice was reported, the patient was prompted to report whether the symptom had worsened during the past week. If a sensation of fever was reported, the patient was prompted to measure and enter the temperature. The thresholds used were ≥38.2°C for temperature and ≥3 kg for weight loss compared with baseline. Finally, a supplementary comments field was added to enable the patients to report other symptoms.
Phase 2: feasibility test
In phase 2, we approached 29 patients in the outpatient clinic, five of whom were ineligible because they had no internet connection at home. Four patients declined participation, feeling that they could not comply with the intervention. The recruitment rate was 69% (20/29).
The baseline patient characteristics are presented in table 1. The median age was 70.5 years (range 54–86 years). Most of the enrolled patients had prior experience with the internet, although one patient categorised herself as a very inexperienced internet user, and she was, nevertheless, able to complete all the four questionnaires in the test period.
Baseline characteristics
Two patients completed none of the questionnaires and were excluded from analysis. Among participating patients, weekly questionnaires were completed 72/80 (93%) times (table 2). The threshold was exceeded by 55% (37/67), leading to further action by a clinical nurse, and in 30% (20/67) action consisted in a phone call. One programming error regarding the severity of dyspnoea unintentionally led to five false notifications. The time spent responding to alarm notifications, including phone calls, was managed by the nurse in a median of 6 min (range 0.2–30 min) per day.
Compliance and notifications (n=18*)
Phase 3: evaluation of the intervention
Patient questionnaires
The intervention and the study set-up were evaluated in a questionnaire completed by all patients participating in phase 2 (n=18) (table 3).
Evaluation questionnaire (n=18), % (n)
Usability
The patients found it easy to log in and to read and answer the questions. The estimated time spent to complete the weekly questionnaire was less than 10 min for 78% of the patients and less than 15 min for the remaining patients. Two needed help from a relative to complete the questionnaires, but nobody experienced any technical problems. One patient reported that ‘the questionnaire is easy to complete, and it is good to be aware of possible side effects’.
Acceptability
Seventeen (94%) patients felt confident that they would be contacted by the clinic if needed and 16 (89%, 2 non-respondents) were satisfied with the questionnaire. Three (17%, 1 non-respondent) patients felt that they got more worried about their cancer when answering the questionnaire. By contrast, 61% disagreed and two patients stated the opposite view in the comments field. The first patient was ‘satisfied with the additional sense of security’, and the other specified ‘it is reassuring to know that one is being watched if complications occur’.
Relevance
Seven (39%) patients felt better prepared for the dialogue with the doctor and 10 (56%) felt more aware of disease-related symptoms. Fourteen (78%) patients found the questions relevant. One patient expressed a need to report more responses that are detailed, and the other would like to be able to report the functional level and the psychological burden. It was possible to report only a broad picture of the situation in the comments field, but the evaluation made it evident that the EORTC item ‘self-rated quality of life’ could usefully be added to the questionnaire. This item allows patients to report their own assessment of quality of life and, in combination with self-rated overall health, enables the calculation of a longitudinal EORTC global quality of life score.
Nurse interviews
Two clinical nurses involved in the management of the notification list were interviewed about clinical perspectives. Both experienced that the phone consultations were very relevant for the patients, and there were only few examples of unnecessary contacts. They felt that the daily task of monitoring patients was acceptable and meaningful. However, one nurse was concerned that the workload could grow and become a problem if many patients were enrolled in the RCT without additional resource allocation.
The clinical relevance of the threshold limits was also explored. One of the nurses thought that the individual symptoms were of different clinical importance and stated that ‘loss of appetite … and also fatigue … often notifies. And the question is how much we actually use it. We only really do something if the fatigue is disabling … or if the loss of appetite is prolonged’. The other nurse agreed and suggested to change the threshold definitions for these two symptoms to a higher degree of severity.
The threshold definitions for fatigue and appetite loss were therefore raised to a higher severity grade. The final design of the electronic questionnaire and threshold definitions can be found in the online supplementary appendix.
Supplemental material
Discussion
In this study, we designed and tested the feasibility of an electronic questionnaire for weekly internet-based symptom monitoring in patients with metastatic lung cancer. We found that the use of an electronic questionnaire based on an EORTC item score for symptom monitoring in lung cancer is feasible for both patients and healthcare professionals. The results pave the way for testing the set-up in an RCT.
The electronic questionnaire had a high usability in both phases 1 and 3, but the instructions for patients had to be simplified. No technical issues arose, although one symptom-specific programming error was identified and corrected. The high usability was consistent with other studies having used the AmbuFlex software to collect PRO data.24 25
The feasibility test demonstrated a need for the software to be supplemented with a functionality ensuring early identification of non-responders as 2 of 20 feasibility testing participants never started filling in a weekly questionnaire. Moreover, the ePRO questionnaire completion rate dropped towards the end of the test period. This could be due to misunderstandings since patients were asked to fill in the questionnaire the day before the next planned treatment, which coincided with the conclusion of the study period. Some patients had already answered other ePRO questionnaires as a part of routine care in the department. They were thus supposed to answer two questionnaires on the same day, which could explain why some forgot to answer the last questionnaire. To ensure compliance in the upcoming RCT, notifications of non-responding patients will be sent to the nurses as a part of the daily routine. Nurses may then contact non-compliant patients, offering them the guidance they need. By introducing a fixed daily work routine where nurses checked notification lists, we ensured proper response whenever score thresholds were passed. Conclusions about the attrition rate cannot be made due to the short study period.
Patients’ questionnaire responses made nurses call patients a total of 20 times (30%) during the 3-week test period. The algorithm was programmed to notify the clinicians only when symptoms grew worse compared with the previous week. The questionnaire responses given in week 1 triggered more notifications than subsequent responses because the system was programmed to always notify clinicians when a symptom threshold was exceeded and no previous response was available for comparison in the first week. The nurses were instructed to contact patients only if patients’ answers were concerning; however, initially, the nurses acted proactively and made more phone calls than they were trained to make. The interviews with the nurses revealed that they acted with a high sense of responsibility but also had some uncertainty about the procedures. Had the instructions been followed strictly, only half as many (viz. 10) phone calls would have been made in the test period. This underscores the need for clear and concise instructions for staff managing the notifications. Accordingly, the training plans for the nurses were updated with relevant clarifications prior to the RCT.
The amount of time spent on managing notifications and contacting patients was a serious concern raised by the nurses and department managers as well as by collaborators in the subsequent multicentre RCT. However, once it was clarified how much time was actually spent on the daily procedures, the initial concerns among all stakeholders were substantially reduced.
Previous studies have tested other electronic systems for patients with lung cancer. Maguire et al found that mobile technology used for monitoring radiotherapy-related toxicity was feasible and had high acceptability in patients with lung cancer.37 An RCT with 253 patients with lung cancer showed that weekly tele monitoring was feasible and acceptable.38 However, the study that used a phone-based interactive voice response technology failed to improve satisfaction or clinical outcomes. This could be due to the fact that follow-up lasted only 12 weeks and that patients were recruited along different treatment lines. The internet-based ePRO systems may offer higher usability and acceptance among both patients and clinicians than a voice response technology as it could ease interpretation of the reported symptoms.
The mechanisms underlying the effect of intensified ePRO-based monitoring are complex. Denis et al found high compliance in a pilot study of web-based symptom monitoring of patients with lung cancer.39 42 This study also showed a potential for detection of early relapse. The six symptoms included in the pilot was later expanded to 12 symptoms used in the previously mentioned RCT where the ePRO intervention improved overall survival.16 17 The authors suggested that early relapse detection was the main reason for the effect. Other potential mechanisms proposed by Basch et al were early responsiveness to symptom management, supportive care and drug dose modifications improving treatment tolerance.18 Additionally, studies have found that early palliative care could improve both health-related quality of life and survival in lung cancer.7 43
The strength of this study was its multidimensional approach conforming with the MRC guidelines for complex interventions.27 All enrolled patients were real-life patients receiving outpatient treatment, some of whom had limited computer skills and moderate educational attainment. It was important to test the system in a setting where patients used their own internet device so that any technical issues could be addressed before launching the subsequent RCT.
The short study period with a relatively low number of participating patients was a limitation to this study. Since the AmbuFlex PRO system has already been widely tested, we may conclude that use of the AmbuFlex software is feasible in this study set-up.25
Conclusions
A study set-up for a national RCT using weekly symptom monitoring based on EORTC items is feasible.
The following trial, ProWide (Patient-Reported Outcomes used for Weekly Internet-based Detection of progressive disease in lung cancer, Clinicaltrials.gov NTC03608410), is a two-arm, open-labelled, multicentre RCT aiming to determine the effect of ePRO-based symptom monitoring added to standard care. This study will include 492 patients diagnosed with lung cancer in Denmark. The power calculation is based on an anticipated effect on overall survival of half the size of the 1-year overall survival in the study by Denis et al and a compliance rate of 90%.16 The study is open and recruitment is ongoing.
Acknowledgments
The authors thank all the participating patients and the dedicated clinical nurses Vibeke Holst May and Anne Marie Hill from the Department of Oncology, Hospital Unit West Jutland, Herning. They also thank Gry Assam Taarnhøj for valuable feedback on the manuscript.
References
Footnotes
Contributors RBF: Designed the draft of the electronic questionnaire. Enrolled all patients, conducted the interviews, monitored the 3-week test period, evaluated and analysed the data and wrote the initial draft of the paper. NHH: Programmed the software. Major input regarding the logistic planning. Did all the software and threshold programming. Large contribution on the design of the item list for the electronic questionnaire. CTM: Had a large say in the design of the interview guide, the analysis of the interview data and the design of both the electronic questionnaire and the evaluation questionnaire. HP: Major influence on the interpretation of data and the structure of the paper. HS: Idea of the project. Primary supervisor. Wrote the initial study plan and made major contributions to the manuscript. Large impact on the design of the electronic questionnaire.
Funding The study was funded by a grant from The Danish Cancer Society (Grant No. R184-A11805) and by the Department of Oncology, Hospital Unit West Jutland Hospital, Central Denmark Region.
Competing interests None declared.
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
Patient consent for publication Not required.
Ethics approval The study was approved by the Danish Data Protection Agency (2017-41-5251). According to Danish law, no approval was required by the Danish Research Ethics Committee (enquiry 266 received on 7 December 2017). All participants received oral information and signed a written consent prior to enrolment.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement All data relevant to the study are included in the article or uploaded as supplementary information. No additional data available.