Overview

We are studying the combination of clinician training and patient videos via a pragmatic SW-CRT and analysing electronic health records (EHRs) for ACP outcomes for patients aged 65 years and older. Using small subsamples of patients, we will also assess patient-centred outcomes using surveys and video declarations in which patients discuss their values and preferences in their own words on video (figure 1). We used the Standard Protocol Items: Recommendations for Interventional Trials reporting guidelines for this manuscript.45

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Advance Care Planning: Promoting Effective and Aligned Communication in the Elderly model.

Study timeline

The ACP-PEACE study has two phases, a characteristic of the funding mechanism. The UG3 phase (year 1) of the study focused on developing and refining the intervention and data acquisition. In this phase, we established our organisational structure, developed the processes and infrastructure needed to conduct the trial and pilot-tested the study intervention in three clinics, one from each participating health system. During the UH3 phase (years 2–5), we planned to introduce the intervention to the 36 remaining oncology clinics in 6-month waves: two clinics per system for a total of six clinics every 6 months (figure 2).

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

Stepped-wedge recruitment and implementation yearly timeline (repeated each year). EMR, Electronic Medical Record.

Sites and randomisation

We will draw participants from disease-based oncology clinics from three unique systems: Duke Health (North Carolina), Mayo Clinic (Minnesota) and Northwell Health (New York). These sites are geographically, socioeconomically and culturally distinct. Each participating clinic has more than one practising oncologist, and to be eligible for randomisation, at least 30% of the patient population must be aged 65 years or older.

For the UH3 phase, we have identified a total of 36 oncology clinics (12 per site) as candidate clinics based on recent data from each system. The pilot clinics that participated in the UG3 phase tested the intervention process and will not be included in the final analysis. In the UH3 phase, we will use stepped-wedge cluster randomisation with the clinic as the unit of randomisation. With the clinic as the unit of randomisation, we avoid the contamination that can occur when randomising individuals within each clinic. The sequence of randomisation was generated prior to initiation of the trial via random number generator. Every 6 months after the baseline, two clinics from each system will be randomised to the intervention (figure 3A).

• Download figure
• Open in new tab
• Download powerpoint

Figure 3

(A) Original stepped-wedge cluster randomisation scheme within each health care system. (B) Modified stepped-wedge cluster randomisation scheme within each health care system.

During the original Step 2, COVID-19 spread throughout the country interrupting the stepped-wedge design in two key respects: (1) the team was unable to conduct the in-person trainings for the step 2 intervention clinics; and (2) ACP activities are likely to increase during this period due to a response to the pandemic, irrespective of the study. On the recommendation of the NIH Collaboratory Statistics Core, we modified the original design to ‘restart’ the trial for the remaining 30 clinics using the original step 2 as the new baseline. The training of the remaining 30 control clinics will be over four steps to keep the trial completion on the same overall timeline (figure 3B).

Population

We will evaluate the outcomes for patients aged 65 years or older with advanced cancer across all 36 clinics. As the intervention will be implemented clinic wide, rather than targeted to specific study patients, all intervention clinic patients can receive the intervention. We will analyse data for patients with advanced cancer aged 65 years or older; patients’ data will be counted towards control or intervention based on the allocation of each clinic at the end of each period of the stepped-wedge design. Therefore, a given patient could contribute data during more than one period and could contribute data to both control and intervention periods.

During the UH3 years, research assistants at each site will conduct in-person surveys with 450 randomly selected patients (150 per site) for our secondary exploratory patient-centred outcomes. Patients selected for surveys will be distributed evenly among clinics within each system and will include an equal number of surveys of patients from clinics in the control and intervention phases. Patients will be surveyed only once as patients surveyed in the control phase will be excluded from completing the later intervention survey. Additionally, from among this group of 450 surveyed patients, a subgroup of 240 will be randomly selected and asked to conduct a video declaration activity. All patients selected for surveys or videos will be excluded from the primary study population to avoid bias rendered from additional contact with the study team.

Intervention design, implementation and adherence monitoring

The comprehensive ACP intervention combines VitalTalk and ACP Decisions, two evidence-based interventions previously used separately, to create an innovative dual approach to improving ACP. These interventions are complementary, as one targets improvement of clinicians’ skills and the other prepares patients for shared decision making. VitalTalk is the most widely disseminated teaching method for effective communication skills training based on practice and feedback on one’s own communication skills. Supported by numerous previous studies,46–52 VitalTalk leverages didactics, demonstration and small group sessions using role play with trained actors portraying patients through which clinicians learn effective delivery of serious news, prognosis discussion and early and late goals-of-care conversations. For this study, the VitalTalk course will be a half-day session that teaches a framework for late goals-of-care discussions, including skills around delivery of serious news, responding to emotion, assessing prognostic awareness, identifying what is most important to patients and making recommendations.

The ACP Decisions programme uses short video decision aids to address the most common issues facing older patients with serious illness. Videos in over 25 languages can be prescribed to patients and caregivers and are easily accessed in a mobile app or through a web-based platform. The ACP Decisions videos have been shown to increase knowledge, decision certainty and the stability of preferences over time and to better inform the way that patients choose healthcare interventions towards the end of life.53–72 The video collection includes certified video decision aids,73 regarding ACP, advance directives, healthcare agents, goals of care, cardiopulmonary resuscitation and hospice, that have been studied in a statewide implementation showing greater patient-aligned medical care.72

We will provide in-person training every 6 months at each new clinic added to the intervention period of the trial. The comprehensive ACP training programme uses the VitalTalk methodology and infrastructure and the ACP Decisions Program tools to instruct clinicians and staff on how to: (1) more effectively communicate with patients with cancer, (2) have ACP conversations with patients, (3) introduce the videos to patients and families, (4) use the videos as an adjunct to ACP counselling by clinicians, (5) select the appropriate video(s) according to patient needs and (6) use the application or electronic platform for viewing videos. The combined programme will involve a half-day face-to-face joint VitalTalk and ACP Decisions training. Any staff member affiliated with the selected facilities will be eligible to participate in training. As staff turnover among the sites is expected, training will be made available on an ongoing basis throughout the trial.

Immediately following the initial training at each site, we will deploy the remainder of the intervention infrastructure. The ACP Decisions videos will be programmed into desktop devices, tablets and password-protected electronic platforms of each health system’s intranet. When clinics initiate the intervention, they will implement the videos with all patients with flexibility as to which providers (physician, nurse and social worker) introduce the videos and exactly which videos are used to meet their patients’ clinical needs. Additionally, the in-person clinician training will be supplemented with emails, pocket cards, offers of coaching and online educational videos. The study team will facilitate dissemination of implementation successes and challenges via a learning network by conducting 1-hour webinars at each of the practices randomised to the intervention every other month to discuss quality improvement activities relating to the study. The intensity of the VitalTalk training implementation will be assessed as the proportion of eligible staff trained, including new staff joining the practice over the implementation period. The intensity of implementation of the ACP Decisions videos will be assessed as the ratio of the number of videos viewed using the site-specific access codes captured at the ACP Decisions website to the number of eligible patients at each site for each 6-month intervention period. Fidelity to the video component of the intervention will be monitored by tracking of video use (which videos are used at each clinic, playthrough rate and frequency). Feedback on video viewing will be shared with each site at the end of each 6-month implementation phase. Last, we aim to evaluate the impact of the study with a novel video declaration process, allowing patients to state their values and preferences in their own terms, which is described in detail in the online supplementary appendix.59

Control condition

Clinics in the control phase will use whatever ACP procedures already exist in place at their respective system. Although current ACP improvement initiatives may be present and vary from clinic to clinic, this heterogeneity reflects the current dynamic state of ‘usual’ care and is therefore appropriate in this pragmatic trial.43

Outcomes

The outcomes of the ACP-PEACE trial can be divided into three main categories: patient level, clinician level and system level. Our primary outcome is the proportion of eligible patients with ACP documentation completed in the EHR. Presence of completed ACP documentation will be defined via one or both of the following two means: (1) structured EHR data: scanned forms including advance directives, living wills or Physician’s Orders for Life Sustaining Treatment (or state-specific equivalent) and code status orders indicating do not resuscitate status (or similar site-specific codes for limitations on treatments) and (2) Natural Language Processing (NLP) extraction (described below in detail): clinical documentation that will include goals-of-care discussion, ACP, hospice discussion, discussion of palliative care or limitations on code status. From the EHR or the local tumour registry, we are also determining demographic covariates and baseline data. Secondary outcomes include resuscitation preferences, palliative care consultations, death, hospice use/utilisation at the end of life and final cancer-directed therapy.

We are deriving patient-centred outcomes from the patient survey and video declarations. The surveys measure our patient-centred secondary outcomes such as patient confidence that their future medical care will match their values, satisfaction with their clinicians’ communication,74 75 satisfaction with their medical decision76 and regret about their medical decision (online supplementary appendix).77 78 Finally, for each of the 450 surveyed patients who die during the study period, we will extract data, via a chart abstraction tool, regarding ACP preferences and care received in the final 3 months of life to explore whether patients receive goal-concordant care.

We are also collecting a small set of clinician data points. Participating clinicians provide information on demographics, clinical experience, prior communication training and socioemotional orientation.52 table 1 lists each data element, with its purpose, proposed source and the target population from whom we need the data for successful completion of the study. System-level data measurement will include measurement of the training and video use as described above as well as exploratory analysis of coaching calls and implementation activities.

Data sources, data elements and linkage

Baseline (ie, preintervention) data for all randomised clinics will include a 6-month period prior to date of intervention delivery. Patients will be identified as having advanced cancer from each site’s tumour registry and/or from clinical ICD codes, which have been studied in some cancers and have demonstrated strong specificities.79 While these methods have lower sensitivity, they capture enough patients with advanced cancer with high specificity for outcome assessment without systematic bias towards intervention or control periods. Demographic information and baseline characteristics relevant to general oncology will be collected from the EHR. Our primary and secondary outcomes will be abstracted from the local EHRs and tumour registries as detailed below in outcomes.

We will also use NLP, a form of computer-assisted abstraction, to detect our primary and secondary outcomes. Our NLP software, ClinicalRegex, identifies predefined keywords or phrases within clinical notes, considering varieties in language and punctuation.80–82 ClinicalRegex also allows for rapid semiautomated review that ensures that keywords have not been taken out of context. For each NLP process (ie, goals-of-care discussion), we have built a keyword library that identifies relevant documentation within clinical notes. Each keyword library was refined and validated by manual review of clinical notes in local EHRs. With NLP, we will collect additional data on ACP documentation, goals-of-care discussions, limitation of life-sustaining treatment, palliative care consultation and hospice assessment. Exploratory patient-centred outcomes and clinician outcomes will be derived from surveys collected through REDCap.83 84

Data use agreements between all systems are on file, and each site maintains and adheres to the process and procedures for the protection of human subjects and protected health information (PHI) for their covered entities. Only the minimum amount of necessary PHI will be collected from participants. Health Insurance Portability and Accountability Act-compliant and password-protected servers will be used to store all collected data. Individual password-protected files will separate participant identifiers and a third password-protected linking file will be maintained. This linking file has restricted access and uses a logging feature that identifies each user and instance of use. All data will be transmitted via secure methods approved by the respective institutions to the Dana-Farber Cancer Institute (DFCI) for data management and to Boston Medical Center for qualitative analysis, and trial investigators will have access to the final data set, and it will be made available on reasonable request. The EHR data will undergo a review-adjudication process whereby DFCI data staff and key, unblinded investigators review the raw data for each variable to identify out of range or unexpected values; a summary is sent to each site, and conference calls are conducted with relevant investigators and programmers to adjudicate any issues. We will also validate a randomly selected subset of data, verifying key demographic characteristics and patient selection criteria against medical records. The EHR data are then uploaded to a REDCap database.

Masking

Blinding for this trial occurs at multiple levels. Research Coinvestigators at each site will be aware of the randomisation order as well as which clinics receive the intervention and when. The investigators leading the trainings will likewise be aware of which clinics receive the intervention. Similarly, due to their roles in working with the data and generating video adherence reports for the intervention clinics, certain members of the implementation and data management teams will be unblinded to clinic assignments and outcomes. All other staff will remain blinded to randomisation scheme and outcomes.

Statistical analysis

Our primary analytical approach uses an intention-to-treat analysis, with no special allowance for non-compliance or non-adherence. With the stepped-wedge design, the outcomes during the intervention (exposed) periods will be compared with outcomes during the control (unexposed) periods. We will conduct two analyses based on the observations included in the analysis: (1) open cohort with repeated measures design: individuals may leave and others may join during the study and the same individuals are allowed to appear in multiple periods, (2) repeated cross-sectional design: subjects will only be included in the period when they first enter the study. Characteristics of the individuals and clusters will be summarised by exposure status.85 We will use generalised linear mixed models to compare outcomes between intervention and control periods. The basic model is depicted in this equation:

where denotes the response from individual k at time j from cluster i. To account for clustering within each clinic, the model includes a random effect for cluster i. Under the stepped-wedge design, calendar time is associated with the exposure to the intervention. We will include a fixed effect to adjust for potential confounding factors from calendar time. In the case that time effect might not be the same for all clusters, we will change the term from a fixed effect to a random effect β_ij . To account for repeated measures from the same subject from the first analysis, we will include a random subject effect . The term represents the treatment indicator in cluster i at time j with θ representing the overall treatment effect. If there is evidence of treatment effect heterogeneity, we will either change the fixed effect θ to a random effect θ_i or change the fixed effect θ to θ_(s) , which allows different treatment effects for different strata. We will also explore heterogeneity of intervention effect for different subgroups by adding an interaction term between treatment status and subgroup to the models. These groups include site, sex as a biological variable, race/ethnicity (white vs non-white) and different types of cancer diagnoses.

If necessary, we will include additional terms δ ₁Z_ijk and δ ₂W_ij to the model, where Z and W represent vectors of patient and cluster characteristics. The index j in the Z matrices allows us to include the time-varying covariates, which correspond to any patient characteristics that could change over time. We will use a logit link ( g ) for the binary outcomes that include our primary outcome of ACP documentation and our secondary outcomes of resuscitation preference and hospice use. Other outcomes such as number of palliative care consults and utilisations are considered as Poisson variables and modelled with a log link.

In adjustment for the COVID-19 19 pandemic, the analysis plan will remain the same for the data collected from the 30 clinics randomised to intervention after the original step 2 (figure 3B). The data collected from the six clinics that received intervention during step 1 will allow us to examine the ACP Programme intervention effect prior to COVID-19 by comparing the ACP rates prior to the intervention (original baseline) and after the intervention (original step 1). Additionally, ACP rates from original baseline, step 1 and step 2 from the 30 clinics randomised to intervention after the original step 2 will be used to estimate the ‘COVID-19 effect’ on ACP.

We also have patient-centred secondary outcomes from survey results for analysis. Since patients will be surveyed in the step immediately before and after the intervention is initiated within each clinic, the number of intervention and control patients will be approximately equal at each time point. We will use linear mixed models that treat time (ie, before or after intervention) as a fixed effect and clinic as a random effect to account for clustering of patients within clinics.

Finally, we will examine care delivery alignment with expressed goals from a subset of deceased patients of those 450 surveyed. Using a chart abstraction tool, two blinded expert investigators will judge whether patients received care concordant with their documented wishes. Coders will make determinations and discuss disagreements; final judgments will be determined by consensus. For qualitative coding evaluation, we will summarise the extent of agreement using kappa statistics and will compare results between those who died before and after receiving interventions.

Statistical power and sample size requirements

We used the Hooper et al 86 87 approach to conduct the power analysis. We originally estimated close to 5000 patients from 36 oncology practices are eligible for the study at each time point and approximately 20% are new patients at each step. With seven time points (baseline plus six steps), we anticipated a total of 11 000 unique patients will be included in the study. With the modified design, we estimate 4160 patients from 30 oncology practices are eligible for the study at each time point, and a total of 7500 unique patients will be included in the stepped-wedge design analysis. With each clinic contributing an average of 139 patients at each step from the cohort design, the design effect due to clustering is 7.9 assuming an intracluster correlation of 0.05, and the design effect due to repeated assessment is 0.12 assuming the cluster autocorrelation coefficient is 0.7 and the individual autocorrelation coefficient is 0.9. These estimates correspond to an effective sample size (ie, sample size required for individual randomization) of 4405. For the repeated cross-sectional design, each clinic will contribute an average of 23 new patients at each step, and the effective sample size is 1628 with the same assumptions on intracluster correlation and cluster autocorrelation. Preliminary estimates indicate the rate for ACP documentation (the primary outcome) ranges from 15% to 30% for the control periods, which requires an effective sample size of 500–954 for detecting a 10% absolute increase in our primary outcome with a two-sided significance level of 0.05. Therefore, the study will have more than 90% power for either analysis using the open cohort with repeated measures design or the repeated cross-sectional design.

For the patient-centred survey outcomes, 225 patients will be surveyed during control periods and 225 will be surveyed during intervention periods. Assuming an intraclass correlation coefficient of 0.05 and an average cluster size of 12.5, the effective sample size is approximately 286. A sample of this size allows for 90% power to detect a small to moderate effect size (Cohen’s d) of 0.39% and 99% power to detect a moderate effect size of 0.5 for outcomes such as patient confidence, decisional satisfaction and regret.