Article Text

Download PDFPDF

Primary Palliative Care for Emergency Medicine (PRIM-ER): Protocol for a Pragmatic, Cluster-Randomised, Stepped Wedge Design to Test the Effectiveness of Primary Palliative Care Education, Training and Technical Support for Emergency Medicine
  1. Corita R Grudzen1,2,
  2. Abraham A Brody3,
  3. Frank R Chung1,
  4. Allison M Cuthel1,2,
  5. Devin Mann2,4,
  6. Jordan A McQuilkin1,
  7. Ada L Rubin1,
  8. Jordan Swartz1,
  9. Audrey Tan1,
  10. Keith S Goldfeld2
  11. The PRIM-ER Investigators
    1. 1 Ronald O. Perelman Department of Emergency Medicine, New York University School of Medicine, New York City, New York, USA
    2. 2 Department of Population Health, New York University School of Medicine, New York City, New York, USA
    3. 3 Hartford Institute for Geriatric Nursing, New York University Rory Meyers College of Nursing, New York City, New York, USA
    4. 4 Medical Center Information Technology, NYU Langone Health, New York, New York, USA
    1. Correspondence to Allison M Cuthel; Allison.Cuthel{at}nyulangone.org

    Abstract

    Introduction Emergency departments (ED) care for society’s most vulnerable older adults who present with exacerbations of chronic disease at the end of life, yet the clinical paradigm focuses on treatment of acute pathologies. Palliative care interventions in the ED capture high-risk patients at a time of crisis and can dramatically improve patient-centred outcomes. This study aims to implement and evaluate Primary Palliative Care for Emergency Medicine (PRIM-ER) on ED disposition, healthcare utilisation and survival in older adults with serious illness.

    Methods and analysis This is the protocol for a pragmatic, cluster-randomised stepped wedge trial to test the effectiveness of PRIM-ER in 35 EDs across the USA. The intervention includes four core components: (1) evidence-based, multidisciplinary primary palliative care education; (2) simulation-based workshops; (3) clinical decision support; and (4) audit and feedback. The study is divided into two phases: a pilot phase, to ensure feasibility in two sites, and an implementation and evaluation phase, where we implement the intervention and test the effectiveness in 33 EDs over 2 years. Using Centers for Medicare and Medicaid Services (CMS) data, we will assess the primary outcomes in approximately 300 000 patients: ED disposition to an acute care setting, healthcare utilisation in the 6 months following the ED visit and survival following the index ED visit. Analysis will also determine the site, provider and patient-level characteristics that are associated with variation in impact of PRIM-ER.

    Ethics and dissemination Institutional Review Board approval was obtained at New York University School of Medicine to evaluate the CMS data. Oversight will also be provided by the National Institutes of Health, an Independent Monitoring Committee and a Clinical Informatics Advisory Board. Trial results will be submitted for publication in a peer-reviewed journal.

    Trial registration number NCT03424109; Pre-results

    • adult palliative care
    • health informatics

    This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

    Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • Novel, pragmatic, cluster-randomised trial using stepped wedge design.

    • Aims to shift the clinical practice paradigm for the emergency care of seriously ill older adults.

    • Interventions tested in geographically and contextually diverse settings across 18 health systems nationwide.

    • Potential limitations of the study design include provider cross-contamination and staff turnover.

    Introduction

    Rationale

    The high intensity of end-of-life care in the USA is now considered an epic public health problem. Persons receiving many life-sustaining therapies do not appear to show a benefit of better health or longer life.1 Emergency departments (EDs) care for society’s most vulnerable older adults who present with exacerbations of chronic disease at the end of life, yet the clinical paradigm continues to focus on treatment of acute illness and injury. Palliative care interventions in the ED capture high-risk patients at a time of crisis and can dramatically improve patient-centred outcomes.2 3

    Half of Americans 65 years and older are seen in the ED in the last month of life, and three-quarters visit the ED in the 6 months before their death.4 Emergency care has not fully adapted to the needs or goals of seriously ill patients who prefer to have care delivered at home.5 6 Palliative care teams are now present in over two-thirds of hospitals, as well as 98% of National Cancer Institute-designated cancer centres; however, consultation by palliative care teams is typically only available Monday–Friday during business hours, and palliative care teams are not routinely available to come to the ED when a patient is in crisis.7

    An ED visit is often described as a sentinel event signifying a breakdown in care coordination for older adults.8 9 Since EDs sit at the crossroads of ambulatory and inpatient care, they can, and often play, a pivotal role in balancing the potential harms and benefits of hospitalisation for seriously ill, vulnerable older adults.10–13Hospitalisation for older adults carries significant risks such as iatrogenic complications, functional and cognitive decline and loss of independence, but emergency providers may be unaware of safe alternatives.14–19

    Emergency medicine developed as a specialty to treat the acutely ill and injured, yet EDs increasingly care for older adults with multiple comorbid conditions who present for acute exacerbations of chronic illness. Visits to the ED by older adults are increasing both in frequency and as a proportion of all ED visits. In 2011, adults aged 65 years and older comprised 15% of total ED visits, had the highest severity of illness and represented 44% of all admissions from the ED.20 The number and rate of admissions to the intensive care unit (ICU) by emergency providers have also increased, especially among older adults.21 The proportion of the US population 65 years and older will continue to grow, and EDs will see an increase in both the number of older adults and the complexity of care they are required to provide.22 The ED presents a key decision point at which providers set the subsequent care trajectory, including whether an older adult is hospitalised and to which setting. Emergency physicians can therefore play an integral role in transforming care for older adults through evidence-based models of care delivery that emphasise tradeoffs between potential benefits and harms.13 Until recently, little attention has been paid to the delivery of goal-concordant care in the ED for older adults with serious illness. The default treatment plan is to deliver treatment intensive care that favours life-sustaining therapies, many of which may contradict the wishes of older adult patients.

    Objectives

    This project seeks to implement and test a novel, highly efficient pragmatic intervention to shift the clinical practice paradigm of emergency medicine. Our generalisable approach will build on the effectiveness of specialty palliative care demonstrated in ED patients admitted to the hospital to increase primary palliative care skills and knowledge among emergency providers. This will improve quality of care and decrease healthcare utilisation for millions of older Americans.

    The intervention itself is multifaceted and highly innovative, employing the latest technology and techniques in clinical informatics and innovations in medical education. Building on lessons learnt from prior studies, this intervention has been designed to employ the most effective pedagogical techniques, including the use of simulated patients and families, role-playing, teaching-to-teach exercises and small group learning with constructive feedback from master clinicians.23 24 Despite recognition of the importance of end-of-life care to individuals, families and society, many interventions have failed to capture its complexity. Interventions have focused on single-level solutions without considering more complex, multilevel approaches.

    Our pragmatic, cluster-randomised stepped wedge design will test the effectiveness of Primary Palliative Care for Emergency Medicine (PRIM-ER) in 35 EDs across the US. PRIM-ER is an education, training, and technical support quality improvement (QI) intervention with four core components: (1) evidence-based multidisciplinary primary palliative care education, (2) simulation-based workshops on communication in serious illness, (3) clinical decision support (CDS) and (4) provider audit and feedback. These core components will be implemented in each participating health system to improve the care older adults with serious illness receive in the ED setting. In the pilot phase of the project, we will tailor PRIM-ER to the emergency provider workforce and a more diverse ED context using an agile implementation framework approach at two sites to test feasibility, fidelity and usability. In the implementation and evaluation (IE) phase, we will implement PRIM-ER in a cluster-randomised, stepped wedge design in the remaining 33 EDs and measure the effect of PRIM-ER on: (A) ED disposition to an acute setting; (B) healthcare utilisation in the 6 months following the ED visit; and (C) survival following the index ED visit.

    Proposed study outcomes leverage existing experiences from prior research and empirical evidence, including but not limited to a Center for Medicare and Medicaid Innovation demonstration project whereby researchers addressed the gap in the emergency care of seriously ill older adults to test the traditional consultative model of palliative care in the ED via a randomised controlled trial.3 Findings demonstrated that early ED-based specialty palliative care consultation for patients with advanced cancer improved quality of life at 12 weeks.3 Results from this demonstration project also showed that ED-based primary palliative care reduced the percentage of geriatric ED admissions to the ICU from 2.3% to 0.9%. The overall geriatric admission rate declined by 9%, from 59% to 50%.3

    While the primary goals of palliative care are to reduce burdensome symptoms and improve quality of life, it is often delivered too close to the end of life. For this reason, palliative care is often associated with end-of-life care, or even (wrongly) with hastening death. Despite this perception, there is no evidence that palliative care shortens life, and in fact, preliminary data show that it may lengthen survival in patients with metastatic cancer.25 26 While it is difficult to establish causality, palliative care may decrease psychosocial stress, which is associated with disease progression and shorter survival.27

    This protocol addresses an important gap in the emergency care of older adults at the end of life that is critically important to patients, families, providers and health systems. A substantial number of older adults continue to visit the ED during times of crisis and emergency providers set the trajectory for their care. Until recently, little emphasis has been placed on delivering goal-concordant emergency care. Palliative care in the ED improves quality of life and reduces healthcare use, and our four-pronged intervention will give emergency providers the knowledge and skills to align care plans with patient goals.

    We hypothesise that it will be feasible to test PRIM-ER at two EDs with high level of fidelity and usability and implement the core components at all subsequent enrolled sites. We also hypothesise that older adult visitors with serious, life-limiting illness cared for by providers with primary palliative care skills will be less likely to be admitted to an inpatient setting, more likely to be discharged home or to a palliative care service, and will have higher home health and hospice use, fewer inpatient days and ICU admissions at 6 months and longer survival than those seen prior to implementation. By testing PRIM-ER in a diverse group of EDs in healthcare systems across the USA, we will be positioned to better understand how a system level intervention will impact healthcare utilisation and what contextual factors will impact the degree of change the occurs.

    Methods and analysis

    Trial design

    This is a cluster-randomised QI programme that will implement and test the impact of PRIM-ER in various healthcare settings. We will first develop and pilot test the QI intervention at two sites in the pilot phase and then use a cluster-randomised, stepped wedge design to implement the education, training and technical support in our network of enrolled EDs over a 24-month period (May 2019–June 2021). Randomisation will occur at the ED level and be completed in advance by the New York University biostatistician to determine the order of the stepped-wedge implementation.

    The overall approach involves ongoing asynchronous learning and technical support to bolster skills, conduct interdisciplinary case reviews and reinforce clinical pathways and protocols via provider audit and feedback. Electronic triggers for palliative care will be embedded in the electronic health record (EHR) to identify patients who may benefit from hospice or palliative care services. Using feedback and lessons learnt from the implementation of electronic triggers in the pilot phase sites, in the IE phase, we will collaborate and provide tailored technical assistance to each participating health system to ensure they adopt an EHR standard clinical workflow. Palliative care champions at each site will facilitate attendance at didactic and workshop sessions, disseminate information about local resources for outpatient palliative care, home care, and hospice, and work with the local informatics team to reinforce protocols and implement trigger criteria to identify older adults who may benefit from further needs assessment and follow-up. Physicians and nurses will receive audit and feedback reports to monitor their performance over time, and a learning monitoring system will track participation in educational activities.

    In the pilot phase, members from the palliative care team, emergency nursing, social work/case management, informatics and ED operations from each of the 18 health systems will participate in workgroups to discuss how to best incorporate primary palliative care into the clinical workflow at each site. Pilot testing of PRIM-ER will also occur at two sites to optimise feasibility, fidelity and usability. Emergency providers at each pilot site will be invited to participate in usability testing of the CDS system.

    In the IE phase, we will engage with eligible providers at the 33 sites based on the random sequential order in which the ED implementation occurs. Throughout the duration of the intervention, we will actively engage each health system by providing all ED staff with audit and feedback reports to monitor their performance over time. These reports will be provided routinely during the study period, as well as incorporated into ED-specific continuous QI processes. By providing this continuous and consistent feedback to ED personnel, we aim to encourage continued participation and active engagement throughout the QI interventions entirety.

    In the evaluation portion of the IE phase, we will use the Centers for Medicare and Medicaid Services (CMS) claims of the beneficiaries in our patient cohort to measure outcomes, including ED disposition to an acute care setting, healthcare in the 6 months following the ED visit and survival following the index ED visit as a result of the intervention. The patient cohort will be extracted via CMS’s Research Data Assistance Center (ResDAC) using a two-step process to maximise diversity and minimise intentional or unintentional exclusions based on risk, age, health literacy, demographics or expected adherence. First, we will provide a comprehensive list of facility codes for the 35 participating sites to identify our cohort of patients 66 years and older who made a visit to any of the EDs between 2012 and 2022. Inpatient and ambulatory claims will be used to identify community-dwelling ED visitors 66 years and over who made a visit to any of the EDs from 2 years prior to study initiation until the last implementation day of PRIM-ER. ED claims will be identified via Revenue Center Code values of 0450–0459 (emergency room) or 0981 (professional fees-emergency room) according to ResDAC. We will then examine all inpatient, ambulatory, and carrier claims for the 12 months prior to each older adult’s index ED visit to calculate each beneficiary’s Gagne Index, a score developed to predict 1-year mortality in community-dwelling older adults.28 The Gagne Index has been adapted from the Romano-Charlson Index and the Elixhauser system.29 30 It calculates a score based on the presence or absence of ICD-9s from inpatient and ambulatory claims in the prior year. Beneficiaries with a 1 year mortality of at least 30% (score >6) based on claims from the previous 12 months will be included in the analysis.

    Prior to intervention implementation, all eligible full-time ED providers (including physicians, physician assistants (PAs), nurse practitioners (NPs) and nurses) will be asked to complete a baseline primary palliative care knowledge and skills survey, using a validated instrument. The survey measures self-rated knowledge about terminal and hospice care, provider’s support for hospice practice and philosophy and attitudes towards patient communication regarding prognosis and hospice.31

    Description of the intervention

    The selected components for our four-pronged intervention are patient-centred and have demonstrated efficacy in randomised controlled trials and observational research. Additionally, they are ready for comparative effectiveness research to test them as new models of care in real-world settings. Table 1 shows a description of each intervention component.

    Table 1

    Description of intervention components 

    Emergency medicine and critical care-specific palliative care education programmes are well developed. A protocol has been developed to leverage existing programmes in an effort to standardise the core elements of PRIM-ER across all 35 sites. This includes primary palliative care education and simulation workshops on communication in serious illness. The educational components incorporate asynchronous learning and interdisciplinary case reviews.

    Provider and nurse education

    Education in Palliative and End-of-Life Care for Emergency Medicine (EPEC-EM) is a high-impact, conference-based education dissemination project that uses the train-the-trainer model developed in the original EPEC and has been shown to be effective when adapted for asynchronous learning.32

    To ensure intervention fidelity, in-person sessions (EM Talk) will be conducted by a national group of emergency medicine physicians with expertise in palliative care who have participated in a 6-day course in communications training around serious illness. The primary palliative care education will be taught through both online modules and didactic presentations. The detailed description of the EM Talk curricula can be found in previously cited literature.33 Palliative care champions at each site will facilitate attendance at didactic and workshop sessions, disseminate information about local resources for outpatient palliative care, home care and hospice and work with the local informatics team to reinforce protocols and implement trigger criteria to identify older adults who may benefit from further needs assessment and follow-up. The End-of-Life Nursing Education Consortium (ELNEC) has developed modules for critical care that are also applicable to the ED setting. This standard set of modules will be delivered via an online platform to ensure intervention content fidelity to all 35 intervention sites.34

    CDS and audit and feedback

    CDS will help reinforce clinical pathways and protocols and will enable providers to receive audit and feedback. Interruptive alerts in patients who are likely to benefit from palliative care will be embedded in the EHR to prompt providers to identify and refer patients with palliative care needs.35 Physicians and nurses will receive individualised audit and feedback reports to monitor their performance over time, and a learning monitoring system will track participation in educational activities and encourage cross-fertilisation and learning among sites.36

    Patient and public involvement

    This research was done without patient involvement and patients were not invited to comment on the study design.

    Outcome measures

    We have chosen the least intrusive method possible to report patient outcomes and therefore selected measures that could be tracked using administrative claims data rather than patient-reported outcomes that would require additional respondent burden. We will estimate the baseline rate of acute care admission, healthcare utilisation and survival following the index ED visit using Medicare claims data for visitors to each ED. We will use the master beneficiary summary file, inpatient, outpatient, home health, hospice and vital status files to monitor acute care admission, healthcare utilisation and survival monthly for up to 6 months after the index ED visit to evaluate whether there is a change before and after implementation. Table 2 outlines the primary and secondary dependent outcomes variables.

    Table 2

    Dependent primary and secondary outcome variables

    The primary outcome is the proportion of eligible patients whose disposition is to an acute care setting (inpatient, non-palliative service). The secondary outcomes include healthcare service utilisation in the 6 months following the ED visit and survival times following the ED visit.

    ED disposition will be measured on the index ED visit and will be a dichotomous variable for an acute care admission (yes/no). Acute care admission will be defined as admission to a non-palliative service, and non-acute care admission will include admission to a palliative care service or unit, discharge to home, observation (without a change to inpatient status) or transfer to inpatient or outpatient hospice. Healthcare utilisation will be measured as ED revisits (count), inpatient days (count), home health use (yes/no) and hospice use (yes/no) in the 6 months from the index ED visit. These will be identified through revenue codes in each site’s administrative data. We developed these measures of healthcare utilisation based on the Dartmouth Atlas Decedent Cohort Care Intensity Measures to monitor the quality of end-of-life care in Medicare patients with serious chronic illness.37–39 Survival will be measured in days from the index ED visit to death or 6 months, whichever is sooner.

    Measurement of what will be considered the baseline rate will continue until the month prior to implementation at each site, and postimplementation rates will be considered 1 month after implementation and continue until 6 months after the last site has undergone implementation. To reduce prevalence incidence bias, we will include a roll-in period of 12 months before we begin to include baseline rates of our outcomes in the analysis.40 The index ED visit will be defined as the first ED visit to one of our 35 facilities during which the beneficiary has 12 months of prior inpatient, outpatient or carrier claims consistent with a Gagne Index >6 or >30% mortality. If a beneficiary’s index ED visit occurs during the roll-in period, they will be excluded from the baseline rate calculations if they return to one of our participating EDs and would otherwise meet our inclusion criteria.

    To account for primary palliative care knowledge and skills on patient outcomes in the analysis, we will use survey data that assesses knowledge and attitudes of palliative and end-of-life care collected before PRIM-ER implementation from the emergency providers and nurses at all 35 participating EDs.31

    Setting

    The PRIM-ER QI intervention will be implemented in 35 diverse EDs ranging from academic medical centres to community hospitals across the US using a cluster-randomised stepped wedge study design (figure 1). All sites begin as part of the control condition and are block-randomised to receive the intervention approximately every 3 weeks over a 2-year period. All sites eventually receive the intervention. All four key core intervention components must be completed by each site during their randomised timeframe. Table 3 identifies the 35 PRIM-ER sites enrolled in the QI intervention and their respective geographical locations.

    Figure 1

    Cluster-randomised stepped wedge study design. PRIM-ER, Primary Palliative Care for Emergency Medicine.

    Table 3

    Enrolled sites and geographical location

    The evaluation will include an analysis of CMS claims, which will be performed solely at the Ronald O. Perelman Department of Emergency Medicine at NYU Langone Health.

    Eligibility criteria

    Inclusion criteria

    Eligible ED patients will be included in the CMS data analysis if they are 66 years or older with serious, life-limiting illness who visited any of our EDs during the implementation of PRIM-ER. Patients must demonstrate 1-year mortality of at least 30% (score >6) according to the Gagne Index, a validated instrument used to measure all cause 1-year mortality in community-dwelling older adults, calculated based on their prior 12 months before the index ED visit of CMS claims.41

    All full-time ED providers including physicians, NPs, PAs and nurses will be invited to participate in the intervention and complete the baseline primary palliative care knowledge and skills survey prior to the start of the intervention.

    Exclusion criteria

    ED patients transferred from a nursing home on the index ED visit will be excluded from the analysis since prediction of mortality and disposition of such patients differs from community-dwelling adults. Patients currently receiving hospice at the time of the index ED visit will also be excluded since they have already received services.

    Recruitment and informed consent

    CMS claims of patients 66 years and older with serious, life-limiting illness who made a visit to any of our EDs during the study period will be used to measure outcomes in our patient cohort. We will seek a waiver of Health Insurance Portability and Accountability Act authorisation for ED patients as this study presents no more than minimal risk and cannot be practicably conducted without the waiver given the study’s geographic breadth and sheer number of participants (>300 000 eligible patients). Obtaining informed consent for participation and use of Medicare claims from all patients in this study is not feasible and will interfere with the conduct of this study.

    Data analysis

    Dependent variables

    Table 2 outlines the dependent variables. ED disposition will be measured on the index ED visit and will be a dichotomous variable for an acute care admission (yes/no). Acute care admission will be defined as admission to a non-palliative service, and non-acute care admission will include admission to a palliative care service or unit, discharge to home, observation (without a change to inpatient status) or transfer to inpatient or outpatient hospice.

    Healthcare utilisation will be measured as ED revisits (count), inpatient days (count), home health use (yes/no) and hospice use (yes/no) in the 6 months from the index ED visit. These will be identified through revenue codes in each site’s administrative data. We developed these measures of healthcare utilisation based on the Dartmouth Atlas Decedent Cohort Care Intensity Measures to monitor the quality of end-of-life care in Medicare patients with serious chronic illness.37–39

    Survival will be measured in days from the index ED visit to death or 6 months, whichever is sooner.

    Independent variables

    Table 4 outlines the independent variables. Independent variables will be assessed at the time of site implementation. Healthcare system and provider-level variables will be collected by a member of the New York University research team (programme manager) and via a provider survey at the level of each participating ED. Patient-level variables will be assessed using the CMS ResDAC Master Beneficiary Summary File, Base (A/B/D) Segment.

    Table 4

    Independent outcome variables 

    Methods

    The analytic plan accounts for the nested structure of the data and addresses issues related to missing data, study participation bias and baseline covariate balance. We address each of these in turn. All analyses will be conducted in SAS V.9.4 and R V.3.3.2 (R Foundation for Statistical Computing, Vienna).

    Prior to conducting the outcome analyses, we will compare patients in each ED cluster with respect to patient, provider and facility characteristics. We will assess whether any adjustments will need to be made in the final statistical models based on whether the differences are clinically meaningful. To account for nesting in the data structure (patients nested in hospitals), we will use mixed effect multilevel models to estimate effect sizes. We anticipate two sources of variation

    The primary outcome is the proportion of eligible patients whose disposition is to an acute care setting (inpatient, non-palliative service). The secondary outcomes include healthcare service utilisation in the 6 months following the ED visit and survival times following the ED visit. Exploratory outcomes may include discharge to skilled nursing facility and days spent at home if available in the claims data. The health utilisation outcomes include receipt of ED revisits (count), home health services (yes/no), inpatient days (count), admission to an ICU (yes/no) and admission to hospice (yes/no). The analysis of the effect of PRIM-ER on ED disposition in the context of a stepped-wedge design will be based on a Generalised Linear Mixed Model (GLMM). In particular, to assess the intervention effect, we will use a generalised linear binomial model with random site level effects. The model will be as follows:

    Embedded Image (1)

    where Embedded Image is an indicator variable, and Embedded Image if patient i  at site j  during month Embedded Image  was admitted to an acute care setting from the ED, Embedded Image  otherwise Embedded Image  is the baseline quarter. Embedded Image  is an indicator variable, where Embedded Image if site j  has received the primary palliative care training before month  t , Embedded Image  otherwise. Embedded Image  is a random intercept for site j  with mean 0 and variance Embedded Image ; Embedded Image  is a random slope for site j  with mean 0 and variance Embedded Image . We are making no assumptions about the variance–covariance structure of the random effects; the model will provide an estimate of the covariance between Embedded Image and Embedded Image . In the primary outcome GLMM model, which is a conditional model, all within-site correlation will be accounted for using the random intercept and slopes.

    Model 1 estimates the intervention effect with the within site difference between disposition rates pretraining and post-training, averaging across sites and accounting for possible secular trends that might confound with timing of the training. The primary outcome model makes no seasonality assumptions, since ED disposition decisions are made independently of time of year.

    Embedded Image is the (log-odds of the) average pretraining disposition rate for a site with both random effects equal to 0. Embedded Image  represents the effect of the general time trend of the compliance rate without any training. Embedded Image  represents the intervention that is our primary outcome of interest. We will conduct tests with these null and alternative hypotheses: Embedded Image  vs.Embedded Image . The models will be fit using PROC GLIMMIX in SAS. The hypothesis will be tested using two-sided level of significanceEmbedded Image .

    The analysis of site, provider and patient-level characteristics and the quality of implementation associated with variation in impact after PRIM-ER is based on the RE-AIM Evaluation Framework for Implementation Research (table 5). We consider implementation outcomes separate from the healthcare utilisation and other patient outcomes, since implementation effectiveness is vital to treatment effectiveness.

    Table 5

    Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) evaluation framework for implementation research by intervention component

    Models for the secondary analysis will all be analogous to the primary analysis. The secondary analysis will be based on extending models described in the analysis plan for primary outcome analysis to include independent variables related to the characteristics of interest. In particular, the models will be 

    Embedded Image (2)

    where all the variables and parameters are the same as in model 1, except for Embedded Image , Embedded Image and Embedded Image . Embedded Image  is a vector that represents all of the site, provider and patient characteristics that we will be exploring. Embedded Image  is a vector of parameters that represent the effects of these characteristics on the outcome prior to the intervention. Embedded Image  is a vector of parameters that represent the effect modification related to each of these characteristics. We will explore models that include site-specific treatment effects rather than site-specific time effects to see which best describes the observed patterns. These secondary models will provide insight into ways that the effect of the training might vary across institutions and individuals based on varying characteristics.

    Ethics and dissemination

    Ethical and safety considerations

    The trial is registered at ClinicalTrials.gov. To minimise research-associated risk and protect the confidentiality of participant data, all investigators and staff involved in this project have completed extensive courses and passed certifying exams on the protection of human subjects in research through Collaborative Institutional Training Initiative training and Health Insurance Portability and Accountability Act certification.

    Dissemination plan

    In addition to the traditional form of dissemination via presentations at academic national meetings and peer-review publications, we will actively engage other patient advocacy, specialty group and healthcare system partners in disseminating the results. This will include our healthcare system partners that are collaborating on the project, patient and family stakeholder groups and specialty societies. We have purposefully designed our primary palliative care intervention to facilitate sustainability and scalability in real-world settings and will prioritise these activities in the last 6 months of the project.

    References

    1. 1.
    2. 2.
    3. 3.
    4. 4.
    5. 5.
    6. 6.
    7. 7.
    8. 8.
    9. 9.
    10. 10.
    11. 11.
    12. 12.
    13. 13.
    14. 14.
    15. 15.
    16. 16.
    17. 17.
    18. 18.
    19. 19.
    20. 20.
    21. 21.
    22. 22.
    23. 23.
    24. 24.
    25. 25.
    26. 26.
    27. 27.
    28. 28.
    29. 29.
    30. 30.
    31. 31.
    32. 32.
    33. 33.
    34. 34.
    35. 35.
    36. 36.
    37. 37.
    38. 38.
    39. 39.
    40. 40.
    41. 41.

    Footnotes

    • Contributors CRG: conception and design of study and drafting and critically revising the manuscript. AAB and AT: design of study and critically revising the manuscript. FRC, AMC, JAM, ALR and JS: drafting and critically revising the manuscript. DM: design of study, critically revising the manuscript. KSG: conception and design of study, drafting the manuscript ‘The PRIM-ER investigators’: development of the protocol and manuscript review. All Authors approved of the final version of the manuscript.

    • Funding Research reported in this publication was supported within the National Institutes of Health (NIH) Health Care Systems Research Collaboratory by cooperative agreement UG3AT009844 from the National Center for Complementary and Integrative Health, and the National Institute on Aging. This work also received logistical and technical support from the NIH Collaboratory Coordinating Center through cooperative agreement U24AT009676. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

    • Disclaimer All content in this report, including findings and conclusions, are solely the responsibility of the authors and do not necessarily represent the official views of the National Center for Complementary and Integrative Health (NCCIH), National Institute on Aging (NIA), or the National Institutes of Health (NIH).

    • Competing interests None declared.

    • Ethics approval Institutional Review Board approval has been obtained from New York University School of Medicine . NYU School of Medicine Office of Science and Research i18-00607.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Collaborators Benjamin Abella, Gallane Abraham, Terri Arlitsch Cridge, Robert Asselta, Brittany Ballaraon, Aveh Bastani, Danielle Bastien, Andrew Black, Shannon Bledsoe, Deidre Bolden, David Bolden, Tracy Bollman, Abby Bonato, Marie-Carmelle Elie, Jeffrey Caterino, Laraine Chiu, Joshua Chodosh, Molly Christenson, Julie Cooper, Ashley Deutsch, Ahmed Elsayem, Natasha Ergorova, Karen Evelyn, Fernanda Bellolio, Charles Feronti, Jessica Fleischer-Black, Natalie Frontera, Tim Fuller, Ellin Gafford, Michelle Gamboa, Maureen Gang, Shelby Garduno, Michael Gartner, Rebecca Goett, Emily Grange, Roshni Guerry, Jennifer Harris, Daniel Herbert Cohen, Christopher Hirtz, Judith Hochman, Leora Horwitz, Eric Isaacs, Mindi Johnson, Deborah Johnson, Karen Jubanyik, Anne Kim, Gail Knight, Regina Kulacz, Sangeeta Lamba, Michelle Lin, Heather Linsata, Meghan Liroff, Elizabeth Long, Caitlin Loprinzi-Brauer, Anne Navarro, Troy Madsen, Kaysea Mclay, Joseph Miller, Catharin Morello, John O’Neill, Ronny Otero, Kei Ouchi, Lynne Richardson, Christopher Richardson, Todd Rowe, Elizabeth Schoenfeld, Melissa Shaw, Donna Shelley, Ashley Shreves, Jennifer Siller, Elizabeth Singer, Lauren Southerland, Laura Stuecher, Robert Swor, Kelly Szabo, Andrea Torre, Karen Toulson, Andrea Troxel, Rachel Urosek, Julie Uspal, Tiny Varghese, Arvind Venkat, Travis Wood, and Erin Zimny

    • Patient consent for publication Not required.