Article Text

Download PDFPDF

Outpatient palliative care referral system (PCRS) for patients with advanced cancer: an impact evaluation protocol
  1. Cinzia Brunelli1,
  2. Ernesto Zecca1,
  3. Alessandra Pigni1,
  4. Paola Bracchi1,
  5. Mariangela Caputo1,
  6. Silvia Lo Dico1,
  7. Viviana Fusetti1,2,
  8. Antonino Tallarita1,
  9. Cristiana Bergamini3,
  10. Marta Brambilla3,
  11. Alessandra Raimondi3,
  12. Monica Niger3,
  13. Salvatore Provenzano3,
  14. Pierangela Sepe3,
  15. Sara Alfieri4,
  16. Gabriele Tinè5,
  17. Filippo De Braud3,6,
  18. Augusto Tommaso Caraceni1,6
  1. 1Palliative Care, Pain Therapy and Rehabilitation Unit, Fondazione IRCCS Istituto Nazionale dei Tumori, Milano, Italy
  2. 2Università degli Studi di Roma Tor Vergata, Roma, Lazio, Italy
  3. 3Medical Oncology Department, Fondazione IRCCS Istituto Nazionale dei Tumori, Milano, Italy
  4. 4Clinical Psychology Unit, Fondazione IRCCS Istituto Nazionale dei Tumori, Milano, Italy
  5. 5Unit of Clinical Epidemiology and Trial Organisation, Fondazione IRCCS Istituto Nazionale dei Tumori, Milano, Italy
  6. 6Università degli Studi di Milano, Milano, Italy
  1. Correspondence to Dr. Ernesto Zecca; ernesto.zecca{at}


Introduction Early palliative care (PC) in the clinical pathway of advanced cancer patients improves symptom control, quality of life and has a positive impact on overall quality of care. At present, standardised criteria for appropriate referral for early PC in oncology care are lacking. The aim of this project is to develop a set of standardised referral criteria and procedures to implement appropriate early PC for advanced cancer patients (the palliative care referral system, PCRS) and test its impact on user perception of quality of care received, on patient quality of life and on the use of healthcare resources.

Setting Selected oncology clinics and PC outpatient clinic.

Methods and analysis A scoping literature review and an expert consultation through a nominal group technique will be used to revise existing referral tools and to develop a new one, the PCRS. 25 patients will be enrolled in a pilot study to assess feasibility of the implementation of PCRS; 10 interviews with patients and healthcare professionals will be carried out to evaluate applicability.

A pretest–post-test quasiexperimental study involving 150 patients before implementation of the PCRS and 150 patients after implementation will be carried out.

Patient satisfaction with care received, quality of life and use of resources, and caregiver satisfaction with care will also be assessed to explore the impact of the intervention.

Ethics and dissemination Ethical approval for the study has been granted by the Institutional Review board of the Fondazione IRCCS Istituto Nazionale Tumori; approval reference INT201/19.

Results will be disseminated through open access publications and through scientific communication presented at national and international conferences.

Trial registration number NCT04936568.

  • Adult palliative care
  • Adult oncology

Data availability statement

No data are available.

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

  • The development of the palliative care referral system (PCRS) is based on published evidence and consensus among oncology and palliative care professionals.

  • Nominal group technique is particularly adequate to reach clinical practice consensus for subsequent implementation.

  • The preimplementation and postimplementation study can provide evidence on the feasibility of the PCRS and its clinical impact.

  • Limitations include the single-centre and non-randomised study design.


Palliative care (PC) is aimed at reducing or preventing suffering and improving quality of life for patients affected by incurable advanced illnesses and their caregivers. Traditionally, PC has been limited to the terminal phase of illness with an unclear impact on overall disease trajectory. In recent years, the term ‘early palliative care’ has been coined to describe an anticipated approach to PC in the care pathway of advanced diseases. In the case of advanced cancer, this means that PC can already start along with treatments such as chemotherapy, radiotherapy and other disease modifying interventions, planned for the management of advanced disease.1

According to the latest American Society of Clinical Oncology consensus on integration of PC into standard oncology care, PC is defined early when administered within 8 weeks from the diagnosis of advanced cancer, a definition that is too generic to be operationally implemented.2 As shown in systematic literature reviews,1 randomised controlled trials demonstrate that the introduction of outpatient PC from the time of diagnosis of advanced cancer is associated with benefit on several clinical and care dimensions.1

On the other hand, early PC is associated with less aggressive cancer treatment at the end of life, such as reduced use of chemotherapy in the last weeks of life and reduced access to intensive care units and emergency rooms (ER).3

Despite the evidence on the benefit of early PC, there is no standard definition of how oncologists should decide to refer patients to the PC specialist once advanced or metastatic disease is diagnosed. To optimise the potential impact of EPC on the overall care pathway, it would be important to combine the capacity of PC teams to participate timely in the shared decision-making process (including patient, family and attending oncologist), and, at the same time, the capacity of the multidisciplinary oncology team to integrate the advice of the PC specialist.4

Without a suitable patient selection process for PC referral, outpatient PC services may be overwhelmed by excessive and perhaps inappropriate requests or, alternatively, resources could be underutilised if referrals do not occur. At present, the volume and timing of referral to outpatient PC for cancer patients vary widely among services. This can be in part explained by the lack of standardised referral criteria for outpatient PC, coupled with variable oncologists’ attitudes and beliefs about PC and differences in models and availability of PC services.4

It would be desirable to provide a personalised care plan for each patient, taking into account different diseases trajectories and identifying the timing and the ways in which patients can be referred to PC. Criteria establishing the right moment for the right patient for referral including are needed to personalise care pathways and optimise resource allocations to improve care outcomes.

This project has a twofold aim:

  1. To develop and study the feasibility of a standardised palliative care referral system (PCRS)

    for outpatient PC in advanced cancer patients.

  2. To evaluate the impact of routine application of the PCRS in a population of patients cared for in a comprehensive cancer centre.

Methods and analysis

The project has been organised into two main sequential phases and second-level steps as reported in figure 1. The study intervention will be applied within several selected oncology clinics covering both frequent and rare cancers (lung, gastrointestinal, genitourinary, head and neck and sarcoma) and the PC outpatient clinic. In our PC clinic, patients regularly encounter PC specialists and are assessed by nurses for symptoms and psychosocial dimension using validated self-reported questionnaires.5 Psychological consult is also available as needed. For each oncology clinic, one oncologist will participate directly in the study procedures.

Figure 1

Project scheme. NGT, nominal group technique; PCRS, palliative care referral system.

Phase I: development and pilot testing of the PCRS

Criteria used for PC referral in advanced cancer patients will be identified through a scoping review of the literature on existing PC needs screening tools and methods (table 1). The choice of the criteria that will constitute the PCRS will be based on their clinical relevance and prognostic value; existing criteria will be modified if needed. A first draft of the PCRS tool will be developed and its appropriateness and feasibility in routine clinical practice will be evaluated using nominal group technique (NGT)6 carried out with different healthcare providers (oncologists, PC specialists, nurses and psychologist) with extensive experience in oncology and PC. NGT is a qualitative research methodology aimed at verifying the possible convergence of a group of experts on an idea, judgement or proposal. NGT consists of six stages7 : (1) presentation of the nominal question; (2) individual generation of ideas or voting; (3) round-robin feedback from participants to record each idea/voting; (4) group discussion of each idea/voting in turn for clarification and evaluation; (5) individual voting on priority ideas with the group decision derived through rank-ordering or rating and (6) sharing of results, further discussion and revoting. Based on the NGT results, the PCRs will be adjusted/modified as needed and standard implementation procedures will be developed.

Table 1

Data collection during study period

Then, a pilot cross-sectional feasibility study on the application of the PCRS will be carried out on five consecutive patients in each of the five outpatient clinics that will be involved in the phase II of the project. Feasibility assessment will include: time needed to screen the patient, number of patients for which the screening was complete and number of missing data for each criterion. Acceptability of PCRS by patients and by healthcare professionals, as well as integration of the assessment procedure with routine clinical practice will also be evaluated through debriefing interviews with five patients and five healthcare professionals. A final revision of the PCRS and of its implementation procedure by the study group will follow the pilot testing.

Phase II: PCRS implementation and impact evaluation

The potential impact of the PCRS will be explored using a quasiexperimental study design measuring study endpoints before and after PCRS implementation (figure 1).

Study design and patient population

A longitudinal pretest–post-test design will be carried out. Two different cohorts of advanced cancer patients will be enrolled before (pretest) and after (post-test) the introduction of the PCRS in outpatient clinics. Patient inclusion criteria are: age>18 years; diagnosis of inoperable locally advanced and/or metastatic cancer. Exclusion criteria: eligibility to anticancer treatment with curative intent; patients already enrolled in a PC programme and cognitive impairment.

PCRS implementation

For implementing the PCRS, patient clinical assessment will be performed initially by a nurse, collecting self-reported questionnaires and partially completing the PCRS. This evaluation will be entered in the electronic medical records and then integrated and validated by the oncologist during the visit. The oncologist will finally decide to refer or not the patient to PC using the PCRS predefined criteria and his or her clinical judgement.

Study outcomes and assessment methods

The following outcomes will be evaluated: patient’s satisfaction with care (main outcome), patient’s quality of life, caregiver’s satisfaction with care, use of healthcare resources and end of life care quality indicators.

The FAMCARE-P13 will be used to assess patient satisfaction with care in the present study. It is a 13-item self-report questionnaire, developed to be used with advanced cancer patients.8 Its items are rated from 1 (very dissatisfied) to 5 (very satisfied) producing a single satisfaction score ranging from 13 to 65.

The European Organisation for Research and Treatment of Cancer (EORTC) Quality of Life Questionnaire Core 15 for Palliative Care (QLQ-C15-PAL)9 will be used to evaluate patients’ quality of life. QLQ-C15-PAL is an abbreviated 15-item version of EORTC-QLQC30 specifically developed for use in a PC setting. It includes two multi-item functional scales (physical and emotional functioning), two multi-item symptom scales (fatigue and pain), five single-item symptom scales (nausea/vomiting, dyspnoea, insomnia, appetite loss and constipation) and a question regarding overall QoL (global health status). Items are rated on a Likert scale from 1 (not at all) to 4 (very much) with the exception of global health status, which is rated from 1 (very poor) to 7 (excellent).

Family satisfaction with care will be assessed with the FAMCARE scale.10 11 It is a self-assessment satisfaction composed of 20 items rated according to a 5-point Likert scale from 1 (very satisfied) to 5 (very dissatisfied). The items are grouped into four subscales: physical patient care, information giving, availability of care and Psychosocial care subscale.

The FAMCARE scale can be administered to family members, while a patient is receiving PC or at some point after a patient’s death.

All tools applied are validated in Italian language but the FAMCARE P-13, which will be culturally adapted from English using ‘forward–backward’ translation method12 as part of the present project; basic psychometric properties will be assessed as well.

Use of healthcare resources during study follow-up will be monthly assessed by a dedicated research nurse with an ad-hoc developed form reporting: access to a PC service (regular outpatient PC visits, home care and hospice admission), number of multidisciplinary team visits (oncology and PC), number of hospitalisation and of emergency department admission, with reason and duration.

For those patients who will die during the follow-up period, a dedicated research nurse will collect the following data relative to the last 30 days of life: active oncological treatments administration (chemotherapy, radiotherapy, immunotherapy, etc) and date of last administration, number of ER visits, hospitalisation (number and length in days), activation of any PC service (home care or hospice) and place of death. The above information will allow us to calculate end-of-life care quality indicators, like the proportions of patients with chemotherapy in the last 14 days of life.3

For all data to be collected, table 1 reports the assessment tools and the timing of various assessments. Eligible patients will undergo FAMCARE P-13 and EORTC QLQC15-PAL evaluation at baseline and then monthly for at least 6 months from enrollment or till death (T1, T2, T3, T4, T5 and T6). The choice of interviewing patients every month is suggested by the need to minimise attrition due to potential patients drop out and also to take into account the frequent possible changes in clinical conditions in a fragile patient population. FAMCARE will be administered to the main caregiver identified by the patient at T0, T3 and T6 or after patient’s death in patients who will die during the follow-up. Patient and caregiver-reported outcomes will be collected through telephone interview. This method was chosen in order to avoid contamination between the pre-PCRS and post-PCRS implementation groups; in fact, the PCRS implementation in the post-test group will be based also on patient-reported symptom assessment which will be electronically collected.13 Healthcare professional reported data will be collected using REDCap electronic data capture tools hosted at Fondazione IRCCS Istituto Nazionale dei Tumori of Milan

The planned duration of the entire project, considering phase 1 and phase 2, is 4 years (15 September 2020–14 September 2024).

Power considerations and statistical analysis

The main endpoint is the variation in patient’s satisfaction between baseline and the average of repeated follow-up assessments. t-test for independent samples on this endpoint will be used to compare prereferral and postreferral patient cohorts. Power calculations14 indicate that 150 patients in each group will allow a two-tailed t-test for independent samples, a power of 0.9 to detect an effect size of 0.375 with alpha error=0.05. In case of a 15% attrition rate, the study power reduces at 85%, all else unchanged. The same analysis scheme will be applied for all continuous-repeated measurement outcomes, while binary outcomes (ie, activation of PC during follow-up or chemotherapy administration during the last month of life) will be analysed using logistic regression models. Linear and logistic regression models will be used for covariate adjustment estimations.

Data sharing

Deidentified data of participants in this project will be available from the project principal investigator (AC) on request, provided that the data reuse is in agreement the European General Data Protection Regulation 2016/679 (GDPR).

Patient and public involvement

No patient nor other member of the public was involved in the development of this research project.


Early PC referral, while concurrent antineoplastic treatment is still possible and indicated, is considered to improve integration of PC in the clinical pathway of patients with advanced disease who cannot achieve cure with the available antineoplastic treatments.2 This general statement is based on several clinical trials1 showing advantages of early access to PC in improving patient’s quality of life, psychological distress, use of healthcare resources at the end of life and caregivers’ distress.

Different early palliative care models

These trials were however based on different clinical models and settings. The model which was more consistently tested as an experimental intervention has been based on the access of patients to specialised PC outpatient clinic. In fact, the implementation of this model15–22 has more consistently shown efficacy if compared with other interventions of care coordination without direct encounters between patients and PC professionals.23–26 This has been also described as an ‘integrated care model’.27 28

Within the integrated model trials, the number and duration of encounters of patients with PC professionals varied from 1 every 1.3 weeks to 1 every 6 weeks with a total mean number of visits during the study period ranging from 2.4 to 8.9. Two of the three negative studies on the integrated model15 21 resulted in very limited if any access to specialised PC visits. The third one acknowledged significant in between trial arms contamination.22

Also qualitative type evidences from the same trials support subjective appreciation of clinical value attributed by patients to integrated PC interventions. Patients and caregivers considered specialised outpatient PC capable of providing personalised and prompt symptom management, holistic support to patients and caregivers, guidance in decision-making and preparation for the future.29 Another study addressed specifically the clinical content of early outpatient PC and showed that interventions based on psychological and cognitive coping, disease understanding, decision-making and care planning were associated with lower psychological distress and better use of healthcare resources at the end of life.30 Finally, the study by Costantini et al 31 shows that an early integration of specialised PC after the diagnosis of advanced cancer is feasible and well accepted by patients, relatives and, to a lesser extent, by oncologists.

Palliative care needs selection criteria

Overall, these data confirm the usefulness of the PC outpatient clinic approach, with specialised clinical skills and multidimensional assessment, for patients with advanced cancer but the selection criteria used in the clinical trials are difficult to translate into clinical practice. The trials aimed at demonstrating efficacy and detecting outcome changes at the group level, while statistically accounting for patients heterogeneity. In clinical practice, the decision to refer a patient to PC outpatient clinic needs instead to be individualised. Therefore, trials’ result cannot clarify how to guarantee timely access at the individual level and to optimise resource allocation. Trials are also not enough to change clinical practice, without taking into account practical and cultural barriers that impact on PC perception by oncologists and patients.28 32

In summary, the best ways to address earlier PC needs identification, timely access to outpatient PC and also to overcome barriers to integration with oncology are still unresolved issues, and yet late referral is considered to be a significant barrier in many cases.28 32–36

The literature discussion about criteria to be used and the domains to be assessed, beyond the stage of the disease, already considered in all trials, includes symptoms, psychosocial distress, prognosis, trigger disease-related events and critical care planning issues.4 37–39 A number of PC need screening and assessment instruments have been developed and sometimes tested,39–50 but only a few of them were specifically designed to improve appropriate selection of cancer patients for referral to outpatient specialised PC.4 51 In the study by Paiva et al, the use of the PC referral tool would have increased referral rate by 3.2-fold.51 There are no studies nor agreed guidelines on criteria to guide timely referral48 52 53 beyond disease stage.2 4

Hui et al 27 published a Delphi study building a consensus on referral criteria among a number of PC expert identifying major and minor criteria and suggesting that referral should occur anytime one major criterion is met. The same authors,54 in a retrospective assessment of the characteristics of patients referred to their PC clinic, found that 85% fulfilled one major criteria and that referral occurred as an average 14 months before death.

Interestingly, in the study by Singh et al,55 prognostic assessment alone using the Surprise question had no effect on referral to PC including the outpatient setting.

The only study looking at implementing an intervention to improve referral to ambulatory PC was planned by a cancer centre in Texas as a quality improvement intervention, including, in a plan-do-study-act cycle, the adoption of a symptom assessment tool and a referral pathway in oncology clinics. Their initial results show a 10-fold increase of referrals after implementation, from 0.07% of all oncology clinic encounters to 0.8%. It is unclear how satisfactory this result can be considered, given the modest number of referrals, in particular because the relationship with high symptom score is not consistent with such a low referral rate. The study is still under development.56

The PCRS project

The primary aim of our study is therefore to develop and implement a PC referral system which takes into account the suggestions from the literature in combination with consensus between oncology and PC specialists. It is anticipated that in an implementation study with the aim of intervening on clinical practices, evidence-based concepts need to be combined with professional interaction, knowledge of service availability, professional trust and resource allocation that avoid extra workload burden and alert fatigue.57

PC integration has been available at our centre since 200158 and we recently described its operational characteristics on a consecutive lung cancer patients population seen at a thoracic oncology clinic. In a 2-year period, 43% of patients were referred to PC clinics, mainly for symptom control, usually pain and or dyspnoea and poor performance status. The mean duration of outpatient PC was 128 days.59

The preimplementation and postimplementation study design in this protocol will give information on the feasibility of the PCRS, its impact on user experience of care, patient quality of life and use of healthcare resources. Ideally, a cluster-randomised trial design would be preferable but at least 6–8 first level units (hospitals) should be involved for the results to be robust, and this is beyond resources allocated to this research. An individually randomised design, instead, may not be adequate for the evaluation of a complex intervention with a high contamination risk. Furthermore, the evidence of the potential efficacy of PC referral implementation are still to be shown. Considering these limitations, the aim of the study is not to provide a final estimate of the effectiveness of PC needs screening but to provide a first estimate of feasibility and impact. Study completion would also allow the subsequent integration of the PCRS, or of some of its components,13 at the institutional level in order to steer a change in clinical practice to augment palliative and oncology integration and improve patient care.

The study is part of a nationally funded programme (Finalizzata di Rete—project ID NET-2018-12367032 to AC—funded by the Italian Ministry of Health and by Regione Lombardia) to address the recognition and response to PC needs in different patients populations and has the opportunity to raise awareness at the national level about the importance of PC integration at the outpatient level. The national programme finalisation could strengthen directions for the Ministry of Health to allocate resources for recognising the lack of specialised PC services at acute hospitals and promoting their implementation.

Patients and caregivers will be enrolled in the pretest and post-test after obtaining informed consent. Dissemination of the study results will occur through publications in peer-reviewed open-access scientific journals and the presentation of data at national and international conferences.

Data availability statement

No data are available.

Ethics statements

Patient consent for publication



  • CB and EZ are joint first authors.

  • Contributors Conceptualisation: CBr, EZ, AP, PB, MC, SLD, VF, AT, CBe, MB, AR, MN, PS, SP, SA, GT, FDB and AC. Writing—original draft preparation: CBr, EZ, AP, PB, MC, SLD, VF, CBe, MB, AR, MN, PS, SP, SA, GT, FDB and AC. Writing—review and editing: CBr, EZ, AP, PB, MC, SLD, VF, CBe, MB, AR, MN, PS, SP, SA, GT, FDB and AC. Supervision: CBr, EZ and AC. All authors have read and agreed to the published version of the manuscript.

  • Funding This study is part of a national programme funded by the Italian Ministry of Health and by Regione Lombardia (project ID NET-2018-12367032 to AC).

  • Competing interests AC has received consultant honoraria from Mundipharma, Pfizer/Eli Lilly Italia Spa, Angelini, Shionogi, Molteni and Kyowa Kirin. He has also received research grants from Molteni & C Soc Esercizio Spa and Ipsen. EZ has received consultant honoraria from Amgen. MN has received travel expenses from Celgene, speaker honorarium from Accademia della Medicina and consultant honoraria from EMD Serono, Basilea Pharmaceutica, Incyte and MSD Italia. FDB has received consultant honoraria from Roche, EMD Serono, NMS Nerviano Medical Science, Sanofi, MSD, Novartis and Incyte. He has received speaker honoraria from BMS, Healthcare Research & Pharmacoepidemiology, Merck Group, ACCMED, Nadirex, MSD, Pfizer, Servier, Sanofi, Roche and AMGEN. He has also received research grants from Novartis, Roche, BMS, Celgene, Incyte, NMS, Merck KGAA, Kymab, Pfizer, Tesaro and MSD. AP, PB, MC, SLD, VF, CBe, MB, PS, SP, SA, AR, GT and CBr declare no conflicts of interest.

  • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting or dissemination plans of this research.

  • Provenance and peer review Not commissioned; externally peer reviewed.