Article Text

Download PDFPDF

Development of a screening tool for the need of specialist palliative care in oncologic inpatients: study protocol for the ScreeningPALL Study
  1. Evelyn Müller1,
  2. Michael Josef Müller1,
  3. Christopher Boehlke2,
  4. Christina Ramsenthaler1,
  5. Helga Jäger1,
  6. Henning Schäfer3,
  7. Christoph Ostgathe4,
  8. Carsten Klein4,
  9. Steffen Simon5,
  10. Gerhild Becker1
  1. 1Department of Palliative Medicine, Medical Center - University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany
  2. 2Department of Palliative Care, University Hospital Basel, Basel, Switzerland
  3. 3Department of Radiation Oncology, Medical Center, Faculty of Medicine, University of Freiburg, German Cancer Consortium (DKTK) Partner Site Freiburg, German Cancer Research Center (DKFZ), Heidelberg, Germany
  4. 4Department of Palliative Medicine, University Hospital Erlangen-EMN, Comprehensive Cancer Center CCC Erlangen, Friedrich-Alexander-University Erlangen-Nürnberg, Erlangen, Germany
  5. 5Department of Palliative Medicine and Center for Integrated Oncology Aachen Bonn Cologne Dusseldorf (CIO ABCD), University of Cologne, Faculty of Medicine and University Hospital, Cologne, Germany
  1. Correspondence to Evelyn Müller ; evelyn.mueller{at}


Introduction A range of referral criteria and scores have been developed in recent years to help with screening for the need of specialist palliative care (SPC) in advanced, incurable cancer patients. However, referral criteria have not yet been widely implemented in oncology, as they usually need to be revised by physicians or nurses with limited time resources. To develop an easily applicable screening for the need for SPC in incurable cancer inpatients, we aim to (a) test inter-rater reliability of multiprofessional expert opinion as reference standard for SPC need (phase I) and (b) explore the diagnostic validity of selected patient-reported outcome measures (PROMs) and routine data for the need of SPC (phase II).

Methods and analysis Inclusion criteria for patients are metastatic or locally advanced, incurable cancer, ≥18 years of age and informed consent by patient or proxy. (Exclusion criteria: malignant haematological disease as main diagnosis). In phase I, three palliative care consultation teams (PCTs) of three German university hospitals assess the SPC need of 20 patient cases. Fleiss’ Kappa will be calculated for inter-rater reliability. In phase II, 208 patients are consecutively recruited in four inpatient oncology wards of Freiburg University Hospital. The PCT will provide assessment of SPC need. As potential referral criteria, patients complete PROMs and a selection of routine data on person, disease and treatment is documented. Logistic regression models and ROC analyses are employed to test their utility in screening for SPC need.

Ethics and dissemination Our findings will be published in peer-reviewed journals and presented at national and international scientific meetings and congresses. Ethical approval was granted by the Ethics Committee of Albert-Ludwigs—University Freiburg, Germany (approval no. 20-1103).

Trial registration number German Clinical Trials Register, DRKS00021686, registered on 17 December 2020.

  • Breast tumours
  • Gastrointestinal tumours
  • Respiratory tract tumours

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

  • The study offers an exploration of the diagnostic validity of patient-reported outcome measures and routine data that are proposed as referral criteria for specialist palliative care (SPC) need.

  • The study focusses on feasibility in everyday practice by selecting criteria that can be implemented in electronic systems to foster widespread use in oncology and thereby access to SPC for those most in need.

  • Successful demonstration of substantial inter-rater reliability of multiprofessional expert opinion on SPC need in phase I is a prerequisite for its implementation as reference standard in phase II.

  • Single-centre study that requires subsequent exploration transferability of results to other care contexts.


There is a broad consensus that palliative care should be integrated in the treatment of patients with advanced tumours.1–4 The background to these recommendations are studies showing positive effects of early integration of palliative care on patients quality of life and symptoms,5–8 their treatment (less aggressive and costly oncological therapies at the end of life9) and cost reductions.10 11

However, recent reviews on the effects of integrating specialist palliative care (SPC) show positive but relatively small effects on quality of life12 13 and symptoms.12 Gaertner et al13 see an important reason in the fact that in the studies specialist care was provided to all patients based on undifferentiated criteria such as disease stage—regardless of whether there was actually a need for specialist care. In most cases, palliative care provided within primary care or oncology is sufficient (generalist palliative care). Experts estimate that 10%–20% of all dying patients have a need for SPC,14 with higher rates for terminally ill cancer patients in inpatient treatment.15 The more complex and uncontrollable the problems the more likely the patients will benefit from SPC.

In order to determine which patients need SPC, various screening approaches have been developed in recent years. The criteria lists and scores are usually meant to be applied by the treating physicians and nurses.16–22 Examples are (1) the screening tool of the National Comprehensive Cancer Network19 23 that adds up 11 criteria (eg, prognosis, comorbidity, need of assistance in decision-making) to a score and (2) a list of 11 major and 36 minor criteria for referral to specialist outpatient palliative care, which was a result of an international Delphi Process by Hui et al.16 These screening approaches based on staff assessment employ a variety of criteria for determining SPC need. Criteria do not compare easily among the screening systems. They include disease-related criteria (eg, prognosis of lifetime, staging, complications), treatment-related criteria (eg, decision-making, hospitalisation) and patient and family needs (eg, physical and psychological symptoms; request for euthanasia).

Initial approaches to using patient reported outcome measures (PROMs; questionnaires completed by patients) are also being pursued to identify patients with complex problems.24 25 Most promising and widely used is the Integrated Palliative Care Outcome Scale (IPOS), a short questionnaire that is completed by patients but also has a proxy version.25 PROMs have advantages; they directly reflect the patient’s perspective rather than the perception of doctors or nurses and facilitate a holistic assessment. However, there is also a considerable disadvantage, if their completion is a prerequisite for screening: Not all patients are willing and able to complete questionnaires. Cognitive, physical and language requirements might lead to the exclusion of patients from screening and make support by SPC less likely for them—possibly excluding the most vulnerable. The use of proxy assessment would be an option in these patients (eg, in IPOS) but seems unrealistic in everyday practice due to the effort required for doctors and nursing staff.

Despite the promising approaches, there is still a need for research as both implementation in practice and research on referral criteria face a number of challenges.

There is no gold standard for the determination of need for SPC,23 due to the multifaceted and complex situation of the concerned patients. In its absence, individual screening instruments have been shown to correlate with various parameters such as questionnaires on needs and burden of patients, contact with SPC or remaining life span.19 23 26 However, while these parameters are correlated with severe illness and sometimes complex need, they have themselves never been validated to indicate SPC need. A further step towards an approximation of a suitable reference standard would be a multiprofessional expert opinion, which we aim for in our study.

Despite attempts for all these screening systems to keep the criteria simple and clear, these instruments are not yet established widely in practice. Typical barriers to implementation are a lack of time resources and a low perception of benefit on the part of the practitioners,23 27 28 especially if doctors or nurses are to carry out the screening.

On the background, we aim to develop a set of referral criteria to determine the need for SPC in advanced incurable cancer inpatients that does not require an extra effort of physicians or nurses in everyday practice but can be implemented and evaluated in electronic systems.

In order to achieve this goal, we investigate:

  1. The inter-rater reliability of multiprofessional expert opinion as reference standard for SPC need and explore the decision-making process of expert teams in assessment of SPC need (phase I).

  2. The diagnostic validity of PROMs as well as person-related, disease-related and treatment-related routine data to detect SPC need as determined by expert opinion as reference standard (phase II).

To ensure wide applicability including patients unable or unwilling to complete PROMs, we aim to test two models, one based on PROMs and routine data and one based on routine data only.

Methods and analysis

Study design

The ScreeningPall Study is an ongoing study that recruits patients since August 2021 and includes two phases: In phase I, a standard procedure for expert assessment of SPC need is developed. Inter-rater reliability of multiprofessional expert opinion as a reference standard for SPC need will be determined by comparing the assessment of three expert teams on 20 cases. In phase II, a prospective, monocentre, cross-sectional study is conducted to explore which PROMs and sociodemographic, disease-related and treatment-related routine data can predict SPC needs identified by multiprofessional expert opinion as reference standard. This protocol was written and reported according to the TRIPOD checklist.29

Study population and setting (study phases I and II)

Cancer inpatients at the University Medical Centre Freiburg, a tertiary care centre and major European hospital with a Comprehensive Cancer Centre, are recruited for the study. As the German ‘Evidenced-based Guideline: Palliative care for patients with incurable cancer’,30 states that incurable cancer is a prerequisite for the offer of SPC, the inclusion criteria for patients are metastatic or locally advanced, incurable cancer, ≥18 years of age and informed consent by patient or proxy. Patients with malignant haematological diseases as their main oncological diagnosis are excluded. These patients differ significantly regarding symptom burden, course of disease, care structures and have a high prognostic uncertainty,31 32 and therefore it is assumed that specific referral criteria are needed for these patients. We aim to recruit patients with wide heterogeneity of oncological conditions, current treatments and sociodemographic characteristics. Therefore, recruitment takes place in four departments that focus on different forms and combinations of treatment (systemic, radiological and surgical) and cover the entire range of solid tumour diseases.

Recruitment and informed consent (study phases I and II)

In both phases, all patients admitted to the recruiting wards are consecutively screened for eligibility criteria by a physician. Eligible patients (or their authorised legal representative) are informed about aims and possibilities of palliative care and the study in detail. If the patient is willing to participate, the written consent of the patient (or authorised representative) is obtained.

Phase I

In the absence of a validated reference standard for assessing the need for SPC, a process of (a) holistic medical history taking and (b) multiprofessional case review by an experienced multiprofessional palliative care consultation team (PCT) is established and documented as a reference standard for assessing the need for SPC.

Development of structured and standardised process/documentation of medical history taking and case review as reference standard

We developed medical history documentation and case review documentation based on existing documents and literature. In a first step, these documents were pretested and explored in individual cognitive interviews with two physicians, three nurses, one social worker, one psychologist and one pastor of PCT at Freiburg University Hospital. The results of the interviews were summarised and presented to the whole team in a joint session. The team consented the wording of these documents, ensured common understanding of terms and chose questions added for the oral medical history taking with patients. The final documents were then employed in everyday practice of the PCT before being used in the study.

Figure 1

Procedure for assessing the reference standard in phases I and II (PCT, palliative care consultation team; SPC, specialist palliative care)

Figure 2

Overview of data to be collected in phase II.

Inter-rater reliability of multiprofessional expert opinion

At Freiburg University Hospital, medical histories of 20 patients are taken by a physician and nurse of the PCT team and will form the basis of standardised case reports. We will extend sampling for maximum variation if the heterogeneity of case characteristics and patient-reported needs on the IPOS33 is not sufficient after the first 20 consecutive participants. The case descriptions are presented to the members of the three PCTs of the Freiburg, Erlangen and Köln University Hospitals for case reviews. Each team independently discusses and consents on need for SPC (yes/no and rationale) and—if yes—discusses treatments or actions that should be taken in a potential consultation with the patient. The discussions are audiotaped and results are documented (based on written informed consent of all participants).

Statistical and qualitative analysis

Fleiss’ Kappa34 is calculated to test the assessor agreement on SPC need between the three teams.35 In the event of a lack of assessor agreement, cases without agreement will be discussed in an online group with representatives of the three teams to further improve the process. If necessary, the process of taking case histories and determining the assessor agreement will be repeated. The audio recording will be transcribed verbatim. A qualitative content analysis36 will extract factors frequently considered in the assessment of SPC needs and their individual influence. Additionally, a linguistic analysis will be performed to gain insight into the patterns of argumentation that lead to the decision on SPC needs. A software for qualitative data analysis (MAXQDA, Verbi GmbH, Berlin) will be used.

For all cases in phase I, the same data are collected on potential referral criteria as described in phase II.

If phase I is successful, the procedure of medical history taking and case review (see figure 1) is employed as reference standard in phase II.

Phase II

In phase II, a prospective, monocentre, cross-sectional study with 208 patients will be conducted. Patient selection and recruitment are congruent with phase I. Patients receive two independent visits within a maximum of 4 days: The PCT takes their medical history; a study assistant independently collects data on the potential referral criteria. Further data are taken from electronic documentation and provided by the ward physician.

Reference standard

The structured and standardised process of medical history taking and multiprofessional case review developed and validated in phase I is employed in phase II to determine SPC need in every patient (figures 1 and 2). The resulting expert opinion is the primary criterion for SPC need. In addition, secondary criteria are (a) the ward physicians’ professional documentation of their assessment of SPC need and (b) initiation of or already ongoing specialised palliative care during current hospital stay. Agreement between the assessment of SPC need of ward physicians and PCT will be explored.

Potential referral criteria

The study aim is to investigate the diagnostic validity of PROMs as well as person-related, disease-related and treatment-related routine data to detect SPC need (figure 2). For this purpose, it is determined whether the potential referral criteria predict the reference standard. The selection of potential referral criteria is based on criteria listed in the literature.16–20 Due to the objective of only testing criteria that can be assessed later in everyday practice based on routinely collected health information from records, criteria that require assessment of physicians or nurses cannot be recorded directly. For criteria that are not documented reliably in electronic documentation (eg, request of euthanasia), we employ surrogate parameters (eg, involvement of psychosomatic or psychiatric consultation service), fully aware of the limitation of this approach. Study assistants will collect data independently from the medical history taking and review process by the PCT.

Patients complete questionnaires for their own assessment of their situation. Two ‘holistic’ questionnaires are employed, the IPOS and Distress Thermometer (DT). The IPOS is a frequently used and validated tool33 37 in international palliative research and practice that enables assessment of the dimensions of physical symptoms, emotional symptoms and communication/practical issues with very little effort on 5-point Likert scales. The IPOS has been shown to correlate with surrogate parameters of SPC need and is therefore promising in screening for SPC need.19 25 Additionally, DT38 is employed as it is an already established screening instrument for psycho-oncological treatment in the recruiting oncology wards and collection of data is no additional effort for patients. DT consists of a single item that measures overall distress on a 0–10 rating scale and a problem list with dichotomous yes/no answer option. Its use for screening for SPC need has not yet been studied. Aiming at low effort for physicians in everyday practice and external validity of the study, no proxy assessment is intended for patients that are not able to complete self-assessment.

A variety of routine data is collected (see table 1 for overview), including disease-related criteria (eg, prognosis of lifespan, complications, comorbidity), treatment-related criteria (eg, duration and frequency of hospitalisation, necessity of certain medical measures) and sociodemographic data. Furthermore, the ward physicians assess the prognosis of the lifetime via the so-called double surprise question.39

Table 1

Data collected and checked for feasibility of use and diagnostic value

All potential referral criteria are collected by a study assistant who asks patients to complete PROM and collects disease-related and treatment-related information from patients (orally), ward physicians (written) and electronic routine documentation. There is no exchange of information on SPC need of the patient between the study assistant and the PCT.

Monitoring of pandemic distress

The COVID-19 pandemic causes stresses in oncologic patients, for example, through visitor restrictions,40 cancellation of treatment/diagnostic measures41 and psychological and social effects of isolation.42 43 The results on the relevance of referral criteria might be influenced, if these stresses result in systematic changes in the profile of patient burden in the patient sample. On that background, we monitor pandemic distress. Patients complete a questionnaire that was developed based on the literature, followed by multiprofessional discussion and feedback with eight professionals including psychologists, physicians, nurses and an expert on PROM development. Face validity and comprehensibility were ensured by a series of 13 cognitive interviews with oncologic inpatients. If the questionnaire indicates a relevant level of distress in more than 20% of patients, we pause data collection (relevant distress level: above middle response option of the Likert scales in two or more assumed dimension).

Sample size

The sample size was calculated according to the guidelines of Hajian-Tilaki,44 for diagnostic test procedures. Prevalence of SPC need was estimated to be 30% based on estimations of physicians of recruiting oncology wards and PCT. Priority was given to sensitivity over specificity, aiming for 80% sensitivity.45

Analysis of results

Two models are aimed for: (1) a model based on PROMs and potentially additional routine data and (2) a model solely based on routine data that can be collected in all patients, even if they are unable or unwilling to complete PROMs. For both models, a preselection of useful referral criteria for SPC need will take place to reduce the number of criteria entered in the regression models. The preselection will take the (a) current state of research on the predictive value of criteria for SPC need at time of analysis, (b) feasibility of data collection (effort rated by study assistants; for example, accessibility of the information) and (c) data quality (missing values, reliability and validity of measurement) into account.

The preselected criteria are entered in an analysis process in which logistic regression models and receiver-operating characteristic curve analysis are used. The procedure is based on the guidelines of the Prognosis Research Strategy-Group (PROGRESS Group46) and the tatement on Transparent Reporting of a Multivariate Prediction Model for Individual Prognosis and Diagnosis (TRIPOD).29 According to the TRIPOD statement, the present analysis is a type 1b study. A binary logistic regression model will be employed for prediction of the reference standard of SPC need yes/no. Selection of referral criteria in the model is based on statistic value for prediction, clinical relevance and ease of availability. The performance of the modelling process will be tested by b (ootstrapping with 200 samples with replacement within the original sample. For the entire model of relevant independent variables, the sensitivity, specificity and the negative and positive predictive values for the prediction of SPC need are determined. In addition, we analyse whether assessments of SPC need of ward physicians and palliative care specialists show concordance (evaluated via Cohen’s kappa).

Percentage of missing data is calculated for all potential referral criteria to see if their use is reliable in everyday practice and for information from different sources, (eg, patient report of previous hospital stay and electronic documentation) congruence is checked. Missing values are not imputed, as we do not expect them to be at random.

Patient and public involvement

Neither patients nor the public were involved in the design or planning of this study.

Ethics and dissemination

The current study was approved by the Ethics Committee of Albert-Ludwigs—University Freiburg, Germany (approval no. 20-1103). All study participants will provide written informed consent before participation. All procedures performed in the study are in accordance with the ethical standards of the institutional and national research committee and with the 1964 Declaration of Helsinki and its later amendments.

Important protocol modifications such as changes in eligibility criteria or outcome will be communicated to the relevant parties, that is, sponsor, trial registry and scientific ethical committee, and explicitly described in future publications.

The results of the study will be presented in peer-reviewed scientific journals. The results of the project will also be disseminated through participation in academic and other conferences.


The absence of a validated reference standard is the key challenge of our study: To determine SPC need, we employ multiprofessional expert opinion that is reached in a standardised process of anamnesis and case review. While this might be a useful approximation to a gold standard, its inter-rater reliability has not been studied before and the results of phase I will determine the implementation and success of phase II.

A key aspect in the planning of the study was feasibility in everyday practice in oncology. Therefore, we considered aspects such as effort of screening for staff, patient requirements (eg, for completion of PROMs) or the completeness of the documentation (eg, we expect missing data, when patients are admitted from other hospitals). For some of the criteria we have to explore feasibility of collection and data quality before deciding on their use, for example, we do not know what proportion of patients is able to complete PROMs.

We are confident to identify criteria for SPC need in oncology. These will have to be validated in multicentre and international studies to ensure transferability as influence of services in institutions and health systems cannot be ruled out.

The study focusses on feasibility in everyday practice. The ultimate goal is the implementation of the criteria in electronic systems to foster widespread use in oncology and thereby access to SPC for those most in need. As the electronic system might ‘raise the flag’ and point out patients that might require SPC, it avoids effort for physicians and nurses while leaving the decision about the referral in their hands.

Ethics statements

Patient consent for publication


We would like to thank all members of the palliative care consultation teams of Freiburg University Hospital—Alexandra Dörle, Dominik Flach, Brigitte Fröschen-Ludwig, Thomas Gelbrich, Junko Hübers, Tina Meer—as well as Luise Elster, Rotraut Heizmann-Schlenker and Andrea Kohler for contributing significantly in the preparation of the study by cognitive interviews, team discussions and feedback. We would also like to thank the directors and senior physicians of recruiting wards in Department of Obstetrics and Gynaecology, Department of Thoracic Surgery, Department of Medicine II and Department of Radiation Oncology. They provided valuable information and expertise in the planning of sample size, recruitment and data collection and will recruit the participants of the study. Furthermore, we thank all patients that contributed by cognitive interviews to the development of the questionnaire on pandemic distress in cancer patients. The article processing charge was funded by the Baden-Wuerttemberg Ministry of Science, Research and Art and the University of Freiburg in the funding programme Open Access Publishing.



  • EM and MJM contributed equally.

  • Contributors MJM, CB and GB initiated the project. EM, MJM, HJ and GB were primarily responsible for the conception and design of the study and acquisition of funding. CB, CR, HJ, HS, GB, CK, CO and SS provided valuable feedback in the conception process and thereby contributed to the study design. MJM and EM coordinate the study. CO, CK, SS, HS, MJM and HJ contribute in data collection. CR was responsible for planning of statistical analysis. EM and MJM wrote the first draft of the manuscript. All authors read, revised and approved of the final manuscript.

  • Funding This work is supported by German Cancer Aid grant number 70114472.

  • Competing interests None declared.

  • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

  • Provenance and peer review Not commissioned; externally peer reviewed.