Introduction Evaluating the quality of dying and death is essential to ensure high-quality end-of-life care. The Quality of Dying and Death (QODD) scale is the best-validated measure of the construct, but many items are not relevant to participants, particularly in low-resource settings. The aim of this multisite cross-sectional study is to develop and validate the QODD-Revised Global Version (QODD-RGV), to enhance ease of completion and relevance in higher-resource and lower-resource settings.
Methods and analysis This study will be a two-arm, multisite evaluation of the cultural relevance, reliability and validity of the QODD-RGV across four participating North American hospices and a palliative care site in Malawi, Africa. Bereaved caregivers and healthcare providers of patients who died at a participating North American hospice and bereaved caregivers of patients who died of cancer at the Malawian palliative care site will complete the QODD-RGV and validation measures. Cognitive interviews with subsets of North American and Malawian caregivers will assess the perceived relevance of the scale items. Psychometric evaluations will include internal consistency and convergent and concurrent validity.
Ethics and dissemination The North American arm received approval from the University Health Network Research Ethics Board (21-5143) and the University of North Carolina Institutional Review Board (21-1172). Ethics approval for the Malawi arm is being obtained from the University of North Carolina Institutional Review Board and the Malawian National Health Science Research Committee. Study findings will be disseminated through publication in peer-reviewed journals and conference presentations.
- Quality in health care
- Adult oncology
- Adult palliative care
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Strengths and limitations of this study
A strength of the study is its aim to develop and evaluate a more culturally universal version of the widely used Quality of Dying and Death (QODD) scale across four North American hospice sites and at a Malawi palliative care clinic. This revised version, called the QODD-Revised Global Version (QODD-RGV), will allow for the assessment of cultural generalisability and relevance of content items and a comparison of QODD across diverse settings.
The North American study arm will compare QODD ratings of bereaved caregivers with those of hospice nurses and physicians who provided care for the deceased patients. Determining the concordance of these proxy ratings will inform the clinical application of the QODD-RGV.
A limitation of the North American study arm is that staff participants will complete the QODD-RGV by themselves, whereas caregiver participants will complete it in an interview with research staff. Variation in whom respondents may consult with prior to completing the measure could bias the scale responses.
The quality of the dying and death experience is a critical outcome to ensure high quality of care near the end of life for people with serious, life-limiting diseases. The quality of dying and death is a multidimensional construct comprising physical, psychological, social and spiritual/existential experiences, the nature of healthcare, preparation for death and the circumstances of death; it is influenced by medical, individual, social and cultural factors.1 Positive experiences in these dimensions contribute to what has been termed a ‘good death’, which has been defined as a death ‘free from avoidable distress and suffering for patients, families and caregivers; in general accord with patients’ and families’ wishes; and reasonably consistent with clinical, cultural and ethical standards.”2 Evaluating the quality of the dying and death is essential to ensure the quality of end-of-life care.3
Palliative care aims to relieve serious health-related physical, psychological, social and spiritual suffering and improve quality of life in people with life-threatening diseases and their families, from diagnosis to the end of life.4 Global mortality rates related to non-communicable diseases continue to rise, particularly in lower-resource countries,5 where palliative care is not available to most individuals and families who are in need of such care.6 As global efforts to expand palliative care services continue, culturally appropriate and generalisable measures of the quality of the dying and death are needed to support advocacy for palliative care, to stimulate local progress in palliative care development and to compare the effectiveness of interventions across different settings.
Strengths and limitations of the Quality of Dying and Death scale
A number of measures have been developed to assess the quality of the dying and death3 7; of these measures, the multidimensional Quality of Dying and Death (QODD) scale is the best-validated and most widely used.3 7 The QODD includes 31 content items assessing different aspects of the dying and death experience and single-item ratings of overall quality of dying in the last 7 days of life and overall quality of the moment of death.8 9 Items are rated on the patient’s experience, from 0 (terrible experience) to 10 (almost perfect experience), in six conceptual domains: symptoms and personal care, time with family, whole person concerns, treatment preferences, preparation for death, and the moment of death. Because it is not feasible for most patients to complete the scale near the end of life,3 the QODD is completed retrospectively by proxy raters, such as bereaved caregivers or healthcare workers who provided care to the patient prior to death.
Despite its widespread use, the QODD has limitations that can affect its validity.10 11 The scale is complex, with at least two questions per each item content (a question about occurrence of the item content, followed by a quality rating), and is thus burdensome to complete. High missing item response rates that appear to be due to ambiguity or item non-relevance are commonly encountered in its use.10 The scale was developed in an American setting, and all validation studies of the QODD or adapted versions have been conducted in higher-resource settings, where palliative care has been more available and accessible.8 12–18
The development of validated outcome measures for use in lower-resource countries has been called for as part of advocacy and initiatives to stimulate palliative care development and expansion.19–21 The generalisability of the QODD outside of high-resource settings is questionable, although formal validity studies have not been conducted in low-resource settings. Our research with the QODD in Kenya, with proxy ratings provided by bereaved caregivers, revealed high missing response rates (ie, >10% omitted ratings) in QODD items that pertained to treatment preferences, preparation for death, end-of-life care discussion with doctors, medical prolongation of life and the moment of death and that suggested their lack of cultural relevance.22
A version of the QODD with cross-culturally generalisable items that are less burdensome to complete would be of value to support palliative care research and clinical care in higher-resource and lower-resource settings. The creation of such a measure is responsive to the call for the development of reliable and valid outcome measures that are generalisable to high-resource and low-resource settings. This measure would be of value to assess the quality of end-of-life care, to evaluate the impact of novel interventions near the end of life, to allow cross-cultural and cross-national comparisons, and to support advocacy for resource allocation for palliative and end-of-life care. Greater resource allocation is urgently needed in low-resource settings where palliative care is not available to the large majority of people in need.
The current protocol details the development and multisite psychometric evaluation of the QODD-Revised Global Version (QODD-RGV), a measure that addresses some of the limitations of the original QODD scale. This validation study comprises two arms: one conducted at four Northern American hospice sites, and the other at a palliative care clinic in Malawi, Africa. The primary study objectives are (1) to determine the content validity and cultural relevance of items in North American and Malawian care settings and (2) to evaluate the reliability and validity of the QODD-RGV with proxy ratings by both bereaved caregivers and primary healthcare providers of patients with advanced cancer in the North American hospice settings and by bereaved caregivers of patients with advanced cancer in the Malawian palliative care setting.
Methods and analysis
Patient and public involvement
Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of our research.
Phase 1: development of the QODD-RGV scale
The QODD-RGV (see online supplemental appendix 1) was developed by the members of our study team, who have extensive clinical and research experience with the QODD in diverse cultural settings. This revision of the QODD is intended to improve the ease of completion by respondents and to enhance its universality. Team members reviewed the original QODD and revised or eliminated items with consistently high nonresponse rates in previous studies. Item revisions included rewording items so that they would be more generalisable across diverse settings. Some new items were added to reflect broadly relevant aspects of the QODD. The format of the scale items was also modified to reduce the length and complexity of the scale. The original questions that ask about occurrence of each item content were eliminated; only the quality ratings for items were retained. The content items were also modified to reflect the extent of the item content rather than the quality of the experience (eg, to what extent did X suffer from pain?), and the corresponding rating scale ranged from 1 (not at all) to 5 (extremely), with options to indicate ‘don’t know’ or ‘no response’. Only the single-item rating of overall dying and death experience retained its original format and 11-point quality rating scale: ‘How would you rate X’s overall experience of dying and death in the last 7 days of life?’ rated from 0 (terrible) to 10 (almost perfect). All proposed changes were discussed among team members until consensus was reached.
Phase 2: psychometric evaluation of the QODD-RGV
The recruitment and data collection periods for the North American and Malawi study arms are summarised in figure 1.
North American arm
Data collection for the North American arm of the multisite study is expected to require 1 year (projected time frame: June 2022–2023) and is to be conducted as part of a collaborative partnership between the University of North Carolina Palliative Care Programme and the Global Institute of Psychosocial, Palliative and End-of-Life Care (GIPPEC) of University of Toronto.
Participants and recruitment
The study includes three Canadian and one American participating hospice sites. In Ontario, Canada, the sites are Kensington Hospice (KH; Toronto, Ontario), Margaret Bahen Hospice (MB; Newmarket, Ontario) and Yee Hong Hospice (YH; Scarborough, Ontario). In North Carolina, USA, the site is the State Employees Credit Union Jim and Betsy Bryan Hospice Home (SECU; Chapel Hill, North Carolina).
Participants will be the bereaved primary caregivers and the primary staff healthcare providers of patients who died at a participating hospice site. Caregiver inclusion criteria include the following: 18 years of age or older; being the identified primary caregiver for the patient; the patient having died in the past four to six months; and being able to complete study questionnaires and interviews in English. During patient admission, clinic staff will provide caregivers with a study introduction letter and ask if they would be willing to be approached four to six months after the death of their loved ones. Caregivers will have the option to opt out of being contacted further. Four to six months after patient death, willing caregivers will be approached by a research team member by telephone, email or videoconference and provided with study details. Caregivers who agree to participate will provide written informed consent through email or mail.
Staff inclusion criteria include the following: 18 years of age or older, being the primary healthcare provider for a patient who died in the past week, and able to complete study questionnaires and interviews in English. A site lead at each participating hospice will provide staff members with a fact sheet detailing the study; interested staff will be asked to let the site lead know, who will then provide staff contact information to the research team. Interested staff will then be contacted by a member of the research team and provided with study details. Those who agree to participate will provide written informed consent through email.
The study will aim to recruit the following number of participants at each North American site: at the Ontario, Canada, sites, 50 participants at KH, 50 participants at MB and 30 participants at YH; and at the North Carolina, USA, site, 100 participants at SECU (N=230). These are the maximum subsample sizes achievable per site within the 1-year data collection period, given the number of patients treated annually at each location. Using G*Power V.18.104.22.168, a power analysis to detect a convergent validity correlation (bivariate normal model) of 0.70 indicates that the total sample size of 230 will provide more than sufficient (power=1.00). Based on recruitment rates of bereaved caregivers in previous studies conducted by our team, we anticipate that approximately 50% of eligible caregivers approached will consent to participate.
Measures and procedures
In addition to the QODD-RGV, measures will assess caregiver, staff, and patient demographic information and patient medical information, palliative care services available, and standardised, validated measures of the end-of-life experience (Good Death Inventory (GDI)23), family satisfaction with care (FAMCARE)24 and caregiver bereavement (Texas Revised Inventory of Grief-Part 2)25 and distress (Posttraumatic Stress Disorder Checklist for Diagnostic and Statistical Manual of Mental Disorders-5).26 Details of these measures are summarised in table 1.
Caregiver participant procedures
At four to six months after patient death, consenting bereaved caregivers will be emailed a unique link to the study questionnaires or mailed a hard copy of the questionnaire package, along with a preaddressed, stamped return envelope. They will be asked to complete the demographic and other questionnaires (except the QODD-RGV) on their own or over the telephone with study staff. Caregiver participants will also be asked to schedule a time to complete the QODD-RGV with a member of the research team by telephone or videoconference. The entire set of questionnaires will take a total of about 30–40 min to complete, including 20 min to complete the QODD-RGV. Participants who request psychosocial support or who are identified by study staff as being significantly distressed will be referred to bereavement support services at each study site.
Across sites, the first 15–20 caregiver participants recruited for the study will be asked to participate in a cognitive interview when completing the QODD-RGV. After rating each item, interview participants will be asked to verbalise their thought processes about the item and their response to it to ascertain the extent of the item’s importance/relevance to the person’s experience of dying and death and the nature of the response. Since verbalisation of thought processes occurs after ratings are provided, quantitative data from this subgroup will be merged with the study database if no changes to the QODD-RGV are made following the interviews. The interviews will be audio-recorded and transcribed for qualitative analysis.
Staff participant procedures
One to two weeks after patient death, the hospice site lead will identify the two consenting staff participants (one physician and one nurse) who spent the most time caring for the patient and inform the research team. Each site lead will also report on the palliative care services available within their site. A team member will contact the staff participant and confirm their consent to complete a self-administered version of the QODD-RGV. The staff participant will be emailed a unique link to the self-administered staff demographic form and QODD-RGV or mailed a hard copy of the questionnaires that can be left after completion in a drop box. Staff completion of the questionnaires will take about 15–20 min.
All quantitative analyses will be conducted using SPSS V.25. Alpha will be set at 0.05 unless indicated. Each analysis will be conducted separately for each site, unless indicated.
Cognitive interview data analysis will be managed using QSR NVivo software. Verbatim interview transcripts will be analysed using content analysis methodology.27 To ensure validity, comprehensiveness and appropriateness of analysis, data will be analysed separately by two or more members of the research team, who will code transcripts, compare codes and review category descriptions. Qualitative responses will additionally be used to refine items that demonstrate a high level of confusion or misunderstanding.
Descriptive statistics will be calculated to summarise the QODD-RGV item ratings and total score for each site. Floor or ceiling effects (>15% indicating lowest or highest rating, respectively)28 and amount of non-responding for each QODD-RGV item will be inspected to identify problematic items that may require further refinement; these data will also be taken as indices of item relevance (eg, high frequency of nonresponses to an item may suggest that it is a poor item or that the item content is not relevant to many individuals).
To assess internal consistency, we will calculate Cronbach’s α for the 25 QODD-RGV content items. Cronbach’s α of 0.70 is considered the minimum level for acceptable internal consistency.28 Inter-rater reliability of the caregiver and clinical staff ordinal item ratings will be evaluated using Fleiss’s kappa (>2 raters), with a kappa of 0.70 being considered the minimum standard for inter-rater reliability.28
Caregiver and clinical staff item ratings/total scores will be compared using independent-samples t-tests. Given the possibly large number of comparisons in these analyses, a more conservative alpha of 0.001 will be set to indicate significance.
To assess convergent and concurrent validity of the QODD-RGV, we will calculate Pearson’s correlation coefficients (r) between the QODD-RGV total score and total scores of the GDI and FAMCARE, respectively. While no standardised criteria for sizes of validity coefficients to support convergent and concurrent validity have been established, we will consider that convergent validity with the GDI is met if r is large (ie, r>0.70) and concurrent validity with FAMCARE is met if r is moderate (ie, r>0.40). We expect this pattern of validity coefficients to be consistent across hospice sites, which would enhance confidence in validity.
Participating hospice site
Data collection for the Malawi arm of the multisite study will be conducted over a 2-year period (projected time frame: November 2022–2024) as part of a collaborative partnership between the University of North Carolina Project-Malawi (UNCPM) Cancer Programme and GIPPEC of University of Toronto. UNCPM is a leader in oncologic research in sub-Saharan Africa and collaborates with Kamuzu Central Hospital (KCH), one of two national teaching hospitals in Malawi that provide cancer treatment. The participating Malawi palliative care site is the KCH Oncology Clinic in Lilongwe, Malawi, which is staffed by the Ministry of Health and UNCPM nurses.
Caregiver participants and recruitment procedures
Participants will be bereaved primary caregivers of patients who died of cancer after receiving care at KCH. Inclusion criteria include being 18 years of age or older, being the primary caregiver of a loved one who died of cancer after receiving care at KCH in the last four to six months, and being able to complete study questionnaires and interviews in Chichewa.
Participants will be recruited from the KCH Oncology Clinic four to six months after the death of their loved ones. A research assistant will contact eligible caregivers by telephone or by a short message service to ask about interest in participation and to provide study details. Caregivers who agree to participate will provide verbal consent that will be documented. Two sets of participants will be recruited: 25 bereaved caregivers for the cognitive interview study, and 200 bereaved caregivers for the validation study. The latter sample size is the maximum achievable within the 2-year data collection period given the number of patients treated annually at the site. Using G*Power V.22.214.171.124, a power analysis to detect a convergent validity correlation (bivariate normal model) of 0.70 indicates that the total sample size of 200 will provide more than sufficient (power=1.00). Based on the recruitment rates for similar studies, we anticipate that approximately 50% of eligible caregivers will consent to participate.
Measures and procedures
All measures will be translated to Chichewa using a rigorous stepwise forward-translation and back-translation approach.29 A panel of bilingual translators will conduct the forward translation, expert panel review and back-translation of measures.
As indicated in table 1, caregiver and patient demographic and patient medical information will be gathered through caregiver or staff self-report and medical chart review. Caregiver participants will complete the QODD-RGV, another measure of quality of dying (APCA African Palliative Outcome Scale (POS)30 31) and a measure of family satisfaction with advanced cancer care (FAMCARE).24
Four to six months after patient death, the research assistant will contact caregiver participants by telephone to complete the study questionnaires in a 30 min interview. All interview data will be collected using an encrypted tablet-based software, REDCap, which is password protected and accessible only to study staff.
After rating each item, interview participants will be asked to verbalise their thought processes about the item and their response to it. Interviewers will also probe to seek greater understanding of responses chosen for each item and of how important or relevant each item is to the patient’s experience of dying and death. If caregivers have difficulty providing a quality rating, they will be asked, ‘Why is this question difficult for you to answer?’ The nature of this difficulty and whether modification of the wording would enhance its understanding or relevance will be explored. Interviews will be audiorecorded, transcribed and translated into English for qualitative analysis.
All quantitative analyses will be conducted using SPSS V.25. Alpha will be set at 0.05 unless indicated. The same qualitative, descriptive and internal consistency analyses as described for the North American arm will be conducted with the Malawi QODD-RGV data as well.
To identify conceptually meaningful item clusters, an exploratory factor analysis will be conducted of the QODD-RGV response data, using principal axis factoring with oblique promax rotation; factors will be identified (eigenvalues>1.0) and retained based on inspection of the scree plot and on their conceptual coherence.
To assess convergent and concurrent validity of the QODD-RGV, we will calculate Pearson’s correlation coefficients (r) between the QODD-RGV total score and total scores of the POS and FAMCARE. While no standardised criteria for sizes of validity coefficients to support convergent and concurrent validity have been established, we will consider that convergent validity is met if r with the POS, as a similar measure of quality of life near the end of life, is large (ie, r>0.70) and concurrent validity with the FAMCARE is met if r is moderate (ie, r>0.40).
To compare the quality of dying between North American and Malawi sites, we will first compare characteristics between the two arms (with pooled North American site data) using univariate statistics. We will conduct analysis of covariance (ANCOVA) to compare the QODD-RGV total scores from the Malawian caregivers and North American caregivers, adjusting for characteristics that significantly differ between sites. We will also conduct ANCOVA to compare the Malawi and North American groups on each of the item ratings; given the large number of comparisons, a more conservative alpha of 0.001 will be set.
Ethics and dissemination
Ethics approvals for the multisite North American study were obtained from the University Health Network Research Ethics Board (#21-5143) and the University of North Carolina Institutional Review Board (#21-1172). The North American arm is currently recruiting caregiver and staff participants. Ethics approval for the Malawi study is currently being obtained from the University of North Carolina Institutional Review Board and the Malawian National Health Science Research Committee. The aim of the study is to validate a revised version of the QODD that addresses some of its main limitations and increases its cultural generalisability to palliative and end-of-life care settings across resource-level countries. We hope that a culturally generalisable, caregiver-reported measure of QODD will stimulate much-needed palliative care research in lower-resource settings and thereby inform the essential expansion of palliative care policies and services. Findings from the studies will be disseminated to palliative care clinicians and scientists through publication in peer-reviewed journals and conference presentations.
Patient consent for publication
EA and AT are joint first authors.
Contributors EA, AT, KM, KW and GR developed the QODD-RGV. EA and GR developed the initial draft of the North American protocol, in consultation with WL, MC, AB, NJ and KM on methodological and statistical details. AT, MC and LT developed the initial draft of the Malawi protocol, in consultation with GR and KM on methodological and statistical details. KM wrote the initial manuscript draft by compiling the North American and Malawi protocols, and EA, AT and GR reviewed the initial draft. CN, EN, SH, CZ and MG reviewed all subsequent drafts and provided critical methodological feedback. All coauthors approved the final draft.
Funding The North American arm was supported by internal funding from the Global Institute of Psychosocial, Palliative and End-of-Life Care (GIPPEC), Princess Margaret Hospital, University Health Network, and University of Toronto, Toronto, Ontario, Canada. The Malawi arm was supported by an internal collaborative research grant from GIPPEC and the Department of Medicine, Division of Palliative Medicine, and the Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada. No award numbers were provided for the internal funds/grant.
Disclaimer Funders were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.