Importance International efforts are being made towards a person-centred care (PCC) model, but there are currently no standardised mechanisms to measure and monitor PCC at a healthcare system level. The use of metrics to measure PCC can help to drive the changes needed to improve the quality of healthcare that is person centred.
Objective To develop and validate person-centred care quality indicators (PC-QIs) measuring PCC at a healthcare system level through a synthesis of the evidence and a person-centred consensus approach to ensure the PC-QIs reflect what matters most to people in their care.
Methods Existing indicators were first identified through a scoping review of the literature and an international environmental scan. Focus group discussions with diverse patients and caregivers and interviews with clinicians and experts in quality improvement allowed us to identify gaps in current measurement of PCC and inform the development of new PC-QIs. A set of identified and newly developed PC-QIs were subsequently refined by Delphi consensus process using a modified RAND/UCLA Appropriateness Method. The international consensus panel consisted of patients, family members, community representatives, clinicians, researchers and healthcare quality experts.
Results From an initial 39 unique evidence-based PC-QIs identified and developed, the consensus process yielded 26 final PC-QIs. These included 7 related to structure, 16 related to process, 2 related to outcome and 1 overall global PC-QI.
Conclusions The final 26 evidence-based and person-informed PC-QIs can be used to measure and evaluate quality incorporating patient perspectives, empowering jurisdictions to monitor healthcare system performance and evaluate policy and practice related to PCC.
- quality in health care
- organisation of health services
- health policy
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
The development of the person-centred quality indicators (PC-QIs) was conducted using a multiphased rigorous scientific process in collaboration with an international team of experts.
The development of the indicators followed the National Quality Forum’s criteria for ‘good quality indicators’.
The perspectives of diverse patients, caregivers and community members were incorporated into the development of the PC-QIs as well as healthcare providers and quality improvement experts.
The study did not include an evaluation of PC-QI implementation.
In 2017, health ministers from Organisation for Economic Co-operation and Development (OECD) countries declared that we need to invest in measuring what matters most to patients.1 2 Ever since patient-centred care (PCC) was first identified as a foundational component of healthcare quality and patient safety by the Institute of Medicine in 2001,3 it has been recognised as a high priority by healthcare systems globally.4–8 The use of valid and reliable measures to monitor and evaluate PCC can provide the data needed to identify gaps in the delivery of PCC and target areas for improvement, and thus drive the changes needed to move towards a true PCC model.
However, there are currently no generally accepted indicators for measuring PCC.9 Moreover, existing indicators do not tend to incorporate the voices of people involved in healthcare, namely, patients, caregivers and healthcare providers.10 This gap means that PCC itself might be measured in ways not relevant to patients and in ways that do not address practical concerns of healthcare providers for person-centred quality improvement.
Evidence also suggests that the delivery of PCC improves healthcare quality, including improvement of patient experiences and outcomes, enhanced involvement of people in their own healthcare decisions, more support for health promotion activities, a decrease in healthcare services utilisation and costs and an improvement in healthcare provider satisfaction.11–14
The overall aim of this research was to ensure that the patient perspective can be used to inform improvements in healthcare quality at the system level by developing a core group of person-centred quality indicators (PC-QIs), based on a synthesis of the evidence and, importantly, includes what matters to patients, caregivers, diverse community members, healthcare providers and quality improvement experts and researchers, when it comes to healthcare.
This study was part of a multiphased programme of research to develop, implement and evaluate PC-QIs for measuring and improving PCC (see figure 1—study at a glance). The development of the PC-QIs included two phases. During phase 1, previously implemented and evaluated PC-QIs were identified and classified using a published PCC framework.10 In phase 2, these identified PC-QIs were refined through a modified Delphi consensus process15 that involved patients, caregivers and diverse community members, clinicians, quality improvement leaders and decision-makers. Phase 3 constitutes future steps of this research and will not be reported in this paper. Phase 3 will involve a feasibility assessment of the newly developed PC-QIs, which will include in-depth interviews with quality improvement leaders and healthcare professionals to obtain their perspectives on the feasibility of implementing the PC-QIs. We will also conduct a survey of healthcare organisations to obtain a system-level perspective on the feasibility of implementation and to obtain greater generalisability of our findings. While this paper provides an overview of the methods used to develop the PC-QIs prior to the consensus process, additional details regarding the first phase of this research (ie, scoping review of the literature, international environmental scan, focus group discussions and interviews with stakeholders) will be published elsewhere.
Patient and Public Involvement
Our research is guided by a transformative framework with the aim of producing knowledge that seeks to improve healthcare for all people, while acknowledging that marginalised groups do not tend to be included in the production of knowledge due to existing power and social relationships within society.16 Thus, consistent with a PCC, we strived to consult with diverse patients and caregivers as active collaborators through a participatory approach—doing research ‘with’ rather than ‘on’ them.
A patient partner (SZ) was recruited to as part of the research team at the study’s inception and to ensure that the study is guided by the patient perspective. The patient partner has played a critical role in the study design, data collection, review of analyses, interpretation of the data and the development of this manuscript, in addition to disseminating the findings of this research. Moreover, in phase 1 of this study, we used a participatory approach to engage diverse patients and caregivers to identify what matters most to them in their healthcare. This would ensure that patient values, needs and preferences are incorporated into the development of the indicators. These focus groups were conducted with the provincial Alberta Health Services Patient and Family Advisory Group as well as ActionDignity (www.actiondignity.org), a community-based organisation that works closely with ethnocultural leaders in Calgary to conduct research and work towards systems and policy change. These organisations supported the development of the focus group discussion tools, recruitment, data collection, analysis and dissemination of the findings. In phase 2 of the study, we used a consensus process with a panel of 29 people of patients, caregivers, diverse community members, providers, researchers and quality improvement leads, which is described in this paper. Participants were routinely asked about burdens to participation to ensure appropriate accommodations.
Phase 1: preliminary review
This first phase involved preparatory work needed for the consensus process. In order to identify, categorise and develop PC-QIs, we developed a conceptual PCC framework10 based on the Donabedian quality of care model (structure, process and outcome).17
This phase also involved a scoping review18 19 to identify 29 previously published PC-QIs, their implementation and evaluation in various settings as well as best practices of PCC monitoring. To be eligible for inclusion, studies/articles had to (1) identify quality indicators for PCC and/or (2) identify PC-QIs in performance measurement (eg, validation).18 Indicators were assessed as being person centred, based on the use of a PCC conceptual framework.10
In parallel to the scoping review,19 an environmental scan was conducted to identify whether healthcare systems in Canada, the UK, Sweden, Australia and New Zealand were using PC-QIs, which PC-QIs were in use, and how they were implemented.20 These countries were chosen as they shared many similarities with respect to healthcare delivery and structures.20 Sixty-one existing indicators were identified. All unique PC-QIs identified through the scoping review and environmental scan were synthesised by the research team.
Focus group discussions with patients and caregivers as well interviews as key stakeholders (ie, quality improvement leads, healthcare providers and PCC researchers) were conducted to inform the development and prioritisation of PC-QIs. With regards to focus group discussions, we employed strategies to attain maximum variation among participants to ensure that the patient and caregiver perspectives represent a greater diversity of people, with considerations for age, race, ethnicity, indigeneity, gender and sexual identities, rural/urban, disease conditions and healthcare sectors accessed. Focus group discussions were conducted in partnership with the provincial Alberta Health Services Patient and Family Advisory Group and ActionDignity (described previously). These organisations supported the recruitment, data collection, analysis and dissemination of findings. We identified healthcare values, preferences and needs from a diverse sample of 65 patients and caregivers. Individual interviews with 22 healthcare providers, quality improvement experts and PCC researchers from Canada, the USA and England were conducted to determine perceptions around feasibility and prioritisation of measuring specific domains of PCC. The findings from these focus groups and interviews were used to identify the most important PC-QIs from those that were found in the scoping review and environmental scan as well as to guide how existing PC-QIs could be modified or refined. The focus groups and interviews also allowed us to identify any PC-QIs still needed for development.
Based on the findings from phase 1 and a review of the literature pertaining to gaps in measurement identified through the focus groups and interviews (ie, access to an interpreter, cost of care etc), 39 unique PC-QIs were developed or modified iteratively by the research team and subsequently classified using the PCC conceptual framework.10 The details and key results from these substudies for phase 1 have either been published or in the process of publication elsewhere.18–20
Phase 2: modified Delphi panel exercise
Phase 2 included a consensus process, using the RAND/University of California LA Appropriateness Method (RAM).21 RAM is a reproducible and valid nominal group technique consensus methodology using the modified Delphi technique.15 This consensus method is used extensively in health services research.21–23 Based on our previous experience,24 25 the consensus method was considered highly appropriate to facilitate the panel’s prioritisation and refinement of PC-QIs. The Delphi technique was modified to include additional people as part of the consensus process, to ensure greater representation from patients, caregivers, community members.
A consensus panel was established consisting of patients, caregivers, diverse community members, healthcare providers, PCC researchers and quality improvement leads. In recognition of the potential power dynamics associated with mixing groups of patients,22 caregivers, healthcare providers, researchers and quality improvement experts, we strove to assemble a panel where at least half of the representation was from patients, caregivers and community members. The community members were representative from the some of the most prominent ethnocultural communities in Calgary (ie, Chinese, South Asian and Filipino). The panellists were identified from their previous participation in the environmental scan, interviews and focus groups conducted in phase 1 as well as our collaborators’ networks. Identified individuals were invited via email and provided with a summary of the project and an overview of the consensus process and expectations (eg, time commitment and activities).
Rating process and materials
Panelists received a package including: (a) a manual that included a monograph of each PC-QI identified and (b) a rating tool used for panellists to rate the PC-QIs (see rating tool here: http://bit.do/PC-QI_RatingTool).
The package outlined the PC-QIs that included descriptors such as: type of indicator, proposed data source (including existing patient-reported experience measures (PREMs) already in use), definition, numerator, denominator, benchmark and risk adjustment. The manual also included definitions to describe what PC-QIs are and what constitutes a ‘good’-quality indicator, as outlined by the National Quality Forum, which states the criteria for evaluating a new measure: importance, scientific acceptability, feasibility and usability.26 Additionally, the rating tool was derived from the Joint Commission Attributes of Core Performance Measures and Associated Evaluation Criteria.27 The rating tool asked panelists to rate PC-QIs on each of the following dimensions:
Was the PC-QI precisely defined?
Does the PC-QI target important PCC improvements?
Does it measure what it is supposed to measure?
Is it a good global PC-QI for overall evaluation?
These questions were designed to assess face, as well as construct validity (ie, whether the PC-QI measures what it is supposed to measure), and appropriateness (whether the PC-QI is an appropriate measure for PCC). The rating materials also included questions related to implementation including feasibility (is data for reporting PC-QI available?) and usability (is the PC-QI actionable and interpretable?). Panelists used SurveyMonkey, a web-based survey tool, to remotely rate the PC-QIs in the first, third and fourth rounds.
Delphi round 1
The first round involved remote rating by panelists. In each of these remote rating rounds, panelists used the rating material described above and the rating scale, a 9-point scale (1=strong disagreement, 9=strong agreement). Overall assessment of the PC-QI scored as: inappropriate (1–3), supplementary (consider as a PC-QI if more resources available) (4–6) and appropriate (7–9).21 Panelists also had the opportunity to provide written comments and suggest additional PC-QIs.
PC-QIs ratings were summarised using medians and IQR for the overall rating included in the globally is it a good PC-QI?’ rating scale. Disagreement on the rating for a PC-QI of at least a third of the panel (n≥9) in the median score 1–3 and at least a third (n≥9) of the panel in the median 7–9. PC-QIs with median overall scores of 1–3 were discarded; PC-QIs with median overall score 4–9 were retained for subsequent rounds. Written comments were analysed using content analysis methods.
Following remote ratings in round 1, the data were analysed, and suggestions and refinements were made to each PC-QI as appropriate. This revised version was shared via SurveyMonkey prior to the face-to face meeting.
Delphi round 2
During round 2, panelists reviewed each PC-QI in a face-to-face meeting as well as the results from the first round. The 2-day meeting was co-moderated by our patient partner (SZ) and a clinician researcher. The moderators led the panel through each of the PC-QIs to review the results from the first round and to note areas of disagreement (indicated by the ratings) as they pertained to: perceived importance/necessity and relevancy for PCC; scientific acceptability (reliability and validity); feasibility of implementing the PC-QIs given different contexts of care and usability to make improvements in care. The research team was available to answer clarifying questions about how the PC-QI was identified/developed and the source of the evidence supporting the PC-QIs. Deliberations were made as a group until agreement on PC-QI specifications was achieved through discussion and subsequent rounds of rerating. Additionally, a number of indicators were proposed for merging and further refinement, particularly for perceived redundancies and in consideration for decreasing the total number of PC-QIs (for feasibility and concerns around indicator fatigue). Modifications were made to the PC-QIs required subsequent rounds of rating.
Remote ratings in rounds 3 and 4 continued as described in the first remote rating to obtain consensus for the modified/merged PC-QIs. Through the remote rounds, revisions of each PC-QI were added to the working document and circulated among panelists for a final rating using paper-based rating tools for final review. Prior to the third round of rating, a working group that was created as a response to the discussions held during the face-to-face meeting, studied the development of an indicator that captures outcomes reported directly by the patients. This working group included five patients, two family members, one physician, two quality improvement leads and two researchers. The group worked on developing the indicator and gathering information to present background knowledge to the rest of panelists. The new indicator and additional information were shared among the rest of the panelists and they rated the new indicator in round 3. Proposed refinements to the PC-QIs during round 3 were agreed on during round 4 of rating, where panelists were also asked specifically about the necessity of each PC-QI, rating either a ‘yes’ or ‘no’ to keeping the indicator.
A total of 29 people participated on the consensus panel. The consensus panel comprised:
Eight patients and three caregivers (37.9%) with various experience with the healthcare system, including primary care, acute care, cancer care and chronic diseases (eg, cardiovascular conditions, diabetes etc).
Five members of diverse ethnocultural communities (17.2%) who are also patients and/or caregivers.
Two healthcare providers (6.9%; an internist and a paediatrician).
Five quality improvement experts (17.2%; representing Canadian Quality Councils and health systems, Cancer Care Ontario, University of Gothenburg Centre for Person-centred Care, Sweden, Picker Institute, and University of Oxford, UK).
Four PCC researchers (13.8%), including: a lead from the Patient-Centered Outcomes Research Institute, a Senior Scientist Collaboration for Leadership in Applied Health Research and Care Oxford Unit, professors from Canadian Universities and leaders of the Canadian Strategy for Patient-Oriented Research.
One representative (3.4%) of the Canadian Institute for Health Information.
One representative (3.4%) from Ministry of Health, British Columbia.
Of the 29 panelists, for round 2, 27 attended in person, 1 attended via videoconference and 1 was absent (with this panelist’s comments shared with the panel). For rounds 3 and 4, 27 panelists participated in the consensus, with two panellists not participating in this phase (one caregiver and one clinician-researcher) due to conflicting commitments.
Person-centred quality indicators
Thirty-nine PC-QIs were identified through phase 1 and were refined through phase 2 of the study, where they were summarised into 26 final PC-QIs. These final 26 PC-QIs included seven structures, 16 processes, two outcome and one global indicators (see table 1). During the first round and based on final ratings (see table 1), four indicators were discarded including:
Timely unplanned readmission from Emergency Department (ER).
Patient-reported outcome multiattribute.
Patient-reported outcome mental health.
Based on the decision to discard the last two PC-QIs, the working group proposed a new indicator related to patient-reported outcomes: ‘using patient-reported outcome measures (PROMs) to deliver PCC.’ This newly proposed indicator was later rated and accepted in Round 3. table 1 summarises the consensus panel ratings as median scores on a 9-point scale and IQR, as reported in the previous studies.21 24 25 The sources of evidence for each of the indicators are also reported in table 1. For additional details on the specific modifications proposed by panelists, please refer to online supplemental appendix 1. During round 2, out of the initial 39 indicators, 11 were merged with other indicators (see table 2). Online supplemental appendix 1 displays the ratings as median scores on a 9-point scale and IQR as as well as details on modifications that were proposed by panellists.
A flowchart of the rating process is found in figure 2.
A complete summary of the final PC-QIs that were developed is available here: https://www.personcentredcareteam.com/s/PC-QIs_Monograph_Santana-et-al-2019.pdf. Each of these newly developed indicators are evidence informed and person centred, some addressing a specific aspect of healthcare quality (eg, safety vs equity).
As Moira Stewart stated in her 2001 editorial: ‘The patient should be the judge of patient-centred care’.28 This article presents a new set of PC-QIs developed and validated through a modified Delphi process that featured the patient perspective on what matters most to them in their care. These PC-QIs are evidence based and patient informed and widely applicable across healthcare sectors and contexts. The use of these standardised metrics to measure PCC can help drive the changes needed to improve the quality of healthcare that is person centred.
The strength of this study is the person-centred approach used to develop metrics to evaluate PCC, which ensures that PCC is evaluated from the perspective of those who provide and receive care. To our knowledge, this is the first study to develop a generic set of PC-QIs using a rigorous evidence-based and person-centred approach and involving the patient and caregiver throughout the research process—from inception to manuscript development. Using a highly participatory approach and a transformative lens, we sought to ensure that the study was guided by the patient perspective, and that diverse and marginalised perspectives were reflected in the development of the PC-QIs. While PC-QIs were identified in the scoping review, first—the vast majority of these was not considered actual indicators in report with quality improvement agencies, such as the Agency for Healthcare Research and Quality (presented as units of measurement, such as percentage or proportion); second—many measures were not developed with significant patient input.29 For instance, previous work conducted by Ouwens et al to develop a person-centred measure for cancer care involved patients in determining what would be important to measure.30 The patient involvement was limited to semistructured interviews to obtain the patient perspective on what guideline recommendations could be used for measuring PCC. In another study related to the development of measures for person-centred cancer care,Uphoff et al involved patients as part of the consensus panel along with medical professionals.31 While this work has been instrumental in demonstrating the value of the patient perspective in developing measures for PCC, only 3 patients were involved, out of 14 experts on the panel. Issues around potential power imbalances were not accounted for. In our study, we strived to have approximately half of our panelists comprised of patients, caregivers and community members, to ensure a balance of perspectives. For most quality indicators that are developed, including those we identified in our environmental scan, PC-QIs tend to be developed based on what healthcare authorities, quality improvement experts or researchers deem as most important for quality improvement. Patients and the public are seldom involved in decisions about quality of care despite being the ones who experience and receive care. How can PCC be truly improved if we continue to measure PCC without the patient perspective on what should be measured? These newly developed indicators present an opportunity to improve healthcare quality in ways that matter most to people. To drive changes in healthcare policy and practice, there is a need to develop and implement standardised ongoing mechanisms to measure and evaluate quality incorporating the patients’ perspectives.14
These PC-QIs offer a tremendous opportunity leverage ongoing initiatives to improve PCC by using data already being collected in many healthcare jurisdictions and helping to standardise the collection, use and reporting of this data. This includes PREMs, such as Hospital Consumer Assessment of Health Providers and Systems, and PROMs. Integrating PC-QIs into performance measurement frameworks can promote actionability for improving PCC. While PREMs and PROMs are often used at the provider level, there is little evidence of use for system-level applications and actionability. These indicators empower jurisdictions to monitor healthcare system performance and evaluate policy and practice related to PCC, while also including the patient’s voice. Finally, the routinised use of standardised metrics, such PC-QIs, to evaluate PCC will help to strengthen the evidence base for the PCC model.32
A key limitation of this research is that these indicators have yet to be operationalised and evaluated in practice. It is only through empirical testing that the feasibility of data collection can be determined and whether they meet the requirements of ‘good-quality measures’—that are acceptable, reliable and valid.33 Moreover, studying the implementation of the PC-QIs can provide important insight into their effectiveness for promoting improvements in PCC as well as patient experiences and outcomes. It is also important to identify any unintended consequences as a result of PC-QI implementation, their use for benchmarking and other issues such as workload and cost-effectiveness.33 Additionally, while this method has generated these 26 PC-QIs using a validated consensus method, they may not necessarily be universally applicable in all countries and settings. Different cultural settings in different healthcare regulatory environments may mean that different measures may be more appropriate for certain settings. Further work can be done to tailor and adapt these PC-QIs, recognising that a consideration of the local context will ensure a more universal relevance.
Future steps for this work include an assessment of the feasibility of implementing these PC-QIs (phase 3, see figure 1). Second, we plan to pilot the use of the PC-QIs in national and international jurisdictions to promote PCC and to evaluate of the use of these measures for improving healthcare quality of care from the perspectives of the patient.
In conclusion, the development of these newly developed evidence-based and person-informed PC-QIs represents an important contribution towards efforts to measure and improve PCC. While these indicators have yet to be evaluated, the PC-QIs are available tools that healthcare systems can use to monitor and evaluate the delivery of PCC, identify the gaps and make the changes needed to improve the quality of care. Importantly, these indicators have the potential to shift our healthcare systems towards a new paradigm for assessing quality by ensuring that we measure what matters most to patients.
The authors are thankful for the contributions and support to this project by Dr V Yiu, Dr G Teare, Dr K Wittmeier, Dr M O’Beirne, Dr N Jette, Dr M Lu, Dr T Sajobi, Dr D Lorenzetti, D LeLievre, L Cuthbertson and members of the consensus panel. The authors thank the investigators, collaborators, and partners of the Person-Centred Care Quality Indicators Study, as well as the research team, who supported data collection and analysis for the consensus process, including: Sadia Ahmed, Chelsea Doktorchik, Ashton Chugh, and Bijan Mohamed. The authors would also like extend their gratitude to Dr Gwen McGhan for her role in facilitating the consensus process, and all of the patients, caregivers, community members, healthcare providers, and researchers who participated in the consensus. The authors would also like to acknowledge Brian Steele, for his support with the submission of this article.
Contributors M-JS, KM, SZ, SB, EG, KL, HQ, and RS contributed to the study concept, study design, and acquisition of funding. M-JS and KM conducted data analysis and the modifications/refinements to the Person-Centred Quality Indicators that are presented in this manuscript. The manuscript was drafted by M-JS, KM, and PF. All authors (M-JS, KM, SZ, SB, EG, JK, LF, SW, PF, KL, HQ, and RS) contributed to the data acquisition and interpretation, critical review and revision of the manuscript for intellectual content. All authors also provided their final approval of this publication and agree to be accountable for all aspects of the work to ensure both accuracy and integrity of this research.
Funding M-JS, HQ, RS, KM, SZ, EG, SB and KL secured funding from the Canadian Institutes for Health Research for this study (Grant# 201709PJT). KM is supported by the Vanier Canada Graduate Scholarship, through the Canadian Institutes for Health Research (Award# 201810CGV). CIHR played no role in the project. The Alberta Strategy for Patient-Oriented Research (SPOR) employs some of the research staff (M-JS, KM, SZ, PF, HQ) to conduct research into patient-centred care, beyond which there was no involvement in the research.
Competing interests None declared.
Patient consent for publication Not required.
Ethics approval Ethics approval was granted from the University Health Research Ethics Boards [(REB15-2846]) at the University of Calgary.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data are available upon reasonable request. Some materials and documentation may be available to other researchers. Please contact the corresponding author with a detailed request.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.