Article Text
PDF
Abstract
Objective To identify the key themes for evaluating the quality of initiatives to engage patients and family caregivers in decision-making across the organisation and system domains of healthcare systems.
Methods We conducted a scoping review. Seven databases of journal articles were searched from their inception to June 2019. Eligible articles were literature reviews published in English and provided useful information for determining aspects of engaging patients and family caregivers in decision-making to evaluate. We extracted text under three predetermined categories: structure, process and outcomes that were adapted from the Donabedian conceptual framework. These excerpts were then independently open-coded among four researchers. The subsequent themes and their corresponding excerpts were summarised to provide a rich description of each theme.
Results Of 7747 unique articles identified, 366 were potentially relevant, from which we selected the 42 literature reviews. 18 unique themes were identified across the three predetermined categories. There were six structure themes: engagement plan, level of engagement, time and timing of engagement, format and composition, commitment to support and environment. There were four process themes: objectives, engagement approach, communication and engagement activities. There were eight outcome themes: decision-making process, stakeholder relationship, capacity development, stakeholder experience, shape policy/service/programme, health status, healthcare quality, and cost-effectiveness.
Conclusions The 18 themes and their descriptions provide a foundation for identifying constructs and selecting measures to evaluate the quality of initiatives for engaging patients and family caregivers in healthcare system decision-making within the organisation and system domains. The themes can be used to investigate the mechanisms through which relevant initiatives are effective and investigate their effectiveness.
- health services administration & management
- health policy
- quality in health care
- rationing
- public health
Data availability statement
Data are available upon reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- health services administration & management
- health policy
- quality in health care
- rationing
- public health
Strengths and limitations of this study
A comprehensive scoping review that produced a list of themes to evaluate initiatives that engage patients and family caregivers in healthcare system decision-making within the organisation and system domains.
Multiple stakeholders, including patients and family caregivers, were involved in the research process to identify and describe the themes.
Experts outside of our research team were not consulted to provide feedback on the identified themes.
Introduction
Healthcare systems have begun to embrace a person-centred and family-centred approach to better meet the priorities of their users.1–3 This approach is an opportunity for healthcare users and their unpaid caregivers, such as family members, to partner with employees, such as managers and professionals, in shaping the healthcare system.4 5 One way the Ministry of Health in British Columbia, Canada is striving to achieve this is through the Patients as Partners Initiative launched in 2008. This initiative is meant to build capacity for and strengthen the engagement of patients and family caregivers in healthcare system decision-making exercises.6–8 The Patients as Partners Initiative operates within the three domains of the health system: direct care or individual domain (micro-level), organisation or community domain (meso-level) and system or policy domain (macro-level).8 Ultimately, the Patients as Partners Initiative’s goal is to support achieving the quadruple aims of optimal patient and provider experience, better health outcomes, and better cost-effectiveness.9
Evaluation is required to determine the extent to which policy-driven patient and family caregiver engagement initiatives like the Patients as Partners Initiative are effective. In this paper, ‘patients and family caregivers’ simply describes the many categories of individuals and groups served by healthcare systems.10 In a 2009 scoping review on public engagement in healthcare priority setting and resource allocation, Mitton et al noted that engagement exercises are rarely evaluated.11 The Manafò et al scoping review a decade later, likewise, noted that limited evaluation has hindered patient and public engagement in decision-making exercises.12 We, therefore, do not have adequate information on what works and does not work for effective, sustainable and productive patient and family caregiver engagement.13
Challenges to evaluating patients’ and family caregivers’ engagement in healthcare decision-making include a lack of consensus on which elements are of importance, reliable and valid measures, and appropriate evaluation tools and methods.14 15 It is not clear the extent to which available questionnaires, checklists and other measurement tools address the key themes of patient and family caregiver engagement since those important themes have not yet been adequately mapped out to define success of these types of initiatives. This is a major gap in enabling evaluation, as these themes would provide a way for thinking about how patient and family caregiver engagement initiatives can be, and should be, evaluated.
We propose the existing gaps can be addressed by a conceptual evaluation framework for determining indicators to monitor and evaluate for initiatives engaging patients and family caregivers. Such an evaluation framework could, furthermore, be used to establish an agenda for research, policy and quality improvement activities on patients’ and family caregivers’ engagement in decision-making in healthcare systems. As a first step, we sought to identify the key themes for evaluating the quality of initiatives to engage patients and family caregivers in decision-making across the organisation and system domains (ie, meso-level and macro-level) of healthcare systems. The individual domain (micro-level) was excluded due to the unique and personal nature of engagement in one’s own healthcare.
Methods
Patient and public involvement
Patient and family caregiver partners engaged as research team members throughout this scoping review.10 16 Patient partners (AK, ALK and VB) joined individually when recruited from a provincial community of volunteers and were offered a CA$35/hour honorarium as reasonable compensation in line with guidelines by regional health research organisations. AK is a retired male with chronic pain for over 10 years. JM, our family caregiver partner, is a Caucasian female in her mid-60s with experience caring for two patients and works for a provincial non-profit dedicated to family caregivers. AK is a Caucasian female in her late 30s with a disability, and who is a former licensed practical nurse and current master’s student who is a patient advocate with extensive engagement experience and interactions with the healthcare system. VB is a visible minority long-term male patient and a survivor of childhood and young adult cancer. He is very active in patient engagement activities, such as making conference presentations and being a community advocate for physical and mental health. ALK and JM joined at the beginning and were actively involved in virtual team discussions and provided feedback on written documents. AK resigned for personal reasons after a year, having participated in three team meetings and copresented the research protocol to the British Columbia Ministry of Health. Two new patient partners (ALK and VB) joined the research team and participated from the fourth team meeting. Patient partners and family caregiver partners contributed to team discussions and manuscript writing by reviewing and commenting on drafts. For example, they influenced the results by suggesting names and descriptions for themes, ways to combine themes and identifying a missing theme.
Approach
Arksey and O’Malley’s scoping review methodology enhanced by Levac et al guided this review.17 18 The methodology consists of six stages and does not require assessment of the methodological quality of included studies. The current paper is based on the first five stages, from identifying the research question through to reporting the results. Stage 6 is ongoing to consult multiple stakeholders to create a conceptual evaluation framework based on our results. We followed the standards of Preferred Reporting Items for Scoping Reviews and Meta-Analysis Extension for Scoping Review checklist.19 We revised our published review protocol in an iterative process to refine the scope of our research question.10 Scoping reviews do not require ethics approval.17 18
Stage 1: identified the research questions
Our question remained unchanged from our published protocol: ‘What key elements define the quality of patient, family, caregiver, and public engagement in decision-making in healthcare systems for use in the evaluation of a provincial engagement initiative?’10 The key elements were framed as themes for evaluation. In contrast to the published protocol, we focused on engagement within the organisation and system domains and excluded individual domain engagement. The individual domain draws on a body of literature pertaining to self-management and self-care,5 which is outside of the direct context of the other two domains and the scope of this study.
Stage 2: identified relevant studies
A university librarian with expertise in systematic reviews helped to develop the search strategy for seven databases: CINAHL, Cochrane Library, Embase, MEDLINE, PsycINFO, Social Work Abstracts and Web of Science from their inception (to capture any foundational study) through to 14 June 2019. See the online supplemental appendix A for our search strategy. Our search was broad because of the semi-organised and evolving nature of the literature on patients’ and family caregivers’ engagement in healthcare system decision-making.
Supplemental material
Stage 3: selected relevant studies
Four authors (CH, MD, ML and HM) were involved in iteratively screening 7747 unique articles within the Covidence online platform (see figure 1).20 CH removed duplicate articles. Each article was screened by CH and either MD or ML. The titles and abstracts screening followed the published eligibility criteria. Selected articles were (1) published in English, (2) described patient and family caregiver engagement within healthcare systems and (3) provided useful information that could inform aspects of patient and family caregiver engagement in decision-making to evaluate. Additionally, we selected studies conducted in upper-middle-income and high-income countries to align with the British Columbia healthcare system context. Our study topic required us to read, in most cases, the complete abstract before deciding on the article selection. Two to three reviewers (CH, MD and/or ML) discussed the selection of the first 300 articles in sets of 100 articles as a training exercise to achieve a common understanding of articles to retain for full-text screening.
Flow diagram for study inclusion.
We refined eligibility to include literature reviews and articles that included models, frameworks, recommendations, guidelines, questionnaires and other knowledge tools, as well as case studies and evaluation studies. An initial round of full-text screening was done by CH and MD. During the second round of full-text screening, training for consistency was done in which MD, ML and HM each reviewed 100 articles in duplication with CH. After resolving conflicting selections, MD, ML and HM provided verbal rating to CH on “How confident are you to proceed with correctly selecting articles” using a 10-point scale (1—not confident to 10—very confident). When each reviewer was below 9 in confidence, each screened another 100 articles together with CH and discussed conflicts to obtain 100% negotiated agreement. A high level of confidence among reviewers provided assurance that each understood the factors contributing to the negotiated agreement and would lead to the appropriate selection of articles. Following the standards for scoping reviews, we did not assess the methodological quality of the included studies.
Stage 4: collected data
Full-text of 366 relevant articles were uploaded into NVivo software (V.12, QSR International). MD, ML and HM coded the articles first by applying the predetermined overarching categories of ‘structure’, ‘process’ and ‘outcome’ as defined in our protocol paper and adapted from the Donabedian conceptual framework.10 21 The Donabedian conceptual framework was used because it is widely accepted as a standard for guiding systematic evaluation of the quality of healthcare and fits well with the overarching categories of a logic model.21 22 The reviewers read each article and then coded appropriate content starting at the results section through to conclusions. The research team then performed open-coding of the content captured within the three overarching categories to identify underlying themes.
Initially, each reviewer read all of the excerpts and independently open-coded the same set of excerpts to an equivalent of five pages. The reviewers then met and discussed the identified codes to create a common understanding of how to label the codes for the purpose of identifying themes appropriate for guiding the evaluation of patients’ and family caregivers’ engagement in health system decision-making. These discussions led to resolving differences in the sections coded, names given to codes and the definition of each code. This training was repeated once again using several more pages of excerpts. After this training on open-coding and creating an initial coding scheme, HM, ML and MD each open-coded the excerpts for structure, process and outcome, respectively. CH met one-on-one with HM, ML and MD to discuss and finalise their codes. They combined and renamed codes as themes by changing terms from the articles to align with concepts that fit the study objective and redistributing some excerpts to more appropriate themes.
Our research team of multiple stakeholders, listed as the authors in this article, met and discussed the preliminary themes identified and their corresponding excerpts. The discussions led to combining some themes within their overarching categories, renaming some themes and adding one new theme. The eligibility criteria were refined to specify literature reviews as the basis for identifying and describing the themes. Furthermore, the reviews included had to have used a systematic search of bibliographic databases. Many of the reviews identified had already synthesised much of the content of other types of articles.
Stage 5: summarised and synthesised results
NC, MD and CH used the themes and corresponding excerpts to draft an initial description for the themes under structure, process and outcome. The summaries were primarily based on literature reviews as higher-order knowledge to create ‘themes for evaluation’ as a useful tool, and were supplemented by non-reviews for richer descriptions. After reading all of the relevant articles, 37 non-review studies (see online supplemental file) were used when they provided additional and unique information for each theme’s description. The research team members further refined the descriptions of the themes through discussions by email, team meetings and one-on-one phone calls.
Results
Figure 1 shows 7747 unique articles retrieved. Eighty-six of the excluded articles, including five reviews, were on low-income or middle-income countries. We selected 42 literature reviews from the 366 relevant articles. The characteristics of these literature reviews are summarised in tables 1–4. They were published between 2002 and 2019 (29 of 42 in 2015 or after) by lead authors from 10 countries (13 from Canada, 8 from Australia, 7 from UK, 6 from USA, 2 from Netherlands, 2 from Austria and 1 each from Belgium, Germany, Iran and Spain). While the titles and objectives of many of the reviews stated a target population of either patient or public, the methods sections typically described a broader inclusion of both patient and public, including caregiver, carer, citizen, consumer, family and services user.
Characteristics of the included literature review articles dated between 2019 and 2018 (n=11)
Characteristics of the included literature review articles dated 2017 (n=9)
Characteristics of the included literature review articles dated between 2016 and 2016 (n=9)
Characteristics of the included literature review articles data between 2014 and 2002 (n=13)
We identified 18 key themes which are summarised in figure 2 and described below. Of the 18 themes, 6 corresponded with the structure category, 4 with process and 8 with outcome.
Infographic of themes for evaluating initiative to engage patients and family caregivers in healthcare system decision-making.
Structure themes
Structure comprises the settings or contexts within which engagement occurs, such as organisational structure, materials and human resources.
Engagement plan
Evaluate engagement initiatives for having clear goals, purpose, guidelines and limits of decision-making for effective engagement.13 23–34 Engagement plans should include guidance needed for initiatives to be successful, such as identifying the level of engagement, addressing power imbalance and outlining clear mechanisms for cultural safety, listening, inclusivity, non-discrimination, representation, transparency of organisation leadership and a safe environment for patients and family caregivers to contribute.15 25 28 35–39 There are several best practices, models and frameworks to use in an engagement plan. Cleemput et al’s40 Preferred General Model for Public and Patient Involvement, as an example, can be used to map plans for engaging patients and family caregivers in healthcare policymaking. As another example, Kohler et al used the co-design methodology to guide patients and family members to partner with employees to co-design a patient engagement strategy for primary healthcare.41
Level of engagement
Assess for the use of appropriate levels of engagement. Many frameworks have varying terminologies to describe levels of engagement. Common among the frameworks is a spectrum of control or influence that patients and family caregivers have on the decision-making process and outcomes.11 15 40 42–47 Typical levels in order of increasing control include inform, consultation, involvement, collaboration, and empowerment or lead, which are suitable for different purposes and strategies for engagement to be meaningful.35 43 48
Time and timing of engagement
Evaluate the adequacy of the amount of time spent engaging patients and family caregivers and appropriateness of the timing of engagement, including scheduling of decision-making exercises.13 35 37 43 49 One can report if sufficient time was available for engagement.29 37 43 49 In terms of timing, evaluation should take into account the frequency and whether or not engagement was done early, one time, recurring, ongoing, or in stages of a decision-making process.11 13 31 35 37 40 43 50–52
Format and composition
Format pertains to decision-making groups within which patients and family caregivers engage. Some examples include advisory panels, governance boards, citizen councils, citizen juries, community forums, community health councils, patient and family advisory councils and boards, mixed advisory committees, patient organisations and other organisation committees, such as steering committees.13 33 42 53–58 Composition pertains to representation based on the format employed, the number of patients and family caregivers engaged, and demographic factors relevant to initiatives.11 13 15 32 37 43 48 57 59–61
Support
This theme involves evaluation of how organisations support patients and family caregivers in their roles.26 35 43 62 63 Support may include training and resources, such as funding, incentives, time, support staff, facilitative leadership, information, information technology infrastructure and tools that enable contributions to decision-making.15 24 29 34 35 41 48 59 63–67 Support processes and strategies established through senior leaders’ buy-in can improve the sustainability of engagement initiatives.34 41 62 Success of initiatives may depend on the training of all stakeholders—both patients/family caregivers and staff/professionals.15 30 37 40 65
Environment
This theme pertains to the physical or virtual location and socio-cultural conditions, which create the spaces of power within which engagement takes place.35 43 68 Environment includes how welcoming the spaces are to input from patients and family caregivers.35 68 69 This could include cleanliness, physical props, accessibility to marginalised groups, such as people with disabilities, and the extent to which stakeholders promote reciprocity, cultural safety and open communication.25 30 36 39 69 Engagement within virtual spaces have benefits and challenges.61 70–72 An enabling or engagement-capable environment that supports teams and removes barriers in engaging patients is considered essential for acting on issues of equity.73 74
Process themes
Process denotes the methods or mechanisms by which engagement occurs.
Objectives
Evaluation should assess the extent to which patients and family caregivers contributed to setting the objectives and agenda of engagement events with respect to the level of engagement.15 43 69 This is different from goal and purpose, which focus on the overall initiative, rather than the engagement process for specific issues. The literature highlights the importance of stakeholder collaboration in developing the objectives and agenda for an engagement event based on the shared values and vision of all partners.43 69
Engagement approach
Engagement approaches range from top-down to bottom-up models and involve a variety of theories and methods used in dynamic ways to engage stakeholders.43 46 48 75–77 Top-down approaches typically promote decision-making led by healthcare system organisations, and are considered less likely to completely represent patients’ and family caregivers’ perspectives.46 75 78 Partnership approaches have multiple loci of power and learning cycles that promote stakeholders’ insights.76 77 Bottom-up approaches seek to promote collaboration and are often led by individuals or communities, such as representatives of advocacy organisations who demand health policy changes.50 76 78
Communication
Information flows through deliberative or non-deliberative processes.11 12 38 59 63 Deliberation is iterative discussions that enable participants to think, reflect, question and provide points of views to uncover knowledge gaps and make consensus-based decisions.38 56 A timely two-way dialogue facilitates problem-solving and allows stakeholders to find common ground.67 69 Respectful and reciprocal discussions among parties occur when they actively interact and present their interests and values equitably in easily understood language.23 43 Characteristics of communication to evaluate include openness, respectful and reciprocal discussions, transparency about the use of stakeholders’ input, timely sharing of information on decisions made and inclusiveness of varying points of view.15 23 36 43 79 Organisations may exhibit a spectrum of responses to engagement that would indicate the extent to which communication was authentic and transparent from basic acknowledgement to detailing the use of stakeholders’ input in decision-making.36 66
Engagement activities
Evaluation should account for types and dynamics of the broad range of engagement activities by patients and family caregivers. Activities may include, but are not limited to, planning, designing, governing and evaluating health services; developing guidelines; allocating resources; reconfiguring health services; setting priorities; voting; completing surveys and doing key informant interviews; and providing feedback on documents or processes.11 12 15 26 27 29–31 33 35 36 41 51 52 57 60 61 66 69 71 80–82 During these activities, they may take on different roles such as being advisors, committee members, jurors or leads.23 34 41 48 81
Outcome themes
Outcome is the effect of engagement activities.
Decision-making process
Engagement may contribute to a more transparent decision-making process. Engagement provides the opportunity for patients and family caregivers to have a substantive influence on decisions. Key elements to evaluate are the establishment of mutual trust, broadened understanding by stakeholders on relevant issues and the identification of trade-offs; all of these make for a more legitimate process.42 50 80 83–85
Stakeholder relationship
Engagement activities could strengthen the relationship between patients/family caregivers and other stakeholders. Establishing good relationships may involve effective interactions among stakeholders, stemming from enhanced communication, mutual understanding and trust.50 59 Improvement in stakeholder relationships affect those directly involved in the engagement initiatives and individuals in the larger population who have vested interested in decisions made.13 45 50 71 84
Capacity development
Engagement can build on the capacity of stakeholders, particularly patients and family caregivers, by reducing their perceived stigma and enhancing their skills, competencies, confidence, knowledge, awareness, self-efficacy (eg, feeling useful), self-sufficiency and social capital as both individuals and groups.15 25 27 31 48 66 69 74 83 86–90 It can enhance their sense of control over decisions, expand their community network and advance career prospects through improved employability.43 86 90 91
Stakeholders (patient, family caregiver and others) experience
An enjoyable and positive experience in which stakeholders feel valued is important. Such benefits could be reflective of positive characteristics in the structure and process of engagement, and other outcomes, such as mutual learning, skills development, feeling a sense of agency (or empowerment) among stakeholders and therapeutic effects among patients.15 30 31 59 69 86 92–94
Shape policy/service/programme
Engagement enables and is a critical tool for shaping (reconfiguring, redesigning, restructuring or developing) policies, practices, priorities, processes, programmes, services, strategic plans, guidelines, initiatives and other discrete products that are responsive to patients’ and family caregivers’ needs, insights and firsthand experiences.15 27 37 42 43 45 48 69 82 89
Health status
Health status could be identified for populations or individuals within healthcare systems.15 59 60 62 86 When health status is investigated for individuals, the patients and family caregivers who engage directly may benefit, such as having improved mental health.86 Studies have found improvement in the health status of populations, for example, after consumer engagement in policy and advocacy in primary care.69 72 89
Healthcare quality
Improvements in the quality of healthcare received would be consequent to engagement having positive effects on earlier outcomes, such as better accountability, decision-making, programmes and knowledge tools.13 42 51 69 83 89 95 Many articles suggest that improvements in healthcare quality are realised through instrumental outcomes, such as increased acceptability, accessibility, credibility, effectiveness, legitimacy, trustworthiness and usability of policies, programmes and services.13 51 81 84 For example, Sharma et al noted that improved accessibility of programmes developed with advisors within healthcare organisations could translate more effectively into primary care outcomes.86
Cost-effectiveness
Engagement may lead to greater cost-effectiveness.15 42 59 69 82 96 This theme pertains to cost of engagement and engagement producing tangible returns on investment in a way that is responsive to the healthcare needs of a population.55 89 97
Discussions
This scoping review identified a comprehensive set of themes and their descriptions as key elements of patient and family caregiver engagement initiatives that should be considered for evaluation. Each theme is useful when assessing the structure, process or outcome of engagement initiatives within the organisation and system domains of healthcare systems. These themes organise more precise underlying elements of engagement, and are not prescriptive in defining best practices for any initiative. Rather, they provide a basic understanding of key elements of engagement that should be evaluated regardless of the current availability of validated measurement tools.
Generic frameworks used for evaluation could be paired with the identified themes. Structure and process themes could be mapped onto constructs of the Consolidated Framework for Implementation Research (CFIR) to investigate facilitators and barriers to achieving effective engagement of patients and family caregivers.98 99 The CFIR is based on the synthesis of multiple implementation science frameworks and provides a taxonomy of constructs important to the implementation of interventions.98 For example, ‘Support’ could be investigated using underlying themes from CFIR such as ‘Patients needs and resources’, ‘Implementation climate’, ‘Leadership engagement’ and ‘Tension for change’ to identify barriers and enablers to patients’ and family caregivers’ engagement in an initiative.98 Similarly, the generic evaluation framework called Reach, Effectiveness, Adoption, Implementation and Maintenance could complement our outcome themes by investigating, for example, the degree of ‘Capacity development’ and ‘Stakeholder relationship’ for achieving the ‘Maintenance’ of patients’ and family caregivers’ engagement in an initiative.100
The themes are relevant for evaluation at different points during and after engagement initiatives. The outcome themes include all four dimensions of the ‘Quadruple Aim’ (ie, health outcome, experience of patients, experience of professionals and cost of care), but changes in those dimensions may be consequent to other outcomes such as ‘shape policy/service/ programme’.9 Outcomes themes were typically aspirational in the literature. The significance of outcome themes could vary over time and relate to each other in dynamic ways, for example, as mediators and moderators. The use of outcome themes and their underlying elements must therefore be driven by theory about the mechanisms for the effectiveness of engagement initiatives.
A systematic review by Dukhanin et al with search conducted in 2015 reported a ‘Taxonomy of Metrics’ as a comprehensive map of process and outcome metrics with domains and subdomains for evaluating patient engagement initiatives at the organisation-level and system-level of healthcare systems.15 The Taxonomy of Metrics complements the themes we identified. The arrangement of our themes extends their findings by mapping themes not only for process and outcome, but for structure as well. Specifying structure themes adds clarity for evaluating the context in which engagement initiatives occur. Mapping the metrics identified by Dukhanin et al onto the 18 themes would provide some precise elements of engagement to assess. Themes not covered by the taxonomy, such as outcomes in shaping policies as opposed to shaping services identified in the taxonomy, would need other sources for the precise elements.15
Our study is limited by not including the grey literature or contacting experts to identify any missed review articles. The systematic search produced a relatively large sample of literature reviews, which was effective in identifying a comprehensive set of themes for evaluation. Future consultation with experts who study and stakeholders who participate in initiatives for engaging patients and family caregivers in healthcare system decision-making could refine the identified themes and their descriptions. By limiting included studies to those about upper-middle-income and high-income countries, our findings need to be confirmed for generalisability to low-income and middle-income countries. A future study could also investigate the levels of importance and feasibility for conducting evaluation within each theme. An international consensus study of experts could determine the precise elements to evaluate within each theme and possibly subthemes. Finally, given the limited availability of measures for themes specific to patients’ and caregivers’ engagement in healthcare system,14 research should fill this gap.
Conclusion
The 18 themes and their descriptions provide a foundation for identifying constructs and selecting measures to evaluate the quality of initiatives for engaging patients and family caregivers in healthcare system decision-making within the organisation and system domains. The themes can be used to investigate the mechanisms through which relevant initiatives are effective and investigate their effectiveness. Consultation on the identified themes with representatives from groups interested in and affected by healthcare decision-making should refine the themes, build consensus on their level of importance for evaluation and create a conceptual evaluation framework.
Data availability statement
Data are available upon reasonable request.
Ethics statements
Patient consent for publication
Acknowledgments
The authors thank Ursula Ellis, health sciences reference librarian, at the University of British Columbia for help with developing the search strategy and for peer reviewing the search strategy using the PRESS checklist. Thanks to Shannon Holms, the former director of the Patients as Partners Initiative, for contributions to the initial conceptualisation of this study. Thanks to Andrew Koster for his contributions in the early stages of this study.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @Dr_Clayon, @LLi_1
Collaborators Patients as Partners Team: Kristen Grant, Shannon Gibson, Shannon Holms, Sherry Bar and Sirisha Asuri.
Contributors CH obtained funding, conceptualised the research and drafted this review. Several authors (CH, MD, MES, NC, ML, JM, HM, TH, PasP and LCL) made intellectual contribution to the development of the protocol and the acquisition of project funding. Authors (CH, MD, MES, NC, ML, JM, HM, AK, VB, TH, PasP and LCL) wrote the manuscript by reviewing and editing early drafts by CH, and by providing input through team discussions, one-on-one meetings and email.
Funding This work is supported by SPOR Evidence Alliance through funding from the Canadian Institutes of Health Research (CIHR) under the Strategy for Patient Oriented-Research (SPOR) initiative grant number GSR-154442. CH was supported by a CIHR-MSFHR Health System Impact Fellowship from the BC Ministry of Health, Canadian Institutes of Health Research, and Michael Smith Foundation for Health Research (MSFHR). CH was also supported by a MSFHR Research Trainee award. LCL holds the Harold Robinson/Arthritis Society Chair in Arthritic Diseases, and is supported by the Canada Research Chair Programme. She was also supported by a MSFHR Scholar Award.
Competing interests JM is employed by Family Caregivers of British Columbia, a paid partner of the Patients as Partners Initiative and a non-profit organisation that provides support and advocates for family caregivers within British Columbia, Canada. The Patient as Partners Team members are employed by the British Columbia Ministry of Health that provide partial support for this project.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.