Article Text

Original research
Strategies for developing and implementing a rheumatoid arthritis healthcare quality framework: a thematic analysis of perspectives from arthritis stakeholders
  1. Claire Barber1,2,3,
  2. Diane Lacaille3,4,
  3. Marc Hall5,
  4. Victoria Bohm1,
  5. Linda C Li3,6,
  6. Cheryl Barnabe1,2,3,
  7. James Rankin5,
  8. Glen Hazlewood1,2,3,
  9. Deborah A Marshall1,2,3,
  10. Paul Macmullan1,
  11. Dianne Mosher1,
  12. Joanne Homik7,
  13. Kelly English8,
  14. Karen Tsui8,
  15. Karen L Then5
  1. 1 Department of Medicine, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada
  2. 2 The Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada
  3. 3 Arthritis Research Centre Of Canada, Richmond, British Columbia, Canada
  4. 4 Department of Medicine, University of British Columbia, Vancouver, British Columbia, Canada
  5. 5 Faculty of Nursing, University of Calgary, Calgary, Alberta, Canada
  6. 6 Department of Physical Therapy, Faculty of Medicine, University of British Columbia, Vancouver, British Columbia, Canada
  7. 7 Department of Medicine, University of Alberta, Edmonton, Alberta, Canada
  8. 8 Arthritis Patient Advisory Board, Arthritis Research Canada, Richmond, British Columbia, Canada
  1. Correspondence to Dr Claire Barber; cehbarbe{at}


Objectives To obtain stakeholder perspectives to inform the development and implementation of a rheumatoid arthritis (RA) healthcare quality measurement framework.

Design Qualitative study using thematic analysis of focus groups and interviews.

Setting Arthritis stakeholders from across Canada including healthcare providers, persons living with RA, clinic managers and policy leaders were recruited for the focus groups and interviews.

Participants Fifty-four stakeholders from nine provinces.

Interventions Qualitative researchers led each focus group/interview using a semistructured guide; the digitally recorded data were transcribed verbatim. Two teams of two coders independently analysed the transcripts using thematic analysis.

Results Perspectives on the use of different types of measurement frameworks in healthcare were obtained. In particular, stakeholders advocated for the use of existing healthcare frameworks over frameworks developed in the business world and adapted for healthcare. Persons living with RA were less familiar with specific measurement frameworks, however, they had used existing online public forums for rating their experience and quality of healthcare provided. They viewed a standardised framework as potentially useful for assisting with monitoring the care provided to them individually. Nine guiding principles for framework development and 13 measurement themes were identified. Perceived barriers identified included access to data and concerns about how measures in the framework were developed and used. Effective approaches to framework implementation included having sound knowledge translation strategies and involving stakeholders throughout the measurement development and reporting process. Clinical models of care and health policies conducive to outcome measurement were highlighted as drivers of successful measurement initiatives.

Conclusion These important perspectives will be used to inform a healthcare quality measurement framework for RA.

  • rheumatology
  • qualitative research
  • quality in health care

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:

Statistics from

Strengths and limitations of this study

  • A strength of this study was the inclusion of a diverse group of arthritis stakeholders healthcare providers, persons living with rheumatoid arthritis, clinic managers and policy leaders.

  • The results provide key insights into the use of quality frameworks in healthcare and helped to inform the development of a national quality framework for rheumatoid arthritis in Canada.

  • The main limitation of this study is that patients were recruited through advocacy organisations and views may not have been representative of all patients.

  • Demographic data were also not collected on participants to minimise any identifiable information.


A variety of frameworks have been developed to assist with conceptualising and structuring the measurement of the quality of healthcare, the most commonly used or adapted are the Institute of Medicine (IOM) framework,1 Donabedian’s framework2 3 and the Triple or Quadruple Aim Frameworks.4 5 The IOM framework1 defines six domains of healthcare quality (safety, effectiveness, patient-centredness, timeliness, efficiency and equity), Donabedian’s framework,2 3 outlining three essential types of measures capturing the structure, processes and outcomes of care.2 3 The triple aim framework from the Institute for Health Improvement includes: improving the patient experience of care, the health of populations and reducing the per capita cost of healthcare as central to optimising health systems.5 A fourth aim includes improving the experience of healthcare providers.6

Quality frameworks of various types are implemented in different countries and health systems and may be associated with incentivisation programmes. For example, in the UK the National Institute for Health and Care Excellence (NICE) publishes quality standards on ‘high-priority’ areas for quality improvement.7 The standards are linked to evidenced-based clinical guidelines and can be used to audit and improve care.8 Primary care practices in the UK can participate in quality improvement incentive programmes based on NICE indicators through the Quality Outcomes Framework.9 In the USA, a National Quality Strategy was published in 2011 built on the triple aim of providing ‘better, more affordable care for the individual and the community’.10 Currently in the USA physician performance on quality measures is linked to incentives for participating physicians.11 Participation in electronic medical record (EMR)-based registries like the national Rheumatology Informatics System for Effectiveness (RISE) is one way rheumatologists in the USA can monitor quality of care while participating in incentive programmes to improve care delivery12 and identify gaps in care.13 In Canada, the Canadian Quality and Patient Safety Framework for Health Services was recently developed and endorses five goals for safe, quality care including people-centred care, safe, accessible, appropriate and integrated care.14

In the early 1990’s Kaplan and Norton developed a performance management framework called a balanced scorecard (BSC).15 The BSC was originally used in the business world and aligns performance metrics and targets to address an organisation’s strategic objectives towards an overall vision. Classically the BSC framework consists of four interacting domains representing different stakeholder perspectives (customer, financial, internal business and innovation and learning).16 Many healthcare organisations have adapted the BSC framework for use in quality improvement.17–19 An adaptation of the BSC methodology was used in our province to transform quality of care in hip and knee replacement with great success in a non-incentivised programme.20–22 This prompted our investigation of the use of a BSC framework in other clinical areas.

Rheumatoid arthritis (RA) is the most common type of autoimmune inflammatory arthritis affecting up to 1% of the population23–25 and results from the international Global Burden of Disease Study suggest that RA prevalence may be rising.26 In 2016, six performance measures for inflammatory arthritis care were developed to monitor access to care and appropriate treatment in Canada.27 Testing the measures in various provinces revealed significant gaps in access and appropriate treatment at a population level.28 29 The impact of these findings on patient outcomes is unknown. Furthermore, current quality measurement programmes in Canada focus largely on acute care or primary care30 and there is little to guide quality monitoring for complex chronic disease such as rheumatoid arthritis. Therefore, as part of a larger project to develop a more comprehensive quality of care framework in collaboration with the Arthritis Alliance of Canada (AAC) and the Canadian Rheumatology Association (CRA), the present study seeks to understand arthritis stakeholders’ perspectives on the structure and content of such a quality framework, while considering specifically the merits and challenges of a BSC framework.



This project was part of a larger, nationally scoped, mixed-methods programme of research aimed at developing, testing and implementing a quality framework for RA. Here, we report on a subset of the findings focused on impressions of quality measurement frameworks, potential measures for inclusion and perspectives on framework implementation.

Participants, recruitment and sampling method

Stakeholders including healthcare providers HCPs (rheumatologists and allied health professionals, AHPs), people living with RA, clinic managers and policy leaders were recruited between December 2017 and June 2018. Participants were recruited through different means depending on stakeholder type: provincial rheumatology leaders (eg, presidents of regional rheumatology societies and/or rheumatology divisional heads) were asked to identify rheumatologists and clinic managers and regional policy leaders for recruitment. AHPs were identified by directors of publicly funded regional arthritis rehabilitation and therapy programmes and the national Arthritis Health Professions Association. People living with RA were recruited through advocacy organisations including the Canadian Arthritis Patient Alliance, Arthritis Patient Advisory Board, Arthritis Consumer experts and The Arthritis Society. We used purposive sampling to ensure representation of participants based on stakeholder type and region.

Focus group and interview objectives

The objective of the study was to obtain stakeholder perspectives on measurement frameworks in general, including their implementation, facilitators, barriers, benefits and risks as well as potential measures for inclusion in an RA quality framework. Participants were provided with Kaplan and Norton’s BSC framework15 16 and it was verbally explained. Participants were then asked to consider the relevance of this framework to RA care (online supplemental data S1).

Supplemental material

Data collection

Focus groups and interviews

Separate focus groups were held with different stakeholder types. Three focus groups were conducted with 6–10 participants and telephone interviews with smaller interview groups (IGs) of between one and four people. Focus groups lasted about 90–120 min and interviews lasted about 45–90 min. A researcher with qualitative research skills (KT) and a research associate (MH) trained in qualitative methods cofacilitated all focus groups and interviews using focus group and semistructured interview guides adapted for each stakeholder type (see online supplemental material), these were digitally recorded, and transcribed verbatim.

Supplemental material

Data analysis

Transcripts were anonymised and imported into NVivo V.12 for data management and analysis. A six-step thematic analysis31 32 was used. We took multiple steps to meet Lincoln and Guba’s33 concept of trustworthiness including: dependability, credibility, confirmability and transferability. Two independent teams of two coders reviewed the decision-making record and made sure the process was logical. The coding teams met for peer debriefing and feedback on coding and analysis. To ensure dependability and confirmability, we used an audit trail to document decisions. Having two teams coding data independently helped control for biases. Credibility was established with frequent updates and discussion of findings to the CRA Quality Care Committee and the AAC during scheduled teleconferences. Transferability was accomplished from diverse purposive sampling of participants. As well, many team members were trained in multiple disciplines (quality of care, rheumatology, nursing, qualitative methods), which provided balanced perspectives.

Patient and public involvement

Two patients (KE and KT) were partners in all aspect of the study from development of the research question, securing funding, review and interpretation of results and drafting the manuscript and are listed as coauthors on this study. Patient perspectives from consenting individuals living with rheumatoid arthritis were obtained as described above to inform our study results.

Ethical considerations

Ethical approval was granted by the University of Calgary Research Ethics Board (REB16-0556) and all participants provided written informed consent.



In total, 54 stakeholders participated (12 from British Columbia, 17 Alberta, 2 Manitoba, 12 Ontario, 3 Quebec and 8 from the Atlantic Provinces). Three separate face-to-face focus groups were conducted: AHPs at a rheumatology clinic (n=10), AHPs (n=9) and patients (n=6) both conducted at a national rheumatology conference. Thirteen participants had individual interviews (2 face to face and the remainder by telephone) and 17 people participated in the smaller IGs of between two and four people of the same participant type based on scheduling availability all by teleconference (eight rheumatologists, seven patients with one patient participating in both a focus group and an interview, four clinic managers, nine healthcare administrators/policy leaders, two AHPs). An overview of our findings is shown in figure 1.

Figure 1

Overview of identified themes to support development of a quality measurement framework for rheumatoid arthritis.

Frameworks for quality measurement

While there was recognition from some participants that a BSC framework might be broad enough to encompass important concepts relevant to RA care, there was concern that the language may not be appropriate for healthcare as the financial domain is prominent in this framework. Participants indicated that although measuring cost of care was important, policy leaders in particular commented that ‘value for money’ was a more important metric and all participants highlighted the overarching importance of patient outcomes.

The policy leaders and HCPs had the most experience with the BSC framework. However, they more commonly endorsed the IOM1 and the Institute for Healthcare Improvement’s Triple/Quadruple Aim4 5 frameworks as being more ‘comprehensive’ for healthcare quality monitoring (IG 13). One advantage to considering a BSC approach, highlighted by policy leaders, was that it lends itself to seeing the relationship between measures and at times the ‘conflict’ between measures, so that it could be used to ‘balance’ potentially competing demands (ie, patient experience, outcomes, costs etc) (IG 16).

Many patient participants had difficulty conceptualising the nature of healthcare measurement frameworks. What was more familiar to patients were websites where patients publicly rated care from HCPs. Some envisioned the concept of a BSC rather like a scorecard for their individual care that could enhance communication between members of their care team to highlight areas in need of attention.

Guiding principles for framework development

As policy leaders were experienced with quality framework development and implementation, they represented the majority of perspectives on this topic. They elucidated key elements for framework development: patients should be at the centre of the framework, and all care should be patient centred, don’t ‘reinvent the wheel’ (leverage existing work), feasibility is important, compare ‘like with like’, develop long-term improvement goals, use a holistic chronic disease approach, ensure relevance to all stakeholders, the importance of patient outcomes and reporting the data in a way that is actionable. All nine elements are described in table 1 with selected quotations.

Table 1

Guiding principles for quality framework development

Candidate themes for quality measures

Some patients reported participating in clinical studies where questionnaires were collected, which they perceived as ‘good because then they’re gathering the information they need to support me…‘ (Patients IG2). In contrast, other patients experienced little information gathering at all ‘I have no questionnaires… there is not any broad-spectrum information gathering at all’ (Patients IG2). Similarly, some HCPs worked in clinics where data collection for research was common, while others perceived challenges in data collection. Table 2 summarises the types of data and potential measures that were highlighted by the participants together with illustrative quotations.

Table 2

Candidate areas of measurement


Measuring wait times was felt to be an important system-level measure of care quality by all participant types. Time to access a rheumatologist was felt to be important as well as time to treatment. Access to medications, in particular to advanced therapies, was highlighted as important by patients. In addition, the work by Arthritis Consumer Experts, a national organisation providing free evidence-based information and educational programming for patients with arthritis, was mentioned as central in tracking access to advanced therapy nationally through their Arthritis Medications Report Card,34 which provides provincial ratings of access to advanced rheumatology therapies. Access to multidisciplinary HCPs was also a major theme (discussed further below).

Knowing the numbers

In relation to healthcare planning and advocacy, the participants indicated that it was important to understand the distribution of the rheumatology workforce and the numbers of patients requiring care as these elements influence access to care, although challenges in tracking these elements were frequent. For example, some policy leaders reported that they were unaware of the numbers of patients with RA in their province/region and had no processes in place to access such information efficiently. HCPs in different provinces also commented on the inability to define the RA population, usually due to inefficient or non-existent systems for capturing patient diagnosis.

Healthcare utilisation and costs

Costs associated with care were important for all participants; however, they indicated that cost was a potentially difficult measure to address, especially in relation to cost savings with early and improved treatments. Policy leaders described having ‘a good grasp of the numbers in terms of dollars and cents’ (Policy Leader IG18); however, they highlighted the ongoing challenges of balancing financial and clinical outcomes. Measures of healthcare utilisation included emergency visits, hospitalisations and healthcare provider access. Policy leaders and HCPs also discussed measuring the number of patients leaving the province for care in other jurisdictions as a cost that should be tracked.

Patient outcomes

Patients and HCPs described a variety of patient-reported outcomes and composite disease activity scores that are used in rheumatology. Tender and swollen joint counts, patient global rating and composite disease activity scores including the DAS28 were commonly reported as useful for measuring outcomes as part of treat to target strategies. As computer calculated disease activity scores are generally required, the feasibility of routine disease activity calculation was limited for some providers. These disease-specific measures were highlighted as important for drug reimbursement.

The Health Assessment Questionnaire (HAQ)35 is a measure of functional status routinely collected in rheumatology clinics with many HCPs and patients describing routine use. However, patients and providers reported some concerns with the HAQ as they felt it did not adequately reflect the patient’s burden and impact of disease. While quality of life measures (eg, 36-item Short Form Survey, SF-3636) were mentioned by healthcare providers, it was only in context of data collection for research studies, suggesting lesser day to day clinical use.

Pain and fatigue were symptoms that patients felt were important to capture; however, providers questioned the relationship of these outcomes to the care provided and also noted that improved disease activity did not always result in improved pain and fatigue. Furthermore, measures such as fatigue were uncommonly used outside of research projects and some HCPs were sceptical as to the utility of such measures especially given time constraints in practice. Cognitive functioning was another important clinical symptom that patients felt should be monitored. For example, a number of patients referred to ‘brain fog’ as a particularly debilitating symptom that had a significant impact on physical function and social interactions.

Mental and emotional health

Capturing the impact of disease on mental, emotional health and coping was of primary importance to patients. However, the application of specific tools for systematically monitoring mental health infrequently occur. Descriptions from a healthcare provider perspective provided insight that mental and emotional concerns are more often captured through patient–provider interactions than through screening surveys or specific measures.

Concordance with treatment plan

Concordance with treatment plan was discussed by HCPs as important, with some stating this was simply discussed with patients, while in other clinics formal questionnaires of treatment ‘adherence’ are used. Patients discussed the cost of medications as well as delays in getting treatments approved and funded which could contribute to undertreatment.

Patient experience with care

While all participant groups felt measuring patient experience with care was important, they reported it was not commonly measured in routine clinical care. When patient experiences were measured it generally was done as part of a research study or if mandated by the hospital. Patients reported that they were infrequently asked about their experience, but some did report their experience with providers using online public reviews.


The burden of comorbidities in patients with RA including osteoporosis and cardiovascular disease was recognised by HCPs, with some reporting routinely screening and monitoring for cardiovascular risk factors and osteoporosis risk factors using specialised online forms in risk reduction clinics. The capacity to do such advanced screening and monitoring in routine clinics was described as ‘limited’ by other HCPs. Many HCPs agreed at a minimum there should be communication of the increased risk of comorbidities to the primary care provider and/or patient.

Radiographs and laboratory testing

While HCPs recognised that radiographs of joints have some utility (especially early on in the disease course when evaluating for erosive disease), formalised scoring systems were not used frequently in clinical practice and were deemed of limited clinical utility. Patients, however, viewed knowing their laboratory and radiograph test results, and communication of the results among their HCPs, as important in following disease progression.

Less frequently discussed themes: safety, workforce participation and healthcare provider satisfaction, healthcare inefficiencies

There were four less-frequently discussed themes including safety, patient workforce participation and healthcare provider satisfaction and healthcare inefficiencies. While safety was infrequently mentioned by participants as an area of measurement in rheumatology, it was noted by HCPs that adverse medication events are sometimes reported through a voluntary process. One healthcare provider routinely conducted a biologics survey for patients ‘…to make sure they know how to safely take them’ (Healthcare Provider IG10). Transitions when patients were moving between settings of care (eg, posthospital discharge) was mentioned as a possible safety concern by a policy leader who suggested measuring aspects of safe transitions, including evaluating the timely sharing of information and medication reconciliation.

HCPs suggested that important metrics to consider would be work productivity and workforce participation. Potentially, tracking the volume of patients on disability and those on provincial low-income support programmes was also suggested as potential measures worth tracking.

Formal measurement of healthcare provider satisfaction was also discussed infrequently. While a healthcare provider commented that both patient and provider satisfaction were important, that they were not necessarily of equal importance and had different meaning. One manager viewed staff opinions as important to collect as a means of addressing suggestions or concerns for clinic and workplace improvement.

One healthcare provider suggested it would be important to measure lost opportunities, such as patient cancelling/not appearing at appointments without being able to reschedule another patient to attend during the allotted time. Similarly, there are inefficiencies in the system when physicians cancel clinics and do not provide a suitable appointment replacement opportunity. The balance of scheduling new appointments over providing follow-up appointment care was discussed. Another healthcare provider commented on patients being on multiple rheumatologists’ waitlists as an example of inefficiency that could be tracked.

Challenges of measurement

A number of challenges for measurement and reporting were identified by HCPs (table 3). Concerns around data access, privacy, accuracy and data linkage were often described. In some centres, data are routinely collected through EMR for patient care, although this is not universal. Additional data collection solely for the purpose of quality improvement was viewed unfavourably, due to clinical demands. HCPs also raised concerns about how the measures would be used, and by whom, and whether there would be any ability to make changes if deficits were identified.

Table 3

Perceived barriers to implementation of a measurement framework

HCPs also expressed concerns about being measured on ‘outliers’, including patients that were ‘particularly complex’, where outcome measures such as low disease activity or remission were difficult to meet. They suggested a number of factors that could be considered when reporting results that would provide context, such as baseline disease activity and disease duration. While the concept of risk adjustment of measures was not directly discussed, it appeared that HCPs were concerned about potential differences in patient populations that could make comparisons between practices challenging and/or misleading. Being measured on processes of care was more acceptable as was using aggregate data for reporting purposes.

Facilitators and strategies for framework implementation

Participants highlighted strategies that were important to ensure effective quality framework implementation. These are described in table 4 along with selected quotations. At a system level, financial and political will were highlighted as major facilitators. Policy leaders in two provinces detailed successes in measuring surgical wait times for hip and knee replacement surgeries following federal incentives for improving access to care and suggested that a similar process might be developed for RA. At a clinic level, nurse-led models of care were described by some healthcare practitioners as being facilitators for data entry which could support tracking of clinical outcomes conducive to measure reporting.

Table 4

Strategies for effective quality framework implementation

Other facilitators included having effective knowledge translation strategies for messaging of results, using quality measure results as sources of constructive feedback, having clinical champions for quality improvement and robust methods for quality measurement to ensure confidence in results.


The perspectives of arthritis stakeholders have been used to inform the development of a healthcare quality framework for RA. Importantly, although we had initially embarked on a project to develop a BSC-based quality of care framework for RA, following the input from arthritis stakeholders we have determined that a BSC framework did not resonate with them and was not reflective of their needs for quality improvement. While the BSC framework has been widely applied in healthcare, there have been challenges including the need to modify traditional domains of the scorecard to ‘fit’ the healthcare context.19 Our participants indicated that the IOM framework (or a similar healthcare framework) had greater relevance as it required no further adaptation and would be more easily accepted by HCPs. This study has additionally provided important perspectives to consider when developing measures to populate the framework including 13 potential areas for measurement and highlights measurement gaps as well as potential implementation challenges including potential physician reticence to measurement and barriers in accessible data. The study also highlights potential strategies to enhance measurement framework success including political will/incentivisation at a health system-level and clinic-level factors including nurse/AHP-led models of care to facilitate data collection to foster quality improvement.

Our study identified 13 aspects of patient care that could be used for quality measurement including patient outcomes. This work is complementary to existing qualitative studies evaluating important outcomes for patients with RA. Ahlmén et al evaluated patient perspectives on RA outcomes and found four main themes including: normal life, physical capacity, independence and well-being.37 In a similar study Carr et al outlines seven themes: physical, general well-being, independence, return to normality, emotional impact, fear of the future and that the relative importance of outcomes changes over time depending on circumstances.38 In our study, all identified patient outcomes were related to these previously described larger themes (eg, physical function is related to the larger themes of independence and physical health). The challenge from a quality measurement perspective is that many of these patient-important outcomes (eg, general well-being, fatigue, quality of life) are infrequently measured at the point of care. Indeed, a recent systematic review of published quality measures in RA outlines very few quality measures addressing patient outcomes such as pain and quality of life.39 We believe future work to incorporate patient-important outcomes in daily clinical care and quality improvement efforts is central to improving patient-centred care.

An additional barrier related to patient outcome measurement is that HCPs were concerned about being measured on ‘outliers’. Indeed, as outlined in a recent American College of Rheumatology White Paper on Performance Outcome Measures,40 risk adjustment and outcome attribution are among a number of important considerations when developing outcome measures.

As previously described when evaluating a set of system-level performance measures across different models of care in a number of Canadian provinces, data to measure access to care and treatment for patients with RA are not always readily available for quality monitoring.41 In some centres disease activity and patient-reported outcomes are not tracked in EMRs, necessitating the development of a secondary platform for tracking patient outcomes.42 With increasing EMR use by rheumatologists and primary care physicians in Canada, it is anticipated that these data sources could be leveraged in the future for more comprehensive monitoring of quality of care in rheumatic diseases.43 44 In the USA, the EMR-based RISE registry provides an excellent example of passive collection of data for quality improvement.12

Frequently cited successful quality framework and measurement initiatives included the evaluation of hip and knee surgical wait times and care in a number of Canadian provinces.21 22 These successes had a common driver: political will and funding. Similarly, in the UK and in the USA, political policy and financial incentives have led to largescale adoption of arthritis quality measures, although sometimes with mixed results.45 In Canada, despite the recognition of RA as a chronic disease with substantial economic impact, quality of care metrics and patient outcomes are not routinely collected and reported.

This study has provided many useful insights for the development of our RA quality measurement framework; however, there are several limitations. It is possible that perspectives of individuals included may not be representative of other patients and HCPs across Canada. We did not collect demographic data on individuals to minimise any identifying information, this could impact transferability of our results to other specific patient populations. However, the involvement of patient partners in all aspects of study design, interpretation and reporting of results helps ensure meaningful patient engagement in this work. Importantly, while we identified many important areas of measurement in RA, it is likely that not all will be included in the final measurement framework due to data availability and the feasibility of measurement. Finally, the list of barriers and benefits to measurement frameworks may not be exhaustive as they represent our stakeholders experiences and perspectives.


The present study summarises stakeholder perspectives to inform the development and implementation of a quality framework for RA in Canada. Importantly, it also highlights that meaningful measurement and quality improvement should be driven with significant stakeholder engagement throughout the process of measure development, testing and reporting. Further work around advocacy for appropriate data sources, analysis, reporting and models of care to support measure collection and quality improvement are required before widespread implementation of the framework. A patient-specific tool to help patients navigate the elements of high-quality care described in the framework may also be a valuable output to consider in future knowledge translation of this work.


The authors would like to acknowledge the support from the Canadian Rheumatology Association (CRA) in helping provide a venue for the focus groups. As well we would like to acknowledge the CRA and the Arthritis Alliance of Canada (AAC) for supporting this work through updates to their committees and membership. We would like to thank the participants for their time and contribution to this study and also our patient partners involved in all phases of this work (KE and KT).


Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.


  • Twitter @LLi_1

  • Contributors CB, DL, MH, VB, LCL, CB, JR, GH, DAM, PM, DM, JH, KE, KT and KLT designed the study. KLT and MH led the interviews. CB, KLT, VB and MH conducted the primary analysis. CB, DL, MH, VB, LCL, CB, JR, GH, DAM, PM, DM, JH, KE, KT and KLT contributed to interpretation of the results and drafting of the manuscript. All authors approved the final version of the manuscript.

  • Funding This work was funded by a Canadian Institutes of Health Research Project Grant PJT 153265. A Canada Research Chair in Health Systems and Services Research, and the Arthur J.E. Child Chair in Rheumatology Outcomes Research to DMar. The Mary Pack Chair in rheumatology research from The Arthritis Society of Canada and the University of British Columbia to DL. A Stars Career Development Award (SI2-169745) from the Canadian Institutes of Health Research Institute of Musculoskeletal Health and Arthritis and The Arthritis Society to CEHB.

  • Competing interests None declared.

  • Patient consent for publication Not required.

  • Ethics approval Ethics approval for this study was granted by the University of Calgary Ethical Research Ethics Board (REB16-0556).

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data availability statement No data are available. Data are not available as this is a qualitative study and participants did not consent to data sharing for purposes outside the present work.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.