Article Text

Original research
Values informing the development of an indicator of appropriate diabetes therapy: qualitative study
  1. Christina M LaVecchia1,2,
  2. Victor M Montori2,3,
  3. Nilay D Shah4,5,6,
  4. Rozalina G McCoy4,5,7
  1. 1School of Arts and Sciences, Neumann University, Aston, Pennsylvania, USA
  2. 2Knowledge and Evaluation Research (KER) Unit, Mayo Clinic, Rochester, Minnesota, USA
  3. 3Division of Endocrinology, Diabetes, Metabolism, and Nutrition, Department of Medicine, Mayo Clinic, Rochester, Minnesota, USA
  4. 4Division of Health Care Policy & Research, Department of Health Sciences Research, Mayo Clinic, Rochester, Minnesota, USA
  5. 5Mayo Clinic Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery, Rochester, MN, United States
  6. 6OptumLabs, Cambridge, MA, United States
  7. 7Division of Community Internal Medicine, Department of Medicine, Mayo Clinic, Rochester, Minnesota, USA
  1. Correspondence to Dr Rozalina G McCoy; mccoy.rozalina{at}


Objectives Despite increasing focus on individualised diabetes management, current diabetes quality measures are based on meeting generic haemoglobin A1c thresholds and do not reflect considerations of clinical complexity, hypoglycaemic susceptibility or treatment burden. Our team observed a multidisciplinary stakeholder panel tasked with informing an appropriate diabetes therapy indicator (ADTI) and analysed their deliberations, seeking to understand what constitutes appropriate diabetes therapy and how it can be captured using an operational quality indicator. We focused specifically on factors the panel valued in an ideal indicator, how they defined appropriateness and how they thought an indicator of appropriateness could be operationalised.

Design Qualitative study examining Delphi panel deliberations as it iteratively refined the ADTI.

Participants and methods The 12-member panel was comprised of clinicians (endocrinology, primary care, geriatrics), pharmacists, nurses, researchers, and representatives of public and private health plans. It met for four teleconference calls and deliberated asynchronously using semi-structured questionnaires following each call to develop the ADTI. These semistructured questionnaires, as well as the meeting minutes, were then analysed using an inductive thematic approach.

Results We identified three themes in panellist discussions that represented the core value systems underpinning the indicator and its formation: (1) promoting individualised, evidence-based and equitable care; (2) balancing autonomy and prescriptiveness in clinical decision-making; and (3) ensuring an accurate, reliable and practical indicator. These three principles were operationalised into definitions of treatment intensity and clinical complexity, and yielded an indicator that participants judged both fair and effective.

Conclusions Better understanding of what multidisciplinary stakeholders perceive as appropriate diabetes management can help develop quality indicators that are patient-centred, evidence-based, equitable and pragmatic across a range of clinical settings.

  • general diabetes
  • qualitative research
  • quality in health care

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

  • Engagement of a multidisciplinary panel of 12 experts representing diverse backgrounds and perspectives in terms of institutions, professions, gender, and race yielded a rich understanding of the values underpinning appropriate diabetes management and an ideal indicator of diabetes care quality.

  • The main limitation of this study is the small number of participants, which may limit the generalisability of study findings.

  • We relied on panellists’ written responses to a series of open-ended and closed-ended questions disseminated after each call, rather than verbatim recordings of the calls themselves.

  • Inferences drawn from this study represent the values of professional experts and may not reflect the values of people living with diabetes.


The goal of diabetes management is to prevent immediate (eg, hypoglycaemia and symptomatic hyperglycaemia) and long-term (eg, microvascular and cardiovascular) complications. In pursuit of this goal, clinical guidelines recommend that glycaemic targets and the medications used to achieve them be individualised to balance the anticipated benefits of these targets with the potential harms of therapy, given each person’s clinical and psychosocial situation.1–4 At the same time, and in an effort to enable high-quality diabetes care, a variety of performance measures have been introduced for use in public reporting and pay-for-performance reimbursement. Such measures can help identify gaps in care quality, spur and support quality improvement efforts, reward high performers, motivate those lagging behind and inform patients’ decisions about their healthcare. Yet, despite the increasing focus on individualisation of diabetes care,1–4 current quality measures are based on meeting a generic glycosylated haemoglobin (HbA1c) threshold5 6 and do not reflect considerations of clinical complexity, hypoglycaemic risk or treatment burden.7 In order to develop more patient-centred measures, we sought to understand what would constitute an ideal quality measure of appropriate diabetes therapy from the perspective of clinicians, researchers and payers.

Grounding quality measure development in the experience of clinicians directly caring for people with diabetes, experts in diabetes management and payers for healthcare is important for ensuring high-quality care. Yet, quality measures are often developed with minimal input from clinicians and others at the frontlines of care.8 9 This can result in measures that do not reflect the totality and complexity of medical care,8 contributing to a potential disconnect between high-quality care and performance on quality measures.

To mitigate these concerns, our team worked to construct an indicator of appropriate diabetes therapy that could more effectively support high-quality and evidence-based care for people with type 2 diabetes across a wide range of clinical complexity.10 However, the first step in developing this indicator was to understand how a diverse multidisciplinary team of stakeholders understood the concept of appropriateness and what, to them, constituted a framework for an ideal quality measure of appropriate diabetes therapy. The indicator had to be consistent with the National Academy of Medicine’s broad definition of quality as care that is safe, effective, patient-centred, timely, efficient and equitable.11 These stakeholders included clinicians, healthcare administrators, researchers and payers. Thus, as the stakeholder panel completed its deliberations on constructing the appropriate diabetes therapy indicator (ADTI), we turned to a qualitative analysis of its process of creation.

Herein, we present this qualitative thematic analysis of the stakeholder panel’s deliberations as they discussed and refined this ADTI. We specifically focused on elucidating the factors the panel most valued in an ideal appropriateness indicator, how they defined appropriateness, and how they envisioned an indicator of appropriateness could be operationalised so as to preserve its primary objective of ensuring high-quality, patient-centred and evidence-based care. This analysis also revealed how participants—stakeholders who may either evaluate with or be evaluated by this indicator—think about quality measures and how the formation process behind the indicator unfolded. Addressing what participants found most important, as well as concerns identified by people with diabetes in subsequent research, may produce a quality indicator that strikes a patient-centred balance between glycaemic control targets and the everyday complexities of being, and treating, people with diabetes.

Research design and methods

This is a qualitative study examining the deliberations of a multidisciplinary team of experts as they came to consensus about what constitutes appropriate glucose-lowering therapy for adults with diabetes, how appropriateness can be measured and how such an indicator should be developed. It was conducted and reported in accordance with Standards for Reporting Qualitative Research reporting guidelines.12

We worked with OptumLabs to convene a 12-member stakeholder advisory panel to represent diverse perspectives in healthcare, including but not limited to clinicians, data analysts, quality improvement professionals in payer and practice settings, practice or facility administrators, public payers who work with electronic medical record and quality reporting systems, private payers who work with diabetes measure reporting (HEDIS, STAR and so on), patients/consumers or advocates representing the interest of patients with diabetes, employers or other group purchasers. Participants were selected after an open call for nominations (self or other) emailed to all OptumLabs partner institutions (academic and payer) and members of the NQF Measure Incubator Network. Applications were independently reviewed and scored by the principle investigator (RGM) and project manager on the basis of their statement of interest (quality, impact) and expertise (background, knowledge and experience). Twelve of 19 applicants were selected; this number of panellists was a priori defined to be sufficient and practical for informative feedback and productive conversations.

Members provided feedback through a modified Delphi process to help develop and refine the ADTI.10 Members (online supplemental table) represented a wide range of areas of expertise and institutional profiles, including clinicians (endocrinology, primary care, geriatrics), pharmacists, nurses, researchers, and representatives of public and private health plans. They also represented diverse gender (four men, eight women) and racial/ethnic (seven White, five non-White) perspectives. Focus group conversations consisted of four 90-minute meetings held over teleconference, facilitated by one of the researchers (RGM) and a project manager, who each kept brief minutes. After each focus group conversation, participants were asked to respond in writing to a range of closed-ended and open-ended questions about the most recent definition of the ADTI and its components; specifically, to reflect on that definition and provide guidance on ADTI definition, framing, and any modifications or changes (please see online supplemental file). Each session covered specific aspects of the indicator, addressed feedback and inquiries from the preceding session, and sought to establish consensus on the indicator definition. Modifications to the traditional Delphi process were as follows: (1) response submissions were not anonymous, though the descriptive data analyses of the panel’s responses presented back to the panel were; (2) the panel deliberated on the ADTI definitions during teleconference calls in addition to the asynchronous questionnaires; and (3) the study team provided just-in-time information to the panel in response to questions raised in the questionnaires or during teleconference calls in order to assist with ADTI development.

We examined the facilitators’ meeting minutes and panellists’ written responses using a thematic analysis approach.13–17 A qualitative approach allowed us to gain insight into the perspectives, attitudes, and values of panel participants,13 15–17 who as front-line clinicians, researchers, and payers have had valuable experiences pertinent to quality measurement and implementation. We explored the current and ideal processes of diabetes care quality assessments and what really matters to stakeholders about the quality of diabetes care. The chosen thematic analysis approach provided a flexible, yet rigorous, approach that enabled us to identify salient patterns across our data and draw a range of conclusions15 18–20 regarding how the indicator development process unfolded, what participants value in high-quality care and what they envision for the final indicator.

Two researchers, a clinician-researcher (RGM) and a qualitative researcher (CML), inductively and independently identified codes, which were refined via consensus and developed in a code book consisting of definitions and example quotations to ensure consistent usage. To yield a trustworthy analysis, both researchers fully coded participants’ responses individually. The data were then recoded by both researchers together to establish consensus; any disagreements in coding were resolved verbally. Following multiple rounds of coding the data, the researchers developed themes from the codes. NVivo V.12 (QSR International, Melbourne, Australia) was used for this process to more effectively manage our data and themes and facilitate our understanding of the relationships among our codes, themes and participants.21

Patient and public involvement

This work was motivated by the need for more patient-centred evaluation of diabetes care quality, though not explicitly informed by individual patients’ experience and preference. Patients were not directly involved in the design or conduct of this study. Results were communicated to all stakeholder participants on the conclusion of the research and again prior to manuscript submission for publication, and will be broadly disseminated through peer-reviewed publication.


Working toward an indicator of appropriate diabetes care, the team of panellists focused most intensively on defining what constitutes appropriateness, grounded in scientific evidence and the realities of clinical care and population health management. There was broad consensus that appropriateness reflects care that is neither overly intensive (ie, not overtreatment) nor inadequate (ie, not undertreatment) when considered in the context of each patient’s specific situation. Thus, appropriateness necessitates an examination of patient complexity, treatment intensity and treatment outcomes.

The resulting ADTI was both a process indicator (how the patient is treated) and an outcome indicator (what is the final HbA1c level achieved) that was inherently adapted to reflect the patient’s underlying complexity.10 As the panellists discussed these features of appropriateness, their conversations and written follow-up responses coalesced to reveal three core value systems, which we saw underpinning the development process (as well as the ADTI itself). Those themes were: (1) promoting individualised, evidence-based and equitable care; (2) balancing clinician and patient autonomy with prescriptiveness; and (3) ensuring an accurate, reliable and practical measurement (figure 1). Ultimately, these three principles were operationalised into the panel’s definitions of treatment intensity and clinical complexity, and yielded an indicator that participants judged both fair and effective.

Figure 1

Themes informing an ideal quality indicator.

Promoting individualised, evidence-based and equitable care

In discussions invoking this theme, panellists considered ways to ensure the indicator promoted care patterns that are individualised, relevant to each patient with diabetes, consistent with contemporary best practices and scientific evidence, and equitable to all. This theme primarily captured comments made by participants with backgrounds in medicine (three participants), nursing (two participants) and pharmacy (two participants; see table 1).

Table 1

Participant characteristics and the concerns that came up most frequently in their responses

Panellists focused extensively on ways the ADTI should promote individualised, patient-centred care. As direct quotes provided in table 2 show, participants repeatedly noted that appropriateness should reflect patients’ situations, values and preferences for their care. These considerations of patient-centeredness were most attentive to patients’ burden of treatment and financial concerns, with some considerations related to comorbidities.

Table 2

Selected excerpts from panellists’ written responses that informed each theme and the definitions of clinical complexity and treatment intensity

However, while panellists wanted the indicator to address patient preferences, they were mindful that it should not deviate from the best available evidence. They frequently referenced research data, confirming that they wanted the ADTI to maintain consistency with both peer-reviewed literature and consensus opinions (table 2). Similarly, participants demonstrated a desire to reconcile existing clinical practice guidelines with each other and with the indicator. Last, participant responses indicated that they used other quality measures as a guide during the ADTI formation process, using phrases like ‘consistent with HEDIS’ or ‘what we have done with our measures looking at concomitant diagnoses’.

Finally, panellists sought to ensure that the ADTI would be equitable and would not marginalise any groups based on their age, clinical complexity or inability to pay for the preferred treatment choice (table 2).

Balancing clinician and patient autonomy with prescriptiveness

Panellist responses invoking this theme indicated that they wanted an indicator that would protect the autonomy and agency of both patient and clinician, while simultaneously ensuring that the minimum standard of care is met and reasonable clinical exceptions are allowed. Comments captured by this theme were almost exclusively made by physicians (three participants; table 1).

Responses reveal that the panel was concerned about how the ADTI would impact clinicians’ abilities to individualise treatment and exercise clinical decision-making and/or actions (table 2). For example, when prompted to choose language for treatment modalities in the indicator, one participant stated a preference for terms that ‘give clinicians some room for individualisation’. Another noted concerns about the indicator’s impact on lower complexity patients. When asked to elaborate on a scenario she used to express her concerns, she reflected: “I was thinking about how to give space for clinical judgement without adding ambiguity. It’s something I have struggled with and don’t have an answer [for].”

In parallel, panellists demonstrated concern as to whether the ADTI would impact the ability of patients to have a say in their care (table 2). Meeting minutes additionally documented a concern that HbA1c targets fail to capture patient preferences.

Ensuring an accurate, reliable and practical measurement

This final theme shows participants were attentive to ensuring ease of implementing the quality indicator in clinical practice, safeguarding accuracy and reliability, and avoiding unintended effects or punitive functions. Comments under this theme, which typically examined whether the indicator was fair and reliable, were the most numerous in our data. Physicians (three participants) by far contributed most to discussions that fell under this theme, followed by those with backgrounds in research (four participants) and nursing (two participants) (table 1).

Many of the comments about the practicalities of implementation considered elements like time, effort and burden (table 2). For example, participants discussed how comorbidities and medications would be defined and captured across different clinical settings. Other comments were concerned with documentation, both in terms of reasonable accuracy (“I think the 12-months period preceding the last A1c will be a better guide as health status change frequently in older population”) and completeness (making sure that all relevant comorbidities and treatment modifiers would be captured). Finally, participants reiterated why it is important to allow for clinical exceptions in ADTI application and enforcement, and considered the potential implications of introducing this quality indicator into practice. They often invoked this theme through scenarios illustrating the kinds of unintended effects for patients and clinicians they wanted to avoid (table 2).

In parallel, participants expressed a desire to reduce unexpected effects the ADTI may have on treatment decisions and patient outcomes, particularly if these might lead to clinically suboptimal care. They also wanted to preclude any malfeasance by clinicians and/or organisations, whereby performance on the indicator would outweigh the quality of care delivered.

Coming to consensus on appropriate and individualised/patient-centered diabetes management

These three themes—promoting individualised, evidence-based and equitable care; balancing autonomy and prescriptiveness; and ensuring an accurate, reliable and practical indicator—were used to inform the definition of appropriate and patient-centred diabetes care. The primary task of the expert panel was to come to consensus on what constitutes appropriate, overly intensive, and inadequate glucose-lowering therapy for people with diabetes across a wide range of clinical, personal and societal contexts. Because no such classification existed to date, all conversations centred on the precise definitions of two important concepts: clinical complexity (ie, in what situations would patients warrant less intensive glucose-lowering therapy?) and treatment intensity (ie, what treatment regimens would be considered overly intensive or inappropriately relaxed across the newly defined clinical complexity levels?).

Capturing clinical complexity

When defining clinical complexity, panellists discussed a range of issues, including the types of comorbidities that contribute to complexity, how to define multimorbidity or clinical complexity, and how age may factor into the definition of clinical complexity (table 2). While participants wanted to relax treatment goals and targets for older adults, as recommended by clinical guidelines,22–24 they were also wary of potentially undertreating otherwise healthy older patients. They ultimately made their final recommendations based on empirical data.25

Overwhelmingly, participants brought up the third theme (ensuring an accurate, reliable and practical indicator) when discussing complexity, reflecting their concerns about the technical aspects of implementing the proposed definitions in an efficient, yet still accurate, way. For example, they noted that ‘dementia can be poorly documented’ and ‘the logic for ≥three chronic conditions may be complex for measure programming purposes’.

Capturing treatment intensity

For treatment intensity, participants discussed whether medication use should be allowed or even promoted independent of glucose-lowering properties, including among clinically complex patients who may not benefit from their glucose-lowering effects (table 2). This related specifically to promoting, not penalising, the use of glucagon-like peptide-1 (GLP-1) receptor agonists and sodium-glucose transport protein 2 (SGLT2) inhibitors by patients with cardiovascular or kidney disease even at lower HbA1c levels. Participants also discussed whether there may be situations where all patients are considered as undertreated, no matter how clinically complex they are or how intensively they are treated. Citing clinical guidelines, the panel endorsed considering patients with HbA1c ≥9.0% as potentially undertreated and benefiting from interventions to address gaps in diabetes care quality.

Participants most frequently invoked the first theme (promoting individualised, evidence-based and equitable care) when discussing treatment intensity, specifically with regard to patient situations. In other words, discussions around treatment intensity commonly referenced patients’ circumstances, goals, and preferences for treatment as a rationale for including different treatment intensities and/or targets in the indicator. Finally, they sought to develop an indicator that would be durable to emerging scientific evidence and best practices, stating that “it would be good for us to think about the meaning and relevance of the measure in the near future.”


To better understand the motivations, reasoning and aspirations underpinning what healthcare professionals perceive to be a patient-centred quality indicator of appropriate diabetes therapy, we thematically analysed the written feedback of the multidisciplinary expert panel that was tasked with developing it. We focused specifically on identifying what the panel valued, prioritised and operationalised in such an indicator. Our themes revealed several points of significance, the foremost being the nuance within which medicine operates and the challenge of ascribing a firm value statement (eg, appropriateness) to a dynamic construct at the confluence of patient preference, medical and psychosocial patient context, evolving scientific literature and realities of everyday practice. Ultimately, the three core value systems defining an ideal quality indicator that emerged from stakeholder deliberations were: (1) promoting individualised, evidence-based and equitable care; (2) balancing autonomy and prescriptiveness in clinical decision-making; and (3) ensuring an accurate, reliable and practical measurement.

The panel sought to develop an indicator that would reflect, facilitate, and reward treatment practices that are, in essence, high quality: safe, effective, patient-centred, timely, efficient and equitable.11 This balanced approach is primarily reflected by the first theme of panellist deliberations (‘promoting individualised, evidence-based and equitable care’). Importantly, individualisation was construed to weigh each patient’s clinical complexity (number and severity of comorbidities, symptom burden, burden of treatment, life expectancy), psychosocial complexity and capacity to adhere to a potentially complex treatment regimen. Panellists recognised that treatment costs and financial barriers to care should be considered, though were cautious not to have a separate standard of care for lower income or under/uninsured patients.

The first theme further reveals that panellists were conflicted about the role chronological age should play in defining clinical complexity. While they recognised the correlation between advanced age and life expectancy, panellists—particularly those with clinical training—were clear that older adults are a heterogeneous population and the balance of benefit/harm from an intensive therapeutic approach should be evaluated on a case-by-case basis. Finally, panellists spoke at length about equity and fairness in the ADTI definition and its implementation, ensuring that patients and their clinicians are not disadvantaged on the basis of their age, race, gender or clinical status. We were surprised, however, that most discussions about fairness related to patient age and financial means, rather than implicit or systemic bias that may exist toward women and minority patients.

The second theme (‘balancing autonomy and prescriptiveness in clinical decision-making’) shows the desire to honour the centrality of the patient–clinician relationship. That relationship is predicated on preserving the autonomy and agency of both patients and clinicians so that they can come to a shared understanding about the most appropriate care. We see this as an attempt to operationalise shared decision-making, with mutual concern for both patients and clinicians, rather than a paternalistic regard for clinicians or an overcorrective focus on patients that burdens rather than empowers.26–28 The ADTI therefore seeks to encourage patients and clinicians to make decisions that best fit patients’ lives practically, intellectually and emotionally. And yet, woven into the discussions about shared decision-making were references to clinical guidelines and randomised controlled trials, calling for some temperance of patient and clinician autonomy by the data and expert opinion. This also represented a strong desire by the panellists to conform to existing and evolving guidelines formulated by experts who were not in the ‘room’.

Responses from this theme also reinforced some concerns raised in the literature regarding the quality measurement process.29 They were made primarily by physician panellists, who drew on their personal experiences in practice to inform the indicator development and evaluation process. Most concerns involved situations that may penalise clinicians for delivering evidence-based care that a quality indicator may deem to be ‘inappropriate’ (eg, a patient with heart failure prescribed an SGLT2 inhibitor despite having a low/normal HbA1c or a patient who cannot tolerate metformin and needs to be treated with a sulfonylurea due to inability to afford other non-hypoglycaemia-prone medications) or where appropriateness cannot be assessed (eg, patients with anaemia or recent hospitalisation for whom HbA1c measurement may not be accurate). Our analysis showed that participants tried to account for clinical and non-clinical exceptions to the ADTI, although within the constraints of a practical quality indicator that would be informative and not burdensome on the practice. This balance of the clinical ideal juxtaposed against the realities of modern healthcare permeated throughout the panel’s deliberations.

The third theme (‘ensuring an accurate, reliable and practical measurement’) centred on the practical and logistical aspects of operationalising the ADTI in the real-world setting. Panellists focused on developing concise definitions that could be implemented across a wide range of practices without undue burden, ensuring accuracy and reliability of the indicator, and minimising unintended consequences. This theme included the practical ‘how’ of quality measurement, including which data are to be captured, how they are to be tracked seamlessly and in real-time, and how they can be best reported. Participants, particularly physicians, were concerned about how the indicator may penalise clinicians not meeting it due to factors outside their control. We were surprised to find little resistance to the need for measurement and reporting more broadly, though clinicians in particular sought to protect the ADTI from being misapplied for better perceived performance at the expense of patient health and well-being. They also felt it to be best used as a quality indicator and a mechanism for quality improvement, not for public reporting and performance-based reimbursement.

These themes informed the operational definition of the ADTI and considerations of appropriateness,10 which weighed who the patient is (capturing complexity), what is done to the patient (capturing treatment intensity), and what can be reliably and reasonably documented by clinicians and extracted by the health system. Panellists further sought to ensure that the indicator would be informative to and actionable by clinicians, providing practical suggestions for actions that clinicians can take to improve care and health outcomes. The final indicator therefore included not only its definition, but also a menu of suggested management items for patients who are flagged as potentially undertreated, overtreated or appropriately treated. In our analysis, the discussions of clinical complexity were more nuanced and frequent than discussions of treatment intensity, which is appropriate as patient complexity can often be considered by clinicians as something not under their control.

Moreover, participants recognised the many non-medical factors in patients’ lives outside the clinical domain that can create complexity. Discussions about treatment intensity similarly recognised non-pharmacological approaches to diabetes management and the importance of addressing social determinants of health. Panellists stopped short of recommending that non-pharmacological approaches be included as a ‘treatment’ in the indicator definition, largely due to practical considerations about what can be reasonably documented in the electronic health record and be within the clinician’s purview to implement, particularly for patients with financial and logistical barriers to lifestyle therapies (eg, healthy food, safe space for exercise). Still, this presents a missed opportunity to reward non-pharmacological approaches to care, including social support (eg, referrals to social services, community health workers, food banks), lifestyle management (eg, dietetics and exercise programmes) and mental health. Nevertheless, panellists stressed that the ADTI implicitly promoted non-pharmacological therapy, particularly for the clinically complex, because it would enable better glycaemic control with fewer medications and hence would promote ‘appropriate’ care. Future development of other measures should similarly include discussions of non-pharmacological interventions when discussing appropriate care.

The main limitation of this study is the relatively small number of participants on the panel, though for its initial goal of forming the ADTI itself, the team deemed the panel to be a sufficient and practical size for informative feedback and productive conversations. Further, the 12 selected panellists represented diverse backgrounds and perspectives in terms of institutions, professions, gender and race. While our sample size may limit the generalisability of study findings to all quality measures, this illustrative case reveals some of the core values that informed our quality indicator and thus lends insight into the formation of quality measures more broadly. Second, because our primary objective in convening the panel was to develop a patient-centred indicator of appropriate diabetes therapy (ie, the ADTI), and it was not until the rich conversations with the panel occurred that we discovered how helpful and productive it may be for others to learn from these deliberations, we did not capture the content of each conference call verbatim. Instead, we relied on panellists’ written responses to a series of open-ended and closed-ended questions disseminated after each call, as these represented the most direct and comprehensive feedback each individual participant provided on the indicator based on their experience and expertise. The panellists’ thoughtful and extensive feedback on ADTI development and the implications of its potential implementation into clinical practice therefore provide a window into their individual perspectives. Finally, while patient perspectives and feedback were considered when developing the indicator, contents of a patient focus group were not included in this analysis.

Themes elicited from the deliberations of this multidisciplinary expert panel reinforce the importance of individualised care, scientific evidence, the patient–clinician relationship and practicality. In other words, an ideal indicator—and good care—lie at the intersection of patients, scientific evidence and reality. While any type of measure could meet these criteria, we envision a hybrid indicator that is comprised of both process (to reward what the clinician is doing) and outcome (to reward preferred health outcomes) metrics that allow for partial, rather than all-or-none, success and has both upper and lower bounds of acceptability as clinically appropriate. As the healthcare system shifts to performance-based reimbursement, population health, and alterative care delivery models that prioritise quality and value (eg, Accountable Care Organizations, Patient-Centered Medical Homes), quality measurement will be increasingly important. The themes elicited herein provide a framework for characteristics that healthcare professionals, payers and quality indicator development experts view as important in an ideal patient-centred quality indicator.


Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.


  • Presented at Preliminary data for this publication were presented as a poster presentation at the American Diabetes Association 79th Scientific Sessions in San Francisco, California, USA (7–11 June 2019).

  • Contributors RGM is the guarantor of this work and, as such, had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. RGM and CML designed the study, interpreted the data and co-wrote the manuscript. VMM and NDS contributed to the discussion and reviewed/edited the manuscript.

  • Funding This effort was funded by an AARP Quality Measure Innovation Grant through a collaboration with OptumLabs and the NQF Measure Incubator (RGM) and the National Institute of Health National Institute of Diabetes and Digestive and Kidney Diseases grant number K23DK114497 (RGM).

  • Disclaimer Study contents are the sole responsibility of the authors and do not necessarily represent the official views of NIH.

  • Competing interests None declared.

  • Patient consent for publication Not required.

  • Ethics approval Mayo Clinic Institutional Research Board deemed this study exempt from review, as participants were subject matter experts who provided general recommendations for quality indicator development and the project posed minimal risk of harm to participants.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data availability statement Anonymised documents used for qualitative analyses are included in the Supplement. Codebook used for the analyses is available upon reasonable request by emailing RGM (

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.