You are here

  • Home
  • Archive
  • Volume 12, Issue 3
  • Primary Care Severe Asthma Registry and Education Project (PCSAR-EDU): Phase 1 – an e-Delphi for registry definitions and indices of clinician behaviour

Article Text

Article menu

Download PDFPDF

General practice / Family practice
Protocol
Primary Care Severe Asthma Registry and Education Project (PCSAR-EDU): Phase 1 – an e-Delphi for registry definitions and indices of clinician behaviour
  1. Katrina A D'Urzo1,
  2. Itamar E Tamari2,
  3. Kenneth R Chapman3,
  4. M Reza Maleki-Yazdi3,
  5. Michelle Greiver3,4,
  6. Ross EG Upshur3,5,
  7. Lana Biro6,
  8. Braden O'Neill3,7,8,
  9. Rahim Moineddin3,
  10. Babak Aliarzadeh3,
  11. Kulamakan Kulasegaram3,
  12. Teresa To5,
  13. Anthony D D'Urzo3
  1. 1School of Medicine, Royal College of Surgeons in Ireland, Dublin, Ireland
  2. 2Stonegate Community Health Centre, Toronto, Ontario, Canada
  3. 3University of Toronto Temerty Faculty of Medicine, Toronto, Ontario, Canada
  4. 4Department of Family and Community Medicine, North York General Hospital, Toronto, Ontario, Canada
  5. 5University of Toronto Dalla Lana School of Public Health, Toronto, Ontario, Canada
  6. 6The Lung Health Foundation, Toronto, Ontario, Canada
  7. 7Department of Family and Community Medicine, St Michael's Hospital, Toronto, Ontario, Canada
  8. 8Li Ka Shing Knowledge Institute, St. Michael's Hospital, Toronto, Ontario, Canada
  1. Correspondence to Dr Anthony D D'Urzo; tonydurzo{at}sympatico.ca

Abstract

Introduction Although most asthma is mild to moderate, severe asthma accounts for disproportionate personal and societal costs. Poor co-ordination of care between primary care and specialist settings is recognised as a barrier to achieving optimal outcomes. The Primary Care Severe Asthma Registry and Education (PCSAR-EDU) project aims to address these gaps through the interdisciplinary development and evaluation of both a ‘real-world’ severe asthma registry and an educational programme for primary care providers. This manuscript describes phase 1 of PCSAR-EDU which involves establishing interdisciplinary consensus on criteria for the: (1) definition of severe asthma; (2) generation of a severe asthma registry and (3) definition of an electronic-medical record data-based Clinician Behaviour Index (CBI).

Methods and analysis In phase 1, a modified e-Delphi activity will be conducted. Delphi panellists (n≥13) will be invited to complete a 30 min online survey on three separate occasions (i.e., three separate e-Delphi ‘rounds’) over a 3-month period. Expert opinion will be collected via an open-ended survey (‘Open’ round 1) and 5-point Likert scale and ranking surveys (‘Closed’ round 2 and 3). A fourth and final Delphi round will occur via synchronous meeting, whereby panellists approve a finalised ideal ‘core criteria list’, CBI and corresponding item weighting.

Ethics and dissemination Ethical approval has been obtained for the activities involved in phase 1 from the University of Toronto’s Human Research Ethics Programme (approval number 39695). Future ethics approvals will depend on information gathered in the proceeding phase; thus, ethical approval for phase 2 and 3 of this study will be sought sequentially. Findings will be disseminated through conference presentations, peer-reviewed publications and knowledge translation tools.

  • respiratory medicine (see thoracic medicine)
  • primary care
  • asthma
  • general medicine (see internal medicine)
http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjopen-2021-055958

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • This is the first primary care severe asthma registry with an embedded educational programme that aims to enhance clinician behaviour as measured by data captured in the electronic medical record (EMR).

    • The development of a Clinician Behaviour Index (CBI) through consensus will allow for an evaluation of how primary care providers manage severe asthma in keeping with guideline recommendations.

    • The development of a CBI will provide opportunities to evaluate how quality improvement initiatives influence data capture in the EMR over time and how this may inform about changes in severe asthma outcomes.

    • The Primary Care Severe Asthma Registry and Education project will provide opportunities for the development of infrastructure necessary to understand the severe asthma care pathway in the real world and how it may be influenced by additional variables, including comorbidities and other relevant determinants of health.

    • Since phase 1 of this project focusses primarily on registry development and validation patient outcomes are not directly addressed.

    Introduction

    The identification and management of patients with severe asthma, defined as requiring a step 4 or 5 treatment to maintain symptom control,1 is challenging in the primary care setting.1 2 The wide variety of health issues managed in primary care and undifferentiated presenting symptoms represent important barriers for early detection. Reports describe missed opportunities to reduce the risk of asthma exacerbations and death, including factors that limit the referral of patients for specialist assessment from primary care.3 4 Indeed, recent evidence from international severe asthma registries suggests a large majority (72%) of individuals with potential severe asthma had neither been referred nor received specialist care within the preceding year.4 Systematic strategies that strengthen interdisciplinary management and monitoring of patients with severe asthma across the disease life cycle are necessary to improve the quality of care received by these patients.4

    Existing initiatives like the International Severe Asthma Registry5 capture important information regarding patients managed in tertiary care; yet, there is limited ‘real-world’ data to inform the role of the primary care practitioner in the detection, referral and co-management of patients with severe asthma. Although patients with severe asthma represent a small proportion of the overall asthma population6 , many patients have a primary care provider such as their family physician or nurse practitioner, who will usually be the first point of contact. Primary care-based ‘real-world’ data are needed in order to describe factors that influence physician decision making, patient behaviour and disease outcomes.7 A severe asthma registry in primary care will define and quantify the population prevalence of severe asthma while identifying best practices at both the clinical practice and population levels.

    To address these existing care gaps, we will develop a severe asthma registry in primary care (phase 1), followed by the implementation of an education-based and integrated support system (phases 2–4) that aims to facilitate (1) collaboration among primary and secondary care providers and allied healthcare providers; (2) awareness of severe asthma management strategies and (3) data capture in the electronic medical record (EMR) reflecting severe asthma management in primary care. This paper will detail the protocol for phase 1, which will develop interdisciplinary consensus criteria for the: (1) definition of severe asthma; (2) generation of a severe asthma registry and (3) definition of a Clinician Behaviour Index (CBI). Phase 1 results will contribute to the development of an education intervention to improve primary care-based severe asthma management.

    Project setting

    This project will be conducted in partnership with the University of Toronto Practice-Based Research Network (UTOPIAN), a network of over 1700 family physicians in practices within the 14 Department of Family and Community Medicine academic sites dispersed throughout the Greater Toronto Area and other parts of Ontario.8 Primary care providers and patients associated with UTOPIAN will serve as the populations studied for the purposes of this project. Currently, UTOPIAN maintains primary care EMR data on over 6 00 000 patients from Toronto and other parts of Ontario.8

    Primary objectives

    The overarching goals of Primary Care Severe Asthma Registry and Education (PCSAR-EDU) are twofold: (1) develop a PCSAR and (2) implement a patient-centred educational programme for primary care providers that supports the collaborative management of severe asthma. Phase 1 activities focus on the development, implementation and validation of a PCSAR and EMR indices of clinician behaviour (ie, the CBI) which will represent an iterative metric of adherence to guideline-based care. The purpose of the severe asthma registry is to aid in the early identification, management, and referral of patients with severe asthma when needed. Registry development will involve:

    1. Using validated case verification approaches9 10 to identify paediatric and adult patients with asthma in the UTOPIAN database.

    2. Establishing criteria through e-Delphi consensus that are relevant for the identification of severe asthma using EMR data.

    3. Using validated database indices of asthma severity11 12 to identify patients with ‘suspected severe asthma’.

    4. Manual clinician review of ‘suspected severe asthma’ cases to establish a reference standard.

    5. Generating algorithms using established e-Delphi consensus criteria to identify the severe asthma population in the UTOPIAN EMR data.

    6. Validating these algorithms against the reference standard (ie, manual clinician review).

    7. Implementing a severe asthma registry within the UTOPIAN database.

    8. Using the registry to estimate the prevalence of severe asthma in a primary care database.

    Methods and analysis

    Project overview

    PCSAR-EDU is a 4-year project (2019–2022) with four phases. Phase 1 involves the interdisciplinary development and implementation of a severe asthma registry within primary care (figure 1). The remaining phases involve subjective and objective needs assessments (phase 2), educational programme development, implementation and evaluation (phase 3) and project refinements through ongoing quality improvement efforts (phase 4). Collectively, phases 2–4 comprise strategies designed to support an ongoing and iterative ‘Clinician Behaviour Modification Cycle’ (figure 2) that integrates continuing medical education, clinical research and quality improvement efforts that will evolve based on the extent of change in the CBI, new knowledge and the need for innovating current practices. As more individuals are entered into the registry, new research will be derived from this database to improve generalisability of research findings at the population level.

    Figure 1
    Figure 1

    PCSAR-EDU project phase 1 overview. CBI, Clinician Behaviour Index; PCSAR-EDU, Primary Care Severe Asthma Registry and Education; SA, severe asthma.

    Figure 2
    Figure 2

    PCSAR-EDU clinician behaviour modification cycle schematic representation. PCSAR-EDU, Primary Care Severe Asthma Registry and Education.

    Project governance

    The proposed project adopts an integrated knowledge translation approach,13 whereby a steering committee (table 1) consisting of thirteen relevant stakeholders (ie, primary and specialist care, pharmaceutical, patient perspectives, as well as educational expertise and representation from The Lung Health Foundation—Ontario) was established to inform all stages of the project and ensure the educational programme adequately targets the audience’s educational and practice-enhancing needs (table 1).

    View this table:
    Table 1

    Pproject governance structure

    Patient involvement

    Patients will be involved in the design and implementation of all phases of the project. In phase 1, patients involved in both the steering committee and patient advisory committee will guide project and registry priorities.

    e-Delphi consensus activity design

    A four-round modified e-Delphi activity will establish expert consensus on specific criteria for the: (1) definition of severe asthma; (2) entry of patients into the severe asthma registry and (3) CBI definition (figure 1).

    Severe asthma definition consensus

    We will consider two definitions of severe asthma as outlined by the GINA 2020 Strategy1 and the Canadian Thoracic Society Position Statement.14 Panellists will establish consensus on a specific definition of severe asthma which may reflect agreement with existing definitions (ie, GINA and/or CTS) or the development of consensus on a novel definition of severe asthma. Our mandate is to establish consensus on a definition of severe asthma that reflects the collaborative effort and expertise of primary, specialty and allied care.

    Severe asthma registry entry criteria consensus

    Panellists will establish the specific criteria used to (1) flag patients for entry and (2) confirm patient entry into the PCSAR. The agreed on definition of severe asthma will determine the type of data, including pharmacotherapy, that should be included in the database to flag a patient for potential registry entry. Additional EMR data criterion under consideration that will be used to flag and/or enter a patient into the PCSAR may include:

    1. Objective documentation of asthma diagnosis.

    2. Report of one or more exacerbation(s) requiring oral steroids per year in the community or hospital setting.

    3. Report of symptoms several days during the week requiring rescue medication use.

    4. Evidence of reduced lung function.

    5. Other (as specified by participants in the open e-Delphi round).

    CBI criteria consensus

    E-Delphi panellists will also establish consensus on the components of the CBI. As previously stated, the CBI will include EMR data elements associated with guideline-based care that are deemed relevant by the expert panellists and will serve as an iterative metric of PCP adherence to guideline-based care. Panellists will vote on whether this metric is represented by a composite score of several CBI items or if there is a single CBI item that should serve as the primary outcome variable for the purposes of evaluating the impact of the proposed education programme on PCP adherence to guideline-based care. The agreed on CBI item(s) will be monitored via ongoing chart reviews and will track changes in PCP management of severe asthma and EMR documentation processes over time with a view to inform future education initiative objectives. The individual components of the CBI under evaluation are outlined in table 2.

    View this table:
    Table 2

    Components of the CBI under evaluation

    Selection of Delphi panellists

    Our approach will involve purposeful sampling of individuals with knowledge of severe asthma identification and management, including those with clinical and/or research expertise in the area. Our goal is to ensure diverse perspectives are represented with regards to severe asthma management priorities and outcomes. Project collaborators including family physicians, paediatric and adult respirologists, allergists, respiratory therapists, and experts in respiratory research (n≥13) will be invited to complete the Delphi consensus activity.

    Delphi panel sampling

    Although no strict sample size requirements exist in the literature, our project will include a minimum of 13 panellists which is in keeping with prior recommendations.15–17 Each participant will be asked to create a unique identification code. Analysis of results will occur through the online survey platform Welphi (www.welphi.com) and via password-protected excel spreadsheets.

    Delphi consensus methods

    As previously stated, this project will adopt a four-round modified e-Delphi consensus approach (figure 1), whereby consensus will be established via soliciting anonymous, iterative rounds of feedback and opinion until consensus is achieved.

    e-Delphi data collection and analysis

    Delphi panellists will be invited to complete a 30 min online survey on three separate occasions (ie, three separate e-Delphi ‘rounds’) over a 3-month period.18 Panellist opinions will be collected via an open-ended survey (round 1; ie, ‘open-round’) and 5-point Likert scale and ranking surveys (round 2 and 3; ie, ‘closed-rounds’).

    Round 1

    In the open round (ie, round 1), participants will be presented with two widely recognised definitions of severe asthma as well as a draft list of criteria for the entry of patients into the severe asthma registry and the definition of a CBI. In addition to panellist demographic information, round 1 will solicit written opinions regarding the appropriateness of existing definitions of severe asthma as well as suggestions for additional criteria items that should be considered by the expert panel for registry entry and the CBI. The resulting list of items will be aggregated and analysed via qualitative content analysis methods19 and anonymously circulated within the second e-Delphi round for ranking (ie, closed-round). This round will be open for a period of approximately 1 month during June 2021. Reminder emails will be sent out twice throughout this period.

    Round 2 and 3

    In each closed round (ie, round 2 and 3), participants will be encouraged to provide opinions (via Likert scales; strongly disagree—strongly agree) and written comments regarding the requirement of each item. Participant responses will be analysed using frequency counts. Criteria items achieving less than 50% consensus (ie, 50% or more participants indicated ‘strongly disagree’ or ‘disagree’ with item being required) in the second e-Delphi round will not be included in the third e-Delphi round. All other criteria items and associated anonymous participant comments will be circulated for the third e-Delphi round. Criteria items achieving 80% agreement or more on ‘round 3’ will be compiled and will represent the ideal ‘core criteria list’ since all core criteria agreed on may not be implementable in every patient. Round 2 and 3 will be open for approximately 1 month, respectively. Two reminder emails will be sent during each period.

    Round 4

    A fourth and final Delphi round will occur via an in-person or virtual meeting, whereby panellists reflect on round 3 results and approve a finalised ideal ‘core criteria list’. Any outstanding disagreements will be documented and addressed where possible using a democratic approach. This fourth round will also involve determining the minimum and maximum number of criteria items to be included in the CBI, assigning weightings to the items and establishing a scoring framework. The resulting criteria list with associated weight and scoring will represent the CBI.

    Severe asthma registry validation

    This project will use EMR data from patients in the UTOPIAN database in Ontario, Canada.

    Establishing the reference population

    The reference population will be obtained from the UTOPIAN database. As previously stated, this database contains medical chart information from almost 600 000 patients which includes all age groups (53% female).20 Available data include the cumulative patient profile (CPP), demographics, progress notes, laboratory test results, allergies, medications, immunisation history and vital measurements.

    Patients within UTOPIAN with an asthma diagnosis will be identified using a validated case definition.9 10 Inclusion criteria will be: age >6 years and use of EMR by the primary care provider for at least 2 years. Exclusion criteria includes: (1) greater than 10 pack-year smoker (ie, current or former smokers with a pack-year history of 10 or less would be included); (2) serum alpha 1 antitrypsin level <11 µmol and (3) other chronic lung disease (eg, Interstitial lung disease, Chronic obstructive pulmonary disease, bronchiectasis, cystic fibrosis).

    To establish the reference population, a random subsample of patients with a high likelihood of diagnosis of asthma will be selected. Given the low prevalence of severe asthma in the general population, this project will select patients more likely to have severe asthma using validated database indices of asthma severity previously described11 12 to then undergo chart abstraction by trained abstractors. This approach will involve a search in patient medication history that categorise patients into ‘suspected severe asthma’ (ie, prescribed prednisone, moderate to high dose ICS/LABA, or reports of asthma-related hospitalisation or ER visit), which will undergo chart abstraction or ‘severe asthma unlikely’ (ie, not prescribed prednisone, ICS/LABA, no reports of asthma-related hospitalisation) whose charts will not undergo abstraction.

    The charts of the patients categorised as ‘suspected severe asthma’ will be manually reviewed by a primary care physician using a standardised abstraction manual. Patients meeting criteria for a diagnosis of severe asthma will represent the reference standard. The EMR criteria for a diagnosis of severe asthma agreed on by Delphi Panellists will attempt to exclude patients with ‘difficult-to-treat asthma’.1 To determine intra-rater reliability, 25 randomly selected charts will be re-abstracted by the same abstractor. To assess inter-rater reliability, an additional 25 randomly selected charts will be abstracted by different abstractors (a respirologist, a primary care physician and an allergist). Uncertain categorisations will be resolved by a consensus committee consisting of a primary care provider and a respirologist. Kappa statistics will be used to measure inter-rater and intrarater reliability.

    Variables for data algorithms

    Registry algorithms will be based on items in EMR data that achieve expert consensus during the e-Delphi consensus activity. These EMR data will be captured through automated searches of free text within the CPP (eg, a record of asthma diagnosis; exacerbations requiring oral or systemic steroids; rescue medication use) and manual review of test results (eg, spirometry reports of reduced lung function) within the electronic medical chart.

    Algorithm evaluation

    Algorithm testing will involve searching the EMR for the registry entry criteria items that achieved e-Delphi consensus, as previously described. Combinations of e-Delphi consensus registry entry items will be evaluated to increase sensitivity and/or specificity. Algorithm examination will involve diagnostic accuracy assessments of specificity, positive predictive value and negative predictive value.

    Clinician Behaviour Index validation

    Since the CBI is a new assessment tool specific to this project, validity evidence relevant to research outcomes will be sought. Specifically, content validity, internal structure (ie, reliability) and response process will be evaluated according to the standards for validity.21

    Content evidence for the CBI

    The resulting CBI established by the Delphi panellists will develop validity evidence through a formal evaluation of content validity from an online questionnaire sent to a content evaluation panel. The content evaluation panel will involve purposeful sampling of 15 individuals according to their professional certifications, experience, accessibility and publications. The content evaluation panel will represent the perspectives of paediatric and adult respirologists, allergists, respiratory therapists and experts in respiratory research who are not otherwise involved in the project. The panel will score each CBI item in a four-point Likert scale in the following three domains: relevance, simplicity and clarity (from 1: not relevant, not simple, and not clear to 4: very relevant, very simple and very clear).22

    Content validity will be assessed quantitatively through calculating the content validity ratio (CVR) and content validity index (CVI) for each item. The CVI will be calculated for all individual items (I-CVI) and the overall scale (S-CVI). For CVI, the content evaluation panel will rate each CBI item in terms of its relevance to the underlying construct (ie, adherence to severe asthma management guidelines). For each item, the I-CVI will be calculated as the number of panellists giving a rating of 3 or 4 divided by the total number of panellists. In keeping with existing recommendations, if I-CVI is >0.79, the item will be considered relevant; between 0.70 and 0.79, the item will be revised by the research team as per content evaluation panellist feedback; and if below 0.70 the item will be eliminated from the CBI.23 The S-CVI average, determined through calculating the average I-CVI of included items, will be computed to ensure the content validity of the entire scale (ie, the CBI). A minimum S-CVI of 0.8 is recommended.24

    The use of Lawshe’s CVR will determine whether an item is necessary. The Content Evaluation Panel will be asked to score each item on a three-point scale ranging from 1=essential, 2=useful but not essential and 3=not necessary. The formula to calculate CVR = (Ne – N / 2) / (N / 2) where Ne is the number of panellists indicating ‘essential’ and N is the total number of panellists. Essential items will be determined according to Lawshe’s table, which states an acceptable CVR value for 15 experts is above 0.49.25

    Internal structure

    Two independent reviewers will score, through chart abstraction, the CBI of the same 50 patient charts. Cohen’s kappa will be assessed to determine the level of agreement between the two reviewers; an acceptable kappa will be >0.7. Additional analyses will explore the degree to which individual components of the CBI drive total score including intercomponent reliability and exploration of underlying factor structure, if appropriate.

    Response process

    The reviewers will report using a structured questionnaire on the process of deriving the CBI, issues of interpretation and quality control processes for recording CBIs.

    Registry implementation and analysis

    The validated severe asthma registry entry criteria will be used to develop a registry abstraction manual. This manual will describe in detail how to flag and enter individuals who meet the validated criteria into an ongoing PCSAR. This information will provide the basis for the calculation of a prevalence estimate based on the total number of individuals with severe asthma26 in the database between 1 September 2018 and 1 September 2020. The registry data will be managed and used based on objectives and priorities established by the PCSAR-EDU Steering Committee and UTOPIAN RHWG with oversight by the UTOPIAN Scientific Advisory Committee. PCSAR will be managed by UTOPIAN scientists and University of Toronto faculty involved in the project. Other stakeholders will be informed of the registry status and progress without compromising patient confidentiality. Clinician investigators that are affiliated with UTOPIAN will have access to deidentified registry data.

    Ethics and dissemination

    Ethical approval

    Ethical approval is required for phases 1–3 of this project. Approval has been obtained for the activities involved in phase 1 from the University of Toronto’s Human Research Ethics Programme (approval number 39695) and will be sought for phase 2 and 3 activities. Future ethics approvals will depend on information gathered in the proceeding phase; thus, ethical approval for phase 2 and 3 of this study will be sought sequentially in order to address phase specific ethical considerations that may emerge.

    Dissemination plan

    Study findings will be shared and discussed at relevant project committee meetings to promote ongoing project evaluation and improvement. In addition, results will be disseminated through conference presentations, peer-reviewed journal publications and knowledge translation tools. With the launch of the validated severe asthma registry in primary care, PCSAR-EDU project collaborators will vote democratically on research proposals submitted to PCSAR-EDU, which will then require final approval from the UTOPIAN Scientific Advisory Committee.

    Ethics statements

    Patient consent for publication

    Acknowledgments

    This project will be conducted in partnership with the University of Toronto Practiced-Based Research Network.

    References

    1. ↵
      1. Global Initiative for Asthma
      . Global initiative for asthma: global strategy for asthma management and prevention (updated 2020). Revue Francaise d’Allergologie et d’Immunologie Clinique 2020.
    2. ↵
      1. Lougheed MD,
      2. Leniere C,
      3. Ducharme FM, et al
      . Canadian thoracic Society 2012 guideline update: diagnosis and management of asthma in preschoolers, children and adults: Executive summary. Can Respir J 2012;19:e81–8.doi:10.1155/2012/214129pmid:http://www.ncbi.nlm.nih.gov/pubmed/23248807
      OpenUrlPubMed
    3. ↵
      1. Price D,
      2. Bjermer L,
      3. Bergin DA, et al
      . Asthma referrals: a key component of asthma management that needs to be addressed. J Asthma Allergy 2017;10:209–23.doi:10.2147/JAA.S134300pmid:http://www.ncbi.nlm.nih.gov/pubmed/28794645
      OpenUrlPubMed
    4. ↵
      1. Ryan D,
      2. Heatley H,
      3. Heaney LG, et al
      . Potential severe asthma hidden in UK primary care. J Allergy Clin Immunol Pract 2021;9:1612–23.doi:10.1016/j.jaip.2020.11.053pmid:http://www.ncbi.nlm.nih.gov/pubmed/33309935
      OpenUrlPubMed
    5. ↵
      1. ISAR
      . ISAR registry. Available: http://isaregistries.org/ [Accessed 4 Aug 2020].
    6. ↵
      1. Chung KF,
      2. Wenzel SE,
      3. Brozek JL, et al
      . International ERS/ATS guidelines on definition, evaluation and treatment of severe asthma. Eur Respir J 2014;43:343–73.doi:10.1183/09031936.00202013
      OpenUrlAbstract/FREE Full Text
    7. ↵
      1. Fajt ML,
      2. Wenzel SE
      . Asthma phenotypes and the use of biologic medications in asthma and allergic disease: the next steps toward personalized care. J Allergy Clin Immunol : 2015;135:299–310.doi:10.1016/j.jaci.2014.12.1871pmid:http://www.ncbi.nlm.nih.gov/pubmed/25662302
      OpenUrlCrossRefPubMed
    8. ↵
      Utopian database report: 2019Q1 data cycle.
    9. ↵
      1. Gershon AS,
      2. Wang C,
      3. Guan J, et al
      . Identifying patients with physician-diagnosed asthma in health administrative databases. Can Respir J 2009;16:183–8.doi:10.1155/2009/963098pmid:http://www.ncbi.nlm.nih.gov/pubmed/20011725
      OpenUrlCrossRefPubMedWeb of Science
    10. ↵
      1. To T,
      2. Dell S,
      3. Dick PT, et al
      . Case verification of children with asthma in Ontario. Pediatr Allergy Immunol 2006;17:69–76.doi:10.1111/j.1399-3038.2005.00346.xpmid:http://www.ncbi.nlm.nih.gov/pubmed/16426258
      OpenUrlCrossRefPubMedWeb of Science
    11. ↵
      1. Firoozi F,
      2. Lemière C,
      3. Beauchesne M-F, et al
      . Development and validation of database indexes of asthma severity and control. Thorax 2007;62:581–7.doi:10.1136/thx.2006.061572pmid:http://www.ncbi.nlm.nih.gov/pubmed/17287299
      OpenUrlAbstract/FREE Full Text
    12. ↵
      1. Jacob C,
      2. Haas JS,
      3. Bechtel B, et al
      . Assessing asthma severity based on claims data: a systematic review. Eur J Health Econ 2017;18:227–41.doi:10.1007/s10198-016-0769-2
      OpenUrl
    13. ↵
      1. Gagliardi AR,
      2. Berta W,
      3. Kothari A, et al
      . Integrated knowledge translation (IKT) in health care: a scoping review. Implementation Science 2015;11:38.doi:10.1186/s13012-016-0399-1
      OpenUrl
    14. ↵
      1. FitzGerald JM,
      2. Lemiere C,
      3. Lougheed MD, et al
      . Recognition and management of severe asthma: a Canadian thoracic Society position statement. Canadian Journal of Respiratory, Critical Care, and Sleep Medicine 2017;1:199–221.doi:10.1080/24745332.2017.1395250
      OpenUrl
    15. ↵
      1. Vogel C,
      2. Zwolinsky S,
      3. Griffiths C, et al
      . A Delphi study to build consensus on the definition and use of big data in obesity research. Int J Obes 2019;43:2573–86.doi:10.1038/s41366-018-0313-9pmid:http://www.ncbi.nlm.nih.gov/pubmed/30655580
      OpenUrlCrossRefPubMed
    16. ↵
      1. Ogbeifun E A-E
      . The Delphi technique: a credible research methodology.
    17. ↵
      1. Okoli C,
      2. Pawlowski SD
      . The Delphi method as a research tool: an example, design considerations and applications. Inf Manage 2004;42:15–29.doi:10.1016/j.im.2003.11.002
      OpenUrlCrossRef
    18. ↵
      1. Rayens MK,
      2. Hahn EJ
      . Building consensus using the policy Delphi method. Policy Polit Nurs Pract 2000;1:308–15.doi:10.1177/152715440000100409
      OpenUrlCrossRef
    19. ↵
      1. Berg K,
      2. Isaksen J,
      3. Wallace SJ, et al
      . Establishing consensus on a definition of aphasia: an e-Delphi study of international aphasia researchers. Aphasiology 2020;42:1–16.doi:10.1080/02687038.2020.1852003
      OpenUrl
    20. ↵
      1. Tu K,
      2. Greiver M,
      3. Kidd MR
      . The University of Toronto family medicine report. Toronto, ON, 2019.
    21. ↵
      1. Downing SM
      . Validity: on meaningful interpretation of assessment data. Med Educ 2003;37:830–7.doi:10.1046/j.1365-2923.2003.01594.xpmid:http://www.ncbi.nlm.nih.gov/pubmed/14506816
      OpenUrlCrossRefPubMedWeb of Science
    22. ↵
      1. Walker S,
      2. Brett S,
      3. McKay A, et al
      . Observational Skill-based clinical assessment tool for resuscitation (OSCAR): development and validation. Resuscitation 2011;82:835–44.doi:10.1016/j.resuscitation.2011.03.009pmid:http://www.ncbi.nlm.nih.gov/pubmed/21481519
      OpenUrlCrossRefPubMedWeb of Science
    23. ↵
      1. Zamanzadeh V,
      2. Ghahramanian A,
      3. Rassouli M, et al
      . Design and implementation content validity study: development of an instrument for measuring patient-centered communication. J Caring Sci 2015;4:165–78.doi:10.15171/jcs.2015.017pmid:http://www.ncbi.nlm.nih.gov/pubmed/26161370
      OpenUrlPubMed
    24. ↵
      1. Polit DF,
      2. Beck CT
      . The content validity index: are you sure you know what's being reported? critique and recommendations. Res Nurs Health 2006;29:489–97.doi:10.1002/nur.20147pmid:http://www.ncbi.nlm.nih.gov/pubmed/16977646
      OpenUrlCrossRefPubMedWeb of Science
    25. ↵
      1. Lawshe CH
      . A quantitative approach to content validity. 28, 1975: 563–75.
      OpenUrl
    26. ↵
      1. Xi N,
      2. Wallace R,
      3. Agarwal G, et al
      . Identifying patients with asthma in primary care electronic medical record systems chart analysis-based electronic algorithm validation study. Can Fam Physician 2015;61:e474–83.pmid:http://www.ncbi.nlm.nih.gov/pubmed/26759847
      OpenUrlAbstract/FREE Full Text

    Footnotes

    • Contributors All authors contributed to the design of this protocol. KAD and ADD initiated the project and drafted the manuscript. IET, KRC, MRMY, MG, REGU, LB, BO, RM, BA, KK and TT significantly contributed to the establishment of project objectives and refinement of study procedures. All authors read, critically revised and approved the final version of the manuscript.

    • Funding Funding support is provided by Novartis Canada (1086896), GlaxoSmithKline Canada (GD0056-2019), AstraZeneca Canada and Sanofi Canada.

    • Competing interests MRMY has received research grants, speaker’s bureau/honoraria and consulting fees from Novartis, Boehringer Ingelheim, GlaxoSmithKline, Pfizer, AstraZeneca, SanofiGenzyme and Merck Canada. ADD has received research, consulting and lecturing fees from GlaxoSmithKline, AstraZeneca Canada, Merck Canada, Novartis Canada, Boehringer Ingelheim (Canada), Pfizer Canada, SanofiGenzyme.

    • Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.

    • Provenance and peer review Not commissioned; externally peer reviewed.