You are here

Article Text

Article menu

Download PDFPDF

Health informatics
Protocol
Integrating equity, diversity and inclusion throughout the lifecycle of AI within healthcare: a scoping review protocol
  1. Milka Nyariro1,2,
  2. Elham Emami2,
  3. Pascale Caidor3,
  4. Samira Abbasgholizadeh Rahimi1,2,4
  1. 1Department of Family Medicine, McGill University, Montreal, Quebec, Canada
  2. 2Faculty of Dental Medicine and Oral Health Sciences, McGill University, Montreal, Quebec, Canada
  3. 3Department of Communication, Université de Montréal, Montreal, Quebec, Canada
  4. 4Mila-Québec AI Institue, Montreal, Quebec, Canada
  1. Correspondence to Dr Samira Abbasgholizadeh Rahimi; samira.rahimi{at}mcgill.ca

Abstract

Introduction Artificial intelligence (AI) has the potential to improve efficiency and quality of care in healthcare settings. The lack of consideration for equity, diversity and inclusion (EDI) in the lifecycle of AI within healthcare settings may intensify social and health inequities, potentially causing harm to under-represented populations. This article describes the protocol for a scoping review of the literature relating to integration of EDI in the AI interventions within healthcare setting. The objective of the review is to evaluate what has been done on integrating EDI concepts, principles and practices in the lifecycles of AI interventions within healthcare settings. It also aims to explore which EDI concepts, principles and practices have been integrated into the design, development and implementation of AI in healthcare settings.

Method and analysis The scoping review will be guided by the six-step methodological framework developed by Arksey and O’Malley supplemented by Levac et al, and Joanna Briggs Institute methodological framework for scoping reviews. Relevant literature will be identified by searching seven electronic databases in engineering/computer science and healthcare, and searching the reference lists and citations of studies that meet the inclusion criteria. Studies on AI in any healthcare and geographical settings, that have considered aspects of EDI, published in English and French between 2005 and present will be considered. Two reviewers will independently screen titles, abstracts and full-text articles according to inclusion criteria. We will conduct a thematic analysis and use a narrative description to describe the work. Any disagreements will be resolved through discussion with the third reviewer. Extracted data will be summarised and analysed to address aims of the scoping review. Reporting will follow the Preferred Reporting Items for Systematic Reviews and Meta-analyses Extension for Scoping Reviews. The study began in April 2022 and is expected to end in September 2023. The database initial searches resulted in 5,745 records when piloted in April 2022.

Ethics and dissemination Ethical approval is not required. The study will map the available literature on EDI concepts, principles and practices in AI interventions within healthcare settings, highlight the significance of this context, and offer insights into the best practices for incorporating EDI into AI-based solutions in healthcare settings. The results will be disseminated through open-access peer-reviewed publications, conference presentations, social media and 2-day workshops with relevant stakeholders.

  • Health Equity
  • Protocols & guidelines
  • MEDICAL ETHICS
  • Health informatics

Data availability statement

No data are available.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjopen-2023-072069

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • To our knowledge, this will be the first scoping review of literature on AI in healthcare that evaluates how studies have considered and integrated equity, diversity and inclusion concepts, principles and practices.

    • Our search will be limited to literature published in English and French, and to 2005 onwards; however, this allows for a focus on the most recent/relevant developments in the field, which is rapidly evolving.

    • A comprehensive search in multiple databases along with evidence in published and grey literature sources will allow us to extensively map the current landscape of EDI throughout the lifecycle of AI within healthcare.

    • To ensure transparency and objectivity, this scoping review will adhere to the Preferred Reporting Items for Systematic Reviews and Meta-analyses standards for conducting scoping reviews.

    Introduction

    Artificial intelligence (AI) is the branch of engineering and computer sciences focusing on development of computer systems that could perform tasks that require human-like intelligence to perform complex tasks. These tasks include learning, reasoning and self-correction.1–4 AI-enabled technologies are increasingly being applied in all fields in our current society including healthcare4–6 where they have been used for classification of diseases7 screening8 and validation of diagnosis.9 10

    AI-enabled technologies are increasingly augmenting healthcare services, supporting healthcare professionals in their decision-making process11 and improving the efficiency of the health systems.12 Health systems are pivotal for effective and efficient healthcare delivery and their ultimate goals are achieving the quintuple aim of care.13 The application of AI-enabled technologies in health systems is anticipated to improve system efficiency and health outcomes for all populations, by alleviating workload on healthcare professionals and improving quality of care through error reduction and increased precision.14

    Despite the benefits of AI in healthcare settings, both AI and health systems are faced with a lack of equity, diversity and inclusion (EDI) principles and practices. The lack of consideration of EDI concepts, principles and practices in AI within healthcare settings can introduce bias and discrimination against some groups of the population.15 The most at-risk groups to face bias and discrimination often belong to minorities that have been historically and systemically marginalised.16 Lack of EDI principles and practices in the lifecycles of AI technologies within healthcare—from design to implementation stages—is emerging in the scholarly discourse as a social, ethical and health concern.

    Although the definitions of EDI concepts keep evolving and will likely continue to do so, it is critical to recognise their context-dependent nature, varying from one setting to another. For instance, in healthcare, equity is perceived as a multidimensional concept and refers to treating people with fairness and recognising the existence of systemic barriers for marginalised groups.17 The concept of diversity is also extremely complex and refers to individuals’ and groups’ unique experiences and perspectives. These experiences are linked to individual traits such as personality and identifiers like race, ethnicity, gender and sexuality among others. Finally, the concept of inclusion means the intentional efforts to implement practices that enable the entire community to be and feel valued, supported and respected, with special attention to under-represented groups. For this work, we adapt a broader perspective of EDI as a conscious effort to incorporate a diverse range of social identities and perspectives, including those normally marginalised, into decision making on social and health issues that impact their lives and well-being.

    The current AI ethics practices and guidelines should be expanded to incorporate equity, diversity and inclusion concepts, principles and practices in AI lifecycle. So far, studies on AI in healthcare have been conducted on some of ethical aspects. For example, studies have been conducted on ethical fairness, transparency18 and explainability.19 20 To the best of our knowledge, there are currently no studies on integration of EDI concepts, principles and practices into the design, development and implementation of AI within healthcare settings. Recent research12 14 21 suggests that studies on the application of AI in healthcare rarely included all stakeholders in the design, development and implementation of the AI systems. Some studies12 14 21 also indicate that there is a lack of consideration and incorporation of EDI concepts, principles and practices in the application of AI in healthcare settings. To bridge this knowledge gap, we are conducting this scoping review.

    Method

    Protocol design, registration and reporting

    The proposed scoping review will be conducted in accordance with the Joanna Briggs Institute (JBI) guideline and the six-step methodological framework developed by Arksey and O’Malley supplemented by Levac et al. methodology.22 The Preferred Reporting Items for Systematic reviews and Meta-Analysis (PRISMA) Statement for protocols and JBI reporting checklists were consulted when preparing this protocol.23 24

    The research questions

    In line with the aims of this scoping review, we have developed the following questions:

    1. What efforts have been made to incorporate EDI concepts, principles and practices throughout the lifecycles of AI within healthcare settings?

    2. Which specific EDI concepts, principles and practices have been integrated into the design, development and implementation of AI within healthcare settings?

    Eligibility criteria

    Participants

    The review will consider sources of evidence on individuals who are users and/or providers of social or healthcare services. There will be no restrictions based on an individual’s health condition(s).

    Concept

    We will include all studies irrespective of which the integration of EDI and AI is the primary focus of the publication.

    Context

    Sources of evidence reporting on social or healthcare settings on the AI and indicators or socio-bio markers of health equity or its related concepts, that is, diversity and inclusion, will be considered.

    Types of sources

    This scoping review will consider all primary research studies including reviews. We will exclude publications including commentaries, conference abstracts, editorials and individual points of view.

    Information sources and search strategy

    The searches will be conducted in two stages: (1) electronic database searching and (2) hand searching the reference lists and citations of included sources to identify further studies for inclusion. The detailed search strategy has been designed with the help of an expert librarian in the fields of Healthcare and Engineering at McGill University. Various electronic platforms such as MEDLINE (Ovid), Embase (Ovid), PsycInfo (Ovid), Scopus, SCI-EXPANDED, ESCI (Web of Science Core Collection), IEEEE Xplore and INSPEC will be searched. Only studies published in English and French languages will be included. The search strategy will include year 2005 to present as this marks the period from which the World Health Organization (WHO) made Health Equity a cornerstone of the Millennium Development Goals.15 A preliminary version of the search was conducted on 29 April 2022, using specific MeSH terms and keywords to assure the accuracy and sensitivity of the search to capture the relevant literature. The start date of the review is June 1st, 2023.

    Selection of sources of evidence

    Following the search, all identified citations will be collated and uploaded into Endnote V.9.3.3 (www.endnote.com) reference management software, and duplicates removed. A team of reviewers (MN, SAR, EE) will conduct title/abstract and full-text screening of all records considering the prespecified eligibility criteria. The study selection will follow a two-step screening process. Titles/abstracts (step 1) and full-text screening (step 2) will be screened by two independent reviewers for assessment against the inclusion and exclusion criteria for the review. All records will be screened independently by a minimum of two reviewers. Any disagreements will be resolved through discussion and a third reviewer will be involved if consensus cannot be reached. All screening will be conducted within Rayan software. For articles excluded during full-text screening, a reason for exclusion will be recorded. The selection process will be recorded in a PRISMA flow diagram.

    Critical appraisal of individual sources of evidence

    Assessment of the methodological quality of included studies is not a requirement of scoping reviews.22

    Data charting process

    Relevant data will be extracted from all included studies in the scoping review by two independent reviewers. A draft structured data recording form developed by the reviewers will be used and the information will be recorded on Microsoft Excel. The draft data extraction form will be modified and revised as necessary during the process of extracting data from each included database. Modifications will be reported in the scoping review. Any disagreements that arise between the two reviewers will be resolved through discussion with the third and fourth reviewers. The data extracted will include specific details on the author(s), year of the study, year of publication, country of study, method applied in the study, participants characteristics, objectives of the study, socioeconomic variables for measuring equity such as desegregation of data by sex, gender, sexuality, race, ethnicity, economic and social status, AI methods and the type of healthcare setting of application and the key findings relevant to the objectives of the scoping review. To synthesise the findings, we will conduct a thematic analysis and use a narrative description to describe the work according to the study design, any emerging patterns identified, ethical implications, as well as legal considerations. Results will be presented using tables and diagrams accompanied by a narrative summary.

    Patient and public involvement

    During this review, we are concentrating on gathering input from a focused group of experts. However, we deeply value the role of patient and public involvement and have ensured to integrate patient and public in the subsequent workshops, where we look forward to benefiting from a rich diversity of perspectives to enhance the quality and relevance of our work.

    Stakeholder consultation

    A stakeholder consultation is planned to validate the findings from the review, add new insights and identify gaps for further research. Stakeholders will include AI system developers, engineers, data analysts, healthcare professionals, EDI experts, researchers, patients and communities. We already have a team of experts in the research team who will further help in identifying external subject matter experts through snowballing technique. The aim of the stakeholders’ workshop will be to consolidate prior and new inter/multidisciplinary research partnerships that will map out literature and contribute to knowledge synthesis on the use of EDI concepts, principles and practices in AI technologies and related applications in healthcare with the aim of co-developing of preliminary data on EDI/AI concepts.

    Ethics and dissemination

    Ethical approval is not required for this study. Future dissemination related to this work will include the publication of the results in an open access peer-reviewed journal, presentations at conferences, social media and a 2-day workshops. We will report findings in line with the ‘PRISMA Extension for Scoping Reviews’.23

    Discussion

    AI has the potential to improve both the patients and health professionals’ experience as well as other quintuple aim of care. However, if EDI is not considered and integrated throughout the lifecycle of AI-enabled technologies within healthcare systems, it can exacerbate the existing social and health inequalities. This scoping review will be the first to offer a comprehensive overview of how EDI considerations have been incorporated into the lifecycle of AI within healthcare settings. It will not only highlight how EDI concepts, principles and practices have been considered and integrated in AI healthcare studies, but also identify current gaps in this area. We aim to pinpoint the indicators and variables of EDI that are not considered by AI studies in healthcare thereby representing a potential area of research.

    Findings from the scoping review study will be: (1) synthesised to identify the knowledge gaps in this important area, (2) disseminated to diverse stakeholders in the fields of AI and healthcare and to the public through workshops and meetings and (3) mobilised towards building a framework on ways to meaningfully integrate EDI concepts, principles and practices in the design, development and implementation of AI within healthcare.

    Data availability statement

    No data are available.

    Ethics statements

    Patient consent for publication

    Ethics approval

    Not applicable.

    References

    1. ↵
      1. UNESCO
      . Artificial intelligence and gender equality:key findings of UNESCO’s global dialogue. In: UNESCO, 7, place de Fontenoy. Paris, France, 2020. Available: https://en.unesco.org/system/files/artificial_intelligence_and_gender_equality.pdf
    2. ↵
      1. Szolovits P
      . Artificial intelligence in medicine. In: Artificial Intelligence In Medicine. Routledge, 2019. doi:10.4324/9780429052071
    3. ↵
      1. Hamet P,
      2. Tremblay J
      . Artificial intelligence in medicine. Metabolism 2017;69S:S36–40. doi:10.1016/j.metabol.2017.01.011
      OpenUrlCrossRefPubMed
    4. ↵
      1. Panda PS,
      2. Bhatia V
      . Role of artificial intelligence (AI) in public health. Indian J Community Fam Med 2018;4:60. doi:10.4103/2395-2113.251442
      OpenUrl
    5. ↵
      1. Rajpurkar P,
      2. Chen E,
      3. Banerjee O, et al
      . AI in health and medicine. Nat Med 2022;28:31–8. doi:10.1038/s41591-021-01614-0
      OpenUrlCrossRefPubMed
    6. ↵
      1. Lin SY,
      2. Mahoney MR,
      3. Sinsky CA
      . Ten ways artificial intelligence will transform primary care. J Gen Intern Med 2019;34:1626–30. doi:10.1007/s11606-019-05035-1
      OpenUrlCrossRefPubMed
    7. ↵
      1. Lakhani P,
      2. Sundaram B
      . Deep learning at chest radiography: automated classification of pulmonary tuberculosis by using Convolutional neural networks. Radiology 2017;284:574–82. doi:10.1148/radiol.2017162326
      OpenUrlCrossRefPubMed
    8. ↵
      1. Chamberlain C,
      2. McNamara B,
      3. Williams ED, et al
      . Diabetes in pregnancy among indigenous women in Australia, Canada, New Zealand and the United States: a systematic review of the evidence for screening in early pregnancy. Diabetes Metab Res Rev 2013;29:241–56. doi:10.1002/dmrr.2389 Available: https://onlinelibrary.wiley.com/toc/15207560/29/4
      OpenUrlCrossRefPubMed
    9. ↵
      1. Tvermosegaard M,
      2. Rønn PF,
      3. Pedersen ML, et al
      . Validation of cardiovascular diagnoses in the Greenlandic hospital discharge register for Epidemiological use. Int J Circumpolar Health 2018;77:1422668. doi:10.1080/22423982.2017.1422668
    10. ↵
      1. McKinney SM,
      2. Sieniek M,
      3. Godbole V, et al
      . Addendum: International evaluation of an AI system for breast cancer screening. Nature 2020;586:E19. doi:10.1038/s41586-020-2679-9
    11. ↵
      1. Biller-Andorno N,
      2. Biller A
      . Algorithm-aided prediction of patient preferences — an ethics sneak Peek. N Engl J Med 2019;381:1480–5. doi:10.1056/NEJMms1904869
      OpenUrlPubMed
    12. ↵
      1. Clark CR,
      2. Wilkins CH,
      3. Rodriguez JA, et al
      . Health care equity in the use of advanced Analytics and artificial intelligence Technologies in primary care. J Gen Intern Med 2021;36:3188–93. doi:10.1007/s11606-021-06846-x
      OpenUrlPubMed
    13. ↵
      1. Frushour C
      . What is the quadruple aim and what role does it play in Healthcare. Privis Health, Mar 2019;13. Available: https://www.privishealth.com/what-is-the-quadruple-aim-and-what-role-does-it-play-in-healthcare/
    14. ↵
      1. Abbasgholizadeh Rahimi S,
      2. Légaré F,
      3. Sharma G, et al
      . Application of artificial intelligence in community-based primary health care: systematic Scoping review and critical appraisal. J Med Internet Res 2021;23:e29839. doi:10.2196/29839
    15. ↵
      1. Fee E,
      2. Gonzalez AR
      . The history of health equity: concept and vision. Divers Equal Health Care 2017;14:3. doi:10.21767/2049-5471.1000105
      OpenUrl
    16. ↵
      1. Seyyed-Kalantari L,
      2. Zhang H,
      3. McDermott MBA, et al
      . Underdiagnosis bias of artificial intelligence Algorithms applied to chest Radiographs in under-served patient populations. Nat Med 2021;27:2176–82. doi:10.1038/s41591-021-01595-0
      OpenUrl
    17. ↵
      1. Lane H,
      2. Sarkies M,
      3. Martin J, et al
      . Equity in Healthcare resource allocation decision making: A systematic review. Soc Sci Med 2017;175:11–27. doi:10.1016/j.socscimed.2016.12.012
      OpenUrl
    18. ↵
      1. Lesot M-J,
      2. Vieira S,
      3. Reformat MZ, et al
      . Information processing and management of uncertainty in knowledge-based systems. In: Lesot M-J, Vieira S, Reformat MZ, et al., eds. Communications in Computer and Information Science. Cham: Springer International Publishing, 2020: 99–113. doi:10.1007/978-3-030-50153-2
    19. ↵
      1. Bhatt U,
      2. Xiang A,
      3. Sharma S, et al
      . Explainable machine learning in deployment. FAT* ’20; Barcelona Spain. New York, NY, USA: ACM, January 27, 2020:648–57 doi:10.1145/3351095.3375624
    20. ↵
      1. Bhatt U,
      2. Andrus M,
      3. Weller A, et al
      . Machine learning Explainability for external Stakeholders. In: ArXiv200705408 Cs. July 2020. Available: http://arxiv.org/abs/2007.05408 [accessed 22 Feb 2022].
    21. ↵
      1. Dankwa-Mullan I,
      2. Scheufele EL,
      3. Matheny ME, et al
      . A proposed framework on integrating health equity and racial justice into the artificial intelligence development Lifecycle. Journal of Health Care for the Poor and Underserved 2021;32:300–17. doi:10.1353/hpu.2021.0065
      OpenUrl
    22. ↵
      1. Levac D,
      2. Glegg SMN,
      3. Sveistrup H, et al
      . A knowledge translation intervention to enhance clinical application of a virtual reality system in stroke rehabilitation. BMC Health Serv Res 2016;16:557. doi:10.1186/s12913-016-1807-6
    23. ↵
      1. Tricco AC,
      2. Lillie E,
      3. Zarin W, et al
      . PRISMA extension for Scoping reviews (PRISMA-SCR): checklist and explanation. Ann Intern Med 2018;169:467–73. doi:10.7326/M18-0850
      OpenUrlCrossRefPubMed
    24. ↵
      1. Shamseer L,
      2. Moher D,
      3. Clarke M, et al
      . Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ 2015;350:g7647. doi:10.1136/bmj.g7647 Available: https://doi.org/10.1136/bmj.g7647

    Footnotes

    • Twitter @RahimiSamira

    • MN and SAR contributed equally.

    • Contributors SAR, MN and EE conceived the ideas for the review and wrote the review protocol. EE and SAR provided methodological guidance and critically reviewed the protocol. PC provided a critical review of the EDI component of the protocol. All authors gave their approval for the final version of the work to be published and agreed to be accountable for the integrity of the work published.

    • Funding This study was funded by Canadian Institutes of Health Research (CIHR) and the Network for Oral and Bone Health Research. We acknowledge support from these institutions. SAR is Canada Research Chair (Tier 2) in Advanced Digital Primary Health Care, received salary support from a Research Scholar Junior 1 Career Development Award from the Fonds de Recherche du Québec-Santé (FRQS), and her research program is supported by the Natural Sciences Research Council (NSERC) Discovery (grant 2020-05246).

    • Competing interests None declared.

    • Patient and public involvement During this review, we are concentrating on gathering input from a focused group of experts. However, we value the role of patient and public involvement and have ensured to integrate patient and public in the subsequent workshops, where we look forward to benefiting from a rich diversity of perspectives to enhance the quality and relevance of our work.

    • Provenance and peer review Not commissioned; externally peer reviewed.