Article Text

Diversity, equity and inclusion considerations in mental health apps for young people: protocol for a scoping review
  1. Caroline A Figueroa1,2,
  2. Nancy J Pérez-Flores3,
  3. Kathleen W Guan1,
  4. Colleen Stiles-Shields4
  1. 1 Policy, Technology and Management, Delft University of Technology, Delft, The Netherlands
  2. 2 School of Social Welfare, University of California Berkeley, Berkeley, California, USA
  3. 3 Brown School of Social Work, Washington University in St Louis, St Louis, Missouri, USA
  4. 4 Institute for Juvenile Research and Department of Psychiatry, University of Illinois at Chicago, Chicago, Illinois, USA
  1. Correspondence to Dr Caroline A Figueroa; c.figueroa{at}


Introduction After COVID-19, a global mental health crisis affects young people, with one in five youth experiencing mental health problems worldwide. Delivering mental health interventions via mobile devices is a promising strategy to address the treatment gap. Mental health apps are effective for adolescent and young adult samples, but face challenges such as low real-world reach and under-representation of minoritised youth. To increase digital health uptake, including among minoritised youth, there is a need for diversity, equity and inclusion (DEI) considerations in the development and evaluation of mental health apps. How well DEI is integrated into youth mental health apps has not been comprehensively assessed. This scoping review aims to examine to what extent DEI considerations are integrated into the design and evaluation of youth mental health apps and report on youth, caregiver and other stakeholder involvement.

Methods and analysis We will identify studies published in English from 2009 to 29 September 2023 on apps for mental health in youth. We will use PubMed, Global Health, APA PsycINFO, SCOPUS, CINAHL PLUS and the Cochrane Database and will report according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses-Scoping Review Extension guidelines. Papers eligible for inclusion must be peer-reviewed publications in English involving smartphone applications used by adolescents or young adults aged 10–25, with a focus on depression, anxiety or suicidal ideation. Two independent reviewers will review and extract articles using a template developed by the authors. We will analyse the data using narrative synthesis and descriptive statistics. This study will identify gaps in the literature and provide a roadmap for equitable and inclusive mental health apps for youth.

Ethics and dissemination Ethics approval is not required. Findings will be disseminated through academic, industry, community networks and scientific publications.

  • Adolescent
  • Health Equity
  • eHealth

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.


  • This is the first comprehensive assessment of diversity, equity and inclusion (DEI) criteria in the development and evaluation of youth mental health apps.

  • We will study DEI criteria (access, content and appearance), according to a framework for evaluating mobile health apps, the representation of marginalised youth and the involvement of youth and other important stakeholders.

  • Our scoping review will conform to the rigorous methodology manual by the Joanna Briggs Institute and the Preferred Reporting Items for Systematic Reviews and Meta-analyses checklist for scoping reviews.

  • The framework of DEI criteria we use is not specific to adolescents and young adult populations.

  • This review will be limited to published research, potentially missing commercialised apps without research evidence.


A pressing global mental health crisis, exacerbated by the COVID-19 pandemic, affects adolescents and young people, particularly those aged 10–24 years.1–4 Annually, approximately 20% of adolescents encounter a mental health condition, and on a global scale, suicide ranks as the fourth leading cause of death among individuals aged 15–19.5 Yet globally, less than 50% of children and young people with mental health problems receive treatment.6 7 Reasons for this treatment gap are barriers, including costs, mental health stigma, low mental health literacy, poor access to appropriate services and inadequate health system structures.8 Individuals from minoritised racial and ethnic backgrounds, those with lower socioeconomic status (SES), and/or those residing in marginalised communities use mental health services even less frequently, even though they are two to three times more likely to develop mental health problems.9 Marginalised youth also often encounter lower service quality than their white and higher SES counterparts.10 11 They face higher barriers than non-marginalised youth, such as concerns about cost, confidentiality, lack of convenience, cultural and language issues, and troubles navigating the system.12 The high burden of mental health problems in young people and the lack of access to appropriate services warrant novel and accessible ways of providing mental health treatment.

Delivering mental health interventions via mobile devices holds promise to address the treatment gap.13 14 Mental health apps offer several advantages, including scalability and feasibility among youth who are avid digital health adopters.15 Additionally, these apps can be personalised, advancing technological capabilities. Various methods, such as ecological momentary assessments/interventions and AI-based methods, such as machine learning, can be used to develop novel and promising personalised and real-time intervention strategies.16 For paediatric and young adult samples, reviews and meta-analyses show that mental health apps are a promising stand-alone self-management tool in mental health service delivery.17 18 Recently, the National Institute for Health and Care Excellence (NICE) guidelines recommended four digital technologies, including apps, for use in the UK’s National Health Service to help children and young people with mild to moderate symptoms of anxiety or low mood once they achieve regulatory approval.19 Thus, Digital Mental health (DMH) could serve as a low-cost, scalable, easily accessible and efficacious approach to delivering mental healthcare which will likely become more important in the future for reducing the treatment gap for youth.

However, several pressing issues impede the equitable use of mental health apps for young people. For instance, commercially available apps often lack evidence-based foundations,14 20 grapple with privacy concerns21 and are hampered by gender and racial biases.22 In addition, mental health apps exhibit low real-world reach and uptake,23 especially in low-resource settings18 and their (cost)-effectiveness and successful implementation is still unclear.18 23 Furthermore, there is a notable under-representation of marginalised youth, including racially minoritised youth(eg, black, Indigenous and people of colour), those with low SES, migration backgrounds and diverse gender identities, in their design, testing and use of DMH.24 For instance, a review on implementation factors in mental health apps for youth found that 71% of studies had recruited university students,23 and a review on preventive digital mental health (mostly internet based) found that demographic factors, such as SES and ethnicity, were often not reported.25 These studies likely point to a lack of inclusive and equitable representation among users.

Several researchers have argued for the need for diversity, equity and inclusion (DEI) considerations in the development and evaluation of mental health apps.26 27 This includes DEI consideration in the design and evaluation of the apps (eg, access, content and appearance),26 representation and cocreation with diverse youth and community partners, and intersectionality analyses to evaluate their effectiveness.27 These steps are suggested as a crucial strategy to increase the uptake and engagement of digital mental health solutions, particularly among minoritised youth and promote equitable access.28 Despite the field’s critique for lacking diversity and addressing health equity issues, comprehensive assessments of how well DEI considerations are integrated into the design and evaluation of mental health apps for young people are lacking.

Review question

Considering these limitations, this review aims to examine to what extent DEI considerations are incorporated in the design and evaluation of mental health apps for youth. Considering the gap in knowledge on this topic, a scoping review is suited to investigate this.

We focus on apps that target crucial and timely subjects, depression, anxiety and suicidal ideation. Depression and anxiety disorders are among the leading causes of illness and disability among adolescents, and a suicide is a leading cause of death in adolescents aged 15–29.29 30 Symptoms of depression, anxiety and suicidal ideation spiked and remained high after the COVID-19 pandemic in young people.1 2 We will apply the DEI framework developed by Ramos et al, adopted from the culturally informed design framework,26 specifically tailored to mental health apps.26 Ramos et al developed this framework as a tool to assess whether mental health apps consider factors such as access to digital tools, diverse representation and culturally adapted content. Using the tool, the authors conducted a systematic review of published mental health app frameworks, and found that, of the 44 frameworks evaluated, only 58% considered at least one DEI criterion. These factors are, however, crucial to develop digital interventions inclusive of individuals with various backgrounds and promote health equity.


Specifically, we aim to examine (1) the prevalence of ethnic minoritised youth (racial/ethnic diversity), low-income youth (socioeconomic diversity) and gender-diverse youth in mobile health apps targeting common mental health disorders (ie, anxiety, depression and suicidal ideation), among children and young adults aged 10–25 years; (2) the proportion of DEI criteria considered by included studies in the design and testing of youth DMH and (3) reported methods of involving youth and other important stakeholders (such as parents) in the development of mental health apps based on participatory research (eg, shared and trustworthy leadership, agility and adaptiveness in research activities and engagement in the community, and use of effective and ongoing communication).

The results of this study will be crucial for exploring gaps in the literature and for identifying future directions in the field for a more inclusive youth digital mental health ecosystem.


  • Most studies lack significant representation of racial/ethnic minoritised participants, participants with low SES and gender minorities.

  • Most studies fail to incorporate DEI criteria (eg, access, content and appearance) into the design and evaluation of mental health apps.

  • Most studies do not include information about the involvement of youth, and other important stakeholders including caregivers, in the development of mental health apps, or when they do, rely on one-time participation models (eg, complete a focus group or single usability testing session).


The proposed scoping review will be conducted in accordance with the Joanna Briggs Institute methodology for scoping reviews.31 We also use the Preferred Reporting Items for Systematic Reviews and Meta-Analyses-Scoping Review Extension (PRISMA-ScR).32


This review title has been registered with Open Science Framework (

Search strategy

Development of the search strategy

The search strategy was developed by the team with the consultation of two research librarians. We used an iterative approach, with two rounds of preliminary searches and refinement of the search strategy based on initial search results. The search strategy, including all identified keywords and index terms, was adapted for each included database and/or information source. The reference list of all included sources of evidence will be screened for additional studies.

The databases searched include Medline, Global Health, APA PsycINFO, SCOPUS, CINAHL PLUS and the Cochrane Database of randomised trials. All searches were conducted on 10 September 2023–29 September 2023. Full-search strings are shown in online supplemental appendix 1. Studies published in English will be included to ensure consistency and feasibility in the review process. Studies published from 1 January 2009 29 to September 2023 will be included. We chose to include studies from 2009, as there was a significant uptake in studies on digital mental health at this time.

Supplemental material

Study/source of evidence selection

Following the search, all identified citations will be collated and uploaded into Covidence Software (, a web-based systematic review platform, and duplicates will be removed. Following a pilot test, titles and abstracts will then be screened by two or more independent reviewers for assessment against the inclusion criteria for the review. The full text of selected citations will be assessed in detail against the inclusion criteria by two or more independent reviewers. Reasons for exclusion of sources of evidence in full text that do not meet the inclusion criteria will be recorded and reported in the scoping review. Any disagreements that arise between the reviewers at each stage of the selection process will be resolved through discussion, or with an additional reviewer/s. The results of the search and the study inclusion process will be reported in full in the final scoping review and presented in a PRISMA-ScR flow diagram.32

Data extraction

Data will be extracted by two or more independent reviewers. We have developed (and piloted in October 2023 with five studies) a data extraction tool (see online supplemental file 1). The data extracted will include the country of the study, the type of intervention, the racial/ethnic composition of patients, sociodemographic background and gender composition. Additionally, we will extract whether studies assessed DEI criteria and whether they reported any youth, caregiver or other stakeholder involvement. To the DEI criteria by Ramos et al, we added cultural adaptation and privacy/security considerations. We will also include brief descriptions of the methods used to assess DEI criteria (if any) and how studies operationalised youth and stakeholder involvement. The draft data extraction tool will be modified and revised as necessary during the process of extracting data from each included evidence source. Modifications will be detailed in the scoping review. Because this study examines how studies report and take into account DEI criteria, we will not contact authors to ask for additional information. Any disagreements that arise between the reviewers will be resolved through discussion or with an additional reviewer/s.

Inclusion and exclusion criteria

As inclusion criteria, articles should:

  1. Have a primary focus on depression, anxiety or suicidal ideation.

  2. Be published in a peer-reviewed publication.

  3. Be published in English.

  4. Involve smartphone applications (with or without wearable sensor, website, etc).

  5. Involve human subjects data.

  6. Have clinical or non-clinical populations as the primary users of the technology developed and/or tested.

  7. Study adolescents or young people (aged 10–25). We will include studies where adolescents are in this age range, or if the mean age of the participant sample is 25 years or less, and above 10 years of age.

Articles will be excluded if the following were met (1) The primary focus was not on depression, anxiety or suicidal ideation; (2) The study exclusively involved SMS/MMS or phone calls, or web-based interventions or wearables that did not involve a smartphone app and (3) The study’s participants were under the age of 10 or above the age of 25.

Types of sources

We will include pilot and feasibility studies, app development studies and randomised trials.

Measured variables and data extracted

Main outcomes of this study

  1. Proportion of studies that provide information on the racial and ethnic composition, SES, and gender identity of the recruited participants.

  2. Where available, the overall prevalence of racial/ethnic minoritised patients (between-study median and range), the median age, education, immigration status (of parents), annual income (of parents) and gender.

  3. Proportion of studies that report at least one DEI criteria in the development and/or assessment of interventions. We will examine access, content and appearance.

    • Access: (1) internet connectivity, (2) data usage, (3) cost, (4) system requirements.

    • Content: (1) language, (2) literacy, (3) tailoring of content-environment, (3) tailoring of content-disability and special needs and (4) tailoring of content-identity.

    • Appearance: (1) diverse visual representation.

  4. Proportion of studies that reported involving youth in the development of mental health apps based on participatory research (eg, shared and trustworthy leadership, agility and adaptiveness in research activities and engagement in the community and use of effective and ongoing communication).

Racial/ethnic minoritised status will be identified on a per country basis. If studies only reported the percentage of white participants, all other participants will be assumed to be ethnic minorities. If participants were categorised as belonging to an ‘other’ group, they will be assumed to belong to ethnic minority groups.

Data analysis and presentation

Our data extraction template inonline supplemental material shows the data we will extract and analyse. Using the template, we will calculate the percentage of included studies that report the racial/ethnic composition of patients, sociodemographic background and gender composition. If available, we will analyse the median percentage of racial and ethnic minority patients, the gender distribution and prevailing education level.

Additionally, we will report the percentage of studies that assessed at least one DEI criteria (access, content and appearance, see above, and the percentage of studies that reported any youth and caregiver involvement). For each study, we will report which of the DEI criteria was assessed, include brief descriptions of the methods used to assess DEI criteria (if any) and how studies operationalised youth involvement. We will present this in a narrative summary and show the main outcomes in tables (see our data extraction template) and/or charts, describing how the results relate to the objectives and questions.

Patient and public involvement

No patients or the public were involved in the review. Results of the study will be discussed with youth and other stakeholders, such as social workers, caregivers and psychologists in the Netherlands and the USA to define future research priorities.

Ethics and dissemination

This study involves neither human participants nor unpublished secondary data. As such, approval from a human research ethics committee is not required. Findings of the scoping review will be disseminated through professional networks, conference presentations and publication in a scientific journal. In addition, we will disseminate findings in our nonacademic networks, including youth organisations and mental health institutions.

This study will make a novel contribution by assessing DEI considerations in research on the development and evaluation of mental health apps for young people. As the digital mental health field is expected to continue to grow in the coming years with advances in technology and the rising demand for mental healthcare, an equitable transformation is crucial. This research will help provide a roadmap for equitable, inclusive and responsible digital mental health development for adolescents and young adults.

Ethics statements

Patient consent for publication


We would like to thank Melissa Vetter and Dirk-Jan Ligtenbelt for their assistance with the search strategy.


Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.


  • X @Colleen_SS

  • Contributors Design of the protocol: CAF, KWG, NJP-F and CS-S. Draft of the manuscript: CAF. Review and final approval of the manuscript: CAF, KWG, NJP-F and CS-S.

  • Funding CAF and KWG are supported through the ‘High Tech for a Sustainable Future’ capacity building programme of the 4TU Federation in the Netherlands (grant number N/A).

  • Disclaimer The funders had no role in the content of the manuscript.

  • Competing interests None declared.

  • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.