Article Text
Abstract
Introduction There is a high prevalence of mental health problems among university students. Better prediction and treatment access for this population is needed. In recent years, short-term dynamic factors, which can be assessed using experience sampling methods (ESM), have presented promising results for predicting mental health problems.
Methods and analysis Undergraduate students from five public universities in Spain are recruited to participate in two web-based surveys (at baseline and at 12-month follow-up). A subgroup of baseline participants is recruited through quota sampling to participate in a 15-day ESM study. The baseline survey collects information regarding distal risk factors, while the ESM study collects short-term dynamic factors such as affect, company or environment. Risk factors will be identified at an individual and population level using logistic regressions and population attributable risk proportions, respectively. Machine learning techniques will be used to develop predictive models for mental health problems. Dynamic structural equation modelling and multilevel mixed-effects models will be considered to develop a series of explanatory models for the occurrence of mental health problems.
Ethics and dissemination The project complies with national and international regulations, including the Declaration of Helsinki and the Code of Ethics, and has been approved by the IRB Parc de Salut Mar (2020/9198/I) and corresponding IRBs of all participating universities. All respondents are given information regarding access mental health services within their university and region. Individuals with positive responses on suicide items receive a specific alert with indications for consulting with a health professional. Participants are asked to provide informed consent separately for the web-based surveys and for the ESM study. Dissemination of results will include peer-reviewed scientific articles and participation in scientific congresses, reports with recommendations for universities’ mental health policy makers, as well as a well-balanced communication strategy to the general public.
Study registration osf.io/p7csq.
- MENTAL HEALTH
- EPIDEMIOLOGIC STUDIES
- PUBLIC HEALTH
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
STRENGTHS AND LIMITATIONS OF THIS STUDY
Prospective multicentre cohort study including participants from five universities, representative for the total undergraduate student population in Spain.
Assessment methods combine web-based self-report surveys with experience sampling methods in order to assess long-term as well as short-term dynamic mental health problems and associated risk and protective factors.
Stakeholders’ involvement is maximised through focus groups with students and academic personnel, during the design phase of the study, prior to recruitment of participants.
Data representativeness of web-based survey data is limited due to lack of personalised invitation in some participating universities, and low participation rates in the web-based surveys.
The use of self-report screening instruments to detect mental health problems rather than in-person clinical interviews to diagnose disorders is another limitation.
Introduction
Around half of Spaniards between 18 and 22 years of age are enrolled in university studies, together with an increasing proportion of young people worldwide.1 2 This period carries risk for the onset or exacerbation of mental health problems, as found in a large epidemiological study in eight countries showed that freshmen screened positive for high rates of 12-month major depressive disorder, generalised anxiety disorder, substance use disorder and suicidal ideation (18.5%, 16.7%, 6.3% and 17.2%, respectively) based on self-report. However, only one-fourth of students reported that they would seek treatment if they struggled with their mental health, with the highest barrier to treatment being preferring to tackle the problem on their own.3 4
In order to better opportunities for treatment among this population, it is crucial to improve the methodologies we use to evaluate and predict outcomes of mental health problems. Existing prediction models use exclusively distal (eg, sociodemographic characteristics, adverse childhood experiences) and proximal risk factors (eg, lifestyle factors, social support, grades), which can be well captured in one-time surveys.5 6 Though these models are promising,7–11 they do not include key short-term dynamic elements such as stress, affect, sleep which are observed in common mental disorders, that can fluctuate over minutes to hours.12–17 Experience sampling methods or ESM, that is, survey data from multiple occasions within the same day and over time,18 is a study design that allows to compile the information needed to describe these short-term dynamic factors. Defining these short-term risk factors and incorporating them into prediction models could be key to create accurate personalised prospective risk prediction models for mental health problems, providing a solid foundation for a more nuanced and effective treatment.
Here, we present the protocol for the observational prospective study within the Promoting Mental Health among University Students (PROMES-U) project, part of the WHO World Mental Health International College Student (WMH-ICS) Initiative.5 The project takes place in five public universities in Spain and combines two substudies (figure 1). The first is a prospective observational cohort study to assess mental health and its associated risk and protective factors. This study will use baseline and 12-month follow-up web-based surveys and a 15-day ESM study. The second study design is a randomised controlled trial of an e-health prevention intervention. The present protocol focuses on describing the design and methodology of the observational cohort study.
Objectives
The objectives of the PROMES-U observational study are: (1) to estimate the frequency of mental health problems (ie, major depression, generalised anxiety disorder, panic disorder, mania and hypomania, ADHD, substance abuse and dependence, eating disorders and suicidal thoughts and behaviours) among undergraduate students; (2) to investigate the associations of a wide range of potential distal and short-term risk and protective factors with mental health problems; (3) to investigate treatment use for mental health problems, including perceived facilitators and barriers to seek treatment and (4) to develop explanatory and predictive models for outcomes of mental health problems and associated treatment use among university students.
An innovative aspect of the project is that data collection will consist of two web-based surveys (baseline and 12-month follow-up) as well as a 15-day ESM study. This will allow to estimate prevalence of various mental health problems, 1-year incidence and persistence as well as the short-term course of key symptoms (depression, anxiety, suicidal ideation and harmful alcohol use) and distal risk factors. Of special interest is the short-term construct of stress sensitivity, that is, the momentaneous correlation between stress and affect, as this construct has been shown to be related to a number of mental health problems.19–22
Methods and analysis
Study design and population
The study consists of a prospective observational multicentre cohort study implementing two web-based self-reported surveys (at baseline and 12-month follow-up) and a 15-day ESM study (figure 1).
The project’s target population are all undergraduate students of five public universities in Spain: University of the Balearic Islands, Jaume I University, Miguel Hernández University, Pompeu Fabra University and University of Zaragoza. There are 69 805 undergraduate students in these universities, which represent a 5.2% of all undergraduates studying in public universities in Spain, and closely mimic the proportion of students across study fields, sex and nationality of the total undergraduate students in Spain23 both in general and stratified by sex (table 1).
Data collection for both the baseline web-based survey and the 15-day ESM study took place from April to June of 2022. Data for the follow-up web-based survey will be collected from May to July of 2023. These data have only been accessed to obtain information concerning the sample and participation, and no analyses have yet been performed.
Participant recruitment
Web-based surveys. All Spanish-speaking undergraduate students aged 18 and older enrolled in any of the five participating universities are eligible to participate in the baseline web-based survey. A convenience sample is formed through a combination of different strategies. In three of the universities and some colleges of a fourth one, students are invited via mass mailing to enlisted students using university administrative lists, with one to three reminder emails. In all universities, this strategy is complemented with on-campus project information sessions, classroom announcements, link on the universities’ websites, poster campaigns and social media posts on the universities’ accounts. All baseline participants that provide an email address at the end of the baseline survey will be invited to the follow-up survey.
15-day ESM study. At the end of the baseline survey, participants are assigned into one of four mutually exclusive groups through an algorithm programmed within the Qualtrics survey platform. The groups are hierarchically created using scores of mental health problems in the 30 days prior to the web-based survey for: (1) suicidal ideation, (2) alcohol abuse and no co-occurring 30-day suicidal ideation, (3) depression and/or anxiety and no co-occurring suicidal ideation or alcohol abuse, (4) students without 30-day depression, anxiety, alcohol abuse or suicidal ideation. This strategy seeks to obtain sufficient statistical power to study all target mental health problems, aiming to recruit approximately 175 per group. See Sample and Statistical Power section for further details.
Conceptual framework for the 15-day ESM study
The conceptual framework for the ESM study is presented in figure 2. This framework is based on a literature search and a series of group discussions of our team members, which include mental health professionals, epidemiologists as well as experts in survey methodology and psychometrics. The search identified 134 studies that used ESM methods to investigate the relationship between stress and mental health problems among adolescents and young adults (see online supplemental annex). Of these, 70 were deemed relevant after screening the title and abstract. Full-text papers were screened to identify known intermediary constructs between stress and mental health problems, focusing on depression, anxiety, alcohol abuse and suicidal ideation. The search showed multiple intermediary constructs that play a role in defining this relationship, such as sleep quality, healthy behaviours, stress coping strategies, negative and positive affect and emotion regulation strategies. Of note, multiple papers focused on the role of stress sensitivity (ie, the momentary relationship between negative affect and stress) as a potentially relevant risk factor for mental health problems.20–22 24 25
Supplemental material
Measures
Web-based surveys. Data collection for the web-based self-reported assessments consists of a web-based survey developed by the WMH-ICS Initiative consortium and implemented using the Qualtrics survey platform. The survey contains 11 sections.
Section 1
‘Your background’ assesses age, assigned sex at birth, current gender identity, current student status, being an international student, type of study, primary and secondary nationalities and parental origin.
Section 2
‘Your health’ assesses general physical and mental health, role limitations associated with physical and mental health problems, social functioning, sleep problems, vitality, manic/hypomanic symptoms, bodily pain and lifetime mental health problem screener items.
Section 3
‘Attention and Concentration’ screens for 6-month attention deficit hyperactivity disorder using the ASRS-V.1.1 scale.26
Section 4
‘Emotional problems’ includes lifetime, 12-month and 30-day screeners for major depressive disorder, generalised anxiety disorder, panic attacks and manic/hypomanic episode using scales from the CIDI-SC;27–29 intermittent explosive disorder using scales from the Army STARRS New Soldier Study Survey;30 post-traumatic stress disorder using the four-item version of the PCL-5;31 eating disorders measured with the PRIME-MD PHQ and panic disorder, manic/hypomanic episode and social anxiety disorder using newly items developed based on DSM-5 diagnostic criteria.32
Section 5
‘Alcohol and Drugs’ screens for substance use disorders using the Alcohol Use Disorders Identification Test (AUDIT33), and the Alcohol, Smoking, and Substance Involvement Screening Test (ASSIST34).
Section 6
‘Self-harm’ includes screeners for lifetime, 12-month and 30-day suicidal thoughts and behaviours, as well as non-suicidal self-injury, assessed with questions from the Columbia-Suicide Severity Rating Scale (C-SSRS35 36), the SITBI37 and the P4 Screener.38
Section 7
‘Seeking Treatment’ assesses lifetime and 12-month treatment use for emotional or substance use problems, motivation to change current problems, the perceived need for treatment and the perceived barriers to seek treatment.
Section 8
‘Childhood background’ assesses parental education, a range of childhood-adolescent potentially traumatic and/or stressful experiences (ie, parental psychopathology, physical abuse, emotional abuse, sexual abuse, neglect, bully victimisation, dating violence) and experienced family love, care and emotional closeness.
Section 9
‘Recent experiences’ assesses 12-month stressful events, current stress severity, social media use, online and in-person social interactions and social network.
Section 10
‘Sexuality’ assesses sexual orientation, sexual attraction, sexual behaviour and relationship status.
Section 11
‘Concept of Self’ assesses personality traits such as disinhibition, detachment, psychoticism, negative affect, antagonism and attachment. See online supplemental table 1 for more information. Measures for three mental disorders (ie, panic attacks and disorder, post-traumatic stress disorder and bipolar disorder) are part of an internal random subsampling scheme in order to reduce the survey response burden. Meaning that only a random selection of participants who screen positive for a certain disorder are asked details regarding symptomatology and lifetime, 12-month and 30-day occurrence. In order to further reduce survey response burden, a short and a long version of the personality measure was developed and assessed using an internal random subsampling scheme.
Supplemental material
Positive screens for all measured disorders are referred to as ‘mental health problems’ throughout this paper.
Students that agree to participate in the ESM study are asked three additional measures as final part of the Qualtrics baseline survey: (1) a healthy behaviours scale assessing healthy diet, sleep schedule, exercise, breaks while working, planning of daily tasks, hobbies, quality time with loved ones, sexual activity satisfaction and personal hygiene; (2) stress coping strategies such as self-distraction, emotional support, active coping, denial, venting, substance use, self-blame, planning, behavioural disengagement, positive reframing, humour, acceptance and religion, using the Brief Cope Inventory (BCI39) and (3) emotion regulation strategies, such as self-distraction, emotional support, active coping, denial, venting, substance use, self-blame, planning, behavioural disengagement, positive reframing, humour, acceptance and religion, using the Brief Version of the Difficulties in Emotion Regulation Scale (DERS-1640).
15-day ESM study. This assessment was developed by our research team and implemented using the Expiwell smartphone-based application. It assesses a range of variables and constructs using different time intervals and recall periods.
On days 1 and 15, the instrument assesses 2-week major depressive disorder, using the PHQ-9;41 2-week generalised anxiety disorder, using the GAD-742 and 2-week Mental Wellbeing, using the short version of the Warwick Edinburgh Mental Wellbeing Scale (SWEMWBS43;).
On days 2 through 15, five constructs are assessed on a momentary basis (ie, 4 times a day) using a single set of 12 7-point Likert-type scales: momentary stress, affect, depression, anxiety and well-being. This set of items consist of an introductory phrase (‘Right now, I feel …’) followed by 12 very short descriptions of momentary states (eg, ‘stressed’, ‘annoyed, irritable’, ‘interested or pleasure in doing things’). Descriptions refer to key symptoms from depression (adapted from the PHQ-941), generalised anxiety (adapted from the GAD-742) and well-being (adapted from the SWEMWBS43) as well as to stress and affect (items taken from the ESM item repository.44 The assessment of affect is based on the two axis circumplex model of affect,45 that is, high/low arousal positive and negative affect. Negative and positive affects are measured combining these items from validated scales. Five items are used to measure positive affect (ie, feeling ‘happy’, ‘interest in doing things’, ‘relaxed’, ‘optimistic’, ‘pleasure doing things’) and four to measure negative affect (ie, feeling ‘nervous’, ‘upset’, ‘depressed’, ‘worried’). A mean value for positive and negative affect will be calculated for each evaluation.
Additionally, participants are asked about social context and interactions, including place, activity and company. Morning and evening surveys contain additional questions in order to collect information sleep duration and quality, 24-hour alcohol use, 24-hour type of stressful events, and 24-hour suicidal ideation.
Participants receive notifications on their smartphones at a random time within 2-hour time intervals in the morning (08:00–10:00 hours), midday (12:00–14:00 hours), afternoon (16:00–18:00 hours) and evening (20:00–22:00 hours). If the participant has yet to complete the survey, they receive a notification 30 min after the first one. Each assessment is open during a 2-hour window after which it cannot be answered and is considered a missed evaluation. See figure 2 for information regarding ESM measures.
Return and incentives for participation
After finalising the web-based surveys, all participants can download a personalised report containing a summary of their results, which focuses on psychoeducation, increasing mental health literacy, reducing perceived barriers for treatment seeking and providing adequate resources to deal with any mental health problems students may experience. Resources included on-campus possibilities of treatment, referral to mental health services within the national health service, and several telehealth resources.
Two months after the assessment, participants are administered an eight-question survey to evaluate level of satisfaction and impact of the personalised report (ie, length and adequacy of design, summary of their results, habit recommendations, mental health services recommendations, implementation of any lifestyle recommendation, having taken steps to access help and general level of satisfaction).
In order to incentivise participation in the ESM study, participants that complete at least 80% of daily surveys as well as mental health screener scales on days 1 and 15 receive a monetary incentive of 30 euros, which is below the mean in ESM studies.46 The budget for the project defined the maximum number of students we could invite to the ESM study, thus the target to enrol 175 students for each of the 4 hierarchical groups of interest, providing high statistical power within each group.
Co-creation of methods, design and dissemination
Stakeholders’ involvement is maximised through focus groups recruited through snowball sampling. Five separate groups were formed: two with students, two with key stakeholders (eg, vice-rector, professor) and one in-depth individual interview. Focus groups were conducted during the design phase of the study, prior to the recruitment, carried out online through a video conferencing software and led by a study researcher and two experts in qualitative methodology. An investigator triangulation, categorisation and content analysis were performed. Sessions pivoted around general acceptability of the project and specificities of the personalised report and ESM study. This permitted to (1) identify students’ drivers of engagement and pathways of dissemination of the study; (2) identify specific needs regarding mental health problems and customise instruments accordingly; (3) assess and increase feasibility of ESM study and (4) improve personalised reports’ content to increase its acceptability and expected impact.
Information gathered through the focus groups shaped multiple aspects of the study. It informed that dissemination should be done through a combination of strategies (eg, email, social media, posters), that it should highlight both the personal benefit that can arise from participating in the project and the anonymity of the data collection process and be done in coordination with the communication departments of the universities. Additionally, informants suggested that participants of the ESM study with a monetary incentive, given its higher burden. Finally, items regarding company and types of stress (eg, academic, financial, relational) were added to the momentary surveys of the ESM study.
As part of the evaluation and the cocreation processes, a 3-week trial run of the ESM study was conducted from 24 November to 14 December 2021, in which a convenience sample of 28 eligible students participated. Majority of participants were women (64.3%) studying social sciences degrees (42%). Given the feedback, the ESM study was shortened from 21 to 15 days and no GPS data would be collected. 84.5% of trial run participants found that the momentary evaluations were relevant to assess their mental health, which aligned with the information provided by the focus groups.
Sample and statistical power
A total of 2427 eligible students finalised the baseline web-based survey, and 2533 completed all the questions concerning their mental health status, which could be considered in some of the planned analysis. For logistic regression prediction models (with approximately 28 predictor parameters), assuming a Cox-Snell R2≥0.1, expected shrinkage of 10% (to minimise potential overfitting) and 7% incidence of any of the mental health problems measured, a sample size of 2427 individuals allows to estimate the overall rate of the population with a margin of error ≤0.05, and difference between apparent and adjusted Nagelkerke’s R-squared ≤0.05.47
782 participants of the ESM study with an average compliance rate is of 76.95%, considering compliance as ‘the proportion of self-evaluations completed by the participants compared with the theoretical maximal number of self-evaluations allowed by the design’.46 The smallest experimental group in the study has a sample of n=167 individuals, with T=56 evaluations each, representing a total number of observations (n*T) higher than 9300. A previous study based on Monte Carlo simulations with 500 replications, of nine different multilevel models and dynamic structural equation models of increasing complexity, showed that with more than 9000 observations, all models assessed presented good performance, with high overall precision (as measured with mean squared errors close to 0), and statistical power close to 1 to detect moderate associations (effect sizes around 0.4 SD) of the between-level parameters.48
Analysis plan
Basic frequency will be used to calculate prevalence, incidence and persistence of mental health problems by measuring occurrence of mental disorders and treatment use. χ2 and Fisher’s exact tests will be used to test bivariate correlations between outcomes and potential risk factors. Furthermore, risk factors will be assessed both at an individual and population-level using logistic regressions (Odds Ratios, OR) and Population Attributable Risk Proportions (PARPS), respectively.
Advanced statistical techniques will be used to accomplish the totality of proposed objectives. Weighting techniques (eg, inverse probability weighting) and full information maximum likelihood (FIML) estimation will enable to address non-response bias and missingness to the maximum extent, in order to obtain population-representative estimates. Multilevel analysis will allow us to carefully differentiate between between-individual and within-individual differences in occurrence of mental health problems and its associations with risk or protective factors. Machine learning techniques will be considered to optimise the prediction accuracy of predictive models. To develop explanatory models, the use of an integrated framework such as dynamic structural equation modelling will be considered, to allow for the integration of multilevel, structural equation, time-series and time varying effects.49
Intraclass correlation (ICC) will be used to describe how much of the variance observed in the ESM study data is due to within-person and between-person differences. An ICC below 0.5 reflects that variance is mostly due to within-person differences, (ie, momentary fluctuations), which are expected to be more important among those symptoms of mental health problems.50 Finally, reliability of ESM study data will be assessed using the ‘multilevel.reliability’ function from R’s psych package.51
Handling missing data
FIML will be used whenever the statistical technique allows for it, which provides a flexible and robust approach to dealing with missing data. In cases where it is not possible to apply FIML, multiple imputation methods (MI) by chained equations will be implemented. Simulation studies provide evidence that both FIML and MI by chained equations generally yields estimates that are unbiased and provide appropriate coverage, particularly under missing at random assumption.52 53
Non-response and attrition bias in ESM study and 12-month follow-up survey will be addressed by applying sample weights through poststratification and inverse probability weighting procedure that matches the final sample to: (1) the population distribution of gender, age, nationality and field of study in each participating university as well as distribution of students across institutions and (2) the full sample of baseline participants according to all baseline survey variables.
Study registration
See osf.io/p7csq.
Patient and public involvement
None.
Ethics and dissemination
Explicit informed consent is asked at the beginning of the web-based surveys, and at the end of this survey in order to obtain consent for participation in the ESM study. The project complies with the principles established by national and international regulations, including the Declaration of Helsinki and the Code of Ethics. The study protocol was approved by the IRB Parc de Salut Mar (2020/9198/I) and corresponding IRBs of participating universities. Respondents were given information regarding access to mental health services within their university and region. Individuals with positive responses on suicide items received a specific alert with indications for consulting with a health professional.
Dissemination of results will be carried out for scientists and clinicians, authorities involved in university and health policies and university students. Scientific articles will be written and sent to high-impact journals, prioritising open access publications. Articles will be presented in conferences and oral communications. In order to reach university and health authorities, conferences, symposiums and exchanges will be held to disseminate the results. Finally, students will be informed of the results through seminars, social media, the project’s website and their university’s website.
Discussion
The observational prospective study within the PROMES-U project provides monitorisation of mental health of university students in Spain considering both distal risk and short-term dynamic risk factors collected using an ESM study design in order to have a more nuanced understanding of incidence and persistence of mental health problems. This will allow to develop more accurate personalised models to describe a person’s mental health and predict how it may develop, a steppingstone in the path in the innovative and revolutionary path that is personalised mental healthcare providing more effective and accessible avenues to treatment through the use of technology.
The current study has a number of strengths and weaknesses. Important strengths include the prospective multicentric cohort study design representative for the total undergraduate student population in Spain, and assessment methods that include both web-based self-report surveys and ESM in order to assess long-term as well as short-term dynamic constructs. Weaknesses include low participation rates in web-based student surveys and lack of personalized invitation in some participating universities which could lead to non-response bias compromising population representativeness. Other weaknesses include the use of self-report screening instruments to detect mental health problems rather than in-person clinical interviews to diagnose disorders and potential difficulties in coordination in timelines between participating universities, due to the multicentric nature of the study.
Ethics statements
Patient consent for publication
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors PM, JA, LB, GV, BPG, MG, JAP, MRR, MR and TRJ were responsible of study conceptualisation and design. AP-VD, PM and GV wrote the first draft of the article. LB, JA, FAA, BPG, HG-M, IAB, MJBC, PCE, RF, IF-R, PG-P, MG, CG-G, FHM, JCMC, GNV, JAP, MRR, MR, TRJ, LR, ER-P and VSS provided critical revisions. All authors read and approved the submitted manuscript.
Funding This work is supported by Instituto de Salud Carlos III (ISCIII) and cofunded by the European Union, grant number PI20/00006. This work was supported by grants from Instituto de Salud Carlos III-Fondo Social Europeo Plus (FSE+), awarded to PM (ISCIII MSERVET Exp. CP21/00078 and CD18/00049); and the Departament de Recerca i Universitats of the Generalitat de Catalunya (AGAUR 2021 SGR 00624).
Competing interests None declared.
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.