Article Text
PDF
Abstract
Introduction Adolescence is a sensitive life stage during which tobacco, alcohol and cannabis are used as ways to learn and adopt roles. There is a great deal of interest in substance use (SU) prevention programmes for young people that work to change representations of these products and help with mobilisation of life skills. Unfortunately, few existing programmes are evidence-based.
In France, a programme called Expériences Animées (EA, Animated Experiences) has been developed, inspired by life skills development programmes that have been proven to be successful. The EA programme uses animated short movies and talks with high school and secondary school pupils about the use of psychoactive substances and addictions. By allowing life skills mobilisation and modifying representations and beliefs about SU, it is aimed at delaying initiation of use of psychoactive substances, preventing adolescents from becoming regular consumers, reducing the risks and harms related to the use of these substances and opening the way for adapted support measures.
We are interested in understanding how, under what circumstances, through which mechanisms and among which adolescents the EA programme works. Therefore, we have developed the ERIEAS study (‘Evaluation Réaliste de l’Intervention Expériences Animées en milieu Scolaire’; Realist Evaluation of the EA Intervention in Schools).
Methods and analysis EA will be conducted in 10 schools. A multi-case approach will be adopted with the aim of developing and adjusting an intervention theory. The study comes under the theory-driven evaluation framework. The investigation methodology will include four stages: (i) elaboration of a middle-range theory; (ii) data collection for validating/adjusting the theory; (iii) data analysis; and (iv) refinement and adjustment of the middle-range theory and definition of the programme’s key functions.
Ethics and dissemination The study will provide evidence-based results to health authorities to help in the rollout of health promotion strategies in schools. It will provide knowledge about the strategic configurations most suitable for leading to life skills mobilisation and change young people’s representations about SU. The project will be carried out with full respect of current relevant legislation (eg, the Charter of Fundamental Rights of the European Union) and international conventions (eg, Helsinki Declaration). It follows the relevant French legislation of the research category on interventional research protocol involving the human person. The protocol was approved by the Comité et Protection des Personnes (CPP), that is, Committee for the Protection of Persons CPP SUD-EST VI n°: AU 1525 and was reported to the Agence Française de Sécurité Sanitaire des Produits de Santé (ANSM) that is, the French National Agency for the Safety of Health Products. It is in conformity with reference methodology MR003 of Bordeaux University Hospital (CNIL n° 2 026 779 v0).
Trial registration details
This research has been registered on ClinicalTrials.gov (No. NCT04110626).
The research project is registered in the European database ID-RCB (No. 2019-A01003-54).
- life skills
- addictions
- realist evaluation
- preventive medicine
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
Consistent with bottom-up approaches, our study—a realist evaluation based on a natural experiment mobilising mixed models methods and a preference stated method (Discrete Choice Experiment)—is an innovative way to evaluate a complex intervention.
The conclusions will be highly replicable and will provide a basis for designing other interventions using identified key functions.
Our study will produce detailed recommendation and further strategies to develop the Expériences Animées (Animated Experiences) programme and adapt it to various contexts.
Introduction
Addictive behaviours are major contributors to human morbidity and premature death, with tobacco and alcohol use being the most prevalent addictive behaviours and cannabis the most prevalent ‘unsanctioned psychoactive drug’ used worldwide.1 Adolescence is a particularly sensitive life stage where tobacco, alcohol and cannabis are used as a means of learning and taking on roles. Consumption of these substances and their associations with cancer risk, chronic disease and health conditions have been clearly discussed in the literature.2 3 As adolescence is a period during which the brain undergoes profound remodelling in a number of areas, alcohol and cannabis consumption can lead to impaired learning ability, memory and brain development and to difficulties in cognitive control and emotion regulation.4–6
The WHO defines life skills as abilities for adaptive and positive behaviour that enable individuals to deal effectively with the demands and challenges of everyday life.7 In 2010, Botvin and Griffin reviewed school-based substance use prevention programmes that have been tested and proven effective.8 These programmes focus on building drug resistance skills, general self-regulation and social skills and/or changing normative expectations regarding inaccurate beliefs about the high prevalence of substance use.8 A major focus is the development of alcohol, tobacco and cannabis prevention programmes for young people that work on these skills and on changing representations about these products and, therefore, their consumption.
Many programmes focussing on addiction prevention have been implemented in schools.9 10 Unfortunately, there are few evidence-based and life skills programmes.9 11 In France, since 2015, the organisation Association Ressources et Initiatives Addictions (ARIA, Association for Resources and Initiatives on Addictions), with a clinical psychologist and an addiction psychiatrist, have developed a programme called Expériences Animées (EA, Animated Experiences in English) inspired by life skills development programmes that have proven to be successful, such as the Life Skills Training programme,12 the Unplugged programme13 and the In Media programme.14 Since January 2020, the Coreadd organisation (regional coordination for addiction prevention) has assumed responsibility for the EA programme.
The EA programme involves animated short movies and talks with high school and secondary school pupils about the use of psychoactive substances and addictions. Priority is given to high schools in priority education areas and to vocational high schools.
The programme draws on the recommendations of Botvin and Griffin8 15 in terms of theoretical origins, goals and means. The EA programme’s overall purpose is to: delay initiation of substance use; prevent adolescents from becoming regular consumers of psychoactive substances; reduce the risks and harm related to consumption of these products; and open the way for adapted support measures.
The EA programme has not been evaluated; to understand how, under what circumstances, through which mechanisms and among which adolescents this programme works, we developed the ERIEAS study (‘Evaluation Réaliste de l’Intervention Expériences Animées en milieu Scolaire’; Realist Evaluation of the EA Intervention in Schools).
Study objectives and location
Our study is aimed at evaluating the EA programme to set out an intervention theory for the programme. To do so, the study has two main objectives:
First objective: To characterise the effects (Outcomes) on the pupils of high schools in terms of:
Tobacco, alcohol and cannabis use.
Consultation with healthcare professionals for problematic substance use.
Second objective: To characterise the context and mechanisms of action (Mechanisms and Context), and to document the conditions of effectiveness of EA in terms of contextual conditions or mechanisms triggered by the programme. That is, to identify the programme’s key functions,16 which may be contextual conditions or action mechanisms, related to:
The pupils’ characteristics.
The practitioners’ characteristics.
The design of materials and, in particular, the techniques employed, their use during programme sessions and the activities for teaching staff and parents.
The macro-intervention (eg, classes, schools, cities) and micro-intervention (eg, supervised short films and talks) environments.
The second objective offers a way to identify the key functions of the EA with a view to transferability13 14 (ie, transferring key features and adapting non-key aspects) and comparison of EA with other implemented or transferred programmes, such as Unplugged.13
The EA programme takes place in 10 schools (three high schools and seven secondary schools) in the department of Charente in Western France. The ERIEAS study takes place in these 10 schools. It began in January 2019 and will run until December 2022. Inclusion of participants began on 30 September 2019.
Methods and analysis
This report is consistent with the RAMESES II reporting standards for realist interventions (compare checklist in online supplementary additional file).17 ERIEAS seeks to evaluate the actual impact and the factors involved of the EA programme using a realist evaluation. For this purpose, a multi-case approach will be adopted considering each school as a ‘case’.18 19 The study adheres to the theory-driven evaluation framework20–23 where the realist evaluation method and contribution analysis24 25 are used to explore the effects, mechanisms and influence of context on the outcomes and to develop and adjust an intervention theory. This case-study method will help to set out the contribution ‘story’: in light of the multiple factors influencing the result, does the intervention contribute to an observed result and in what way?25
Supplemental material
The study also comes under the rubric of natural experiments in intervention research. This is because the intervention has little standardisation and is not undertaken for the purposes of research.26
Conceptual framework
In realist evaluation, developed by Pawson and Tilley,27 the effectiveness of the intervention depends on the underlying mechanisms at play within a given context. The realist evaluation is about identifying context-mechanism-outcome configurations (CMOs). The aim is to understand how and under what circumstances an intervention works. A middle-range theory (ie, a theory that is aimed at describing the interactions between outcomes, mechanisms and contexts) is set out to highlight the mutual influences of intervention and context.28 29 This approach is linked to the black box paradigm30 and differs from the experimental paradigm, which evaluates effectiveness without looking at the mechanism by which an intervention works or the influence of the context. Realist evaluation asks whether the intervention worked in a way consistent with its underpinning theory. The generative causality works on three assumptions:31 (i) an intervention does not work in and of itself, and it is not what produces an outcome; (ii) all interventions trigger a mechanism or a set of mechanisms that produce an outcome; and (iii) all interventions are delivered in a context.
Hence, the evaluation is about identifying middle-range theories. Hypothesised and validated by empirical investigations, these CMO configurations help to understand how an intervention brings about change, considering context and target group.28 29 The recurrence of CMOs is observed in successive case studies or in mixed protocols, such a realist trials.29 Indeed, to consider context, realist evaluators observe in successive cases what Lawson (quoted by Pawson in 200631) calls demi-regularities of CMOs (ie, regular although not necessarily permanent occurrences of an outcome when an intervention triggers one or more mechanisms in a given context).29 Studying these recurrences in different contexts allows the isolation of key elements that are replicable in a family of contexts. This gives rise to middle-range theories that become stronger as progress is made through the cases. ‘These middle-range theories, in certain conditions, predict possible intervention outcomes in contexts different from the one in which the intervention was tested’.29
Applied to our case
As the realist principle is suitable for studying non-linear interactions in complex systems, we adopted this approach. The intervention under investigation applies to an operational programme and it is therefore important to identify its key functions,32 33 that is, its interventional or contextual components underpinning its effectiveness. This will allow us to hypothesise about (1) the programme’s effectiveness, (2) its added value compared with other existing programmes and (3) its specificities. Ultimately, it should enable replication.34 35
Here, each institution involved in the EA programme, with its own context, will constitute a case. For each case, the intervention will be studied to identify the mechanisms at play in the given context along with the variation in outcomes. CMO configurations will be identified through analysis of each case. A cross-case analysis will highlight recurrent CMO configurations and thus identify key features for possible replication.
In our study, outcomes are related to using tobacco, alcohol and/or cannabis and obtaining support from a health professional.
Drawing on the literature and on the experience of professionals delivering the intervention, we will first set out initial middle-range theories,27 31 which we will test in each case (ie, schools), by collecting qualitative and quantitative data.29
The mechanisms will be identified qualitatively according to the definition of Ridde et al: ‘a mechanism is an element of reasoning and reaction of an agent with regard to an intervention productive of an outcome in a given context’;36 37 and the definition of Cambon et al: ‘What characterises and punctuates the process of change and hence, the production of outcomes’.16 The mechanisms will be divided into those related to the session leaders and teachers in attendance and those related to the pupils. For pupils, the literature allows us to define two categories of mechanisms: (1) representations about alcohol, tobacco and cannabis consumption and (2) life skills developed/mobilised.
Contextual elements will be included among all the elements collected qualitatively that satisfy the following definition: elements located in time and space that may affect the intervention and the outcomes produced, and whether they relate to the session leaders, teachers, pupils, session delivery or the operational setting. In a realist approach, interventional elements are part of the context. Therefore, we could distinguish between Ci (for Contextual factors linked to the Intervention) and Ce (for Contextual factors not linked to the intervention, that is, external factors).
Figure 1 synthesises the principles of the realist evaluation.
The realistic approach and the way to refine middle range theories applied to ERIEAS study.
The Expériences Animées programme
The EA medium: the short-animated movies
The EA programme involves showing short-animated movies followed by discussion sessions in high schools about the use of psychoactive substances and, more generally, about all types of addiction. To reach its target audience, the EA programme uses short-animated films produced by student filmmakers (second-year students at the EMCA animated filmmaking school in Angoulême, France). The films are made specifically for the programme and every year new students are selected to produce new short films.
A filmmaking workshop for students (20 to 26 years old) has been organised by the EA programme managers and the EMCA school board every year since 2015. During the 4-day workshop, the students are tasked with looking at the use of psychoactive substances, and their functions, and sharing their thoughts about how people become addicted. They are asked ‘What do you have to say about it? What would you like to convey to a young audience about this issue?’ The programme managers do not intervene in the actual screenwriting and aesthetics of the films. At the end of the workshop, the students show their proposals as filmed storyboards. Then, the programme managers select several projects for inclusion in the EA programme. Some 45 films have been produced since 2015 focussing on subjects such as food and drug addiction, as well as also emotional dependence and bullying. The mean duration of the films is 3 min.
Delivery of the sessions
The interventions during the sessions consist of four phases.
Presentation and instruction: In the first session, session leaders present the programme and the instructions to follow during the sessions. After the first session, and at the beginning of each subsequent session, pupils are briefly reminded about the previous session (ie, what the session leaders noted and remarked) and the instructions. Pupils are then told, ‘We will watch a short film and then discuss the content together, sharing our thoughts and feelings about what we have seen’.
Film: Two or three films are shown during each session. After watching each film, pupils are encouraged to discuss freely the issues raised. Films are selected by session leaders according to the previous discussion and the specific context of the session.
Interactions about the films: These include reflective dialogue, sharing thoughts and experiences (experiential approach) with everyone able and willing to contribute. After watching each film, pupils are given time for critical reflection, positioning, reformulating and summarising.
End of the session: At the end of a session, pupils go over the keywords from the session and summarise what was said.
The sessions are facilitated by two psychologists as session leaders. Their demeanour when delivering the sessions is important. They should act as clinicians who are there to listen, to help pupils reflect on the issues raised, to encourage them to express themselves freely while allowing others to do so and to facilitate dialogue. A teacher, school nurse or other staff member from the school also attends the sessions. Classroom tables are arranged in a U shape or in a rectangle (if there are many pupils in the class).
In a given high school, one session per term is delivered for each class, that is, three film showings (sessions) per academic year.
Study population
Different groups will be followed in the ERIEAS study:
Pupils: The sample involved in the EA programme will be studied. From the 2019/2020 academic year until the 2021/2022 academic year (three academic years), the EA programme will be delivered in 10 institutions (a total of 40 classes and around 1000 pupils).
Education community members: One teacher whose class is involved in the sessions, the school nurse and the headmaster, that is, a total of three people per institution will be interviewed in the study.
Session leaders: Three persons will be interviewed.
Funders: Three funders will be interviewed.
The inclusion criteria will be:
For pupils:
Pupils from the 10 secondary and high schools involved in the EA programme, in 7th grade and 10th grade in the 2019/2020 academic year, willing to participate and whose parents/guardians have given consent for the child’s participation.
Pupils of these 10 schools who will enter 8th grade and 11th grade in September 2020, and 9th grade and 12th grade in September 2021 who are willing to participate and whose parents/guardians have given consent for the child’s participation.
For education community members, session leaders and funders: willingness to participate.
The only exclusion criteria will be to be under legal protection.
Patient and public involvement
The ERIEAS study does not include any patient or public involvement in terms of setting research priorities, defining research questions or outcomes, providing input into the study design or disseminating the results. The research participants will be called on to answer questionnaires or interviews.
Study design
According to theory-driven evaluation methodology,20–23 the study will be composed of four stages, as outlined in figure 2.
Stages of the ERIEAS study. Ce,contextual factors not linked to the intervention, that is, external factors; Ci, contextual factors linked to the Intervention;CMO, context-mechanism-outcome; QUAL, qualitative; QUAN, quantitative.
Stage 1: literature analysis
The first stage involves a literature analysis to design and adapt investigation tools and contribute to elaboration of the initial middle-range theories (stage 2).
A review has been conducted to identify strategies that are effective in school settings in helping pupils to change their representations of alcohol, cannabis and tobacco use, and to develop or mobilise life skills to prevent addiction. The data sources used were: PsycINFO, PsycArticles, Psychology & Behavioral Sciences Collection, SocINDEX, Cairn and Web of Sciences.
The keywords were ‘life skills’ and ‘adolescents’ or ‘young adults’ or ‘teenagers’. The inclusion criteria were: English or French language publications from 2014 to 2019; original or methodological articles focussing on the evaluation or exploration of intervention techniques applied to school settings; interventions aimed at changing representations; life skills as a way of delaying experimentation with alcohol, tobacco and cannabis, and reducing their use.
Using the software Covidence, two researchers conducted a double-blind review of the selected reports and extracted information regarding the intervention (name, location, population, design), the evaluation design (method, main and second outcomes), the result of the intervention, its key functions and context of implementation. The purpose of this review was to extract information about the contexts, mechanisms and outcomes that were the focus of our study. Our analysis questions were: What are the most effective intervention techniques? What representations and skills are they effective for? What contextual conditions influence the effectiveness of the techniques?
The results have been used to design and adapt our tools (questionnaire, interview and observation grids, scorecards) and will be used for the initial development of middle-range theories (stage 2). Review results will be presented in a separate paper.
Stage 2: data collection and elaboration of the initial middle-range theories
Stage 2 involves data collection to appraise the outcomes, mechanisms and contextual elements (including the techniques). There will be two rounds of data collection. The first round of data collection will help to elaborate initial middle-range theories (to establish how the intervention works in context) together with the results of the literature analysis from stage 1. The second round of data collection will contribute to verifying the initial theories (contribution analysis). Table 1 shows the variables that will be collected and the manner of collection.
Mechanisms data (M) expected and time of collection
To collect data, the following tools will be used:
A questionnaire to collect data on mechanisms and outcomes on pupils at T0 (first round) and T1 (second round).
It will contain: descriptive variables (gender, age, socio-professional category of parents); questions on tobacco, alcohol and cannabis use; a question relating to the ‘use of support services’; questions related to knowledge and perception of tobacco, cannabis and alcohol use; and scales assessing peer pressure and life skills.
This questionnaire will be administrated twice, at T0 (ie, at the end of September/beginning of October 2019) and T1 (June 2022) to all pupils participating in the ERIEAS study. Table 2 lists the questions/scales and their previous use in other studies.
Sources of questions included in T0 questionnaire
Pupil interviews: (IP1 & IP2)
A total of 160 pupil interviews will be conducted during the study in two sessions.
The first session of non-directive interviews will be held in November/December 2019 (IP1). These interviews will collect contextual factors, mechanisms and outcomes on pupils (IP1), and help to elaborate the initial middle-range theories. In the first academic year, two pupils per class will be randomly selected and interviewed, until saturation (ie, when the collected data do not add anything new to the understanding of the research topic), meaning at least 80 pupils will be surveyed.
These interviews will collect large amounts of information allowing the elaboration of the second round of interviews with pupils (IP2) and assist the configuration and design of a questionnaire including a Discrete Choice Experiment (DCE) set of questions.
The second session of interviews will validate CMO configurations in pupils (IP2). At the end of the second academic year (June 2021), a session of semi-directive interviews with pupils will be held (80 pupils randomly selected, and different from those of the previous session). This qualitative material may validate the framework expressed through the initial middle-range theory (CMO configurations). For each hypothesised configuration in the initial middle-range theories, there will be an open-ended question that will not allow the respondent to be guided by the expected answer (presence or absence of C and/or M).
Professionals’ and EA session leaders’ interviews (IE1 & IE2)
Semi-structured interviews with the education community and among the session leaders will collect information on mechanisms and contextual factors related to school. Three session leaders and three people per institution will be surveyed: a nurse, the headmaster and a teacher whose class is involved in the intervention. Interviews will be held twice: at the end of the first academic year (June 2020) and at the end of the 3-year intervention period (June 2022). A total of three session leaders and 30 professionals will therefore be interviewed at each session. In total, 66 interviews will be performed.
This first session of semi-directive interviews will help to elaborate the initial middle-range theories, to design the second round of interviews (IE2), and to configure and design the DCE questionnaire. The second session of semi-directive interviews may validate the framework expressed through the initial middle-range theory.
Funders of the programme interviews (IF)
Semi-directive interviews with funders of the programme will collect information related to context parameters (Ce exclusively). A total of three funders will be interviewed once, in October 2019. These interviews will help to elaborate the initial middle-range theories.
Observation of the sessions
The objective is to collect the following contextual elements, specific to each institution: the intervention climate within the institution and in each class, the reception given by the school staff as a whole, the conditions of session delivery, the motivation levels of the session leaders and teachers in attendance and the characterisation of the feedback and sharing sequences (spatial organisation, relations between pupils/session leaders/others).
Observations of at least two sessions per institution will be conducted in each academic year. Twenty sessions per year will therefore be observed, making a total of 60 observations over the 3-year intervention period. Furthermore, the session leaders of non-observed sessions will produce a report following a reporting grid.
Before the programme starts (in mid-October 2019), each programme presentation for the school staff has been observed, thus adding 10 observation sessions. A total of 70 observation sessions will therefore take place during the study.
Scorecard to collect institution characteristics (contextual elements, Ce)
Data related to the characteristics of the institutions will be collected from the education authorities using a scorecard: level of urbanisation of the town or city where the institution is located (urban, quasi-urban, quasi-rural, rural), the deprivation index of the school area (FDep index from Rey et al, 200938), the number of pupils, male/female ratio, number of repeaters per institution, the availability of a nurse, the type of institution (general/vocational, priority education areas), the proximity to a centre for young substance users and its availability, where appropriate, the occurrence of occasional awareness campaigns on addiction in the institution and having been previously involved in the EA programme.
DCE questionnaire
The questionnaire will have variables based on a stated-preference method,39 known as DCE.40 It will enable us to gain a better understanding of the pathway and the circumstances and determinant combinations that could trigger initiation, maintenance or re-engagement of consumption of alcohol, tobacco and/or cannabis use. It may help to validate the final middle-range theories. that is, the CMO configurations.
Figure 3 presents the different modalities of data collection.
Different modalities of data collection and their links with the middle-range theories. Ce,contextual factors not linked to the intervention, that is, external factors; Ci, contextual factors linked to the Intervention;CMO, context-mechanism-outcome; DCE, Discrete Choice Experiment.
Figure 4 presents the data collection timelines.
Timeline of the data collection. Ce,contextual factors not linked to the intervention, that is, external factors; Ci, contextual factors linked to the Intervention;estab., establishments; CMO, context-mechanism-outcome; DCE, Discrete Choice Experiment; prof., professor.
Survey instruments: questionnaires, interview guides, observation logs and observation checklists have been designed, consistent with the results of literature analysis (stage 1) and will be pre-tested.
Using quantitative analyses, the following outcomes will be identified at baseline and after nine EA sessions:
Tobacco, alcohol and cannabis use: current frequencies of tobacco, alcohol and cannabis consumption (every day/week/month, rarely, never); number of episodes of heavy drinking (at least five drinks per occasion) in the last 7 days/30 days/12 months; number of episodes of drunkenness in the last 7 days/30 days/12 months.
Consultation with a healthcare professional to discuss problematic use of alcohol and/or tobacco and/or cannabis (number of times in the last 12 months).
At this stage, the literature review will have helped to identify some of the mechanisms and contextual elements that could lead to greater effectiveness of an intervention. These factors need to be confirmed during the second stage (ie, seminar and elaboration of the initial middle-range theories). Here, we present the potential M and C we aim to investigate at this stage of the study:
Representations and attitudes:41 normative expectations or perceived norms about use, intentions, refusal and resistance skills, risk-related attitudes and behaviours, positive and negative beliefs about consequences, reasons to use and perceived peer influence and friends’ use.
Life skills: ability to mobilise self-management skills (self-esteem, problem-solving abilities, reducing stress and anxiety), general social skills (overcoming shyness, communicating clearly, building relationships) and drug resistance skills (defenses against pressures to use alcohol and other drugs, identify the consequences of substance use, risk-taking and the influences of the media).
Both of these mechanisms have been shown in the literature to be effective in terms of addiction prevention.7 8 Here, we use the definition of Cambon et al: ‘What characterizes and punctuates the process of change and hence, the production of outcomes’.16 In this definition, a mechanism can be psychological (motivation, self-efficacy, self-control, skills, and so on) or social (values shared in a community, power sharing perception, and so on).
Table 1 presents the details of the expected/ hypothesised contexts and mechanisms and the time and modalities of collection.
Stage 3: data analysis
Data analysis will include analysis of each case (each school) and a cross-case analysis allowing us to compare and see potential effects of some contexts. The analysis will answer the question: In what contextual conditions and through which mechanisms does the EA programme produce outcomes? The validation of initial middle-range theories (CMOs) will allow us to answer the question. This validation will combine and compare data from quantitative and qualitative analyses in monographs (analysis of each case (institution)) and by cross-case analysis (analysis comparing these cases). We will combine a QUAN/QUAL design42 to use quantitative and qualitative approaches in tandem, to provide new insights and a more refined approach. This analysis will identify the recurrent CMOs that will therefore be replicable.
Quantitative data (from T0 and T1 questionnaires): Analyses of the evolution of mechanisms and outcomes
A first descriptive analysis will be conducted on the representations, mobilisation of life skills, tobacco, alcohol and cannabis use, and the use of support services at each measurement point. Trajectory modelling will be performed using latent class growth modelling to study changes in representations, life skills, product use and use of support services between the two questionnaires. The analysis will be adjusted according to the characteristics of the pupils (age, gender, socio-professional category of parents) and institutions (level of urbanisation, index of social disadvantage, number of pupils, male/female ratio, number of teacher per institution, proximity to a centre for young substance users/nurse availability, earlier involvement in the EA programme, involvement in occasional awareness campaigns on addiction, type of institution).
This analysis will enable verification of the programme’s impact in context across all the pupils by studying the changes between the two collection times (ie, before and after nine sessions of EA). As some of the variables used here are the same as for the Life Skills Training (LST)43 and Unplugged13 programmes, the findings will be compared and contrasted with these programmes conducted in the Nouvelle-Aquitaine region.
Qualitative data
The qualitative data can be divided in two groups (ie, for the two rounds), with different uses and aims. The first interview sessions and observations (ie, IP1, IF, IE1, scorecard and first observations) will be used to set the frame of the initial middle-range theories. Following this first set of qualitative investigations, a 2-day seminar will be organised to set out the initial middle-range theories. The aim of this seminar will be to discuss the findings of the literature review and the first round of data collection. This seminar will be interdisciplinary, and will involve epidemiologists, prevention experts, addiction experts, psychiatrists, psychologists, sociologists, EA session leaders and Coreadd staff. Drawing on the participants' experiences, the literature review, the observations and the interviews, the seminar will enable us to set out hypotheses about the mechanisms (M) linking Ci and Ce to the outcomes.
The second set of qualitative investigations (ie, IP2, IE2 and lasts observations) will be used to validate the final middle-range theories.
All the qualitative data (ie, data collected from pupils, funders, professionals’ interviews and observations) will be examined by content analysis,44 which refers to ‘a set of techniques for systematically and objectively analysing and describing the content of communication. The aim is to obtain indicators allowing inferences to be made about the messages and how they are produced and received (inferred variables)’. Content analysis encodes, classifies and ranks the communications to examine patterns, trends or distinguishing features; in our case, the recurrence of C-M-O configurations in each case (institution) and by cross-case analysis.
Quantitative data from DCE questionnaires: analyses of DCE questionnaires
A DCE will be used to: (i) understand combinations of circumstances and determinants that impact the decisions of teenagers to start substance use and/or prevent them from cessation and/or lead them to relapse; (ii) validate CMO configurations.
All previous materials collected and analysed will be used to identify attributes and potential scenarios.
The DCE will allow quantification of the weights individuals attach to various attributes of a situation to finally predict their independent impacts on decisions. In other words, the DCE approach will allow analysis of individual stated preferences in response to hypothetical choices and will enable estimation of the relative importance of each level of attribute during the decision-making process. When presented with hypothetical options (ie, choice scenarios) that describe alternative specifications of a situation, it is assumed respondents will choose the behaviour that they would most likely adopt. The higher a respondent’s preference for a certain attribute level, the more likely they are to choose that scenario over any alternative. This method will allow the researcher to highlight situations and configurations allowing the implementation of life skills. Mixed logit models will be used to establish whether the attributes’ levels presented in the scenarios are statistically significant predictors of adolescents’ choices.
Combination of quantitative and qualitative data
The ERIEAS study will combine quantitative and qualitative methods to achieve mixed methods research. This study has a multiphase design with first data collection combining QUAN (T0) + QUAL (ie, IP1, IF, IE1, scorecard and firsts observations). These first data will contribute to and guide a second qualitative phase (ie, IP2, IE2) and the DCE questionnaire elaboration by identifying some C, M, O and, possibly, certain configurations for investigation. This design seemed to be particularly relevant as it allows flexibility to address a set of research questions that will arise from the separate parts of the study.
The CMO configurations obtained from the qualitative data will be compared with those obtained from the DCE questionnaire. All these CMO configurations will be discussed during a 1-day seminar in stage 4, which will lead to a complete analysis of the EA complex intervention.
Stage 4: refinement and adjustment of the middle-range theories, and definition of the programme's key functions
The different modalities of investigation and analysis will allow us to progressively refine the middle-range theories. We will compare the primary theories with the CMO configurations observed in each institution and the recurrent CMOs to consolidate and adjust the initial middle-range theories. Furthermore, the DCE questionnaire will allow the research team to formulate strong hypotheses about the triggers involved in initiating, maintaining or re-engaging in consumption of alcohol, tobacco and/or cannabis among adolescents and the impact of the EA intervention.
The material will be synthetised and used to refine and adjust the final middle-range theories and the programme’s key functions. This work will be conducted during a second seminar (1 day) with the whole project team, that is, the researchers, EA professionals and Coreadd staff members.
The team will carry out the following activities:
Definition of the EA programme’s key functions (configurations underpinning the programme's success).
Comparison with the elements used in other addiction prevention programmes designed to develop life skills, such as the LST programme43 and Unplugged.13
Exploration of further strategies to complete the EA programme wherever this is necessary to ensure optimal outcomes.
Drafting of detailed recommendations (1) to scale-up the EA programme’s key functions to other areas with other stakeholders and (2) for a quasi-experimental large-scale evaluation of the programme (where only the key functions will be retained and freely adapted) if required (judgement criteria may not be precisely determined at this stage).
Ethics and dissemination
This article describes a protocol using a realist design to understand how an innovative prevention programme works, and what contexts, mechanisms and outcomes are involved in this intervention. Realist evaluation is a valuable approach that highlights the triggers of an intervention and guides its transferability.
The project will be carried out in full accord with current relevant legislation (eg, the Charter of Fundamental Rights of the European Union) and international conventions (eg, Declaration of Helsinki). It follows the relevant French legislation on interventional research protocols involving the human person (Jardé law, category 3 research on prospective data). The methods development, data collection and analysis will take into account the following issues:
Anonymity of study respondents will be preserved and ensured at all times. Unnecessary collection of personal data will be avoided and respondents will have the right to review outputs and withdraw consent. All personal data will be coded, removed from the data for analysis and stored separately. Only designated research staff will have access to the keys linking the data with the personal information.
Information regarding the study and the right to refuse to participate will be distributed to all study participants and their parents or guardians and, in the case of refusal, alternative means of data collection will be explored (eg, alternative respondents).
The protocol was approved by the Comité et Protection des Personnes (CPP), that is, Committee for the Protection of Persons CPP SUD-EST VI No. AU 1525 and was reported to the Agence Française de Sécurité Sanitaire des Produits de Santé (ANSM), that is, the French National Agency for the Safety of Health Products. It is in compliance with reference methodology MR003 of Bordeaux University Hospital (CNIL No. 2 026 779 v0).
This research has been registered on ClinicalTrials.gov (No. NCT04110626).
The research project is registered in the European database ID-RCB (No. 2019-A01003-54).
This research has received funding from a national recognised research agency; the INCa. This funding has been obtained via a national competitive peer review grant application process, named ‘2018 Call for projects - Population health intervention research: Addressing all dimensions of cancer control’.
From a research viewpoint, our proposed methodology is consistent with the bottom-up approaches advocated in health promotion, starting with a real-world response to a pressing problem.45 As this approach allows better reflection of stakeholders’ views and concerns, and makes external validity workable, it therefore becomes a preferable alternative for evaluation of health promotion or programmes.45 Our study is a realist evaluation based on a natural experiment mobilising mixed model methods and a preference stated method. Therefore, it is an innovative way of studying the process of a complex intervention.46 Due to its specific methodology and large sample size, this study will provide strong and detailed information regarding consumption of tobacco, alcohol and cannabis among young people and their representations of the consumption of these products. Using the stated preference methods, this study will highlight how pupils use their competences and life skills in relation to addictive products.
Our study has some limitations related to its design. First, even if our study provides insights into pupils’ behaviours before and after the intervention it will not allow us to take into account all factors that may play a part in the consumption of psychotropic agents in adolescence. Nevertheless, the two rounds of quantitative data acquisition and the large amount of qualitative data collected will provide a better understanding of how such an intervention could have an impact on consumption and perceptions regarding psychotropic agents. Second, our results are declarative and the ERIEAS study will not use any kind of biological or medical information. Even if declarative data could lead to underestimation, the use of an auto-administered questionnaire on drug consumption would reduce this under-declaration.47
Tobacco, alcohol and cannabis consumption among adolescents constitute a worrying trend, especially in France,48–51 and interventions aimed at prevention should be tailored to this specific population. This study will explain and pinpoint the precise impacts of the EA programme and the conditions for this impact. It will allow definition of the EA programme’s key functions and how they work in different contexts and, possibly, how they could be adapted in form. We will be able to compare and contrast the programme with other programmes being implemented in France, with a view to creating fine adjustments of solutions for optimal outcomes. Finally, guidelines will be set out, to implement EA elsewhere. The conclusions will be highly replicable and offer a basis for designing other interventions using identified key functions. We will publish different papers describing the addictive behaviours of this population, and then go on to analyse the impacts and key functions of the EA programme. We will eventually focus on what triggers consumption of psychotropic agents among young people exposed to a prevention programme. Finally, a report on the study will provide health authorities with evidence-based results to help with the rollout of health promotion policies in schools. In conclusion, this project will be of great interest to policymakers, authorities and field professionals involved in the substance use prevention and health promotion sector.
Acknowledgments
The authors are very grateful to all those who took part in the project.
References
Footnotes
JM-F and AA are joint first authors.
Contributors JM-F and AA drafted this article and all authors revised the manuscript. The project design was developed by LC. JM-F, AA, EM, RG, MT, LM, LM, VD and FA were involved in implementing the project and in developing the evaluation design, under the supervision of LC. All authors read and approved the final manuscript.
Funding This research has received funding from a national recognised research agency; the INCa. This funding has been obtained via a national competitive peer review grant application process, named ‘2018 Call for projects- Population health intervention research: Addressing all dimensions of cancer control’ (No. CAMBON-2019–008).
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.