Article Text
PDF
Abstract
Introduction In Europe, alcohol consumption is responsible for many diseases, disabilities, injuries and premature deaths. In France, alcohol consumption represents an important health burden, due to its frequency, scale and the serious damage it causes. One of the keys to addressing the problem would appear to be the adoption of harm and risk reduction approaches. In order to operationalise this strategy, the Nouvelle Aquitaine Regional Health Agency is funding three different programmes to reduce the harm and risks associated with alcohol consumption: Alcochoix, Iaca and ETP (Education Thérapeutique du Patient) Conso-repère. We are interested in understanding how, under what circumstances, through which mechanisms and for which population the different programmes work.
Methods and analysis The ECIAE study (a cross-evaluation of the 3 programs Iaca/Alcochoix/ ETP (Patient Therapeutic Education) is a theory-based evaluation where the realist evaluation method is used to explore effects, intervention mechanisms and the influence of context on outcomes. This realist evaluation is based on multiple case studies in two nested levels. At the first level, each centre implementing the programme will represent a case. At the second level, each programme will represent a case in which a set of activities is conducted to achieve risk reduction objectives.
Ethics and dissemination The project will be carried out in full compliance with existing legislation and international conventions. It was subject to analysis, including a privacy impact assessment conducted by the Data Protection Officer of the University of Bordeaux. The University of Bordeaux has ensured that all the regulatory procedures related to the ECIAE study have been carried out. The dissemination plan includes scientific papers, seminars, a report and recommendation and a public restitution. The study will provide evidence-based results to help health authorities roll out strategies to reduce risks and harm associated with alcohol use.
- PREVENTIVE MEDICINE
- PUBLIC HEALTH
- MENTAL HEALTH
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
Our proposed methodology is consistent with the bottom-up approaches advocated in health promotion, starting with a real-world response to a pressing problem.
Our study applies realist evaluation at two levels at the same time: a single programme in several application situations and a single objective of reducing the harm and risks associated with alcohol consumption addressed by several different programmes.
Our study combines quantitative and qualitative investigations.
Our study mobilises multiple modes of data collection: literature review, interviews, observations, seminars and hetero-administered questionnaires.
The ECIAE study will not use any kind of biological or medical information and will rely on declarative data
Introduction
In 2016, about 80 000 people died of alcohol-attributable cancer, and about 1.9 million years of life were lost due to premature mortality or to disability in the EU.1 Alcohol use is a predominant risk factor in disease and injury.2 3 A large contribution to this burden is due to alcohol use disorders. In France, alcohol consumption represents an important health burden, due to its frequency, scale and the serious harm it causes. Even if a drop in consumption has been recently observed,4 alcohol remains a major source of mortality and morbidity in France. It is one of the three leading causes of avoidable mortality. Indeed, alcohol is responsible for 7% of deaths in France. This is more than in many other European countries.4 In the Nouvelle Aquitaine region in the southwest of France, 12.3% of adults between 18 and 75 years of age were thought to consume alcohol daily and 45.3% weekly in 2017 vs 10.0% nationally.5
Damage to health from alcohol consumption can appear from one drink per day.6 7 It is therefore in the interest of public health to find ways of reducing alcohol consumption, whether in a situation of low or high consumption. One of the keys to meeting this challenge would seem to be the adoption of approaches to reduce harm and risks. Until the 1930s, only excessive alcohol consumption was singled out as a health risk, and moral and pathological patterns of alcoholism were predominant.8 The only way to deal with the problems associated with excessive alcohol consumption was abstinence.8 9 Within this framework, the prevention of excessive alcohol consumption advocated a ‘zero tolerance’ or ‘just say no’ approach.9
Nowadays, an approach seeking to reduce harm and risks associated with alcohol consumption has been developed gradually in France. It is a pragmatic approach to alcohol consumption and alcohol-related problems based on three pillars: ‘(i) reducing the harmful consequences associated with alcohol consumption; (ii) offering an alternative to zero tolerance approaches by incorporating drinking goals (abstinence or moderation) that are compatible with the needs of the individual and (iii) promoting access to services by offering low-threshold alternatives to traditional alcohol prevention and treatment’.9 This alternative approach has been shown to be effective in numerous studies.9–15 Although it is still met with some resistance, the harm and risk reduction approach is now strongly supported and encouraged.
The Law on the Modernisation of the French Healthcare System (Law n°2016–41) has reinforced the harm and risk reduction (HR) policy and extended it to the field of legal psychoactive substances.16 This approach is now the subject of recommendations issued by the French National Authority for Health, for supporting alcohol users in Drug User Harm Reduction Support Centres (CAARUD) or in Addiction Treatment, Support and Prevention Centres (CSAPA).17 18
In the Nouvelle Aquitaine region, the Regional Health Agency (ARS) in charge of regional steering of the health system intends to develop this strategy by reinforcing proactive approaches to prevention, support, care and harm and risk reduction.19 In order to operationalise this strategy, the Nouvelle Aquitaine Regional Health Agency is funding three different programmes to reduce harm and risks associated with alcohol consumption: Alcochoix, Iaca and ETP (Education Thérapeutique du Patient) Conso-repère.
The Nouvelle Aquitaine Regional Health Agency has appointed our team to conduct a comprehensive evaluation of these different programmes. The purpose of this article is to present the protocol adopted to conduct this study.
Methods and analysis
Study objectives and location
The objective of the ECIAE (cross-evaluation of the Iaca, Alcochoix and ETP Conso-repère programmes—in French ‘Evaluation croisée des programmes Iaca, Alcochoix et ETP Conso-repère’) project is to evaluate the operational modalities of three different programmes conducted in the Nouvelle Aquitaine region—Iaca, Alcochoix and ETP Conso-repère—and centred on the principles of HR and mental health recovery. Recovery in mental health is understood as a process with five components: connectedness, hope and optimism about the future, identity, meaning in life and empowerment.20 Recovery is ‘a comprehensive mental health experience based on restoration from mental illness and optimisation of positive mental health’.21
The ECIAE project has four objectives: (i) to characterise the effects (outcomes) on the beneficiaries in terms of: addiction, use of alcohol and other addictive substances, medical-psycho-social path and empowerment; (ii) to identify the mechanisms associated with their impact, referred to as the ‘key functions’ of each of the three programmes, (iii) to identify the conditions for the impact of each of the three programmes: for whom, how and under what circumstances are these interventions effective? (iv) to highlight the specificities and complementarities of these three programmes developed with risk and harm reduction in mind.
This study will enable us to identify guidelines for the deployment and development of interventions to reduce the harm and risks associated with alcohol consumption.
The ETP Conso-repère programme is being conducted in one centre, the Alcochoix programme in three centres, and the Iaca programme in four centres in the Nouvelle Aquitaine region. The ECIAE study will therefore investigate eight centres implementing one of the three programmes listed above. These centres are located in Agen, Bordeaux, Limoges and Pau. In addition, several types of centre are represented: CSAPA (Addiction Treatment, Support and Prevention Centres), CAARUD (Drug User Harm Reduction Support Centres) and CHRS (Accommodation and Social Rehabilitation Centres).
It began in May 2021 and will last until February 2023.
Methods
The ECIAE study employs theory-based evaluation, where the realist evaluation method is used to explore effects, intervention mechanisms and the influence of context on outcomes. This method allows the formalisation of intervention theories that are specific to each programme on the one hand, and common to a number of programmes developed from the same perspective on the other. This report is consistent with the RAMESES II reporting standards for realist evaluation.22
This assessment is based on multiple case studies at two nested levels (see figure 1).
Case consideration in ECIAE study. ETP, Education Thérapeutique du Patient.
At the first level, each centre implementing the programme (differently) is a unique application situation (with a set of activities conducted according to the constraints and opportunities of its environment) and will represent a case.
At the second level, each programme will represent a case in which a set of activities is conducted to achieve risk reduction objectives.
In order to study the three programmes and their different implementation in the centres, this study uses two methods of data collection and analysis: one qualitative and the other quantitative.
The qualitative part includes a study of the documentary corpus, working seminars, observations and semistructured interviews.
The quantitative component includes several hetero-administered questionnaires at different times: M0, M6, M9, M12.
Conceptual framework
Theory-driven evaluation
Theory-driven evaluation is dedicated to complex interventions and explores the how and why of certain effects observed as a result of or during the intervention. It is intended to provide ‘an in-depth view of how things work if we are to accurately assess problems, intervene effectively and achieve beneficial results’.23 It leads to the mechanistic representation of an intervention or set of interventions contributing to the achievement of similar objectives through the development of intervention theory. We chose to join the theory-based evaluation movement because it allowed us to touch on the objectives we have in this study. Indeed, theory-based evaluations explore how and why a programme works or does not work, thus providing valuable assistance for programme or policy improvement.24 The identification of the ‘black box’ of interventions takes shape in the establishment of detailed theories. ‘Theory provides a means of abstracting and schematizing this complexity so that we can better grasp what aspects are important, how things are interrelated and what should be done under various circumstances.’.23 Theorising interventions also facilitates knowledge transfer.23 Thus, the use of theory-driven evaluation allows us to: (i) approach the complexity of interventions involving many interacting components; (ii) explore how interventions do or do not achieve the targeted results and thus identify the ‘key functions’ that are essential to their effectiveness; (iii) reflect on their potential for transferability and scaling up, and on the related conditions.
Realist evaluation method
Realist evaluation complements and goes beyond the traditional question of evaluations based on a positivist epistemology of ‘does the intervention work?’ by also asking the question of how, for whom, for what and in what circumstances the intervention works.25 This theoretical framework underpins the production of knowledge to improve interventions and enrich theoretical and empirical knowledge.26 27 In realist evaluation, as developed by Pawson and Tilley,28 the effectiveness of the intervention depends on the underlying mechanisms at play within a given context. The realist evaluation is about identifying context-mechanism-outcome configurations (CMOs). The aim is to understand how and under what circumstances an intervention works. A middle-range theory (ie, a theory that is aimed at describing the interactions between outcomes, mechanisms and contexts) is set out to highlight the mutual influences of intervention and context.29 30 This approach is linked to the black box paradigm31 and differs from the experimental paradigm, which evaluates effectiveness without looking at the mechanism by which an intervention works, or the influence of the context. Realist evaluation asks whether the intervention worked in a way that is consistent with its underpinning theory. The generative causality works on three assumptions:27 (i) an intervention does not work in and of itself, and it is not what produces an outcome; (ii) all interventions trigger a mechanism or set of mechanisms that produce an outcome and (iii) all interventions are delivered in a context.
Hence, the evaluation is about identifying middle-range theories. Hypothesised and validated by empirical investigations, these CMO configurations help to understand how an intervention brings about change, considering context and target group.24 25 The recurrence of CMOs is observed in successive case studies or in mixed protocols, such as realist trials.25 To consider context, realist evaluators observe what Lawson (quoted by Pawson in 2006)27 calls demi-regularities of CMOs (ie, regular although not necessarily permanent occurrences of an outcome when an intervention triggers one or more mechanisms in a given context) in successive cases.25 Studying these recurrences in different contexts allows key elements that are replicable in a family of contexts to be isolated. The observation of recurring patterns allows generalisable recommendations to be made on the key functions of interventions.28 29 This gives rise to middle-range theories that become stronger as progress is made through the cases. ‘These middle-range theories, in certain conditions, predict possible intervention outcomes in contexts different from the one in which the intervention was tested’.25
Applied to our case
In the case of the ECIAE study, the realist evaluation method will be applied to two subsets: each programme implemented on several sites, and several programmes pursuing a common set of harm and risk reduction objectives.
This consideration is consistent with the interventional paradigm of theory-based and realist evaluation, which proposes to view complex interventions as 'black boxes' made up of many interacting intervention, population and context components.
This will allow us to hypothesise about 1) a programme’s impacts, 2) its added value compared with other existing programmes and 3) its specificities.
For each case, the intervention will be studied to identify the mechanisms at play in the given context, along with the variation in outcomes. CMO configurations will be identified through analysis of each case. A cross-case analysis will highlight recurrent CMO configurations and thus identify key features for possible replication.
Drawing on the literature and the experience of intervention professionals, we will first establish initial middle-range theories,27 28 which we will test in each case, collecting qualitative and quantitative data.30
The mechanisms will be identified qualitatively according to the definition of Ridde et al: ‘a mechanism is an element of reasoning and reaction of an agent with regard to an intervention productive of an outcome in a given context’;32 33 and the definition of Cambon et al: ‘what characterises and punctuates the process of change and hence, the production of outcomes’.34
Contextual elements will be included among all the information collected qualitatively, in line with the following definition: elements located in time and space that may affect the intervention and the outcomes produced, and whether they relate to the professionals or beneficiaries, session delivery or the operational setting. In a realist approach, interventional elements are part of the context. We may thus make a distinction between Ci (for contextual factors linked to the intervention) and Ce (for contextual factors not linked to the intervention, ie, external factors).35
Programmes
Alcochoix
The ‘Alcochoix’ programme was originally developed by Canadian researchers from RISQ (the University of Quebec’s programme for Research and Intervention on Psychotropic Substances) and has been adapted to the European context and rolled out in Switzerland, France and Belgium. The Alcochoix programme aims to restore people’s ability to choose where, when, with whom and how much alcohol they consume. It is aimed at those who are concerned about the consequences of their alcohol consumption and who wish to change their habits. This programme is based on a cognitive-behavioural and motivational approach and proposes to choose one’s drinking goals and then pursue them.
Iaca
The ‘Iaca’ programme was created in 2018 in Marseille, in the southeast of France, and develops a harm and risk reduction approach for people who consume alcohol. Iaca is a programme for community workers who work with people who use alcohol, offering a method of action to organise the various stages of support. In particular, it applies the principle of psychosocial recovery, as used in the ‘Housing First’ programme.36 Iaca aims to reintegrate the person into a path of care by removing the barriers that cause medical and social isolation (shame, guilt, feelings of failure), stabilising consumption, ensuring safety and supporting psychosocial recovery. The support is intended to be comprehensive, acting on the factors that influence consumption, based on the person’s improvement priorities and proposing objectives aimed at well-being and recovery.
ETP Conso-repère
The ‘Conso-Repère’ therapeutic patient education programme was created by the Addictions France 87 association and offers support arrangements based on the control of consumption for alcohol users who do not have a state of (physical, psychological and social) health that requires total abstinence. It combines collective and individual approaches in order to support and equip users according to their choices, desires and constraints, with a view to improving their health and quality of life. The Conso-Repère programme thus promotes a harm reduction approach based on cognitive-behavioural therapies, mainly used in the context of individual support and also the development of psychosocial skills, which is used more in group workshops.
Study populations
This study focuses on two target populations and concerns nine centres.
The first corresponds to the beneficiaries of the different programmes supported within the centres.
For the beneficiaries, who are regular alcohol users or users reporting problematic alcohol use, the inclusion criteria will be: over 18 years old, willing to participate, having started one of the three programmes within the previous 15 days max and being followed by one of the eight centres in the study. The beneficiaries will be excluded if they have a severe somatic or psychiatric pathology that is incompatible with understanding the assessment tools; difficulty understanding and/or writing French; or if they are unreachable by telephone.
In total, the ECIAE study will involve 45 beneficiaries broken down as follows: 15 benefiting from the Alcochoix programme (ie, 5 beneficiaries per centre providing the programme), 20 benefiting from the Iaca programme (ie, 5 beneficiaries per centre providing the programme) and 10 benefiting from the ETP Conso-repère programme (ie, 10 beneficiaries from the centre providing the programme). The number of beneficiaries expected to participate in the study was determined in consultation with each programme’s representative. The different centres estimated a plausible number of participants based on their experience. Indeed, various programmes studied only involve a small number of users in the active file of each centre.
The second population are the professionals working in the centres or care facilities.
For the professionals, the inclusion criteria will be: willing to participate in the ECIAE study, having been trained in the programme studied (ETP Conso-repère, Iaca or Alcochoix), working in the centres participating in the ECIAE study.
In total, the ECIAE study will involve 10 professionals: 3 providing the Alcochoix programme (ie, 1 professional per centre providing the programme), 4 providing the Iaca programme (ie, 1 professional per centre providing the programme) and 3 providing the ETP Conso-repère programme (ie, 3 professionals per centre providing the programme). The number of professionals expected to participate in the study was determined in consultation with each programme’s representative. It should be noted that the participating centres do not have the same number of staff trained for the programme concerned, providing it and willing or available to participate in this study.
Patient and public involvement
The ECIAE study does not include any patient or public involvement in terms of setting research priorities, defining research questions or outcomes, providing input into the study design or disseminating the results. The research participants will be called on to respond to questionnaires or interviews.
Study design and timeline
The main steps of the study are to:
Develop CMO hypotheses.
Collect the data necessary to validate them.
Analyse the data.
Amend and refine the CMO configurations.
To do this, several collection methods will be used (see figures 2 and 3).
Study design. ASI, Addiction Severity Index; CMO, context-mechanism-outcome configuration; ETP, Education Thérapeutique du Patient; MINI, Mini International Neuropsychiatric Interview; TSR, Treatment Service Review.
ECIAE study timeline.
Stage 1
Stage 1 involves data collection to become familiar with each of the three programmes and identify hypotheses about the outcomes, mechanisms and contextual elements (including the techniques). A literature review will be conducted to identify potential key elements and mechanistic explanations for alcohol-related harm reduction interventions. The documentary corpus analysis and first round of interviews with informants will help to elaborate initial middle-range theories (to establish how the intervention works in context).
Literature review
The first stage involves a literature analysis to contribute to the elaboration of the initial middle-range theories.
A scoping review will be conducted to identify strategies applying harm and risk reduction strategies with alcohol users. It will give us an overview of intervention strategies used in this approach as well as of potential effective levers. Our research objectives correspond to the following indications of the scoping review: to identify the types of evidence available in a given field, clarify key concepts/definitions in the literature, examine how research is conducted on a certain topic or field, identify key characteristics or factors related to a concept and identify and analyse knowledge gaps.37 We will conduct our search via bibliographic databases: MedLine using the PubMed portal, PsycInfo, PsycArticles, Behavioural Sciences Collection and Web of Sciences. The keywords will be ‘harm reduction’, ‘manage alcohol’, ‘intervention’, ‘addiction’ and ‘alcohol’. The inclusion criteria are: English or French language publications from 2011 to 2021; original articles focusing on the evaluation or exploration of HR intervention strategies with alcohol users. Using the Covidence software, two researchers will conduct a double-blind review of the selected reports and extracted information regarding the intervention (name, location, population, design), evaluation design (method), result of the intervention, its key functions and context of implementation. The purpose of this review will be to extract information about the contexts, mechanisms and outcomes that are the focus of our study. Our analysis questions are: what are the most effective intervention techniques? What contextual conditions influence the impact of the techniques? What are the conditions for the success of these interventions? For which populations? The results will be used for the initial development of middle-range theories.
Documentary corpus analysis
The analysis of the corpus of documentation concerns all the documentation available at each centre on the subject of the programme studied as well as documentation available on the internet. These documents provide information on the programmes delivered, their construction, delivery and evaluation. They contribute to causal hypotheses about how each programme works.
Informant interviews
These interviews will allow us to identify the intervention theory guiding the actions of professionals towards beneficiaries for each programme studied. Using the Astaire tool and TDR,38 39 the aim of these interviews will be to identify the main active elements of the programmes by putting forward causal hypotheses: what are the activities of the workers? How do they do them? In what context? Why or by what mechanism is the desired effect achieved? The aim will therefore be to gather as much information as possible on the description of each programme and the analytical views of key informants on each programme.
Two seminar sessions were initially planned, one at the beginning of the study and one at the end. Due to the current health constraints caused by the Sars-Cov2 virus, the first face-to-face seminar was replaced by telephone or face-to-face individual interviews with the people who created or initiated the development or adoption of the programmes. Interviews will therefore be conducted with programme creators, centre directors, programme referents within the centres or professional trainers, depending on the situation. These exchanges will make it possible to put forward hypotheses linking the intervention and its context to the supposed or observed effects.
Data collection from informants will be focused on the data described in online supplemental table 1.
Supplemental material
These first three steps will enable the research team to develop an initial intervention theory for each programme. These sets of hypotheses will then be tested against reality through data collection from beneficiaries and professionals.
Stage 2
Qualitative data
This round of data collection will contribute to verifying and amending the initial theories (contribution analysis). The research team will use three data sources: data from observations carried out in the centres participating in the study, data from semistructured interviews with professionals providing the programmes and data from semistructured interviews with the beneficiaries of the various programmes.
Observations
Observations will be carried out in each centre participating in the project, implementing one of the three programmes studied. They will allow the identification of the environmental contextual elements that may contribute to the impact of the interventions. We chose here to conduct non-participatory observations without any intervention on the part of the observer or participation in the activities carried out in the centres.40
Data collection from observations will be focused on the data described in online supplemental table 1.
Eight observations will be carried out: one in each participating centre.
Interviews with professionals
Semistructured interviews will be conducted with professionals implementing one of the three programmes studied. They will allow collection of data concerning the perception of the programme (usefulness, accessibility, feasibility, salient features, difficulties and so on). We aim to use a non-directive process, with open-ended questions to encourage the subject to express themselves freely. In this way, we defined some themes to be addressed and some follow-up questions.41 Data collection from professionals will be focused on the data described in online supplemental table 1.
These interviews will be conducted approximately 6 months after inclusion in the programme in question.
Ten interviews will be conducted with professionals (3 Alcochoix, 4 Iaca and 3 ETP Conso-repère).
Interviews with beneficiaries
Semistructured interviews will be conducted with beneficiaries of one of the three programmes. They will allow the collection of data concerning perceptions of the programme (usefulness, accessibility, feasibility, salient features, difficulties and so on). As with the interviews with professionals, we also chose a non-directive process for these interviews.41 Data collection from beneficiaries will be focused on the data described in online supplemental table 1.
These interviews will be conducted approximately 6 months after inclusion in the programme concerned.
45 interviews will be conducted with beneficiaries (15 Alcochoix, 20 Iaca and 10 ETP Conso-repère).
The next three steps will allow the research team to enrich and complete the initial intervention theory. They will allow us to validate or invalidate certain mechanistic hypotheses about the different programmes. The research team will then be able to propose a middle-range theory for all the programmes.
Quantitative data
The aim of this data collection is to gather longitudinal data concerning the effects of different interventions.
All participants who meet the eligibility criteria will be offered to participate in the study. A meeting will then be organised between the patients and the research team, in order to collect data:
The Baseline M0 will then be scheduled (maximum 15 days after starting the intervention).
Data will be collected from 45 beneficiaries, prospectively by trained clinical research staff. During baseline inclusion (M0), participants will be interviewed using the following:
the Addiction Severity Index (ASI),
the Treatment Service Review (TSR),
the Mini International Neuropsychiatric Interview (MINI),
the Empowerment Scale.
At each follow-up, participants will be assessed with a follow-up ASI, TSR interview, Craving Evaluation Scale and Empowerment Scale.
Addiction Severity Index (ASI)
The ASI is a semistructured interview designed to assess impairments that commonly occur due to substance-related disorders.42 A modified, validated, 45 min French version of the ASI will be used to take account of tobacco and addictive behaviours.43 The ASI explores six areas that may be affected by addiction: medical status, employment/support status, substance and behavioural addiction, family and social relationships, legal status and psychological status. These data will be used to generate Composite Scores (CSs) for each domain, thereby reflecting the severity of the subject’s condition. CSs range from 0 to 1, with severity worse when scores are closer to one.44
Mini International Neuropsychiatric Interview (MINI)
The Mini International Neuropsychiatric Interview is a structured diagnostic interview providing standardised assessment of 18 major psychiatric disorders defined according to Axis I DSM-IV (anxiety disorders, mood disorders, psychotic disorders, addictive disorders, eating disorders) and the diagnosis of antisocial personality disorder.45 A 30 min version of MINI adapted for DSM-5 criteria will be used.
Craving Evaluation Scale
The Craving Evaluation Scale developed by the University of Bordeaux Addiction Team from the SANPSY Laboratory will be used. It is a 5 min hetero-evaluation of craving for all substances and of addictive behaviours now or in the past. This tool explores the frequency of craving, corresponding to the number of days on which craving was reported during the last 30 days as well as mean and maximum intensity on a scale ranging from 0 (no craving) to 10 (extreme craving).
Treatment Service Review (TSR)
The Treatment Service Review, 6th version, is an inventory of the medical, psychosocial and psycho-educational contacts of the subject over the last 30 days.42 46 This instrument allows for quantitative evaluation of a subject’s effective medico-psycho-social management. It was validated in French and is now integrated into the ASI evaluation, as it was developed by the same group that developed the ASI.
Empowerment Scale
The Empowerment Scale measures personal empowerment by examining the concepts of hope, social acceptance and quality of life. It is a 28-item scale with four points each, ranging from ‘Strongly Disagree’ to ‘Strongly Agree’. The total empowerment score is a quantitative variable, ranging from 28 to 112. This scale can be divided into subdimensions measuring self-efficacy and self-esteem, power and powerlessness, community activism and autonomy, optimism and control over the future and righteous anger.47 48
Stage 3
Qualitative data
Data analysis will include analysis of each case (application situation at the first level and programme at the second level) and multiple cross-case analyses comparing different applications of the same programme and different programmes aiming for similar objectives. The analysis will answer the question: in what contextual conditions and through which mechanisms does the programme (Alcochoix, Iaca, ETP Conso-repère) produce outcomes? The validation of initial middle-range theories (CMOs) will allow us to answer the question. This validation will combine and compare data from quantitative and qualitative analyses in monographs (for each application situation and for each programme) and by cross-case analysis (analysis comparing these application situations and programmes).
This analysis will identify the recurrent CMOs and this set of qualitative investigations will be used to validate the final middle-range theories.
All the qualitative data (interviews with professionals and beneficiaries, and observations) will be examined by content analysis,49 which refers to ‘a set of techniques for systematically and objectively analysing and describing the content of communication. The aim is to obtain indicators allowing inferences to be made about the messages and how they are produced and received (inferred variables)’. Content analysis encodes, classifies and ranks the communications to examine patterns, trends or distinguishing features; in our case, the recurrence of CMO configurations in each case and by cross-case analysis.
Quantitative data
Quantitative evaluations will make it possible to assess the impact of these interventions on the main judgement criterion (ie, the change in alcohol use severity at 12 months) and to describe the evolution of the subjects over 12 months.
Descriptive analysis will be performed to describe the severity of the subjects' alcohol use after 12 months of intervention. This change in alcohol use severity corresponds to the delta of the composite scores between M12 and M0. Alcohol consumption, alcohol craving and severity of addiction variables will be described over the 12 months of the intervention in relation to the initial assessment. They will also be compared between centres and types of intervention.
Qualitative variables will be described according to their frequency and percentage. Quantitative variables will be described according to their means and SD.
Second, to determine the factors impacted by the intervention, we will perform repeated measurements to analyse variance and determine whether the variables changed during the intervention. For variables showing a change, we will use a comparison test on repeated measurements controlling for sociodemographic variables: age, gender, work in the last 3 years, presence or absence of current mood and anxiety disorders, the centre in which the intervention was carried out and the type and quantity of intervention (applying the Bonferroni correction). All statistical analyses will be performed with JMP software (V.Pro 15.2.0, SAS Institute, North Carolina, USA).
Stage 4
Final seminar
A final half-day to 1 day seminar will take place during the ECIAE project after the various data have been collected in order to validate, refine or amend our hypotheses and to define the intervention theory for each programme. This will be held face-to-face or remotely according to the health recommendations in force. All members of the research teams involved, representatives of each programme and professionals participating in the study will be invited.
This last step will allow us to finalise the realist evaluation by proposing a middle-range theory of alcohol harm and risk reduction interventions based on three programmes (Alcochoix, Iaca and ETP Conso-repère) conducted in the Nouvelle Aquitaine region. The research team will then be able to propose recommendations on the topic based on the key functions identified as well as the conditions of impact highlighted by this study. The specificities and complementarities of these three programmes will also be presented. A policy brief will thus be presented to the Regional Health Agency (ARS) that commissioned the study.
Ethics and dissemination
Ethics
The project will be carried out in full accord with current relevant legislation (eg, the Charter of Fundamental Rights of the EU) and international conventions (eg, Declaration of Helsinki). The project respects the obligations related to the EU General Data Protection Regulation. The study protocol was submitted to the University of Bordeaux’s Data Protection Officer (DPO) for analysis and did not qualify as falling under the rules applying to ‘Research Involving the Human Person’. This protocol did not therefore require the solicitation of an ethics committee but only registration in the DPO register and the implementation of a Privacy Impact Analysis. All sensitive data collected in the framework of the study are routine data, the collection of which is part of National Authority for Health (HAS) recommendations. The University of Bordeaux has ensured that all the regulatory procedures related to the ECIAE study have been carried out.
The methods’ development, data collection and analysis will take the following issues into account:
Anonymity of study respondents will be preserved and ensured at all times. Unnecessary collection of personal data will be avoided, and respondents will have the right to review outputs and withdraw consent. All personal data will be coded, removed from the data for analysis and stored separately. Only designated research staff will have access to the keys linking data with personal information.
Information regarding the study and the right to refuse to participate will be distributed to all study participants., In the case of refusal, alternative means of data collection will be explored (eg, alternative respondents).
Dissemination
The dissemination plan of this study will follow the participative methodology. The results will be disseminated on various academic and non-academic platforms. Several papers will be published in international peer-reviewed journals (literature review, results). The results will be presented at international and national conferences. A final seminar will be held in order to discuss and validate some of the work’s hypotheses. A public report will describe all the steps of the study, results and recommendations. Eventually, a general restitution will be held in order to present the final results of the study to all the participants and funders.
Discussion
This article describes a protocol using a realist design to understand how three programmes in southwestern France work to reduce the harm and risks associated with alcohol consumption. This protocol also aims to identify the contexts, mechanisms and outcomes involved in these interventions. Realist evaluation is a valid approach that highlights the triggers and mechanisms of an intervention and guides its transferability.
From a research viewpoint, our proposed methodology is consistent with the bottom-up approaches advocated in health promotion, starting with a real-world response to a pressing problem.50 As this approach provides a better reflection of stakeholders’ views and concerns and makes external validity workable, it therefore offers a preferable alternative for evaluation of health promotion or programmes.50
The ECIAE study mobilises realist evaluation based on multiple case studies at two nested levels. At the first level, each centre implementing the programme (differently) is a unique application situation that will represent a case. At the second level, each programme represents a case in which a set of activities is conducted to achieve risk reduction objectives. This represents a challenge and a particular methodological interest for our research team, which aims to develop intervention research methods.
By developing comprehensive research, the ECIAE study aims to include its perspectives in that of so-called formative evaluation. It should then make it possible to make readjustments, adaptations and improvements to the intervention proposals present in each of the three programmes.
The ECIAE study has some limitations. First, the fact that the three programmes have not all reached the same stage of maturity or deployment can lead to difficulties in the cross-case analysis and must be taken into account during phases II/III and IV. The Alcochoix programme is an intervention that has been worked on and disseminated for a long time. It has been the subject of much reflection and many readjustments (depending on the context and audience). It has been evaluated in depth several times, in particular as regards its impact. On the other hand, the Iaca programme has already benefited from an evaluation of its impact and a realist study by our team in 2019, which made it possible to define the intervention theory of the programme. It is currently being conducted in 10 different centres located in the Nouvelle Aquitaine region, in southwest France as well as in the Provence-Alpes-Côte-d'Azur region, in the southeast of the country. It is also the subject of a pilot study to assess its viability before being tested by a large-scale trial. Finally, the ETP Conso-repère programme is a very recent intervention that is still under development and has not yet been tested and evaluated. In addition, the ETP Conso-repère programme is being dispensed only in a single centre. We will therefore be very cautious in terms of comparative analyses and in our recommendations.
Another limitation is that the ECIAE study concerns a small number of subjects (only 10 professionals and 45 beneficiaries will be recruited). It will also face some challenges and limitations insofar as it started during the COVID-19 crisis, which is impacting the follow-up and implication of beneficiaries and the professionals. We also expect a significant risk of loss of follow-up during the ECIAE study for various reasons. The sector of support, prevention and care in addiction and alcoholism is experiencing significant turnover of its personnel, according to most of the centres participating in the study. In addition, the target group for these programmes is a difficult group to retain. It is common for these beneficiaries to discontinue monitoring while the programmes are being dispensed. Finally, the quantitative part of the study, made up of various repeated hetero-administered questionnaires, represents a significant effort on the part of the beneficiaries. These interviews require them to give up 1.5–2 hours of their time on one occasion and then half an hour on three occasions, without anything in return for the time they devote to the study. This study will therefore only be able to consider a small variation in points of view, uses, experiences and situations among its sample, by focusing on describing each study case thoroughly.
Nevertheless, we hope that the ECIAE study will contribute to building or consolidating the ‘bridge’ between implementation of interventions in the field on the one hand, and the research and decision-making world on the other. The study is sponsored and funded by the Regional Health Agency of the Nouvelle-Aquitaine region and will contribute to defining health policy based on scientific data. It will allow us to draw up guidelines for the deployment and development of alcohol HR interventions, with particular attention to the specificities and complementarities of the three programmes. Reducing harm and risks associated with alcohol consumption is a topic of major interest for public health policies. While the theoretical foundations of the study are fairly clearly established, the practical and effective intervention mechanisms to be used are still a grey area. ECIAE will therefore help to highlight a certain number of key ingredients that can be mobilised with this in mind.
Ethics statements
Patient consent for publication
Acknowledgments
The authors are very grateful to all those who took part in the project.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
NS and JM-F are joint first authors.
Contributors NS and JM-F drafted this article and all authors revised the manuscript. The project design was developed by LC and MA. JM-F, NS, SM and FS were involved in implementing the project and in developing the evaluation design, under the supervision of LC and MA. All authors read and approved the final manuscript.
Funding This study has received funding from Regional Health Agency of New Aquitaine (ARS Nouvelle Aquitaine). Award/grant number : N/A.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.