Article Text
PDF
Abstract
Introduction The Modification of Diet, Exercise and Lifestyle (MODEL) study aims to examine the impact of providing visualisation and pictorial representation of advanced structural vascular disease (abdominal aortic calcification), on ‘healthful’ improvements to diet and lifestyle. This paper reports the protocol for the process evaluation for the MODEL study.
Methods and analysis The overall aim of the process evaluation is to understand the processes that took place during participation in the MODEL study trial and which elements were effective or ineffective for influencing ‘healthful’ behavioural change, and possible ways of improvement to inform wider implementation strategies. A mixed-method approach will be employed with the use of structured questionnaires and semistructured in-depth interviews. All 200 participants enrolled in the trial will undertake the quantitative component of the study and maximum variation sampling will be used to select a subsample for the qualitative component. The sample size for the qualitative component will be determined based on analytical saturation. Interviews will be digitally recorded and transcribed verbatim. Qualitative data will be analysed thematically and reported according to the Consolidated Criteria for Reporting Qualitative Research (COREQ) guidelines.
Ethics and dissemination The MODEL study process evaluation has received approval from Edith Cowan University Human Research Ethics Committee (Project Number: 20513 HODGSON). Written informed consent will be obtained from all participants before they are included in the study. The study results will be shared with the individuals and institutions associated with this study as well as academic audiences through peer-reviewed publication and probable presentation at conferences.
Trial registration number ACTRN12618001087246.
- public health
- medical ethics
- social medicine
- qualitative research
- statistics & research methods
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
A comprehensive evaluation of all components/elements of a complex intervention will be achieved using a mixed-methods approach.
Maximum variation sampling will be used to select participants for interview to maximise the diversity relevant to the research objectives.
A reliable method of inquiry will be employed using standardised set of questions for the survey (quantitative component).
Qualitative findings will give rich insights into perspectives of participants engaged in the Modification of Diet, Exercise and Lifestyle study intervention.
A limitation of this study is the risk of recall bias (unintentional and intentional responder bias) due to poor memory or the life-threating/life-changing nature of cardiovascular disease.
Introduction
Suboptimal lifestyle choices and risky behaviours are the leading causes of atherosclerosis which, in turn, precipitates most cardiovascular disease (CVD) events, such as heart attacks and strokes.1–3 Most CVD-related events can be prevented or delayed by improvements to lifestyle factors including diet, physical activity and the cessation of smoking.2 Despite the known benefits of these factors, few people take up or adhere to existing lifestyle recommendations. Therefore, strategies to encourage individuals to initiate and adhere to long-term dietary and lifestyle changes are urgently needed. One strategy that offers promise in this regard is to provide individuals with visual information about their blood vessel health using vascular imaging modalities. New technologies have enabled information about blood vessel health to be provided to study participants,4–7 and a randomised controlled trial (RCT) of the impact of such information on behaviour change is forthcoming. This RCT holds great promise for modifying behaviour in older individuals with no history of clinical CVD. The purpose of this protocol is to overview the process evaluation that will be embedded within the Modification of Diet, Exercise and Lifestyle (MODEL) randomised control trial.
Critics of RCTs contend that there’s a set of ‘positivist’ assumptions that drive RCTs which are discordant with understanding the context of complex interventions.8 Berwick,9 Clark et al,10 Pawson and Tilley11 opined that there is an oversimplification of cause and effect in RCTs of complex interventions and investigators often ignore the agency of participants and implementers as well as the context in which the intervention is experienced and implemented. There is emerging evidence to support the line of reasoning that a more critical realist framework should guide the conduct of RCTs of complex interventions. This will enable methods to be applied and interpreted critically while social realities are viewed as valid objects of scientific study.12 The Medical Research Council (MRC) framework13 14 does not support the arguments against RCTs but acknowledges that ‘effect sizes’ alone are not sufficient, and that process evaluations should be conducted alongside of RCTs to limit biases when estimating effects. Process evaluations provide insight into implementation processes and mechanisms of impact in complex interventions, assisting with interpretation of overall study outcomes.13 15 16 They can also provide detailed information that could support the interpretation of causality by a systematic reviewer, practitioner or policymaker.13 14 Process evaluations have been demonstrated to be useful at the time of explaining trial results for complex interventions.17–20
For example, van Dongen et al17 used a comprehensive process evaluation plan to examine the delivery and receipt of a diabetes prevention intervention by evaluating the intervention components that contributed to effective prevention of type 2 diabetes.17 They concluded that it is feasible to implement a diabetes prevention intervention in Dutch primary healthcare after completion and reporting results of the process evaluation.17 Another process evaluation assessed the quality of the execution of a programme for a self-management intervention for people with polyarthritis from the participants’ perspective.12 The process evaluation results identified the extent to which specific exercises and programme were highly valued and therefore the need to use various components such as writing exercises, use of role models and combined individual trajectory and group training to create an attractive intervention for a broad audience.18 Also, the ProActive study (a physical activity intervention) process evaluation19 20 identified various reasons for trial outcomes using an explicit a priori hypothesised causal model while the Welsh National Exercise Referral Scheme intervention21 process evaluation reported that there were limitations in communication, training and support which impacted the fidelity of some components.21 Moreover, a process evaluation for an adolescent sexual health programme intervention in Tanzania reported the extent to which young people were engaged with the programme and quality of programme implementation.22 All of these process evaluation examples have reported on the impact of contextual factors on the effectiveness of an intervention22 as well as contextual factors and implementers’ actions that shaped delivery21 and the fidelity of implementation19 using mixed-methods21 22 or quantitative approaches.19
This study will evaluate the implementation, mechanisms of impact and key contextual factors involved in outcomes of the MODEL study using a mixed-method approach. This will enable the investigators to better understand how and why the intervention was effective or ineffective, as well as identify contextual factors involved in outcomes to inform wider implementation strategies. It will also be useful in the interpretation of trial results.
The MODEL study
The MODEL study will investigate whether providing individuals with visualisation and pictorial representation of structural vascular disease in the abdominal aorta can influence short-term fruit and vegetable intake (primary outcome), adherence to other dietary recommendations (eg, sodium, fibre, whole grains, seeds and nuts intake), physical activity, gut health, physical function and psycho-emotional and mental health outcomes (motivation to initiate behavioural change, perceived risk of CVD, depression, quality of life). All participants will have their abdominal aortic calcification (AAC) assessed from a lateral spine image captured using dual-energy X-ray absorptiometry at baseline. The MODEL study will include a total of 200 (n=100 control group; n=100 intervention group) ambulant community-dwelling Australian men and women, aged 60–80 years, recruited from the general population in metropolitan Perth, Melbourne and surrounding areas in Australia. A detailed explanation of the methods for the MODEL study is provided in the protocol for the MODEL study (Radavelli-Bagatini et al in press).
Process evaluation
The process evaluation will ascertain the participants’ views on the counselling session (including information about atherosclerosis and diet and lifestyle advice provided in videos and summarised in a booklet) and reaction to their blood vessel disease results (image and illustrative information). It will also be useful in terms of evaluating the factors in the community, socioeconomic context, participant characteristics or other situational issues, that may influence the process of changing behaviour. This will inform future methods, intervention designs and theories23–25 in addition to ascertaining the direction of the intervention’s key components to produce the anticipated results.13 14
Aim
The overall aim of the process evaluation is to understand the processes that took place during participation in the MODEL study trial and which elements were effective or ineffective for influencing ‘healthful’ behavioural change, and possible ways of improvement to inform wider implementation strategies.
Specific objectives :
1. To evaluate the resources, structures and the procedures used to deliver the MODEL study intervention from the perspective of participants.
2. To assess participants’ responses to the MODEL study intervention and mediating processes which may influence the process of changing behaviour and subsequent changes in outcomes.
3. To better understand the contribution of external factors which may influence intervention outcomes (ie, behaviour change).
The research objectives for the process evaluation were structured around the three domains of implementation, mechanisms of impact and context. This is required to assess the intervention using a standardised process evaluation framework.14 The conceptual framework will aid us to address the three objectives of the process evaluation.
Conceptual Framework for the Process Evaluation
This process evaluation design was informed by the guidance for process evaluations as specified by the MRC.13 14 Specifically, the process evaluation will examine three key features—implementation, mechanisms of impact and context—to understand the processes through which one can achieve outcomes (figure 1). Table 1 further illustrates the domain/constructs, objectives and how the objectives will be addressed.
Domain/constructs, objectives and how the objectives will be addressed
Key functions of MODEL study process evaluation and relations among them. Adapted from Moore et al14 and modified for the MODEL study process evaluation. AAC, abdominal aortic calcification; CVD, cardiovascular disease; MODEL, Modification of Diet, Exercise and Lifestyle.
Process evaluation methods
Design considerations
The intervention is expected to influence behavioural change based on certain mediators/moderators such as perceptions of severity and susceptibility. Factors in the community, social/political context or other situational issues have been associated with tobacco use, physical inactivity and poor diet.26–31 Therefore, in the course of the intervention, situations which may influence the outcome of the intervention such as family, friends, general practitioner (GP), cultural differences, finances as well as access to information (internet, social media) will be part of the context to be explored. Participants perceived risk of CVD, perceptions of CVD severity and susceptibility and perceived self-efficacy is also expected to be key mediators of behaviour change. While we anticipate that these influences will be relevant contextual factors and mediators/moderators, we remain open to other potential contextual factors and mediators/moderators obtained from the qualitative interviews where participants describe their experiences in their own words. Health-related behavioural change will be explained and predicted in this study using the social-psychological health behavioural change model known as the Health Belief Model.32
Overall design
The process evaluation will employ a mixed-method approach using both qualitative and quantitative methods of data collection and analysis. This will include the use of a structured questionnaire and semistructured in-depth interviews (to be administered to participants). There are several reasons for focusing on the perspectives of participants. The intervention is intended to act on the perspective of participants; their perception of the effectiveness of the components is critical to identify key components and effective techniques. In other words, the intervention is likely to depend on participants’ interpretations of, and reactions to, the intervention; hence, it is important to consider those perspectives. Also, the participants will not be passive receivers of the intervention and it will likely influence their circumstances, attitudes, beliefs, social norms and resources.14
All participants recruited for the MODEL study will respond to a questionnaire that has been designed for the process evaluation. Maximum variation sampling (also known as maximum diversity sampling or maximum heterogeneity sampling),33 a form of purposeful sampling, will be used to select participants with characteristics that maximise the diversity relevant to the research objectives. This sampling will be used to assess what influences behaviour change among participants at Perth and Melbourne study sites. Participant characteristics such as ethnicity/culture, age, profession, household income as well as sources of income will be considered in the selection. The sample size will be determined based on analytical saturation.34 This is commonly taken to indicate that, based on the data that have been collected or analysed, further data collection and/or analysis are unnecessary.34 We anticipate achieving saturation with 15–20 trial participant interviews.
The research team will be composed of investigators with diverse backgrounds, such as psychology, nutrition, exercise physiology, social work, with some being part of the core team of the RCT (MODEL study).
Data Collection
Qualitative data will be collected using a semi-structured in-depth interview. A semistructured interview guide (online supplemental appendix 1) will be used to enquire about experiences of participants in terms of clarity of information, counselling, reaction to their blood vessel disease results (image and illustrative information) and cardiovascular risk factors. Interviews will be conducted approximately 1 month after participants complete the baseline component of the intervention. Participants must complete a 30 min counselling session at baseline (including watching three educational videos, receiving a booklet with diet and lifestyle information), and receive their AAC results and baseline biochemistry results. Quantitative data will be collected using a questionnaire (Postcounselling health status questionnaire—online supplemental appendix 2). This questionnaire will be used to obtain information on the perceived risk of CVD, perceptions of CVD severity and susceptibility and perceived self-efficacy. It will be administered immediately after participants complete their baseline counselling session.
Supplemental material
Supplemental material
The use of semistructured interviews will provide flexibility in exploring relevant and interesting matters as raised by participants. This will enable prespecified areas to be explored and remain open to exploring other ideas and thoughts that will arise in the interview.35 Table 2 presents information on study objectives, sample, data collection tools and what data will be gathered at each stage of the trial.
Methods for objectives
All consenting trial participants will be invited to respond to a questionnaire with a sub-sample invited to participate in an interview.
Investigators involved in data collection will discuss the aims of the questionnaire/interviews and provide information on any potential benefits and harm of participation. Participants will be assured of the confidentiality of the information they will provide. Interviews will be conducted at a mutually convenient site. The first author will administer the questionnaires and conduct the interviews. Each interview will be audio recorded and transcribed verbatim later.
The research team will develop the questionnaire, and the interview guide based on the objectives of the process evaluation, secondary data on the topic and further discussions and brainstorming among the research team. The questionnaire and interview guides will be piloted in the initial stages of the study to assess suitability for the study. As suggested by Given36, interview guides will be amended as necessary by the research team.
Management of data
Questionnaire data will be entered into SPSS V.21.0 data management and analysis software. Interviews will be digitally recorded and transcribed verbatim. All identifying aspects will be removed to maintain anonymity and confidentiality and pseudonyms will be assigned.
Analysis
The quantitative data will be analysed using SPSS (V. 21.0). The analysed data will be organised into frequency tables and represented on pie charts and tables. The analysis of the primary data will be entirely descriptive (summaries, frequencies and cross-tabulation tables).
The qualitative data will be analysed thematically. The analysis and interpretation of the interviews will be guided by Huberman and Miles’s framework for thematic content analysis.37 The stages will involve the identification of meaning units, an initial grouping of meaning units into categories, and the creation of emergent category names. Following this stage, initial themes will be developed using a constant comparison method to ensure those meaning units are reflective of emergent themes. This will also focus on examining intra-theme coherence/consistency and intertheme distinctiveness. The first author will lead the analysis and other authors will review that analysis and NVivo V.12 software will be used to assist the data analysis. Using this software will enable the investigators to examine themes and structure in the content as well as visualise the findings and support findings with detailed evidence. An experienced qualitative researcher (MSt) will be engaged for peer debriefing and member checking will be conducted to enhance rigour. Investigators undertaking the MODEL RCT’s assessment and counselling (SR-B, CPB, MSi, LB, EC, JTS, MPS, JG and BDR) will not be involved in the process evaluation data analysis or interpretation. Qualitative data will be collected and reported according to COREQ guidelines.38
Integration of process and outcomes data
Survey data on contextual factors (participant characteristics) and mediators (perceived risk of CVD, perceptions of CVD severity and susceptibility and perceived self-efficacy) will be analysed prior to analysis of outcome data. After the interviews (on the impact of contextual factors such as family, GP, etc) are conducted and analysed, the process evaluation investigators will be able to conclude that the MODEL study intervention has been successful by communicating clear information on CVD risk and prompting lifestyle/behavioural change. The process data will also highlight the role of contextual factors and mediators enabling participants to change lifestyle/behaviour or not. These data will be used for post hoc explanation after trial outcomes are known.
Discussion
This is a detailed protocol for a process evaluation embedded within a randomised control trial, the MODEL study. The process evaluation will provide useful information on the MODEL study intervention and how and why the key components/elements (provision of information on CVD risk) impacted on lifestyle/behaviour change or not. This process evaluation will complement and add value to the MODEL study by providing a better insight into study results. The investigators of the MODEL study will, therefore, be confident after the report of the process evaluation data that it is feasible or otherwise to use similar approaches to conduct this type of study or influence lifestyle/behavioural change. The researchers will also derive insight into possible methods for improvement to inform wider implementation strategies as demonstrated in previous process evaluations.17 18 39
This process evaluation will employ a comprehensive approach to evaluate the resources, structures, and the procedures used to deliver the MODEL study intervention. Interviews will be conducted to gather information on participants experiences throughout the intervention. This would be useful in identifying reasons for lack of intervention effect (if any) or any significant changes in lifestyle/behaviour. This is in contrast with some other process evaluations such as the ProActive study (a physical activity intervention)19 ,20, which did not include any qualitative component to identify reasons for lack of intervention effect and a significant increase in physical activity among participants.19 ,20
Although a mixed-method approach was employed for the process evaluation for the Welsh National Exercise Referral Scheme intervention,21 the logic model focused more on links between intervention activities and mechanisms of impact and only limited focus on delivery mechanisms. The MODEL study process evaluation aims to focus equally on delivery mechanisms (ie, application of resources such as videos and counselling to ensure implementation), intervention components, mechanisms of impact and intended outcomes (behavioural change).
The MODEL study process evaluation also aims to gather extensive data on theoretical determinants of behaviour change such as risk perception and self-efficacy. However, a process evaluation for an adolescent sexual health programme intervention in Tanzania22 gathered inadequate data on the impact of the intervention on the theoretical determinants of behavioural change.
Evaluating and reporting what works for which group and what constitutes an effective intervention is an essential consideration for practitioners, researchers and policymakers.40 41 The MODEL study process evaluation will contribute to existing knowledge and understanding of the processes that took place during participation in the MODEL study trial. It will also serve as a guide for future studies that will be conducted for such complex trials.
Strengths and limitations
This study will employ a comprehensive mixed-method approach to evaluate the resources, structures and the procedures used to deliver the MODEL study intervention. The process evaluation will assess participants responses to the MODEL study intervention and mediating processes which may influence subsequent changes in outcomes and identify key contextual (external) factors which may influence the process of changing behaviour. Core intervention components that were effective in influencing lifestyle/behavioural change will be identified, forming the basis for guidance for replication in future studies and implementation in other programmes.
This process evaluation will not evaluate the fidelity of the MODEL study and the associated challenges in delivery from the perspective of the study investigators. Another limitation is the risk of recall bias specifically referring to responder bias (unintentional or intentional) or possible difficulties on the part of participants recalling all information gathered from the intervention. Unintentional responder bias may be attributed to incomplete or poor memory recall and intentional responder bias may be attributed to embarrassment with admitting truth about previous event or nature of disease under investigation. The MODEL study intervention will use several resources and procedures in its delivery and it is anticipated that recalling all information gathered from the intervention may be a challenge. Also, some participants may intentionally give inaccurate details about their lifestyle/behavioural change due to the life-threating/life-changing nature of CVD or embarrassment associated with not changing behaviour.
Ethics and dissemination
The MODEL study process evaluation has received approval from the relevant Ethics Committee (Edith Cowan University Human Research Ethics Committee approval, Project Number: 20513 HODGSON). The study results will be shared with the individuals and institutions associated with this study as well as academic audiences through peer-reviewed publication and presentation at conferences.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @AdMandydevine, @stanley_mandy
Contributors RA, SR-B, LCB, MSt., JMH and JRL developed the study concept. RA, MSt., JRL, JMH drafted the manuscript. RA, MSt, JRL, LCB, JD, BJ contributed to the design of the study and are responsible for study coordination. RA, SR-B, JRL, JD, BJ, DPK, JTS and JMH contributed to the design and development of the data collection instruments. RA will implement the protocol as well as oversee the collection of the qualitative data and will code all transcripts. RA and CPB will oversee the collection of the quantitative data. RA, MSi and NPB will be involved in the analysis of quantitative data. RA, SR-B, MSi, CPB, EC, RW, KZ, MSt., WHL, PS, RMD, KLC, AD, PLT, JG and BDR contributed to the writing of the study content. All authors contributed and approved the final manuscript.
Funding The salary of JRL is supported by a National Heart Foundation of Australia future leader fellowship (102817). DPK’s time was supported by a grant from the National Institute of Arthritis, Musculoskeletal and Skin Diseases (R01 AR 41398). The salary of JMH is supported by a National Health and Medical Research Council of Australia Senior Research Fellowship (ID 1116973). The salary of LCB is supported by an NHMRC of Australia Emerging Leadership Investigator Grant (ID: 1172987) and a National Heart Foundation of Australia Post-Doctoral Research Fellowship (ID: 102498). RA is supported by an Australian Government Research Training Program Scholarship (International).
Disclaimer None of the funding agencies had any role in the conduct of the study; collection, management, analysis, or interpretation of the data; or preparation, review, or approval of the manuscript.
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.