Introduction Community-based weight management programmes are important in addressing childhood obesity. However, the mechanisms that lead to behaviour change within the programmes are rarely studied within the context of the programmes themselves once they have been implemented. This means that further potential gains in the effectiveness of the programme are often not made and any potential losses of efficacy are often not noticed. Qualitative research alongside randomised controlled trials (RCTs) can tell us the context in which these programmes are implemented and elucidate potential mediators or modifiers of the programmes' effectiveness. The aim of this evaluation is to determine the barriers and enablers to the delivery and impact of an incentive-based behaviour change strategy targeting child obesity to inform future translation.
Methods and analysis Qualitative analysis, including stakeholder and family interviews, focus groups and a survey, will be used. The research will be conducted in collaboration with policymakers, researchers and community health professionals. Participants will be selected from programme providers, and parents/carers and children participating in an Australian community weight management programme during an RCT examining the effectiveness of incentives for improving behaviour change. A maximum variation sampling method based on participant demographics and group characteristics will be used. Thematic analysis will be carried out inductively based on emergent themes, using NVivo V.9.
Ethics and dissemination This research is approved by the South West Sydney Human Ethics Committee review body (HREC/14/LPOOL/480). The evaluation will provide information about the contextual and influencing factors related to the outcomes of the RCT. The results will assist researchers, community health practitioners and policymakers regarding the development, implementation and translation of behaviour change strategies in community initiatives for obese children. Insights gained may be applicable to a range of chronic conditions where similar preventive intervention approaches are indicated.
Trial registration number ACTRN12615000558527, Pre-results.
- PUBLIC HEALTH
- QUALITATIVE RESEARCH
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
Qualitative data collection of multiple perspectives allows for triangulation of findings.
This process evaluation will enhance our understanding of implementation and identify causal pathways explaining behaviour change.
Comparison between the intervention and control group sites will inform generalisability of the findings.
Considering behaviour change theory will increase the possibility of identifying specific active components of the incentive scheme and how they were effective.
There will be no opportunity in this design to carry out structured observations of the intervention during the implementation phase.
Childhood obesity is a global priority that many countries are currently trying to address. In 2013, the number of overweight children under the age of 5 was estimated to be over 42 million.1 Overweight and obese children are likely to stay obese into adulthood2 and are more likely to develop chronic diseases like diabetes and cardiovascular diseases at a younger age.3 The increasing prevalence also has implications for current and future health services.4 Therefore, improving the management and prevention of childhood obesity is extremely important.
Community-based weight management programmes are an important response to address childhood obesity; however, although the specific behaviours required for effective weight loss and long-term behaviour change are well established,5–10 facilitating health-related behaviour change remains an on-going challenge. The reasons for this are broadly twofold in that: (1) individuals (and particularly children) find it difficult to make lasting health-related behaviour changes11 ,12 and (2) associated implementation challenges mean that health behaviour change interventions based on behaviour change theory are often not upscaled and translated into a natural context.8 ,12–14
Difficulties associated with facilitating health-related behaviour change have led to an increased emergence of research investigating whether incentive schemes, based on behavioural theory such as operant conditioning,15 might be a potential solution. As such, promising research in adults has found that incentives can positively influence health-related behaviour change in the short term.16–18 For example, several systematic reviews have demonstrated positive outcomes as a result of financial incentives in terms of healthy eating19–21 and positive effects on exercise behaviour.22 ,23 Many of the studies included in the reviews have acknowledged the need for more research to determine specific information on the type, timing and magnitude of incentives needed to motivate individuals to change their behaviour, as well as disincentives.19 There are also mixed findings on whether incentives are more useful for simple one-off behaviours (eg, attendance at a vaccination) rather than complex health behaviours such as dietary behaviour change, and if specific groups may benefit more from incentive schemes.17 This highlights the need to fully understand not only the cognitive influences on behaviour change, but also social and environmental factors as well when designing and evaluating behaviour change interventions.
Despite the growing body of evidence in adults, robust research investigating the value of incentives focused on improving health-related behaviours in children is still lacking. Several (uncontrolled) studies have highlighted the potential value of behavioural incentives in children, and these studies have been summarised in a recent narrative review.24 One non-randomised study (n=1589) has demonstrated that an incentive programme, based on lottery-style tickets, increased the probability of children bike riding to school by 16%.25 Three randomised studies (across 40 elementary schools) have reported that small rewards (such as stickers or low-value financial rewards) doubled the number of children consuming a serving of fruit or vegetables with their school lunch,26–28 and two of these studies found evidence of sustained effects at 227 and 6 months28 after the intervention. However, these studies are based on small samples, the duration of the intervention and follow-up tends to be short and inconsistent between studies (eg, intervention duration ranged from 2 to 5 weeks, and follow-up duration ranged from 4 weeks to 6 months), they investigate a single health-related behaviour (ie, either diet or exercise) and there is no known qualitative research associated with the trials. Before a systematic review can be performed and provide meaningful information on the effectiveness of incentives for health-related behaviour in children, there is a need for robustly designed trials to provide more evidence on for whom and in what context incentive schemes might be most effective.
While randomised controlled trials (RCTs) are regarded as the gold standard for establishing the effectiveness of interventions, RCTs are incompatible with understanding how complex interventions work in context.29 Effect sizes alone do not provide policymakers with information on how an intervention might be replicated in their specific context, whether intervention outcomes will be reproduced or the broader impact on participants' lives.30 Conducting a process evaluation alongside a RCT provides a deep understanding of how interventions are implemented, what worked and did not work, and in which contexts it was most and least effective and why.31 In particular, qualitative research is extremely valuable for understanding the more subjective nature of participants' experiences and determining what kind of change has occurred.31 ,32 As a relevant example, there is a debate about whether extrinsic incentives can discourage the development of intrinsic motivation and undermine the development of longer term habit formation.33 Qualitative research can provide invaluable information about the disadvantages of using rewards to modify eating and exercise behaviour, which should be fully considered in the design of behaviour change schemes for children. Therefore, completion of a detailed qualitative process evaluation will facilitate the translation of knowledge from clinical research with children, and help guide how effective incentive strategies could be optimally implemented into routine practice for addressing childhood obesity. Furthermore, using a framework for deconstructing and specifying various intervention components within a process evaluation will enable the active components of the intervention that change behaviour to be isolated, along with underlying mediators of action, effective modes of delivery and the most receptive populations.34 This will facilitate replication of the interventions into community health settings, inform resource allocation and advance intervention science.
In summary, managing childhood obesity is a global priority. There is a gap in the evidence base on the impact of incentives on health behaviours in children, and qualitative research can provide rich contextual information to help the interpretation of child-focused obesity intervention delivery and outcomes. The overall aim of this research is to provide detailed information on the barriers and enablers to the intended delivery and outcomes of a child-focused incentive-based scheme, and determine whether the scheme is an acceptable intervention to improve behaviour change in overweight and obese children. This qualitative information will be used to build the evidence base and inform policy and practice. Specific aims of this research are to:
Identify what factors influenced implementation fidelity of the scheme to:
A. determine whether the incentive scheme was implemented as intended;
B. determine whether the model of goal setting and incentives enhances an existing community weight management programme and
C. understand how the implementation process might be improved.
Identify what factors influenced intervention outcomes and the broader impact on participants' lives to:
A. determine for whom and in what context the incentive scheme was effective.
Identify the active components of the intervention that influenced behaviour to:
A. determine which intervention components were the most effective;
B. determine any components that are unnecessary and
C. identify components that were non-effective but may be useful.
This qualitative evaluation research will be conducted shortly after the follow-up 6-month data collection point of an RCT conducted in New South Wales, Australia, from 2014 to mid-2016. The details of the RCT design are described elsewhere.35 In summary, the RCT intervention (n=524 children and 38 community-based programme sites across 5 local health districts) is a cluster RCT which tested the use of incentives linked to goal setting for sustained behaviour change in overweight and obese children aged 7–13 years. The RCT was set within the context of an existing Australian community-based weight management programme.
The weight management programme, ‘Go4Fun’, is a free, voluntary programme run by health professionals, addressing weight-related behaviour and self-esteem for overweight or obese children aged 7–13 years. Go4Fun is based on the Mind Exercise Nutrition Do-it (MEND) child weight management programme in the UK, which has demonstrated efficacy in weight outcomes (reduced waist and body mass index measures, and improvements in physical activity and self-esteem).36 The programme is multidisciplinary and evidence-based, and incorporates elements recognised as important to achieve long-term behaviour changes, such as family involvement, practical education in nutrition and diet, and increasing physical activity.7 ,8 Control sites in the trial delivered the standard weight management programme content consisting of weekly 2-hour group sessions for 10 weeks during the school term. Intervention sites delivered the standard programme together with an enhanced goal setting and the structured incentive scheme for goal achievement. This was supported in the 6 months following the programme by weekly text messages and a lucky draw-style incentive.
The participants in this evaluation will come from two groups.
First, existing participants in the RCT (parents/carers and their children), who have consented to being invited to further research. They were recruited for the evaluation via a brief survey given to parents/carers during the 6-month follow-up health assessments in the RCT. All children who participated in the RCT were aged 7–13 years at the time of the trial, had a body mass index >85th centile for their age and gender (according to the Centre for Disease Control classification of overweight/obesity in children37), were enrolled in and met the criteria to participate in the community weight management programme at one of the sites that participated in the study. Families self-referred via a toll-free phone number, text message or online registration to the programme, and secondary referrals were accepted from health professionals, organisations and community members. Eligibility was assessed at the time of referral or contact with local health districts and based on anthropometric measures and a medical questionnaire completed by a parent/carer, who also provided written consent for their child to participate in the research.
A maximum variation sampling method based on the community weight management programme participant demographics and programme group characteristics will be used in the evaluation. In the sample of parents/carers and children, we will aim to include high and low attendance (</>50%) in the weight management programme, single child and multiple sibling families, participants from smaller programme groups (less than six children) and larger programme groups (more than eight children). A variation in ethnicity, culturally and linguistically diverse (CALD) and socioeconomic status (SES) characteristics will also be included.
Second, participants in the evaluation shall be stakeholders who were involved in the delivery of the RCT, including public servants, policymakers, behavioural specialists, programme managers for local health districts and community health professionals. These shall be selected, beginning with core stakeholders who were directly involved in the design and implementation of the RCT, and informed by the research data collection as it progresses.
The logic model (figure 1) shows the aspects most critical to the success of the RCT, to guide this evaluation. The logic model shows the intended inputs and activities involved in implementing the incentives-based intervention as well as the intended outcomes and broader impacts.
Levels of evaluation and how they relate to the logic model
To address each of the three aims, three levels of qualitative evaluation will be conducted: (1) process, (2) impact and (3) intervention deconstruction. The evaluation plan is summarised in table 1.
Aim 1: process evaluation
The process evaluation will determine the degree the incentive-based intervention was implemented as intended (implementation fidelity). The evaluation will document factors influencing how the different components of the intervention were delivered and received from the perspectives of those delivering and receiving the intervention, and compare this with intended implementation by the trial Stakeholder Working Group (see Acknowledgements).
The framework used to guide the design of the process evaluation was the Normalisation Process Theory.38 Normalisation Process Theory provides a conceptual framework for understanding the processes by which interventions are implemented and integrated into everyday practice. The model explores factors involved in the dynamics between people delivering and receiving an intervention in order to account for outcomes of the implementation process, and differences between expected and observed outcomes in real settings. In implementing an intervention in a natural setting, Normalisation Process Theory argues that people need to continuously make sense of the work they are doing (coherence), engage with it (cognitive participation), enact it (collective action) and reflect on it (reflective monitoring). With an emphasis on exploring these concepts, the process evaluation will look at three evaluation components. These are: (1) content delivery, (2) content engagement and (3) reach, and are presented in table 1. Content delivery refers to the delivery and receipt of each individual input component of the intervention from the perspective of those delivering the intervention. Content engagement refers to the delivery and receipt of each component of the intervention from the perspective of those receiving the intervention. Reach refers to the proportion of the intended target audience who participated in the intervention, and the process evaluation will specifically explore reasons expressed for discontinuing the programme.
Information collected from stakeholders (those implementing the intervention) will include perceptions on the roles and activities involved in the implementation of the intervention, from initial discussions to intervention design, set-up, project management and facilitation. Perceptions on the acceptability, including integration and disruption to the standard community programme, benefits and impacts of the intervention and suggestions for improvement, will also be collected.
Information collected from participants (parents/carers and children receiving the intervention) will include perceptions on how specific components of the intervention were understood and engaged with, and recounts of their experiences of interacting with the intervention materials, other families and facilitators. Perceived benefits, strengths/weaknesses and suggested improvements will also be collected.
Aim 2: impact evaluation
The impact evaluation will determine the contextual factors that influenced trial outcomes and broader impacts, and identify for whom the incentive scheme was most and least effective in terms of having an effect during the community weight management programme and sustained impacts after the programme. Investigations will include the environmental and social context associated with participation in the intervention, individual attitudes and beliefs about healthy living, and associated barriers and enablers to adapting to new behaviours and forming new habits.
The Realistic Evaluation model39 was used to inform the design of the impact evaluation, and table 1 outlines the three evaluation components the investigation will focus on. These are: the broader environment, participant characteristics and programme factors. The broader environment refers to other health activity, particularly other interventions and healthy eating programmes the family was involved in at the time of the intervention, as well as barriers and enablers to participation and engagement in the community weight management programme associated with the home environment and the families' lifestyle. Participant characteristics were identified at baseline in the trial and include: age, gender, single or separated family status and single or multiple sibling family. This information will be used to recruit participants for focus groups and family interviews (with the aim of including a mixture of characteristics) and will also be factored into analyses. Programme factors to be explored include: size of the community weight management programme group, regional or metro site location, venue type and day the programme sessions were held (weekday or a weekend day), and will also be informed by the process evaluation.
Information collected will draw from those receiving the intervention (children and their parents/carers), and include lifestyles, attitudes and health behaviours, perceptions on what has changed in their lives since the programme, habits retained and behaviours that have proved difficult to integrate into their lifestyles. Specific impacts of the individual intervention components will be explored in depth.
All intended primary and secondary outcomes of the trial will be considered in the investigation of contextual factors. The primary measure in the RCT was body mass index at the end of the community weight management programme (10 weeks) and at 6 months. Secondary outcomes included: waist circumference; nutrition and physical activity behaviours and self-esteem (by survey) at the end of programme (10 weeks) and at 6 months. Attendance rates during the programme (completion of >75% of sessions) were also measured in the trial. The evaluation will also compare perceptions of the incentive scheme with the outcomes in the absence of the intervention (control group). This will help determine how the intervention influenced outcomes, and whether the effects were intended.
Aim 3. Intervention deconstruction
The intervention deconstruction will explore and describe which specific components of the incentive-based intervention were most impactful and why. The incentive intervention was developed iteratively in 2014 drawing from a review of the relevant literature, workshops, focus groups and field visits. Combined literature review and advice from community programme leaders highlighted the importance of incentivising goals and activities (eg, having healthier lunches) rather than outcomes (eg, weight loss) for greater impact on behaviour change.11 ,40 For example, in a series of RCTs across 203 elementary schools, it was reported that providing incentives for reading books was more effective than those for outcomes such as scores on a test.40 Community programme leaders indicated during field research that while goal setting as part of the programme was usually specific, measureable, achievable, relevant and timely (SMART), it was likely to be of value to enhance the goal setting process, including resetting/stretching goals if they are achieved too easily, and linking goal achievement to incentives. The importance of choosing the appropriate size, type and timing of incentives was also highlighted as critical for motivating change. For example, a study in children26 found that a larger financial reward (a US quarter rather than a nickel) combined with receiving it on the same day as the behaviour was performed produced the largest behaviour change. The key behavioural concepts that informed the design of the individual intervention components (inputs in the logic model) are summarised in table 2.
The Behaviour Change Technique Taxonomy (BCTTv1)41 will be used as a framework for deconstructing the behaviour change intervention components used in the trial. The purpose of this is threefold: (1) to optimise the robustness and replication potential of the research by using a common language to recognise and specify the components of the intervention that are changing behaviour (use of the taxonomy supports the CONSORT guidelines for the reporting of behaviour change interventions,42 (2) to facilitate translation of the interventions for the greatest effect on child obesity outcomes by separating the active ingredients within the intervention components and the conditions under which they are effective, from components that may be less necessary and (3) to understand more about any intervention components that may be useful but were not sufficient in dose to produce an impact on outcomes.34 It will also be useful to refer to the full list of 93 items in the BCTT41 in the analysis of how and why specific components worked or did not work. The behaviour change technique mappings, provided in table 2, have been mapped in retrospect as a starting point to guide analysis. For this exercise, careful attention was paid to the coding definitions specified in the BCTT,41 and the ‘BCT taxonomy’ app was downloaded through Apple to guide coding.
The evaluation will use a qualitative analysis, which will explore multiple groups of people's perspectives on the benefits, acceptability and other ways in which incentives could be used for behaviour change. Data collection will include a combination of family and stakeholder interviews (see online supplementary 1: Discussion guide_family interview and see online supplementary 2: Discussion guide_stakeholders), focus groups (see online supplementary 3: Discussion guide_parent focus-group) and a survey (see online supplementary 4: Survey_parents). Table 3 provides details of the data sources and their aims.
The focus groups and interviews will follow a semistructured approach, and facilitation will be supported by bullet-pointed topic guides to facilitate natural conversation between moderator and participants by emphasising topics, objectives and flow. Each focus group and interview will be recorded and transcribed verbatim. Participants will be made aware of this at the start of the group or interview and their consent obtained. All discussions will be confidential. GE will conduct the focus groups and interviews in a private room within a facility near to the participants' residence (such as a community centre) or place of work. A second researcher will also be present in the room in an observatory capacity to take notes during the group discussion. Refreshments and parking reimbursement will be provided for focus group participants. To optimise attendance a $40 (eg, a supermarket voucher) will be offered to each family who participates in an interview.
Data analysis and synthesis
Data analysis and synthesis will be based on the Grounded Theory approach.49–51 Qualitative data from focus groups, family and stakeholder interviews, and the survey will be transcribed and systematically coded, drawing out the key points. Similar codes will be grouped into concepts and categories based on emergent themes. The coding process will begin with the first interview and be repeated for each subsequent interview or focus group with the use of field notes, memos and constant comparison to accumulate ideas about how concepts relate to each other. For each transcript, examples will be extracted that either confirm or contradict the emerging themes. Once thematic saturation occurs, no further interviews and focus groups will be necessary. Two researchers will conduct the analysis, and if at any stage consensus cannot be reached, a third researcher will review those aspects. NVivo software will be used to assist with qualitative data management. The overall output will be a set of probability statements about the relationship between concepts, or a set of conceptual hypotheses50 about the barriers and enablers to implementation of the incentives scheme, and its impact on the behaviour of participants.
The analysis will be conducted in three waves, corresponding to the three levels of evaluation in this research—the process, impact and behaviour change component evaluations. Specifically, for the process evaluation, the coding process will refer back to the individual input and activity components in the logic model. Each focus group and interview transcript will be scanned for themes relating to the delivery and receipt of each component of the incentives scheme. Project management materials will also be reviewed, and examples will be identified to support or contradict themes emerging from the transcribed data. For the impact evaluation, the coding process will be related back to the intended outcomes and impacts of the intervention, as per the logic model. Each focus group and interview transcript will be scanned for themes associated with broader environmental influences on the effects of the incentive scheme, and in relation to different participant characteristics and programme factors. The output will include typologies to describe for whom and how incentives may impact behaviour. The intervention deconstruction will be guided by the BCTT41 as a basis for describing the active intervention components and exploring whether the perceived impacts of the intervention support or contradict the behavioural concepts underlying the design of intervention. The specific mechanisms of the incentives-based behaviour change scheme that lead to behaviour change may be associated with behavioural concepts outside of those identified in the literature and influenced by unanticipated factors, which the researchers will explore throughout the analysis.
The findings will be triangulated, drawing on the variety of data sources and perspectives, and with comparison to the trial control group to determine whether the trial outcomes can be attributed to the intervention. Issues and potential biases in the design of the trial will be carefully factored into interpretations.
Timing of data collection
The timing of the data collection is given in table 1. Data will be collected at two points: (1) during the 6-month follow-up assessments in the RCT (which is complete) and (2) 1–2 months after the 6-month assessments. Conducting the fieldwork close to the follow-up assessments will minimise corruption of the RCT as the evaluation data collection will prompt people to remember their experiences during a prescheduled intervention point. Focus groups and stakeholder and family interviews are estimated to start in August 2016.
This study details the protocol for a qualitative evaluation to be conducted shortly following a RCT aimed at increasing behaviour change in overweight and obese children participating in a community-based obesity programme. The research aims to address the challenges of public health interventions and provide information about the barriers and enablers to the implementation of an incentives-based intervention and its impact on health-related behaviour change in children. This research will determine the relationships between intervention delivery, contextual factors influencing outcomes and the mechanisms of behaviour change, and provide detailed information on the acceptability of the incentives-based scheme and its potential as an obesity-reducing strategy in children in a range of contexts. It has potential to add a considerable value to the interpretation of the quantitative-based RCT outcomes and to inform future implementation and translation as a behaviour change strategy for managing obesity in overweight children, should the interventions be deemed acceptable. The knowledge will also advance the development of further strategies for health-related behaviour change in children, supporting positive change in tackling the growing global problem of obesity.
This evaluation would not have been possible without the contributions of the RCT investigator team and Working Group. Investigators and Working Group members who are not coauthors on this paper, and their affiliates are listed below: Office of Preventive Health: Anita Cowlishaw; Santosh Kanal; Nicholas Petrunoff. Behavioural Insights Unit, Department of Premier and Cabinet: Shirley Dang. Western Sydney LHD: Christine Newman; Michelle Nolan; Deborah Benson, Kirsti Cunningham. South Western Sydney LHD: Mandy Williams; Leah Choi; Kate Jesus; Stephanie Baker. South Eastern Sydney LHD: Myna Hua; Linda Trotter; Lisa Franco. North Sydney LHD: Paul Klarenaar; Jonothan Noyes; Sakara Branson. Hunter New England LHD: Karen Gillham; Dr John Wiggers; Silvia Ruano-McLerie. Mid North Coast LHD: Ros Tockley; Margo Johnson. Better Health Company: Madeline Freeman; Bec Thorp. The George Institute for Global Health: Sarah Eriksson; Caroline Wu. In addition, the authors thank the Go4Fun program leaders and representatives from our funding partner organisations, including the Heart Foundation, who have contributed to the development and implementation of the RCT.
Contributors GE led the drafting of all sections of the article in consultation with all the coauthors. JR/GE led the application for funding for this work. All authors provided substantial contribution to the concept and design of the evaluation, drafted the protocol paper and reviewed critically for important intellectual content and final approval of the version for publication.
Funding This research is funded in-kind provided by the George Institute for Global Health and the Department for Premier and Cabinet. GE is funded by a PhD scholarship through the George Institute for Global Health within a National Health and Medical Research Council (NHMRC) program grant ID1052555. JR is funded by a Career Development and Future Leader Fellowship co-funded by NHMRC and the National Heart Foundation. AR is funded by an NHMRC Principal Research Fellowship APP1124780. JR and AR are investigators on NHMRC program grant ID1052555.
Competing interests None declared.
Patient consent Obtained.
Ethics approval South West Sydney Human Ethics Committee.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement The findings of this study will be disseminated via the usual scientific forums including peer-reviewed publications and presentations at international conferences. The study will be administered by the George Institute for Global Health, with the design and conduct overseen by a Steering Committee. Unpublished data from the study such as anonymised transcripts and coded survey data set may be requested from the corresponding author at firstname.lastname@example.org. Consent will be obtained from study participants for transcripts prior to dissemination.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.