Parenting programme for parents of children at risk of developing conduct disorder: cost effectiveness analysis
BMJ 2007; 334 doi: https://doi.org/10.1136/bmj.39126.699421.55 (Published 29 March 2007) Cite this as: BMJ 2007;334:682
- Rhiannon T Edwards, senior research fellow in health economics1,
- Alan Céilleachair, research officer in health economics1,
- Tracey Bywater, project trial coordinator2,
- Dyfrig A Hughes, senior research fellow in pharmacoeconomics1,
- Judy Hutchings, research director2
- 1Centre for the Economics and Policy in Health, Institute of Medical and Social Care Research (IMSCaR), University of Wales Bangor, Bangor LL57 1UT
- 2School of Psychology, University of Wales Bangor
- Correspondence to: R T Edwards r.t.edwards{at}bangor.ac.uk
- Accepted 23 January 2007
Abstract
Objective To investigate the cost effectiveness of a parenting programme.
Design An incremental cost effectiveness analysis alongside a pragmatic randomised controlled trial of the effectiveness of a group parenting programme delivered through Sure Start in the community.
Setting Sure Start areas in north and mid Wales.
Participants Parents of 116 children aged 36-59 months (87% of the clinical sample) at risk of developing conduct disorders defined by scoring over the clinical cut off on the Eyberg child behaviour inventory). Children were identified by health visitors and recruited by the research team.
Intervention The Webster-Stratton Incredible Years basic parenting programme or a six month waiting list control.
Main outcome measure Incremental cost per unit of improvement on the intensity score of the Eyberg child behaviour inventory.
Results The bootstrapped incremental cost effectiveness ratio point estimate was £73 (€109, $142) per one point improvement on the intensity score (95% confidence interval £42 to £140). It would cost £5486 (€8190, $10 666) to bring the child with the highest intensity score to below the clinical cut-off point and £1344 (€2006, $2618) to bring the average child in the intervention group within the non-clinical limits on the intensity score (below 127). For a ceiling ratio of £100 (€149, $194) per point increase in intensity score, there is an 83.9% chance of the intervention being cost effective. The mean cost per child attending the parenting group was £1934 (€2887, $3760) for eight children and £1289 (€1924, $2506) for 12 children, including initial costs and materials for training group leaders. When we categorised the sample into relatively mild, moderate, and severe behaviour groups based on intensity scores at baseline the intervention seemed more cost effective in those with the highest risk of developing conduct disorder.
Conclusion This parenting programme improves child behaviour as measured by the intensity score of the Eyberg child behaviour inventory at a relatively low cost and was cost effective compared with the waiting list control. This parenting programme involves modest costs and demonstrates strong clinical effect, suggesting it would represent good value for money for public spending.
Introduction
Conduct disorder1 is estimated to affect 5-10% of children aged 5-15 years in the United Kingdom and the United States.2 3 For those children with early onset in preschool years, conduct disorder often persists into adulthood4 and predicts poor employment prospects, marriage breakdown, and self harming or antisocial criminal behaviour.5 6 7
The economic implications of severe behavioural problems in childhood are serious. The costs of publicly resourced services for those aged 28 with conduct disorder in childhood were estimated to be 10 times higher (£70 019; €104 416, $137 450) than for those with no behavioural problems (£7423; €11 069, $14 571).8
Parenting is a key determinant in child behaviour. Parents who encourage prosocial behaviour have children with fewer behaviour problems.9 Parenting training programmes are effective in helping families with children at risk of developing conduct disorders.10 The Incredible Years basic parenting programme is described in more detail elsewhere10 and in the accompanying paper.11 The programme strengthens parenting competencies and reduces the risk of developing conduct problems. It is delivered by two trained leaders in 12 weekly sessions.
The recent National Institute for Health and Clinical Excellence (NICE) appraisal on parent training programmes for conduct disorders highlighted the dearth of evidence on cost effectiveness.12 In a review of 1600 papers, Romeo et al13 found only two that qualified as true economic evaluations of treatments for behavioural disorders.14 15 Muntz et al found that an intervention with parents of children with conduct disorder could be cost effective under conditions of resource trade off.16 We carried out a full cost effectiveness analysis, alongside a pragmatic randomised controlled trial, of the Incredible Years basic parenting programme in the UK. We present the programme costs and consequences for public sector resources and child behaviour outcomes.
Methods
Study population
This economic evaluation took place alongside a pragmatic randomised controlled trial.11 Health visitors in 11 Sure Start areas administered the Eyberg child behaviour inventory17 to parents of children aged 3 and 4 years. The parents who reported their child to be above the clinical cut off on either the intensity or problem scale (127 and 11, respectively) were invited to participate. Of these, 153 consented and were randomised to the intervention group or a six month waiting list control. We chose this comparator because families would receive usual care and then have access to the parenting programme. Of the 153 families, 86 in the intervention group and 47 in the control group were available for follow-up. Researchers collected data on service use and measures of effectiveness during home visits at baseline and six month follow-up. The sample for the economic evaluation is smaller than for the clinical trial as we had incomplete data on costs and service for 13 participants in the intervention group and four in the control group. We had full economic data for 73 families in the intervention group and 43 in the control group. A t test showed no significant differences in Eyberg intensity scores between the 116 on whom the economic analysis was based and the 17 families missing from the economic sample because of incomplete or missing data (t=−0.79, P=0.43).
Measurement of costs
We examined costs from a multiagency public sector perspective, including health and special educational and social services.18 Effectiveness was measured by reductions in the intensity and problem scores of the Eyberg intensity scores, a commonly used outcome measure in clinical trials of child behaviour interventions.19
Group leaders from four of the 11 groups completed weekly cost diaries to enable us to measure group running costs (including training and weekly supervision for group leaders). We used a client service receipt inventory20 administered to parents at baseline and follow-up to collect data on children's use of health, social, and special educational services. The inventory asked about their child's use of services in the preceding six months. National costs were applied to these services, drawn from a number of published sources including Unit Costs of Health and Social Care 2004,21 NHS reference costs for 2003-4,22 and local NHS trusts and councils. All costs are in 2003-4 pounds sterling unless stated otherwise. As all costs fell within a one year time horizon we did not discount costs or effects.
Analysis strategy
We calculated an incremental cost effectiveness ratio point estimate with a 1000 replication bootstrap to provide a confidence interval.23 We used cost effectiveness analysis because the clinical research trial used a condition specific outcome rather than a utility measure. We considered it inappropriate to try to measure utility in these young children to facilitate a cost utility analysis. A cost effectiveness acceptability curve24 provided probabilistic analysis for a range of cost effectiveness ceilings for policy makers.
Sensitivity analysis with the training costs of group leaders removed produced an estimate of the incremental cost effectiveness ratio for rollout of the programme. A further subgroup sensitivity analysis, with children categorised into risk groups for conduct disorder (relatively mild, moderate, and severe), tested whether cost effectiveness varied with intensity of the risk at baseline. We collated and analysed the cost and effectiveness data using SPSS v12.0 and computed the incremental cost effectiveness analysis, 1000 replication bootstrap, and cost effectiveness acceptability curves using EXCEL XP Pro.
Results
Characteristics of the study families
Table 1⇓ summarises the characteristics of participating families at baseline. Independent samples t tests found no significant differences between the intervention and waiting list control on any demographic variable tested at baseline.
Demographic characteristics of the families. Figures are numbers (percentages) of families unless stated otherwise
Clinical effectiveness
The clinical effectiveness findings are based on the sample used for the economic analysis (n=116). At baseline, the mean intensity and problem scores were 144.46 (SD 24.18) and 16.18 (SD 6.88) in the intervention group and 140.74 (SD 24.47) and 14.70 (SD 7.45) in the control group (table 2).⇓ An independent t test showed no significant difference in mean intensity or problem scores between conditions at baseline (t=1.20, P=0.23). At the six month follow-up, mean intensity had fallen to 117.17 (SD=35.99) in the intervention group when we controlled for regression to the mean.25 Likewise, the mean problem score had fallen to 12.09 (SD 9.83). This represents a significant reduction in problem behaviour. Behaviour remained constant or deteriorated among children in the control group with the mean intensity score unchanged at 140.74 (SD 40.77). The mean problem score fell to 12.95 (SD 10.87) (table 2)⇓. After we determined that the data met the assumption of equivalent slopes (the decision variable was a test/non-test dummy variable), an analysis of covariance revealed that the intervention delivered a 27.29 point reduction in mean intensity score (95% confidence interval 16.41 to 36.42) when we controlled for any differences in scores before treatment between the groups.25 26 At follow-up 45 (62%) children in the intervention group and 18 (42%) in the control group had Eyberg intensity scores below the clinical cut off (127).
Eyberg child behaviour inventory* at baseline and six month follow-up
Group costs
Table 3⇓ summarises the reported running costs based on weekly cost diaries completed by leaders of four groups and the cost information supplied from the Incredible Years Welsh office. Costs shown are divided into non-recurrent initial training and group setup costs and recurrent costs. When possible, we have shown means and standard deviations, reflecting the costs reported by group leader diaries. Employment overheads of 25% were added to reported salary costs to reflect national insurance and superannuation. A further 25% was added to reflect capital and managerial overheads. The time spent recruiting families in this study, as recorded in diaries by group leaders, may understate the time necessary in other settings where this parenting programme has not previously been available. Supervision costs stated here, however, may be higher than would normally be expected as supervision was provided in Bangor as a condition of participating in the research. In some cases this necessitated six hours' travel each week. The mean cost per index child (based on eight families per group, as in our trial), including non-recurrent and recurrent costs, was £1933.56. We used this figure as the cost of the intervention per child in calculating the point estimate for the incremental cost effectiveness ratio. Table 3 also shows the reduction in average cost per index child with 12 families in the group.⇓
Total costs and cost per child of running parenting group over 12 session programme
Service utilisation by children
Table 4 summarises the results of the client service receipt inventory for children in the two groups for the six months between baseline and follow-up⇓. Service use costs at baseline differed between the groups but this was not significant and remained unchanged at follow-up (see www.bangor.ac.uk/imscar/healtheconomics/reportspublications.php.en?menu-3&catid=2389&subid=0 for further details). Published national costs routinely include capital, staff time, training, and travel.21 Costs obtained from local sources were total unit costs, and providers were unable to supply a breakdown of these costs into fixed and variable costs.
Health, social care, and special education services used by children. Figures are mean total cost per child (£)
Cost effectiveness
Using the cost and clinical data gathered for 73 participants in the intervention group and 43 in the control group, we calculated a point estimate for the incremental cost effectiveness ratio of £71 per 1 point change in the Eyberg intensity score (£1992.29−£49.14/27.29=£71.20), where £1992.29 is the change in service use costs (including costs of the parenting programme, £1933.56, tables 3 and 4) for the intervention group, £49.14 is the change in service use costs for the control group, and 27.29 (27.29−0) is the incremental change in the Eyberg child behaviour index (table 4).⇑
Bootstrapping and cost effectiveness acceptability curve
The 1000 replication bootstrap yielded a bootstrapped mean estimate of £73 (95% confidence interval £42 to £140), close to our basic calculation figure of £71. A one unit improvement in the Eyberg intensity scores over and above that provided by the waiting list control would cost £73.
Figure 1⇓ shows the cost effectiveness plane for the 1000 replication bootstrap. The density of the intervention can be seen to fall within the north east quadrant of the cost effectiveness plane. Figure 2⇓ gives the cost effectiveness acceptability curve for a range of cost ceilings. The curve intercepts the y axis at 0 and asymptotes at 1.
Fig 1 1000 replication bootstrap cost effectiveness plane
The points on the cost effectiveness plane fall in the north east quadrant, signifying that the programme is more costly and more effective than the control. The parenting intervention, therefore, does not dominate the waiting list control condition (that is, is clearly more effective at a lower cost). From a policy perspective, the cost effectiveness acceptability curve shows that, for example, if a cost ceiling of £100 was set, then the intervention would have an 83.9% probability of being cost effective. The point estimate of £73 per point improvement on the Eyberg intensity scores is a relatively low financial figure.
Sensitivity analysis
Considerable training and supervision costs are needed to establish capacity to deliver this parenting programme effectively. Because economists aim to focus on marginal costs we performed a sensitivity analysis to recalculate the incremental cost effectiveness ratio excluding initial non-recurrent costs of training and materials. This would be the appropriate ratio once group leaders had been trained to deliver the programme. In this analysis, the ratio decreased from £73 to £60 (95% confidence interval £32 to £119) per one point improvement on the Eyberg intensity score. We categorised the groups into three based on thirds of intensity score (relatively mild (24 in intervention group, 14 in control group), moderate (25 and 15), and severe (24 and 14)) to explore how the cost effectiveness of the intervention varied with severity, as measured by baseline intensity scores. The incremental cost effectiveness ratio point estimates for the high intensity group were much lower than for the low and medium groups, estimates being £92 in the mild group and £37 in the severe group. The samples for this subgroup analysis are small and provide only an indication of where resources might usefully be targeted, though at baseline the behaviour of all these children was of clinical concern.
Discussion
The Incredible Years basic parenting programme delivered a point estimate for the incremental cost effectiveness ratio of £73 per one point improvement on the Eyberg intensity scale compared with parents and children who remained on the waiting list. The cost effectiveness acceptability curve showed that for a cost ceiling of £100 the intervention was 83.9% likely to be cost effective. A sensitivity analysis of the sample subdivided by children's intensity score showed that the intervention became more cost effective in children at greater risk of developing conduct disorder. Mean intensity for the intervention group at baseline was 144.46. This means that a 17.46 point fall would be required to return the average child in this intervention to below the clinical cut-off point at a cost of just over £1344, while it would cost £5486 to bring the child with the highest intensity score in our study to below the clinical cut-off point. The clinical study also found important benefits to the parents' mental health and the behaviour of siblings that are not reflected in the incremental cost-effectiveness ratio.11 Policy makers and commissioners within the NHS and local government must balance the modest extra costs involved in providing this parenting programme against the clinically significant amelioration in intensity scores in deciding whether or not to introduce or roll out the intervention.
There are few analyses of cost effectiveness of parenting interventions. Thompson et al also used a six month waiting list control in their evaluation of the common sense parenting intervention.14 Their study was not a true economic evaluation, however, and made only basic cost calculations in terms of staff time. The short follow-up period of both our study and the work of Thompson et al14 is a characteristic of research in this topic.27 Hutchings et al19 and Muntz et al16 conducted a four year follow-up of a similar parenting programme and found sustained benefits.
The cost effectiveness of parenting programmes continues to be under-researched,13 with wide ranging implications.8 While there is some evidence of the short term effectiveness and cost effectiveness of parenting programmes, little is known about their longer term effectiveness. Future research should look to increase both the number of follow-ups and the length of follow-up.
Our findings are timely given the government's “Respect” agenda. The UK government has prioritised support for young children and their families in high risk communities via the Sure Start programme but until recently has failed to specify the use of evidence based programmes, resulting in failure to achieve effective outcomes.
The recent National Institute for Health and Clinical Excellence (NICE) appraisal on conduct disorder in children states that programmes should be clinically effective and cost effective.12 Delivery of evidence based programmes in the way that they were originally researched and developed is vital to ensure that the effectiveness (and indeed the cost effectiveness) is replicated in service settings.10 We have shown that if the Incredible Years parenting intervention with proved effectiveness is delivered with fidelity, it can provide value for money in terms of public resources within the Sure Start framework.
What is already known on this topic
The costs of conduct disorder in childhood and associated costs in later life are potentially high to the family, the NHS, social care services, and the legal justice system
Parenting is now at the top of government policy agendas
The National Institute for Health and Clinical Excellence has argued that investment in parenting programmes should be based on evidence of effectiveness and cost effectiveness
What this study adds
Compared with children remaining on a waiting list, the Incredible Years basic parenting programme improved children's behaviour at a cost of £1344 (€2005, $2638) per child
Footnotes
We thank the extended Incredible Years Wales team, particularly Pam Dewis, Karen Jones, Catrin Eames, Dilys Williams, Pam Martin, Bridget Large, Nicola Armstrong, Claire Davis, Cara Rogowski, and Michelle O'Brien. We also thank all participating families, Sure Start staff, and group leaders; Ian Russell for his efforts in guiding the authors with respect to the submission process; and Daphne Russell and Chris Whitaker for their statistical support.
Contributors: RTE was principal investigator on the economic evaluation and contributed to design, implementation, management of the economic evaluation, analysis and interpretation of data, and writing the paper. AÓC was the lead health economics researcher and was responsible for designing and implementing the data management plan and entering, verifying, analysing, and validating data. He contributed to implementing the trial design, analysing and interpreting data, and writing the paper. TB managed the clinical project, conducted randomisation, collected and coded data, and redrafted the paper. DAH confirmed the final findings and assisted in the presentation and wording of the final draft. JH was primary investigator on the main clinical trial funded by the Health Foundation. She was a co-applicant on the bolt-on economic evaluation grant proposal and helped to redraft the paper. RTE is guarantor.
Funding: Research grant from the Health Foundation, grant No 1583/2594.
Competing interests: JH is paid by Incredible Years for running occasional training courses in the delivery of the parent programme and has served as an expert witness for the NICE appraisal on parenting and conduct disorder.
Ethical approval: North West Wales research ethics committee.