Article Text
PDF
Abstract
Objectives Computer-administered cognitive–behavioural therapy (CCBT) may be a promising treatment for adolescents with depression, particularly due to its increased availability and accessibility. The feasibility of delivering a randomised controlled trial (RCT) comparing a CCBT program (Stressbusters) with an attention control (self-help websites) for adolescent depression was evaluated.
Design Single centre RCT feasibility study.
Setting The trial was run within community and clinical settings in York, UK.
Participants Adolescents (aged 12–18) with low mood/depression were assessed for eligibility, 91 of whom met the inclusion criteria and were consented and randomised to Stressbusters (n=45) or websites (n=46) using remote computerised single allocation. Those with comorbid physical illness were included but those with psychosis, active suicidality or postnatal depression were not.
Interventions An eight-session CCBT program (Stressbusters) designed for use with adolescents with low mood/depression was compared with an attention control (accessing low mood self-help websites).
Primary and secondary outcome measures Participants completed mood and quality of life measures and a service Use Questionnaire throughout completion of the trial and 4 months post intervention. Measures included the Beck Depression Inventory (BDI) (primary outcome measure), Mood and Feelings Questionnaire (MFQ), Spence Children's Anxiety Scale (SCAS), the EuroQol five dimensions questionnaire (youth) (EQ-5D-Y) and Health Utility Index Mark 2 (HUI-2). Changes in self-reported measures and completion rates were assessed by treatment group.
Results From baseline to 4 months post intervention, BDI scores and MFQ scores decreased for the Stressbusters group but increased in the website group. Quality of life, as measured by the EQ-5D-Y, increased for both groups while costs at 4 months were similar to baseline. Good feasibility outcomes were found, suggesting the trial process to be feasible and acceptable for adolescents with depression.
Conclusions With modifications, a fully powered RCT is achievable to investigate a promising treatment for adolescent depression in a climate where child mental health service resources are limited.
Trial registration number ISRCTN31219579.
- Randomised Controlled Trial
- computerised Cognitive Behaviour Therapy
- Depression
- Adolescents
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
The study explores the feasibility of using a range of mental health and health economics measures in a population of adolescents with low mood.
The study produces important feasibility information to inform a fully powered randomised controlled trial.
Feasibility data on acceptability, completion rates and attrition are examined.
The sample of participants is obtained from only one National Health Service Trust resulting in potential bias.
Introduction
Rates of adolescent depression appear to be rising1 ,2 with the 1-year prevalence suggested to be between 2–4%.3 ,4 Early treatment is important because adolescent depression has high levels of future morbidity including further emotional disorders, suicidality, physical health problems, substance misuse and problems in social functioning.4 ,5
Research examining antidepressant medication for treating adolescents with depression has been mixed. A meta-analysis6 including two randomised controlled trials (RCTs) comparing antidepressants and cognitive–behavioural therapy (CBT)7 ,8 found no statistically significant differences between antidepressants and CBT for the majority of outcomes, including no differences between the groups on self-rated depression symptoms post intervention or at 6–9 months follow-up. Significantly, fewer participants had suicidal ideation in the CBT group than the antidepressant group at post intervention and at 6–9 month follow-up. Although antidepressants clearly have an important place in a stepped-care approach, the National Institute for Health and Care Excellence (NICE) Guidelines (2015)9 state that they should not be used for the initial treatment of children and adolescents with mild depression. Thus, provision of talking therapies (ie, CBT) is an important area for research and practice.
Reviews of CBT for adolescent depression show that it is effective and currently one of the main treatment options recommended for this group.10 ,11 Trials comparing CBT to family therapy or supportive psychotherapy suggests it is better at improving mood and achieving remission.12 It has also been shown to prevent depression in high-risk adolescent groups, including the offspring of depressed adults.13 With a low side-effect profile, CBT is an attractive option for treating adolescent depression. Despite this and recent government focus on improving access to psychological therapies, reduced staff numbers in Child and Adolescent Mental Health Services (CAMHS) make CBT delivery challenging, especially in a context where local authority and National Health Service (NHS) child services funding is tight.
Despite the evidence base for the effectiveness of CBT for adolescent depression,10 ,14 ,15 adolescents often avoid face-to-face therapy because of stigma.16 Given their affinity with computers, treatment accessibility may be improved by the availability of computer-administered CBT (CCBT) which could be provided at an earlier stage of illness,17 particularly for those with a high degree of familiarity with technology.18 CCBT represents an autonomous form of therapy delivery with the potential to provide a realistic alternative, or potentially preventative, intervention.
A systematic review19 found six studies examining CCBT for adolescent depression. Only three were RCTs and none were conducted in the UK. Although the countries where this research occurred (USA,20 ,21 Australia22 ,23 and Holland24) may be culturally similar to the UK, differences on a systemic level (eg, mental healthcare provision) as well as on a societal level (eg, language use, norms) mean their results cannot be generalised to a UK population. To date, although current literature suggests potential benefits of CCBT for adolescents with low mood/depression,25 more RCTs need to be conducted with depressed adolescents in the UK.
The UK-developed CCBT package Stressbusters showed positive results in a case series17 where 95% of a UK adolescent sample met diagnostic criteria for depression at baseline, falling to 22% post treatment. This, alongside good completion rates (70% completing all eight sessions), suggests that Stressbusters is a potentially effective CCBT package warranting further investigation within an RCT.
More recently, an RCT examining the effectiveness of Stressbusters within a school setting was conducted.26 The study showed a significant reduction in adolescent depression and anxiety compared with a waiting list control. However, more research is required to examine the effectiveness of CCBT in comparison to an attention control to ensure any effects observed are a result of the CCBT intervention.
We aimed to assess the feasibility of delivering an RCT comparing Stressbusters (CCBT) with an attention control (accessing low mood self-help websites) for adolescents with low mood/depression to establish:
the feasibility of recruiting adolescents with low mood/depression;
the feasibility of delivering CCBT as a treatment for adolescents including retention rates, compliance with the CCBT program, completion rates for outcome measures and withdrawal from intervention/trial rates;
the parameters, outcomes and cost-effectiveness of the study to inform a full-scale RCT.
Methods
Participants and recruitment
Our target population was 12–18-year olds with low mood/depression living within the areas covered by a CAMH service in a Northern City in England.
Trial referrals were made by nine Primary Mental Health Workers (PMHWs) who screened adolescents referred to them with low mood/depression. Young people gave fully informed consent (with that of their parent/guardian if under 16). Eligibility was defined by a score of ≥20 on the Mood and Feelings Questionnaire (MFQ)27 (validation research28 proposes a score of ≥20 indicates any depressive disorder and ≥29 a likely current major depressive episode). Those with comorbid physical illness were included but those with psychosis, active suicidality or postnatal depression were not. Those with severe depression were referred to the local CAMHS team for assessment and treatment in the first instance, clinicians then decided whether trial participation would be suitable for them.
Study design and methodology
The study was conducted between June 2011 and December 2014. Consented participants attended a baseline visit with the researchers where they completed the following measures:
Short Beck Depression Inventory (BDI) (primary outcome measure)
A 13-item self-reported measure used to assess depression severity among adolescents by measuring cognitive, behavioural, affective and somatic dimensions of depression.29
Mood and Feelings Questionnaire (MFQ)
A 33-item questionnaire, based on Diagnostic and Statistical Manual of Mental Disorders (DSM-III-R) criteria for depression, comprising descriptive phrases regarding how an individual has been feeling or acting in the preceding 2 weeks.27 A Cronbach's α of 0.95 for the MFQ has been reported28 suggesting high internal consistency.
Spence Children's Anxiety Scale (SCAS)
A 45-item, self-report measure used to assess the severity of anxiety within six subgroups (generalised anxiety, panic/agoraphobia, social phobia, separation anxiety, obsessive–compulsive disorder and physical injury fears) alongside providing an overall anxiety score.30 An analysis of the internal consistency of the SCAS31 produced a coefficient α of 0.92 and a Guttman split half reliability of 0.90.
Quality of life (QoL), including the EQ-5D-Y and HUI-2, and service use questionnaire
Self-report questionnaires were used to obtain information about health-related QoL and service use. This comprised: (1) the EuroQol EQ-5D-Y32 and (2) Health Utility Index Mark 2 (HUI-2), both instruments for capturing health-related QoL in young people,33 and (3) a Service Use Questionnaire at individual level. The Service Use Questionnaire collected data on the following services: (1) consultation with the general practitioner (GP) or nurse at home, surgery or by telephone; (2) other community health service use, including CAMHS, clinical psychologist, community psychiatric nurse, counsellors, family therapist, social worker and helplines (eg, MIND, Childline); (3) hospital-based services, including psychiatric or non-psychiatric appointments, psychiatric and non-psychiatric admissions and emergency attendances (including Accident and Emergency attendance) and (4) use of medication.
Preference scale (treatment allocation)
A preference scale was used to determine participant preference for the trial arm they would like to complete if the research did not involve randomisation. This was completed by participants at the baseline visit. Participants were asked to complete a scale (0–100) to indicate their preference for either Stressbusters (100), websites (0) or no preference (50). Participants were informed that their responses on this scale would have no impact on trial allocation.
Demographic information
Participants were asked to supply demographic information including their age, gender and ethnicity, their education and employment status and their family life and social relationships.
Following the completion of baseline measures, participants were randomised to one of two trial arms using remote computerised single allocation (provided by the University of York Trials Unit). The trial arms were as follows:
Arm 1: CCBT intervention: ‘Stressbusters’
Stressbusters is a CCBT program comprising eight 30–45 min sessions of CBT designed for 12–18-year olds. The program is based on the manualised treatment programme from an RCT designed to evaluate the effectiveness of CBT compared with a placebo control.34 Each Stressbusters session is an interactive presentation featuring videos, animations, graphics and printouts.
Sessions are completed in linear progression with each building on knowledge gained in previous sessions and tasks carried out at home between sessions. Homework tasks typically involved the completion of mood diary sheets, with compliance measured by young people entering whether they had completed the tasks into the Stressbusters program at their next session. Sessions contain flexible ‘add-ons’ such as written fact sheets (eg, about bullying, sleep problems) which can be printed out and taken home alongside practice-related hand-outs from the program (eg, mood diary sheets).
Video inserts (case vignettes) of three teenagers feature throughout. Participants hear about the lives of the teenagers and watch them themselves using the program in a combination of short video sequences and voiceovers. The participant inputs information (eg, mood ratings, activity plans, quiz answers, etc) which is stored and used throughout the program.
The session content is organised into the following:
Session 1: Introduction to the program and goal setting
Session 2: Getting activated
Session 3: Emotional recognition
Session 4: Noticing thoughts
Session 5: Thought challenging
Session 6: Problem solving
Session 7: Improving social skills
Session 8: Relapse prevention.
Arm 2: attention control: self-help websites
Participants in arm 2 spent an equivalent time accessing currently available self-help websites. These were chosen by an expert clinical panel, with user and carer involvement, based on them being suitable for use with the participant age range, not being heavily laden with information about self-harm and having no or minimal CBT content. All selected websites provided information about low mood/depression in a combination of texts, narratives and videos. These were:
Participants were introduced to a new website at each of the first four sessions. After introduction of all four websites they could spend subsequent sessions returning to the sites/areas that they found most helpful.
Procedure
A researcher met individual participants at each session to provide instructions and practical support with accessing the computer but did not provide any therapy. Participants were offered a choice of venue to complete trial sessions; including their school, CAMHS site, GP surgery or community centre. All sites provided private spaces that protected confidentiality. Sessions were typically once per week with flexibility offered to fit around participants' other commitments. The methodology is described in more detail elsewhere.35
Participants completed a modified version (15 items) of the MFQ36 at the beginning of each session to monitor mood and assess risk as adopted in the Abeles et al17 study. If a participant responded ‘true’ to the question ‘I thought about killing myself’ on the short MFQ their PMHW/CAMHS clinician was contacted immediately and asked to speak with the individual to discuss their response. If a PMHW/CAMHS clinician was unavailable, this request was made to the duty clinician. Where none of the above could be contacted, the individual was advised to contact their GP. (This procedure also applied if a participant selected ‘true’ to the same question on the full MFQ or selected the response ‘I have definite plans about committing suicide’ or ‘I would kill myself if I had the chance’ during any completion of the BDI). No serious incidents took place during the trial.
The MFQ, BDI, SCAS and QoL/Service Use Questionnaire were then subsequently completed at 4 and 12 months after completion/withdrawal from the intervention (results of the 12-month follow-ups will be reported separately).
The acceptability of the trial and its processes were evaluated through face-to-face qualitative interviews with 20 participants. Interviews followed a topic guide to ensure consistency and included experiences of depression, care pathways, responses to depressive symptoms, acceptability of treatment/location and priority outcomes (ie, what individuals hoped their participation would achieve). These results will be reported in a subsequent publication.
Sample size
No formal power calculations are undertaken in feasibility studies; rather sufficient participants are recruited to determine factors such as recruitment and attrition rates in relation to feasibility outcomes.37 However, in the initial planning stages, we based our sample size on previous work examining the Stressbusters program in 28 young people.17 Here, 70% of participants who completed at least one session of Stressbusters continued with the program and completed all eight sessions. We calculated that, to detect a difference of 10 points post treatment on the MFQ between the two groups at 80% power and 5% significance, 26 participants per group were required: 48 per group (n=96) to allow for those not completing treatment. We were therefore fully powered for the MFQ but had no power calculation for the BDI or other outcome measures selected, and no pre-existing feasibility data or attrition and acceptability data for computerised therapy. Hence, the reason for this feasibility study.
Data analysis
Feasibility data included: number of eligible participants, willingness of clinicians to recruit and young people to participate, adherence to interventions and completion of outcome measures including follow-up, time needed to collect data and SDs of the outcome measures to estimate sample size for a full-scale RCT. Treatment adherence was calculated as the percentage of participants in the Stressbusters group completing all eight sessions, and of the website group completing all four websites over a minimum of four sessions. Data are summarised for the baseline characteristics using means (SD), medians (IQR) and n (%) for the each of the outcome measures (BDI, SCAS, MFQ).
Consistent with recommendations about good practice in the analysis of feasibility studies38 feasibility analysis was descriptive with descriptive statistics calculated for follow-up rates, withdrawal from intervention/trial rates and adherence. Summary statistics were also calculated for the outcome measures (BDI, MFQ and SCAS) at baseline and 4 months follow-up. Analyses were performed in SPSS (V.21). Completion rates of the EQ-5D-Y,32 HUI-233 and Service Use Questionnaires were assessed. Changes in costs and outcomes were combined in the form of Incremental Cost Effectiveness Ratio (ICER) and the uncertainty in ICER was estimated. Full details of the economic analysis will be reported separately.
Results
Overall 136 individuals were assessed for eligibility. Thirty-eight did not meet the inclusion criteria and seven declined to participate. Ninety-one young people consented and were randomised to either Stressbusters (n=45) or websites (n=46) (see CONsolidated Standards Of Reporting Trials (CONSORT) Statement diagram in figure 1).
CCBT CONSORT flow diagram. CCBT, computer-administered cognitive–behavioural therapy.
Feasibility outcomes
The groups were similar in age, but the Stressbusters group had a lower proportion of males (27%) than the website group (41%). More individuals in the websites group had physical health problems (22%) than the Stressbusters group (4%). A higher proportion of the Stressbusters group were currently or had been bullied (83%) than the websites group (72%) and were more likely to occasionally use alcohol (62% vs 41%). Similar proportions in each arm had previously sought help with low mood (table 1).
Baseline characteristics
At 4 months post intervention, self-report questionnaires were sent to 41/45 (91%) participants in the Stressbusters group and 42/46 (91%) of participants in the website group (table 2). The five participants who did not receive a follow-up questionnaire had withdrawn completely from the trial (figure 1). The overall return rate was 56% (25/45) for the Stressbusters group and 65% (30/46) for the website group (p=0.346). There was no significant difference between those who returned their 4-month questionnaire (n=55) and those who did not (n=36) by age (15.2 (1.4) vs 15.4 (1.3), p=0.690) or gender (male: 40% vs 25%, p=0.140).
Completion rates for questionnaires across trial time points
Adherence to treatment was calculated in both groups; 62% (28/45) of participants in the Stressbusters group completed all eight sessions, and 76% (35/46) of participants in the website group completed all four websites. Overall, 8/91 (9%) participants did not complete any treatment sessions (four randomised to Stressbusters and four to websites) (table 3).
Summary of retention and completion of measures
The length of time between key dates in the participants' journeys throughout the trial was similar between groups (table 4). The mean time from randomisation to the start of intervention was 19.6 days. The average duration of the intervention was 54.6 days in the Stressbusters group and 49.9 days in the website group. Overall the average number of days between randomisation and sending out a 4-month follow-up questionnaire was 209.8 days while return of the questionnaire took an average of 9.6 days.
Length of time between key events in the RCT process
Outcome measures
Table 5 shows the scores for those who completed the BDI, MFQ, SCAS (total and subscale scores) and the QoL and Service Use Questionnaire at baseline and 4 months. Higher scores on the BDI, MFQ and SCAS represent greater levels of depressive (BDI, MFQ) or anxiety (SCAS) symptoms.
Scores on the BDI, MFQ, SCAS, utilities (EQ-5D-Y) and costs at baseline and 4 months
There was no statistically significant difference between those who returned their 4-month questionnaire and those who did not in the baseline BDI (15.9 (7.3) vs 17.5 (7.1), p=0.303), MFQ (35.7 (9.4) vs 36.2 (8.6), p=0.820) or SCAS total scores (43.8 (17.) vs 41.5 (17.5), p=0.548).
The mean MFQ scores fell by 6.7 (15.5) in the Stressbusters group, while website group scores increased by 1.7 (11.5) from baseline to 4 months, a difference of 8.4. Eligibility was defined by a score of ≥20 on the MFQ at baseline. Since a score of 29 represents a likely current major depressive episode and 20 any depressive episode, this change difference has some clinical meaning. At 4 months, 20% (5/30) of the Stressbusters group scored below 20 compared with 20% (6/24) in the websites group. Hence, there was no substantial reduction in the proportion with no evidence of a depressive disorder at 4 months. For participants completing all eight sessions in the CCBT group, MFQ scores reduced from a mean of 36.6 at baseline to 28.6 at 4 months.
At baseline, 86% (37/43) scored 29 or more in the Stressbusters group which fell to 60% (15/25) at 4 months. In the websites group, 72% (33/46) scored 29 or more at baseline, falling to 63% (19/30) at 4 months. Hence, in the Stressbusters group, there was a substantial reduction from baseline in the proportion of participants with a likely current major depressive episode at 4 months.
To ensure that participants who did not complete 4-month follow-up were not different to those who did, only those participants for whom paired data were available were examined. In the Stressbusters group, 91% (21/23) of participants scored 29 or more on the MFQ at baseline which fell to 57% (13/23) at 4 months. In the websites group, 66% (20/30) of participants scored 29 or more at baseline which fell to 63% (19/30) at 4 months. Again in the Stressbusters group, there was a substantial reduction from baseline in the proportion of participants with a likely current major depressive episode at 4 months. The baseline proportions were not significantly different to the overall sample.
In the Stressbusters group, mean BDI scores fell by 2.8 (6.6) from baseline to 4 months, whereas the website groups increased by 1.2 (8.5). The changes in SCAS total scores and subgroup scores were similar between groups, with the total score seeing a slight fall (0.2 (13.7)) in the Stressbusters group and a slight increase in the websites group (0.2 (10.6)). For participants completing all eight sessions in the CCBT group, BDI scores reduced from a mean of 15.8 at baseline to 12.8 at 4 months.
Weekly session short-form MFQ scores are shown in table 6. The Stressbusters group had high scores at visit 1 (17.4(6.5)), which reduced over the first four sessions to plateau after that (12.3 (7.0) at session 4). The website group had lower scores at visit 1 (14.8(6.2)) compared with the Stressbusters group, but their scores remained fairly consistent over the sessions (13.7 (8.0) at session 4).
Session MFQ scores (mean (SD), n)
When asked about their preference for treatment, prior to knowledge of treatment allocation, four participants indicated that websites would be their preferred treatment if randomisation was not used, while the remainder opted for Stressbusters (n=44) or had no preference (n=41).
Sample size required for a full-scale RCT
To conduct a fully powered RCT of the clinical and cost-effectiveness of CCBT, based on a 4-month return rate of 60%, the changes in MFQ and BDI scores from baseline to 4 months have been used to calculate sample size requirements (table 5).
If the MFQ were the primary outcome measure, to detect a difference of 8.4 points (pooled SD=13.37) (the change score detected on the MFQ across both groups in this study), at 80% power and 5% significance, 41 participants would be required per arm. Based on a 60% completion rate, this represents 68 per group (a total of 136 participants).
If the BDI were the primary outcome measure, to detect a difference of 4.0 points (pooled SD=7.70) (the change score detected on the BDI across both groups in this study), at 80% power and 5% significance, 60 participants would be required per arm. Based on a 60% completion rate, 100 per group are required (a total of 200 participants).
The MFQ can monitor risk, has wide use in other sudies for comparison in this age group39–41 and is the outcome measure recommended by NICE. Thus, we suggest that the MFQ would be the appropriate outcome measure of choice in a larger fully powered study.
Health economic analysis
Completion rates of the QoL and Service Use Questionnaire were assessed and responses analysed using STATA 13.
Completion rates for the economic questionnaire were >60% in both groups (table 2), in terms of QoL questionnaires (ie, EQ-5D-Y and HUI) and the Service Use Questionnaire. This was the same as other questionnaires at 4-month follow-up. The utility analysis suggested small changes in both groups between baseline and 4 months (websites: baseline =0.61 (SE: 0.10) and 4 months =0.65 (SE: 0.13); Stressbusters: baseline =0.52 (SE: 0.09) and 4 months =0.58 (SE: 0.12)). The cost of delivering the Stressbusters program to the 45 participants allocated to receive this treatment was £4557.40, thus indicating a provisional cost per participant of £101.20.i The cost analysis of service use suggested that there was a difference in mean costs at baseline between the two groups (websites: baseline =£950 (SE=£193.1) and 4 months =£597 (SE=£63.0)). This difference in baseline costs is likely to be due to the small sample size which may result in large differences in costs due to a small number of heavy service users (as shown by the large SE). However, more importantly, this difference in costs between groups remained similar at 4 months (websites: 4 months =£950 per patient (SE: £542.8); Stressbusters: 4 months =£499 per patient (SE: £147.6)). Baseline differences in costs and utility between groups should be taken into account in a regression analysis to estimate difference between groups (ie, the treatment effect) and to calculate ICER. However, due to the small sample size, there was not enough power to conduct a regression analysis to adjust for baseline differences and estimate ICER. Hence, we suggest that a fully powered study with adequate sample size is required to conduct a full cost-effectiveness analysis.
Funding an intervention should be based on whether it is likely to represent value for money to the NHS. This decision might be guided by the expected health gain required within NICE reimbursement thresholds (£20 000–£30 000 per quality-adjusted life year (QALY)). Based on the treatment delivery cost of Stressbusters (assuming other service use costs are not significantly different between groups), the incremental QoL gain due to Stressbusters would need to be in the range between 0.00337 (at willingness-to-pay threshold of £30 000 per QALY) and 0.00506 QALYs (at willingness-to-pay threshold of £20 000 per QALY) for the Stressbusters intervention to be cost-effective. This implies that the Stressbusters program would need to achieve an improvement in the overall QoL of children by at least 0.6% to 0.8%, to be cost-effective (based on the conventional willingness-to-pay thresholds used in the UK).
Discussion
This study aimed to assess the feasibility of delivering an RCT comparing Stressbusters with self-help websites for adolescents with low mood/depression. Rich information regarding the application of CCBT as a treatment for this group in the community (notably in schools) has been yielded through this feasibility study. Furthermore, reductions were seen in depression scores, as measured by the MFQ and BDI for those who received CCBT.
Those involved in the trial including adolescents, teachers (who assisted with setting up the trial in schools) and CAMHS clinicians (responsible for referring young people and dealing with risk) were largely supportive of CCBT as a treatment for adolescent depression. Recruitment was successful within the specified time frame.
We were able to develop a strong infrastructure across schools and the community to deliver the program. Although, we initially set out to deliver the trial in seven local schools, this infrastructure expanded to include 10 schools, two clinics, one GP practice and a community centre to accommodate the needs of the young people wanting to take part.
Clinicians were positive about the use of CCBT as a treatment for adolescents with low mood/depression, demonstrated through their enthusiasm in referring young people to the trial. Schools also actively requested access to CCBT with requests being made during the trial period when they heard about the trial (and schools requested involvement) and following completion of the trial (where schools requested their own copies of the CCBT program).
Despite positive reactions to the trial only 60% of participants returned follow-up questionnaires (despite reminders) at 4 months. Although disappointing, this figure is comparable to the 4-month follow-up return rates found in several similar studies of computerised therapies for adolescents with depression (eg, 51%,42 60%43). Future studies could use improved mechanisms for collecting follow-up data, such as electronic reminders, availability of remote electronic data completion and small rewards for time taken. Other studies44 suggest that these methods can improve completion rates.
There was no significant difference in the response rate at 4 months between the groups or a significant difference between those who returned their 4-month questionnaire and those who did not by age or gender. In addition, there was no statistically significant difference between those who returned their 4-month questionnaire and those who did not in the baseline BDI, MFQ or SCAS total scores. Hence, there is no evidence that bias in the estimation of parameters is presented (now with 95% CI for the outcome measures) due to response bias. In addition, there is evidence that missing data are missing completely at random (MCAR). The statistical advantage of data that are MCAR is that the analysis remains unbiased. Power may be lost in the design, but the estimated parameters are not biased by the absence of the data. Where response rates are low for future studies, helpful methodologies could include analysis of last observation carried forward (LOCF) or mixed-effect model repeated measure (MMRM) analysis.
In total, 62% of participants in the CCBT group completed all eight sessions of the program. This figure compares favourably to other studies examining computerised therapies with completion rates of 39%24 and 57%.45 These figures are, however, lower than those reported with face-to-face CBT,45 suggesting that the presence of a clinician may encourage completion and thus reduce attrition. It is important to note that eight of the 91 participants who originally consented to the trial withdrew before starting treatment. These eight received alternative NHS treatments. We note that many studies only include consented participants after they have started the first research treatment session. Our percentage for completion of 4-month follow-up questionnaires and completion of all treatment sessions would be improved if we reported in this way (66% and 76%, respectively). The high completion rates for self-help websites suggest that they are acceptable to participants but no notable improvement in depression scores took place in this group.
For participants completing all eight sessions in the CCBT group, MFQ scores reduced from a mean of 36.6 at baseline to 28.6 at 4 months (BDI: 15.8 to 12.8). As not all participants completed all eight sessions (62%), this may suggest that in a treatment care pathway, some young people may need redirecting to alternative services (eg, face-to-face support). Research into the place of CCBT in the care pathway is warranted.
Regular monitoring was necessary in the completion of this study. Responses to items on the MFQ and BDI were checked as part of this monitoring with good responses available in the event of participants needing additional support. The method employed here was feasible to successfully support the trial and will be repeated in any further studies we complete within this context
Our economic analysis suggests that Stressbusters is relatively inexpensive to implement and may require only modest health benefits to be within the NICE's willingness-to-pay threshold per QALY. This supports the need for a fully powered RCT. Further economic analysis may consider the potential to reduce the volume, costs and length of treatment received by adolescents with low mood/depression.
Limitations and lessons learnt
Exploring mechanisms to improve data collection in a technological age is likely to yield higher return rates for outcome measures. Online data collection (currently being trialled in other local research) would potentially make outcome measure completion quicker and easier for participants. Currently, we are carrying out further feasibility work to examine the collection of more complete outcome measure data using increased face-to-face research assistant time, using text message reminders and providing appropriate ‘thank you’ rewards that do not contravene ethical boundaries. Our original trial timeline was based on participants completing treatment sessions within 8 weeks (ie, one session per week). This rarely occurred because of participant availability (eg, examination periods) which often caused delays. Although most participants reported that they did not find this problematic, this would need to be considered in a full-scale trial.
The trial was only conducted within one NHS trust resulting in the under-representation of certain minorities and a lack of varied geographical localities and demographic characteristics. A larger RCT would need an extended geographical footprint that includes wider ethnic and sociocultural diversity. Furthermore, some of those recruited had received other services for their low mood/depression prior to the trial. In future research, a community sample should be recruited to investigate the effectiveness of CCBT for those who have not accessed services previously.
Conclusion
This feasibility study has provided rich information about recruitment, attrition and acceptability of outcome measures, interventions and involvement in an RCT. These feasibility findings and encouraging trends in cost-effectiveness data offer encouragement to warrant a fully powered RCT to evaluate the clinical and cost-effectiveness of this intervention for this population. Quantitative data have provided robust power calculations, while health economic analysis highlights the level of uncertainty in the ICER given small numbers and further supports a larger study, with a larger sample, to address this.
Improvements in methodology could focus on additional support for session attendance and outcome measure completion but these are not insurmountable challenges. For a full trial, we would recommend using MFQ scores as the main outcome measure given that NICE guidelines now recommend this outcome measure for adolescent depression.
Acknowledgments
For their assistance with aspects of the research reported here we thank: Steven Grigg, Tina Hardman, Christine Godfrey, Chrissie Verduyn, Simon Gilbody, David Torgerson, Ben Alderson-Day, Sophie Bennett, Joy Adamson, Lisa Dyson, Paul Dempster, Naomi Hooke, Isobel Barlow, Rebecca Hargate, Danielle Varley, Holly Taylor and Catherine Arthurson. The active involvement of all participants within the research is also gratefully acknowledged.
References
Footnotes
Contributors BW was responsible for the overall development of an ethically sound protocol. BW and EL were involved in the conception and production of the study and the development of the initial protocol. PA was one of the developers of the Stressbusters CCBT program and provided advice and support throughout the study. VA provided statistical expertise while SA and DT advised on the design and conduct of the health economic analysis. LT assisted with the day-to-day running of the trial. All authors made substantial contributions to the drafting, critical revision and final approval of the paper.
Funding This paper presents independent research funded by the National Institute for Health Research (NIHR) under its Research for Patient Benefit (RfPB) Programme (grant reference number PB-PG-0609-19 295).
Disclaimer The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.
Competing interests None declared.
Ethics approval The trial was designed to protect the human rights and dignity of the participant as reflected in the 1996 version of the Helsinki Declaration. Ethical approval for this trial was received from Leeds (West) Research and Ethics Committee (Reference: 10/H1307/137).
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.
Trial Sponsor Leeds and York Partnership NHS Foundation Trust, North Wing, St Mary's House, St Martin's View, Leeds, LS7 3JX.
↵i The licence fee had not been finalised at the time of this study and was waivered for the trial so this figure may be an underestimate.