Article Text
PDF
Abstract
Objective To evaluate the impact of child and adolescent mental health services (CAMHS) transformation in South East England on patient access, resource utilisation and health outcomes.
Design In an observational study, we use difference-in-differences analysis with propensity score matching to analyse routinely collected patient level data.
Setting Three CAMHS services in South East England.
Participants All patients attending CAMHS between April 2012 and December 2018, with more than 57 000 spells of care included.
Main outcome measures The rate and volume of people accessing CAMHS; waiting times to the first contact and waiting times between the first and second contact; and health outcomes, including the Strengths and Difficulties Questionnaire (SDQ) and the Revised Child Anxiety and Depression Scale (RCADS).
Results The intervention led to 20% (incidence rate ratio: 1.20; 95% CI: 1.15 to 1.24) more new patients starting per month. There was mixed evidence on waiting times for the first contact. The intervention led to 10% (incidence rate ratio: 1.10; 95% CI: 1.02 to 1.18) higher waiting time for the second contact. The number of contacts per spell (OR: 1.08; 95% CI: 0.94 to 1.25) and the rereferral rate (OR: 1.06; 95% CI: 0.96 to 1.17) were not significantly different. During the post intervention period, patients in the intervention group scored on average 3.3 (95% CI: −5.0 to −1.6) points lower on the RCADS and 1.0 (95% CI: −1.8 to -0.3) points lower on the SDQ compared with the control group after adjusting for the baseline score.
Conclusions Overall, there are signs that transformation can help CAMHS achieve the objectives of greater access and improved health outcomes, but trade-offs exist among different performance metrics, particularly between access and waiting times. Commissioners and providers should be conscious of any trade-offs when undertaking service redesign and transformation.
- child and adolescent psychiatry
- mental health
- organisation of health services
- health economics
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
With little prior evidence from the evaluation of child and adolescent mental health services, this paper is a novel attempt to understand, describe and evaluate these services.
We used data from across one large NHS Foundation Trust, which provides consistency.
We employed state-of-the-art econometrics in a quasi-experimental design.
However, we cannot eliminate residual confounding.
A further limitation is data availability—specifically, health outcomes data were insufficient for including in a DiD design.
Introduction
Mental health problems impose a large health and economic burden on individuals, their families and society.1 Globally, mental illness has been associated with reduced life expectancy and is the single biggest cause of years lived with a disability.2 3 In the UK, it is estimated that the costs associated with mental illness totals 4.5% of gross domestic product.4 Childhood and adolescence is crucial to any attempt to alleviate this burden. The prevalence of mental disorders in children and adolescents aged up to 18 is estimated at 13%.5–7 Untreated mental health conditions negatively impact on development throughout the life course, making access to ‘adolescent-responsive’ and high-quality health systems crucially important.8
Child and adolescent mental health services (CAMHS) in England have come under increasing pressure and scrutiny in recent years. Between 2012/2013 and 2014/2015, there was a sharp rise in referrals to CAMHS,9 alongside a growing number of young people presenting to emergency departments with self-harm.10 However, over this period, many local budgets for CAMHS were reduced in real terms, resulting in reports of the perception of rising thresholds for care and limiting the number of young people able to access care, with longer waiting times in some services and considerable variation in local service provision.11 Although effective interventions exist,8 it is estimated that less than a quarter of young people in England with clinically significant mental health problems receive help from specialist health services.12
Amid rising concerns about the adequacy of CAMHS, a 2015 UK Government review, Future in Mind,13 made recommendations as to how services should change. This, along with other directives, such as the Five Year Forward View For Mental Health,14 highlighted the need to ‘transform’ services to improve accessibility, increase quality of care and improve health outcomes. NHS England tasked Clinical Commissioning Groups (CCGs) to develop ‘Local Transformation Plans’ in order to access a £1.4 billion CAMHS transformation fund committed by the government to be spent from 2014/2015 to 2020/2021.
Across England, commissioners have used this additional funding to spur the local implementation of changes aligned with the transformation of CAMHS. Many CAMHS have moved from a traditional ‘tiered’ model, where a young person is allocated to a specific tier of service aligned with complexity/need, towards a more ‘integrated’ service model that allows young people to benefit from the full range of support available, recognising that needs can be complex. Other changes include new ways of working with the third sector, with some CAMHS contracting the third sector to provide support; new ways of working with schools and frequently the introduction of a single point of access (SPA) to make it easier for parents and young people to contact services.
Monitoring improvement using performance indicators is a key element of the Local Transformation Plans and a prerequisite for accessing funding. Although these indicators provide commissioners with information about trends in time, they cannot provide conclusive evidence about whether CAMHS transformation has been successful in achieving its objectives. With only one financial year left to spend the transformation funds, evidence is needed to inform future investment in CAMHS and any adaptation or redesign of transformation strategies.
In this study, we investigated the impact of the transformation of CAMHS in Buckinghamshire compared with neighbouring CAMHS in Oxfordshire and Swindon, Wiltshire, Bath and North East Somerset (collectively referred to as SWBaNES) on patient access, resource use and health outcomes, using a series of service and individual level outcomes. We used data for all patients and contacts with CAMHS between 2012/2013 and 2018/2019. The study is part of a larger mixed methods evaluation.15
Methods
Study design
The development and implementation of local transformation plans occurred according to a range of factors, such as local decision-making and capacity. Since the timing and content of the transformation was not subject to randomisation, we adopted a quasi-experimental study design by using difference-in-differences (DiD) analysis with propensity score matching (PSM) to reduce observed and unobserved bias.16 17 This type of study design is suggested by the Medical Research Council in England, among others, to overcome the significant challenges inherent in the evaluation of complex health interventions.18–21
Setting
The study was conducted in the South of England (online supplementary figure 1) and included CAMHS provided by Oxford Health NHS Foundation Trust (Oxford Health). Oxford Health is one of the largest CAMHS providers in England. Delivering services for NHS Buckinghamshire CCG, NHS Oxfordshire CCG, NHS Swindon CCG, NHS Wiltshire CCG and NHS Bath and North East Somerset CCG (NHS BaNES CCG). As the latter three are commissioned under one contract, they are referred to collectively as SWBaNES. For more information on the study setting, see online supplementary table 1.
Supplemental material
Intervention and control groups
The transformation of CAMHS across the study geography differed in intensity, timing and pace. Transformation started in 2015/2016 in Buckinghamshire (intervention). The core components of transformation were similar across the sites. All services adopted some version of the THRIVE model.22 This involved a shift towards treatment decisions based on level of need, or capacity to benefit, rather than specific diagnoses or interventions. It required redesigning pathways, including introducing ‘Getting Help’ and ‘Getting More Help’ pathways broadly in place of tiers 2 and 3, and a redesignation of staff. Another core shift was the introduction of an SPA for referrals. In conventional services, primary care, in the form of general practitioners (GPs), acts as the gatekeeper to CAMHS. The introduction of an SPA, including the ability for young people (primarily aged 16–17) to self-refer, was intended to make it easier for young people and parents to engage with CAMHS by allowing them to contact services directly. A final core element of transformation was new ways of working with the third sector. In Buckinghamshire, third sector involvement was primarily with Barnardo’s, one of the largest charities providing care to children in England. Barnardo’s workers were placed into the SPA as Contact Support Workerswhere they would take initial calls and liaise with clinicians about referral decisions; and into the new ‘Getting Help’ pathway as ‘Buddies’. Buddies were trained to provide a low-intensity intervention of up to six sessions for children and young people with low-to-moderate difficulties.
While recognising that service transformation is a process rather than a discrete event, we dated the transformation to the financial year in which it started in the intervention group, 2015/2016. The study therefore has a pretransformation and post-transformation period.
Transformation did not start in Oxfordshire and SWBaNES until 2018/2019 (control). In the interim, the control group continued to deliver services based on a ‘traditional, tiered system’ in which CAMHS were accessed primarily through GPs with some referrals from school staff and social services. Many of the goals and the core components of transformation were similar in Oxfordshire and SWBaNES to those in Buckinghamshire, although some aspects of delivery were shaped according to local assets and capacity (eg, the role of the third sector).
Data
Data on referrals to CAMHS, contacts (ie, appointments) with CAMHS, diagnosis and health outcomes of all patients who started their contact with CAMHS between April 2012 and December 2018 were obtained from the electronic patient record systems of Oxford Health. Data prior to April 2012 were incomplete and were not considered valuable for the purposes of this study. The Strengths and Difficulties Questionnaire (SDQ) and the Revised Children’s Anxiety and Depression Scale (RCADS) were the only widely recorded health outcomes available. Health outcome data were available from 2014, but recording was very limited until 2015. We also obtained sociodemographic data including gender, ethnicity, age at referral to CAMHS and socioeconomic deprivation based on the postcode of patient’s residence or patient’s GP practice (which is most often based on the residence). The data were grouped in distinct spells of care. A spell of care was defined as an overlapping bundle of referrals to and contacts with CAMHS. Specifically, based on clinical input, we considered a referral to be overlapping if the new referral date fell within 28 days weeks of the previous discharge date. More detail is provided in online supplementary figure S2.
Outcomes
A set of outcomes was defined to assess the impact of the CAMHS transformation on patient access, resource use and health (table 1). For access to care, the rate of new patients per month per 1000 people aged under 18 was used to capture overall access to the service. This was the only outcome defined at service level (ie, Buckinghamshire—intervention; Oxfordshire, SWBaNES—control).
Outcomes
In addition, we analysed waiting times from referral to the first contact with CAMHS and waiting times from the first to the second contact. In total, 5% of spells did not have any attended contacts and 19% of spells did not have a second contact. We further excluded 189 spells with a waiting time to the first contact of more than 355 days (<1%) and 437 spells (<1%) with a waiting time for a second contact over 235 days. This was based on visual inspection of the distribution of the data which revealed a long tail of extreme values and was further tested using Cook’s Distance, according to which all values over the respective number of days were considered extreme.
With regard to resource use, two outcomes were used: the number of contacts per spell and the rate of rereferrals—a binary outcome recording whether a patient has more than once distinct spell of care (the first spell is coded 0, any further spells 1). Furthermore, the total scores on the SDQ (range: 0–40) and RCADS (range: 0–141) questionnaires were used as health outcomes. Both RCADS and the SDQ can be completed by the child or parent. We created a single measure for each that combined the child and parent scores, giving priority to the child response where both were recorded and controlling for respondent type as an additional covariate.
Propensity score matching
We used PSM to account for observed differences between the intervention and control group that would influence the treatment effect and for changes in the composition of the groups over time. Specifically, when analysing spell level outcomes, we accounted for potential differences by estimating a multigroup propensity score to weight each spell.23 Multinomial logistic regression was used to predict the probability of each spell being in one of four groups (precontrol, postcontrol, preintervention and postintervention) as a function of observed covariates. We then ascribed each spell a weight according to its likelihood of being in the intervention group in the preperiod. For spell i, the weight was calculated using the following formula:
where the intervention group in the preperiod is denoted 1 and the group the spell starts in is denoted g. Weights were included in the regression models outlined above to balance the selected covariates across the four groups. Given the limited recording of health outcomes, we performed the PSM separately for the SDQ and RCADS. Since data on health outcomes were almost entirely restricted to the postperiod, there were only two groups (control and intervention).
In selecting the covariates for inclusion, we sought to avoid post-treatment bias by excluding factors that we suspected the intervention would directly address.24 We therefore included only demographic variables, rather than service factors such as the level of care or urgency of referral. The following covariates featured in our analysis of all outcomes: gender (male/female), ethnicity (White British/White Other/Black, Asian and minority ethnic), age at referral received and deprivation index. When analysing health outcomes, we included the baseline score from the respective health questionnaire to control for differences in initial severity. Among our covariates, ethnicity was the only major source of missing data (n=9963, 17.3%). To account for the missing data in the analysis, we used multiple imputation. Our approach is detailed in the online supplementary table S2.
For the PSM, covariate balance was assessed based on the mean and median percentage standardised bias as well as Rubin’s B (the absolute standardised difference of the means of the linear index of the propensity score in the treated and (matched) non-treated group) and Rubin’s R (the ratio of treated to (matched) non-treated variances of the propensity score index).25 Following Rubin’s recommendation,26 we considered B less than 25 and R between 0.5 and 2 to indicate sufficient balance.
Regression analysis
We included all of the same covariates in our regression analysis as for PSM: gender, ethnicity, age at receipt of referral, deprivation and, for health outcomes only, initial severity. When analysing data at a service level, we took the average for deprivation and age and calculated proportions for other covariates, specifically the proportion male and the proportion White British. At the service level we also included the quarter the spell started in to account for any seasonality.
Following common practice, we first specified a DiD model that assumes parallel trends in the outcomes between the intervention and control group (model 1).
(1)
In equation 1, the dependent variable 
denotes one of the nine outcomes for spell (or service) i at time t. 
denotes the intervention group (taking the value 1 for the intervention group and 0 for the control group) and 
accounts for the transformation period (taking the value 0 before transformation and 1 after transformation). The interaction term 
is the variable of interest (ie, the difference-in-difference estimator) and captures the impact of the transformation on the outcome. The vector 
includes the covariates described above, and 
, a time variable indicating the quarter of each year and capturing any linear time trends. The error term 
captures unobserved variations in the outcome.
To relax the parallel trends assumption, which is critical for producing unbiased DiD estimates,23 we specified a model with differential trends between the intervention and control group (model 2). This model included the term, 
, which accounts for different linear trends across the service areas in the study.
(2)
To test whether the parallel assumption holds, we limited the analysis period to observations in the pretransformation period (ie, before 2015/2016), assigned an arbitrary date in 2013/2014 as the pseudotransformation date, and ran model 1. If the DiD estimator was statistically significant in this test, the parallel trends assumption was violated.
Finally, we were unable to employ a DiD design for health outcomes. Recording of health outcomes was only recently introduced in CAMHS and we had insufficient data to form a reliable preperiod. Instead, we performed a regression analysis with PSM to compare the health outcomes measures at follow-up (ie, the latest recorded health outcome of a patient in a spell) between the intervention and control group adjusting for differences at baseline measures (ie, the first recorded health outcome in the same spell), using the following model:
(3)
where 
is the last outcomes score. The vector, 
, includes a range of covariates defined at baseline including the first score on the outcome scale. The variable of interest here is 
.
All regressions were fitted using generalised linear models with clustered standard errors at patient level (or service level for the service level outcome) and included the weights from the PSM. The distribution family in each regression models was chosen based on a Modified Park Test (using the user-command ‘GLMdiag’ in Stata) and the Akaike and Bayesian information criteria. The link function was selected with reference to the link test (‘linktest’ command in Stata).
Sensitivity analysis
In order to assess the sensitivity of the results to the specified pretreatment period, we replicated the analysis with a different timeframe. While in the main analysis we utilised all complete data for our preperiod and postperiod, in our sensitivity analysis, we followed the approach of Slaughter in assessing the robustness of the results to a shortened pretreatment period.27 In our case, we restricted the data to starting from October 2013. We specified this time point following visual inspection of trends in the preperiod (see online supplementary figures S3 and S4).
Results
Observed differences between intervention and control
There were 57 501 spells of care initiated between April 2012 and December 2018. They differed between the intervention and control group in terms of sociodemographic characteristics (table 2). In the preintervention period, the intervention group included younger patients and patients from less socioeconomically deprived areas, and consisted of proportionally more males and fewer patients with White British ethnicity. These differences were similar in the postintervention period.
Outcomes and covariates for intervention and control, preintervention and postintervention (unweighted)
Table 2 also shows the change in the outcomes from the preperiod to postperiod. The rate of new patients starting per month per 1000 under 18 increased in the intervention group (from 1.05 to 1.22), but not in the control (1.24 in both). Waiting times for the first contact were lower in the intervention group (39 days) than the control (41 days) in the preperiod. This gap widened in the postperiod, with the wait remaining 39 days for the intervention, but increasing to 61 days for the control. By contrast, waits for the second contact were higher in the intervention group (27 days vs 23 days for the control). This gap also increased in the postperiod (35 days vs 27 days). Attended contacts per spell were higher in the intervention group (17.0 contacts) in the preperiod compared with the control (15.8). In the postperiod, the gap narrowed (10.2 in the intervention compared with 10.4 in the control group). The difference in the rereferral rate was consistent in the preperiod and postperiods. In the preperiod, the re-referral rate was higher in the intervention group (0.35) than the control (0.32); the difference narrowed in the postperiod, at 0.17 for the intervention and 0.16 in the control. In terms of health outcomes, the last RCADS score was 4.6 points lower in the intervention group and the SDQ score was 1.2 points lower.
Achieved covariance balance
Following PSM, observed confounding was reduced to acceptable levels according to Rubin’s B (ie, it was lower than 25 in the preperiod and postperiods) and Rubin’s R (ie, it was between 0.5 and 2 in the preperiod and postperiod) across all imputed data sets. Likewise, the mean and median percentage standardised biassed were also substantially reduced to values closer to 0. The results of the PSM in terms of covariate balance from one imputed data set are presented in table 3 and illustrated graphically in the online supplementary figure 5. The results across each imputed data set were very similar.
Covariate balance: matched versus unmatched, preintervention and postintervention
Results from the regression analysis
Table 4 provides the results of the difference-in-difference analysis with PSM under the selected model, as well as whether the parallel trends assumption held. The intervention led to 20% (incidence rate ratio: 1.20; 95% CI: 1.15 to 1.24) more new patients starting per month. Waiting times were lower in the intervention group under the parallel trends model, however the parallel trends assumption was violated. With differential trends, the waiting times for the first contact instead increased 83% (incidence rate ratio: 1.83; 95% CI: 1.67 to 1.99). The intervention led to 10% (incidence rate ratio: 1.10; 95% CI: 1.02 to 1.18) higher waiting time for the second contact with CAMHS compared with the control group. The parallel trends assumption was also violated for the number of contacts per spell. Under the differential trends model, the point estimate was 1.08, but inconclusive (OR: 1.08; 95% CI: 0.94 to 1.25). The rereferral rate was unchanged following the intervention (OR: 1.06; 95% CI: 0.96 to 1.17). During the post intervention period, patients in the intervention group scored on average 3.3 (95% CI: −5.0 to −1.6) points lower on the RCADS and 1.0 (95% CI: -1.8 to -0.3) points lower on the SDQ compared with the control group after adjusting for the baseline score. Table 4 also shows the distribution family and link function that provided the best goodness-of-fit to the data. The results for each model and from the test of the parallel trends assumption are presented in the online supplementary table 3, and graphical visualisation of the observed trends in outcomes is included in online supplementary figures S3 and S4.
Main results
Sensitivity analysis
Table 5 shows the results of the sensitivity analysis. The intervention led to 16% (incidence rate ratio: 1.16; 95% CI: 1.13 to 1.20) more spells starting per month. Waiting times for the first contact fell 14% (incidence rate ratio: 0.86; 95% CI: 0.81 to 0.91), but rose for the second contact compared with the control group (incidence rate ratio: 1.11; 95% CI: 1.03, to 1.19). The parallel trends was again violated for the number of attended contacts per spell; under the differential trends model, the number of attended contacts was 12% higher (OR: 1.12; 95% CI: 0.96, 1.31). The re-referral rate was unchanged (OR: 1.03; 95% CI: 0.91 to 1.15). The results for each model and from the test of the parallel trends assumption are presented in the online supplementary table S4.
Sensitivity analysis—restricted preperiod (starting October 2013)
Discussion
This empirical evaluation provides the first evidence about the impact of CAMHS transformations in South England on access to care, service use and health outcomes. There are mixed signs on access to care, resource use was unchanged and there is tentative evidence that health outcomes are better under a transformed service.
There is increased access to care in the transformed CAMHS relative to the control as measured by the rate of new patients accessing care. However, waiting times for the second contact increased following transformation. For waiting times for the first contact, our various specifications produced widely different estimates for the effect, with a decrease under the parallel trends model but a large increase when adjusting for differential trends. Visual inspection confirms divergent trends in the earliest periods of the study: prior to October 2013 waiting times for the first contact fell in the intervention group while rising for the control group, after this point both groups exhibit a similar trend (see online supplementary figure S3). In our sensitivity analysis, from October 2013 onwards, the trends were parallel and we found a fall in waiting times for the first contact. Given the inflexion in waiting times observed in the intervention group in the post-treatment period after a steadily rising trend in the immediate pretreatment periods, it seems plausible that the intervention reduced waiting times in the intervention group. Yet, it is also possible that an earlier reduction in waiting times—prior to October 2013—in the intervention group is one reason for the earlier adoption of the transformation plan in Buckinghamshire, which raises doubts as to whether other areas would be able to replicate this success. Taken together, this amounts to evidence of an increase in waiting times for the second contact alongside more people accessing the service, but greater uncertainty over the impact on waiting times for the first contact.
In terms of resource use, the rereferral rate was unchanged. There was also no significant change in the overall number of attended contacts per spell. Again, the results diverge depending on the model, with a reduction in contacts per spell found in the parallel trends model, although this was also inconclusive. A fall in attended contacts per patient would be expected under the Thrive model, which looks to curtail over-lengthy treatment. Our results again hint that Buckinghamshire, as an early adopter of the transformation, may have differed from the control group. However, contacts per spell as an outcome could be explored in greater detail. Data from this study reveal that some young people attend CAMHS only once and many patients are seen less than three times as part of a spell of care.28 The average says little about changes in the pattern of use among frequent or infrequent attenders and future research could seek to understand whether changes exist at different levels of service use. This paper is part of a wider mixed-methods evaluation that may be able to shed further light on this.15
Finally, this study also finds evidence that patients leave with statistically lower scores on the two main health outcomes questionnaires—SDQ and RCADS—when controlling for the baseline score and the type of questionnaire. This suggests higher quality of life in the intervention group. However, the evidence is weaker since we have much less comprehensive recording and insufficient data in the preperiod to test this in a DiD framework. It could be that the intervention and control groups have been consistently different prior to the intervention, but we cannot test for this. The differences reported are also at the average; future research could look more closely at final outcomes scores in order to understand the clinical significance of this shift. Improved collection of routine outcomes data will remain important for the evaluation of CAMHS in the future.
Overall, this analysis highlights some of the trade-offs inherent in changing a complicated organisation such as CAMHS. We found an increase in activity in the intervention group relative to other services, which would be expected given the increased funding associated with the transformation. Such overall increases in access are needed to meet recently introduced targets on the numbers of people estimated to have a mental health condition receiving care.14 However, attempts to improve one may have consequences elsewhere in the system. Such trade-offs have been illustrated with reference to, for instance, liaison mental health services.29 In particular, this study finds evidence of an increase in access alongside an increase in the waiting time for a second contact, suggesting the increased workload brought about by the transformation was not fully matched by sufficient capacity further along the system. Commissioners and providers should work together to anticipate such changes but must also consider the relative importance of different outcomes.
Strengths
Strengths of this study include the use of all data across one large NHS Foundation Trust. The scale of the research means the analyses are highly powered. With little prior evidence from the evaluation of CAMHS, this paper is a novel attempt to understand, describe and evaluate these services. An advantage of having data from one trust is that the recording system is consistent, increasing the comparability of the data. We employed a DiD framework, as recommended by MRC among others.16 19 Examples of the use of a DiD framework to evaluate service level interventions include stroke unit centralisation and policy changes such as the introduction of payment by results or liberalising bar closure hours.30–35 An additional concern with repeated cross-sectional analysis is that the composition of the populations may evolve over time, exogenous to the intervention, biassing results. Given this risk, we combined the DiD framework with PSM to reduce the bias in our estimates.23 36 37 A number of studies have applied this approach, including evaluations of NHS Health Checks, multidisciplinary teams and extended access to primary care.38–40
Limitations
Limitations of the study include residual confounding. We have applied techniques appropriate to the non-experimental design, but cannot entirely eliminate the potential for residual confounding.41 A further issue is data availability, most particularly the lack of data on health outcomes or patient experience—specifically, health outcomes data were insufficient for including in a DiD design. We were also limited in the number of available comparator groups, with a greater number of CAMHS as controls we could employ other techniques, such as synthetic matching which would permit us to match on outcomes in the preperiod.42 Although, trends appeared broadly consistent across the groups, the major exception was waiting times for the first contact. We sought to control for this using a differential trend, although this may be too restrictive an assumption.43 We also conducted a sensitivity analysis restricting the pretreatment period. Future research could explore a more flexible difference-in-differences model to better understand the sensitivity of the estimate to pretreatment dynamics.44 Finally, the intervention took place in a relatively affluent part of England, raising questions as to the extent to which results can be extrapolated for other CAMHS in the country.
Implications
This study looked at one of the early adopters of CAMHS transformations. As such, the findings should be used to help steer transformations going forward. In addition, there is the need for embedded monitoring of transformations informed by richer data and detail of the transformation objectives. The set of techniques proposed in this study could be applied to evaluate the impacts of transformation elsewhere where the same opportunity exists relating to differential adoption of transformation plans, and also to other service level interventions at various levels.
Conclusions
This study provides the first robust evidence of the impact of CAMHS transformation in England. There are signs that transformation can help CAMHS achieve the objectives of greater access and improved health outcomes, but trade-offs exist among different performance metrics, particularly between access and waiting times. Commissioners and providers should be conscious of any trade-offs when undertaking service redesign and transformation. Improved capture of routine patient outcomes and experience would enhance our understanding of the differential impact of the various components of the transformation.
Patient and Public Involvement
We conducted extensive Patient and Public Involvement (PPI) activities ahead of this study. We presented our plans and questions to the Oxon Young Person’s advisory group (25 members) who gave feedback on the current transformation plan, in particular around how they wanted to access services, concerns about privacy when being seen outside of traditional CAMHS settings and about seeing non-CAMHS professionals. These same questions were asked to a further 15 secondary students. In addition, 60 teachers at different schools were asked what they and their students with mental health problems needed from services. This helped inform our understanding of the context for changes in services.
Acknowledgments
We would like to thank staff at Oxford Health NHS Foundation Trust for their help in acquiring the data for this study.
References
Footnotes
Contributors AT and MF conceived the study. AT was responsible for the study design and analytical methods. SR performed the analysis and drafted the manuscript. All authors have made substantial revisions to earlier drafts and approved the final manuscript.
Funding This research was funded by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care Oxford at Oxford Health NHS Foundation Trust, now recommissioned as NIHR Applied Research Collaboration Oxford and Thames Valley. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care. Additional funding came from the Clinical Commissioning Groups in Oxfordshire and Buckinghamshire.
Competing interests None declared.
Patient consent for publication Not required.
Ethics approval This project was approved as a 'research and service evaluation' by NHS Oxford Health Foundation Trust. Anonymised routinely collected data, which is exempt from requiring formal ethical approval, was analysed retrospectively. A local data sharing protocol governed the transfer of the data.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data may be obtained from a third party and are not publicly available.