Article Text
PDF
Abstract
Introduction Young people in contact with the youth justice system are more likely to present with complex ongoing needs than young people in the general population. To address this, the Framework for Integrated Care (SECURE STAIRS) is being implemented in the Children and Young People's Secure Estate: a ‘whole systems’ approach to support secure settings to develop trauma-informed and relationally based environments, supporting staff to provide consistent, therapeutic care. This paper aims to present the protocol for a national cohort study examining the impact and implementation of this cultural transformation programme.
Methods and analysis A mixed-methods realist evaluation will be conducted. Data collection will take place between August 2018 and December 2020. Eighteen sites will collect routine service activity data and questionnaires completed by young people, parents/guardians and staff. Semi-structured interviews and non-participant observations will be conducted across five qualitative focus sites with young people and staff. An economic evaluation will examine value for money. The results will be triangulated at the analysis stage to gain an in-depth understanding of experiences.
Ethics and dissemination Ethical approval was granted by the Health Research Authority, Her Majesty’s Prison and Probation Service and UCL Ethics Committee. Findings will be disseminated via project reports, site feedback, peer-reviewed journal publications and conference presentations.
- public health
- health economics
- forensic psychiatry
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
The prospective and longitudinal design of the study will allow for more robust generalisations and will increase ecological validity.
The mixed-methods approach with triangulation of quantitative and qualitative results at the analysis stage will contribute to a more integrative and reflective interpretation of the results.
The sampling method and recruitment techniques such as opportunity sampling, are more prone to bias (eg, oversampling particular healthcare professionals, those who are more engaged in the research or less burnt out).
There is a risk of high attrition rates due to young people and staff disengagement or release at later stages of the projects which may impact the data quality.
The duration of the study is limited, and therefore the long-term outcomes may not be captured within the study timeframe.
Introduction
The youth justice system is the part of the overall justice system that focuses solely on children and young people (hereafter ‘young people’), particularly on the prevention of offending behaviour. Young people can also be placed in Secure Children’s Homes (SCHs), by local authorities under a secure welfare order for the protection of themselves and/or others, under Section 25 of the Children Act 1989.1 The Children and Young People Secure Estate (CYPSE) is a collective term for settings where young people sentenced or remanded to custody can be placed: SCHs, Secure Training Centres (STCs) and under-18 Young Offender Institutions (YOIs). There was a monthly average of 859 young people in custody in the CYPSE and 11 900 first time entrants to the youth justice system in 2018–2019.2 ‘First time entrants’ refers to young people entering into the Youth Justice System for the first time, receiving their first reprimand, warning, caution or conviction.
The mental health needs of young people in secure settings have been identified as numerous, complex and severe.3 4 There is considerable agreement in the literature that young people in the youth justice system have high levels of mental health needs. For example, studies in the UK5–7 and internationally8 9 have identified a higher prevalence of mental health difficulties among young people in custody compared with their peers in the general population, leading to poorer short-term and longer-term life outcomes.5 10
Unmet mental health needs are often compounded by other psychosocial adversities, which can be complex and enduring in nature. These include, but are not limited to, self-harm and risk of harm to others, communication difficulties, challenging peer and family relationships, unstable accommodation, being looked after by the local authority, social and school exclusion, and poverty.3 4 11–13
There is a disproportionately high number of young people from Black, Asian and minoritised ethnic groups represented in custodial settings.14–16 This over-representation should not go unnoticed in its impact on young people’s experience of the youth justice system, as young people from minoritised ethnic groups are less likely to be recorded as experiencing mental health and learning difficulties on admission and may therefore have unmet needs in custody.17
With this range of overlapping risk factors, combined with restrictions on liberty, it is unsurprising that young people in custody have been identified as having multiple, severe and ongoing needs.18 Although there have been significant improvements since the early 2000s, there remain concerns about ‘punishing disadvantage’19 and ‘double discrimination’,20 with the potential of the custodial experience to reproduce the trauma associated with adversarial childhood experiences, exacerbating mental health difficulties.21 Understanding care provision within the CYPSE through a trauma-informed lens,22 suggests there may be a risk of exacerbating previous experience of trauma, with practices such as restraint23 and seclusion3 increasing young people’s risk of re-traumatisation and mental health difficulties.18
Past research has highlighted limited resources within the youth justice system leading to significant challenges in meeting the complex needs of this vulnerable group.24 Historically, many of the mental health needs of young people in custodial environments were not addressed, with difficulties often going undetected at reception10 and treatment lacking for those in need.25 Secure settings may be instrumental for change, and an opportunity for young people to engage with services.26 Psychologically informed services are suggested to have an enhanced capacity to influence change through improving relational security between staff and young people.27 Over the last decade, mental health provision for young people in the CYPSE has improved, with significant investment in both resources and practice developments. However, there remains scope for improvement in meeting the needs of young people in secure settings who present with complexity and high risk and whom systems struggle to engage with effectively.28
More recently, there has been a recognised need to shift care delivery for young people within the CYPSE, for example, the ‘Five Year Forward View for Mental Health’.29 The Five Year Forward View is a policy document, which sets out recommendations for change within the National Health Service (NHS) healthcare system in the UK. It represents the views of the NHS’ national leadership, patient groups, clinicians, local communities and frontline NHS leads. Within it, there is a clear rationale to support the development of trauma-informed care, enhancing further the sensitivity and responsiveness to the complex needs of young people in secure settings. As part of the Health and Justice and Specialised Commissioning Children and Young People Mental Health Transformation Workstream,30 a new Framework for Integrated Care, SECURE STAIRS, (SECURE STAIRS is an acronym representing the following principles: staff with the skill sets appropriate to the interventions that are needed. Emotionally resilient staff who are able to remain child-centred in the face of challenging behaviour. Cared for staff; through supervision and support. Understanding across the secure setting of child development, attachment, trauma and other relevant key theories. Reflective system; staff who are able to consider the impact of trauma at all levels. ‘Every Interaction Matters’; a whole system approach. Scoping: the presenting situation is assessed with clarity around the young persons’s pathway and life narrative. Targets: staff, young people and the ‘home’ environment agrees on the goals for the young persons’s time within the secure setting. Activators: all young people have an agreed psycho-bio-social, developmentally informed, multi-factorial formulation (not based on diagnosis) that clarifies what activates problems for the young person. Interventions: specialist and core interventions, driven by the formulation and incorporating the risk assessment. Ensuring interventions are tailored to each young person’s risks and needs with content, intensity and timing of the intervention specified. Review and revise: clear ‘real-life’ outcome monitoring by the secure setting and ‘home’, including the frequency and severity of high-risk behaviours and of movement towards goals, regularly evaluated using a formulation-based approach at multidisciplinary reviews. Sustain: sustainability planning from the outset around maintaining goals upon release and the transition to ‘home’ or other services) is being implemented in 18 secure settings across England. The aim of this transformation programme is to influence and support an organisational culture change to better support trauma-informed and formulation-driven, evidenced based, whole system approaches to creating positive change for young people. This includes training and supervision to support staff to provide trauma-informed, developmentally attuned, psychologically based care, underpinned by multi-agency collaboration and coproduced formulation.18 Culture change in this study describes the fundamental shift in the way care and intervention for young people within the CYPSE is delivered,18 which the SECURE STAIRS Framework for Integrated Care facilitates. It will be driven through improved understanding of trauma and attachment in staff and reflected through improved staff well-being and staff–young people relationships, shifting more towards a trauma-informed therapeutic environment.
While encouraging evidence exists regarding the implementation of innovative trauma-informed care in secure settings, such as the Sanctuary Model,22 the Developmentally informed Attachment, Risk and Trauma approach31 and Trauma Systems Therapy,32 there is insufficient empirical evidence on effective ways to meet the needs of this complex cohort of young people in the CYPSE, leading to the improvement of their well-being and life trajectories. Recent policy in children and young people's mental health calls for more research involving young people with high risk of harm to self and/or others.29 33 The current study aims to help address this gap in the literature by evaluating the implementation of this cultural transformation programme, to explore effectiveness and process. This is the first study of its kind to collect national longitudinal data on young people in the CYPSE.
Aims
The overarching aim is to examine whether the implementation of SECURE STAIRS changes organisational culture in secure settings, to focus on whole-systems approaches to creating positive change for young people. This study aims to explore both the effectiveness and process of the implementation of SECURE STAIRS.
The primary research questions are:
What are the characteristics of young people in settings implementing SECURE STAIRS?
What are the outcomes and experiences of young people in settings implementing SECURE STAIRS? (Includes organisational change).
What are the experiences of staff in settings implementing SECURE STAIRS? (Includes organisational change and processes, eg, training).
What is the initial evidence of cost effectiveness of SECURE STAIRS?
Methods and analysis
Study design and recruitment
To address the aims, a mixed-methods longitudinal prospective Realist Process Evaluation approach will be taken. Realist Process Evaluation is a theory-driven framework,34 focusing on the ways that the context of an intervention interacts with a mechanism of action, such as underlying processes or structures, to produce outcomes. Process evaluations are appropriate for the exploration of complex interventions and structures, typically consisting of both quantitative and qualitative data collection, from multiple data collection strands, allowing for mixed-methods analysis and exploration of findings. A particular strength of process evaluations is the ability of researchers to ascertain what elements of an intervention work, under what circumstances and with whom. This is through an exploration of the main elements of an intervention: (1) the implementation; (2) mechanisms of change and (3) the impact of the context on how the intervention works.35 Process evaluation methodology is suitable for the present study due to the multi-faceted nature of SECURE STAIRS, along with the multi-site organisation of the CYPSE. Underpinning this is the realist approach, which is important, because it enables the researcher to explore theory-driven practices. This position actively addresses the recognition that interventions work for some people in certain circumstances; realist researchers explore the theories, or mechanisms, underpinning interventions to explore the contextual influencers on the mechanisms to engender different outcomes.36
A logic model has been created, in collaboration between the researchers, funders and the appointed steering group. The logic model underpins all elements of the Realist Process Evaluation, through classification of the target, context, mechanisms of change and the expected outcomes of the Framework. The logic model is the foundation for the associated areas of data collection and subsequent data analysis. For example, the routine site data set has been derived from the consideration of what data fields are most appropriate to explore the target and outcomes as identified in the logic model; the interviews topic guides and surveys contents are underpinned by the context, mechanisms and outcome elements of the logic model; and the components of the economic evaluation are focused on the mechanisms and outcome elements of the logic model (see figure 1). To take an example, the target is young people in secure accommodation (research question 1 seeks to explore the characteristics of the young people). The intervention is the Framework for Integrated Care, which seeks to support an organisational culture change by improving staff–young people relationships (context) and environment safety (mechanism) through increasing staff understanding of trauma and the histories of young people in their care (mechanisms). These will be explored within the intervention and change mechanism strands of the logic model; research questions 2 and 3 seek to explore the processes through the experiences of young people and staff, with emphasis on understanding staff–young person and staff relationships, and perceived change over time. The expected outcomes relate to a reduction in risk and offending and an increase in mental health and well-being. This will be explored within the outcome strand of the logic model; research questions 2, 3 and 4 will explore this via exploration of quantitative risk and incident data, as well as interviews with staff and young people.
Logic model.
All 18 sites have been commissioned to fully implement the new Framework for Integrated Care, via a process of mobilisation, transition and transformation. (Oakhill STC was outside of the scope of the project due to not being within the operating regulations of NHS England and NHS Improvement and therefore not receiving NHS England and NHS Improvement commissioned healthcare. Medway STC became a secure school and is no longer a study site). Some sites have been able to implement the Framework at an earlier timepoint than others. Therefore, some sites are considered ‘early implementers’ and others ‘late implementers’.
Quantitative data will be collected at all 18 sites. This includes routine service activity data and questionnaires examining the characteristics, outcomes and experiences of young people in the CYPSE, as well as staff experiences of working in the CYPSE. All young people within the CYPSE will be eligible for inclusion in the routine service activity data. Young people aged ten and older, and their parents/guardians will be eligible to complete questionnaires and will be approached by staff within the CYPSE to participate. Young people unable to provide informed consent (ie, are not Gillick competent) will not be eligible to take part in the questionnaires or interviews. All staff (eg, custodial officers, residential workers, psychologists, administrators, teachers, governors and managers) working in the CYPSE will be eligible to participate in the staff questionnaires.
Qualitative data will be collected from five qualitative focus sites, selected based on their progress (early or late) implementing SECURE STAIRS, site type and geographical spread. The qualitative strand will be used to examine each focus site’s journey through the stages of the phased approach to implementation,37 culture change, and staff and young people’s views and experiences. Young people aged 16 and over, and their parents/guardians, will be eligible to participate in interviews or focus groups. All staff in these sites will be eligible to participate in interviews or focus groups. Non-participant observations will be completed to examine the extent to which services are implementing SECURE STAIRS, and both observations and interviews will be used to understand whether cultural changes are sustained over time. The data collection strands are demonstrated in table 1.
Data collection summary
Local collaborators at the qualitative focus sites will support with recruitment by identifying staff, young people and parents/guardians eligible to participate in the study. Opportunity sampling will be used to recruit staff from sites based on availability and willingness to participate. Additionally, an economic evaluation will take place to examine whether SECURE STAIRS is good value for money.
Realist Process Evaluation is method-neutral, therefore multi-method data collection and analysis will enable us to determine what works for whom, in what circumstances and to what extent.34 Unlike analysis techniques which entail a process of building theory, the chosen framework analysis and succeeding thematic analysis sit within the Realist paradigm; entering into the analysis with a preliminary conceptual framework about the underlying structures and mechanisms, which has been developed in the form of the logic model. Further, exploration of the quantitative data will also be considered in this vein. This reduction of data to the conceptual framework is indicative of analysis which is positioned within the realist paradigm such that the researcher seeks to explore evidence that fits with the prior theory. Further, in realism research, the mixed-methods approach provides numerous sources of information which seek to explore ‘several contingent contexts, to capture a single, external and complex reality’.38 In the present research, this entails the combination of routinely collated data, survey responses and qualitative interviews. In the present research, this also includes the triangulation of the data derived from qualitative and quantitative sources to construct a picture of the young people, the settings and experiences of the implementation of the Framework. The analysis will be conducted in such a way that the combinations of—and interactions between—the underlying mechanisms and contexts are given suitable consideration. Ultimately, the data analysis will be concentrated on interpretations of the data to focus on underlying structures and mechanisms in the logic model. The aim of the analysis itself, and the interpretation, is not to understand causality, given that all findings will be strongly influenced by the context, rather the data will generate generalisable knowledge based on an exploration of the reality of young people and staff, drawing out key elements of the Framework which act as the ‘tip of the iceberg’39 of the unobserved and unobservable reality in the CYPSE across England.
Sample size
For the quantitative data, with 5% significance and 80% power, we will need to recruit 39 young people in the early implementer sites and 39 young people in the late implementer sites to be able to detect a clinically meaningful difference in mental health using our primary outcome measure, the Child/Outcome Rating Scale (C/ORS).40 Young people in the late implementer sites have an estimated score in the clinical range (ie, 28) and in the early implementer sites have a predicted score in the non-clinical range (ie, 33) (note: higher scores indicate better mental health). As the data are clustered within services and we estimate the intraclass correlation coefficient to be 0.05, we would need to inflate our sample size calculation by a design effect of 2.76. We would need to recruit a minimum of 7 sites and 140 young people in the early implementer sites and 7 sites and 140 young people in the late implementer sites (ie, 14 services and 280 young people) in order to detect a clinically significant difference in the primary outcome measure.
For qualitative data collection, anticipated recruitment is 7–10 staff and 10–15 young people in each focus site at three time points over the study timeframe. The sample size was determined based on the research team’s rich experience of conducting similar interviews to achieve data saturation.
Patient and public involvement
Young people and local collaborators will be involved in all stages of the project, from designing materials, informing data collection, analysis and reporting. An expert panel, including young people with lived experience, along with a steering group with expert clinicians, will be included in all stages of the study. Quarterly steering group meetings and frequent parent and young people’s panel meetings will take place across the study duration. Young people and local collaborators will be involved in the dissemination of findings.
Hypotheses and independent and dependent variables
The hypotheses that are being tested explore both the effectiveness and process. These are: (1) Young people in the CYPSE present with very high levels of risk and contextual factors. (2) (a) Young people in early implementer sites will have more positive outcomes than those in late implementer sites; (b) risk and harmful behaviours will be reduced over the course of the implementation. (3) (a) Staff will have positive experiences of implementing SECURE STAIRS including direct examples of organisational culture change; (b) there will be unique barriers and facilitators of implementation for different settings. (4) SECURE STAIRS will demonstrate initial cost effectiveness in terms of quality adjusted life years.
The independent variables being explored are: (1) site status (early/late implementer); (2) staff relationships with young people (explored through qualitative data collection); (3) time (three data collection time points); (4) staff burnout (emotionally resilient staff explored through survey data collection); (5) adherence to formulation protocol (explored through qualitative data collection); (6) costs (based on literature reviews and implementation data).
There are several dependent variables which include: (1) reduction in risk at the individual and site level (as coded by the number of risks and incidence data); (2) increased mental health and well-being outcome (as coded by aggregate scores on the relevant measures); (3) quality adjusted life years (as coded by the EQ-5D-Y); (4) experiences of implementation (as coded in qualitative data).
Data and analysis plan
Figure 2 and table 2 provide a summary of the assessments and assessment schedule. Further detail is provided in figure 2, table 2, including whether data assesses contexts, mechanisms or outcomes.
Sources of data collection.
Assessment summary
Quantitative data
Routine service data
Service activity data will be collected by each site as part of routine practice and submitted at three timepoints: early, mid and late implementation stages. All data will be anonymised and shared in line with data sharing agreements between sites and the research team. The routine data will be broadly split into individual-level and event-level data, with all events linked to an individual. The dataset is divided into seven sections: demographics (ie, ethnicity, gender and placement-related information) (context), risk assessment (ie, self-harm, violent behaviour and substance misuse) and information about formulations (mechanisms). Information about moves, incentives and earned privileges, incidents, separations and restraints including length and reason will be collected.
The routine data will include some clinical data from the Comprehensive Health Assessment Tool (CHAT41) (context), a standardised screening and assessment tool for all young people (aged 10–18) throughout the Youth Custody Service, aiding early identification of needs, improving continuity of care and reducing duplication during transition periods. The CHAT consists of five broad assessments: reception screening, physical health, mental health, substance misuse and neuro-disability, from which some items will be drawn.
Young person and parent/guardian questionnaires
The young person and parent/guardian questionnaires will be administered at two timepoints: on entry and 4–6 months later or at release, whichever is soonest. They will address quality of life, well-being and experience of care. Bespoke young person and parent/guardian questionnaires have been designed by the research team and steering group to capture demographic and background information such as age, gender, ethnicity and prior contact with services. Additionally, the questionnaire includes standardised outcome and experience measures, including a one-item help-seeking questionnaire.42 The following key domains and their associated standardised self-report measures are included in the questionnaire and collected at the two timepoints, except the Experience of Service Questionnaire (ESQ) which is collected at one timepoint only.
Mental health and well-being (outcome)
To measure mental health and well-being, the Child/Outcome Rating Scale (C/ORS)40 will be used. The C/ORS is a four-item self-rated visual analogue scale that assesses symptom distress, interpersonal relationships, functioning and global well-being. Young people are asked to indicate their responses to four questions targeting these areas of functioning, for example, ‘How am I doing?’, by marking a cross (‘x’) on four different lines, each 10 cm in length. It is widely used in youth mental health research, has established clinical cut offs, and has demonstrated reliability and validity.43 The C/ORS displays strong evidence of reliability, with coefficient alpha estimates of 0.97 for internal consistency.43 This is a very high coefficient of reliability for such a brief measure, suggesting the ORS taps into global distress.40 43
Quality of life (outcome)
The EQ-5D-Y44 is a version of the EQ-5D self-report measure of health status and quality of life adapted for young people, containing the EQ-5D-Y descriptive system and the EQ Visual Analogue Scale (EQ VAS). The descriptive system comprises five dimensions of quality of life: mobility, looking after myself, doing usual activities, having pain or discomfort, and feeling worried, sad or unhappy. These dimensions are scored on a 3-point scale: ‘no problems’, ‘some problems’, ‘a lot of problems’. The EQ VAS records the respondent’s overall health on a vertical, visual analogue scale where the endpoints are labelled ‘the best health you can imagine’ and ‘the worst health you can imagine’. Research has tested the correlation and convergent validity of the EQ-5D-Y, yielding good results.45 Several studies have used the EQ-5D to describe the health of a population, effects of interventions and cost-effectiveness analyses.46 The EQ-5D-Y will be used for the economic analysis.
Service climate and experience (mechanism)
The EssenCES47 is a measure of social and therapeutic climate on psychiatric wards and prison units. The five-item Therapeutic Hold subscale will be included in the current study, for example, ‘Staff know young people and their personal histories very well’. Respondents are asked to score each item on a 5-point scale ranging from ‘not at all’, to ‘very much’. Research has found satisfactory internal consistency for all EssenCES subscales and supports its construct, convergent and divergent validity.48 49
Seven additional bespoke items relating to experienced safety, care involvement and access to services have also been included by the research team and steering group. Sample items include: ‘I feel involved in my care’ and ‘I feel safe in this setting’.
Satisfaction with care (mechanism)
Satisfaction with care will be measured using the age 9–11 self-report ESQ,50 which assesses young people’s satisfaction with their care and the environment.51 Eight of the original 12 items (statements 1–7 and 12) were deemed appropriate for use in the current study, for example, ‘Were your views and worries taken seriously?’. Respondents are asked to rate their agreement with questions on a 4-point scale of either ‘yes’, ‘only a little’, ‘not really’ or ‘don’t know’. Four of the included items will be used as a proxy for measuring shared decision-making, as administered in previous research.52 Two additional bespoke questions relating to experience of care are also included, for example, ‘Did you know why everyone was there and what their job was?’.
Staff questionnaires
Questionnaires will be collected from staff at three timepoints during the study: early, mid and late implementation stages. Demographic information and bespoke items about staff understanding of SECURE STAIRS principles, training, supervision and self-efficacy will be included, for example, ‘SECURE STAIRS has increased my understanding of young people’s needs’. The questionnaire also includes items drawn from standardised measures in the following key domains.
Staff satisfaction (mechanism)
Measured using the NHS Staff Survey 2017, an independent survey of employees’ experience of working. It captures background information, attitudes towards work, information on their health, well-being and safety at work, personal development and organisation. For the current study, items on staff satisfaction and experience have been extracted.
Service climate (mechanism)
To measure team function and service climate, the ‘Participatory Safety’ and ‘Support for Innovation’ subscales from the shortened version of the Team Climate Inventory (TCI53) will be used, for example, ‘we have a ‘we are together’ attitude’. Respondents are asked to score each item on a 5-point scale ranging from ‘strongly disagree’ to ‘strongly agree’. Research has supported the internal homogeneity, reliability and normality of the scales, and suggests comparative predictive validity between the shortened TCI and the original version.54 To measure social and therapeutic climate, staff will be asked to complete the Therapeutic Hold subscale of the EssenCES.47 See the Service climate and experience subsection within the Young person or parent/guardian questionnaires section for a description of this measure and its psychometric properties.
Burnout (mechanism)
The Copenhagen Burnout Inventory (CBI55) is a 19-item tool for the measurement of burnout consisting of three scales: personal burnout, work-related burnout and client-related burnout. Respondents are asked to score 12 items on a 5-point scale ranging from ‘always’, to ‘never/almost never’. Respondents are asked to score the remaining seven items on a 5-point scale ranging from ‘to a very high degree’, to ‘to a very low degree’. All three scales of the CBI have been found to have very high internal reliability, validity and low non-response rates.55
Quantitative data analysis plan
Descriptive statistical analysis will be conducted using the anonymised routine service data and questionnaire responses. The primary analysis will be a multilevel regression predicting change in mental health using the Child/Outcome Rating Scale (C/ORS)40 with early versus late implementation as a predictor variable, accounting for the clustered structure of the data (with time clustered within young people clustered within sites) and controlling for covariates (eg, age, gender, severity and ethnicity). Site-level characteristics (such as proportional characteristics of the group of young people) may be included depending on data quality.
Qualitative data
Qualitative data will be collected from staff and young people at the five focus sites at three timepoints during the study: early, mid and late implementation stages. Qualitative data will primarily collect information on contexts and mechanisms.
Young people
Semi-structured interviews, or focus groups where preferable, will be conducted with young people. The topic guide will explore young people’s perspectives on culture change, their views and experiences of the secure settings, formulations and relationships with staff and peers.
Staff
Semi-structured interviews and focus groups will be conducted with staff. The interview schedule will explore services’ journey through the stages of implementation, barriers and facilitators to implementation, the extent to which culture change is occurring, as well as staff experience of working in secure settings, using formulations, relationships and well-being.
Qualitative data analysis plan
Staff and young people interviews and focus groups will be audio-recorded and transcribed verbatim. Transcripts will be analysed using a thematic analysis approach56 and coded using NVivo software according to established methods for qualitative analysis validation.57 A combined inductive and deductive coding approach will be used58 as themes will be derived from interviews with staff and young people or parents/guardians, as well as the study’s logic model (see figure 1). Data will be compared based on implementation stage, as well as across site type to identify any similarities and differences between these groups.
Implementation data
Non-participant observations
The research team will conduct non-participant observations at early, mid and late evaluation stages using a bespoke observational tool modelled on the Huddle Observation Tool.59 The tool assesses team processes during meetings (eg, SECURE STAIRS Implementation Group, Multidisciplinary Team (MDT) and Case Review meetings) and consists of four domains, each of which is rated on a 5-point scale, from 1 (strongly disagree) to 5 (strongly agree), with free text response sections for notes: structure (ie, ‘the MDT meeting followed a clear structure’), formulation-driven (ie, ‘the meeting was focused on understanding the young person’s story’), collaborative culture (ie, ‘everyone had the opportunity to contribute and all points of view were respected’) and risk management (ie, ‘there were opportunities to identify risks and discuss concrete plans to mitigate these risks’).
Economic evaluation
An economic evaluation will be conducted alongside the current study to assist policy decision-making and resource allocation and will take a pragmatic societal perspective using a mixed-methods approach.
Economic evaluation data
The primary economic outcomes for young people are: health-related quality of life defined as a state of complete physical, mental and social well-being, and not merely the absence of disease or infirmity; and other indicators of life chances.60
Appropriate structure, process, service data and/or clinical outcome data collected in the study or by the service will inform the analysis. The analysis will also draw on data from two systematic reviews to find: published utility scores for this population; and costs, taking a pragmatic societal approach, outside of direct services, in relation to prioritised outcomes.
Economic evaluation analysis plan
A partial economic model/s will be developed that demonstrates the impact on health and well-being due to the new culture compared with before/late implementation using cost–utility analysis to compare part of the impact of the framework SECURE STAIRS. Cost–utility analysis allows consideration of new or changed resource use (eg, staff time) to asses if they are good value for money in relation to improved outcomes for young people.
A development panel of representatives from young people with lived experience, clinical experts and staff in the service will be formed to support development of the analysis, interpret findings and collaboratively make value judgments about the initial cost effectiveness of the framework SECURE STAIRS on improving outcomes for staff and young people.
Ethics and dissemination
Ethical approval has been granted by the London–South East Research Ethics Committee and the Health Research Authority for the data collection with young people or parents/guardians (IRAS project ID: 242383; REC reference: 18/LO/1569). Her Majesty’s Prison and Probation Service granted ethical approval for data collection with young people and parents/guardians (2018-274) and staff (2018-335). Staff data collection has received ethical approval from UCL Ethics Committee (6087/007). Local approval was also given by each NHS Trust’s Research and Development office. All data will be deposited and curated within the UCL Data Safe Haven.61
At the time of publication, the study has progressed to phase 3 of data collection, due to delays in manuscript preparation and sign off procedures. The results will be disseminated via reports to the funder (NHS England and NHS Improvement), and more widely through peer-reviewed journal publications and conference presentations (regional and international). Participating sites will receive aggregated reports at key points during the project, which will feedback key findings and recommendations. It is anticipated that findings will contribute to commissioning decisions nationally and add to the evidence base on the characteristics and needs of young people and staff in the CYPSE.
Discussion
The limitations of this study include the sampling method and recruitment techniques such as opportunity sampling, which while the most feasible for the study setting, are more prone to bias. This may result in recruitment of participants that are innately more interested and engaged in the research and thus the Framework being evaluated. There is also a risk of high attrition rates due to young people and staff disengagement, or release, at later stages of the projects which may impact the data quality. Finally, the duration of the study is limited, and therefore the long-term outcomes may not be captured within the study timeframe, although the economics evaluation will seek to explore the likelihood of prospective outcomes. However, the longitudinal design of the study will allow for more robust generalisations and will increase the ecological validity. A further considerable strength of the study is the mixed-methods design, which will triangulate the quantitative and qualitative results at the analysis stage. This will in turn contribute to a more integrative and reflective interpretation of the results.
Acknowledgments
The authors would like to thank the young people at Ardenleigh Hospital for their help refining the study PIS and consent forms. The authors would also like to thank all members of CORC, its Board at the time of writing—(includes DL) Ashley Wyatt, Beth Ingram, Ann York, Kate Dalzell, Isobel Fleming, Kate Martin, Mick Atkinson, Amy Mary Rose Herring and Rebecca Lewis—and the CORC team at the time of writing: (includes JJ, SD, ARashid, RS and JEC) Kate Dalzell, Ben Ritchie, Nick Tait, Martha Reilly, Sally Marriott, Lee Atkins, Rachel Piper, Anja Teichert, Luís Costa da Silva and Florence Ruby.
References
Footnotes
Twitter @JennaMBJacob, @dickonb
Contributors The protocol was conceived and developed by all authors. The manuscript was drafted by SD, RL, ML and WR, reviewed and edited by JJ, ARogers and JEC and reviewed by RU, ARashid, RS, JW, DB, JD, PFonagy, PFuggle and DL. Revisions to the manuscript were made by JJ, SD, RL and JEC.
Funding This work is supported by NHS England and NHS Improvement and is sponsored by University College London (UCL). No grant number is available.
Competing interests ARogers is engaged by the NHS England and NHS Improvement, Health & Justice, Children & Young People Team, to support the development and implementation of the Framework for Integrated Care across the children and young people secure estate. He also acts in an advisory, non-decision-making role to the national evaluation of SECURE STAIRS.
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
Provenance and peer review Not commissioned; externally peer reviewed.