You are here

  • Home
  • Archive
  • Volume 8, Issue 4
  • Children and Young People-Mental Health Safety Assessment Tool (CYP-MH SAT) study: Protocol for the development and psychometric evaluation of an assessment tool to identify immediate risk of self-harm and suicide in children and young people (10–19 year…

Article Text

Article menu

Download PDFPDF

XML
Paediatrics
Protocol
Children and Young People-Mental Health Safety Assessment Tool (CYP-MH SAT) study: Protocol for the development and psychometric evaluation of an assessment tool to identify immediate risk of self-harm and suicide in children and young people (10–19 years) in acute paediatric hospital settings
  1. Joseph C Manning1,2,3,4,
  2. Gemma M Walker1,
  3. Tim Carter2,
  4. Aimee Aubeeluck2,
  5. Miranda Witchell1,
  6. Jane Coad3
  7. The CYP-MH SAT study group
    1. 1 Family Health Division, Nottingham Children’s Hospital, Nottingham University Hospitals NHS Trust, Nottingham, UK
    2. 2 School of Health Sciences, Faculty of Medicine and Health Sciences, University of Nottingham, Nottingham, UK
    3. 3 Centre for Innovative Research Across a Life Course, Faculty of Health and Life Sciences, Coventry University, Coventry, UK
    4. 4 Institute of Nursing and Midwifery Care Excellence, Nottingham University Hospitals NHS Trust, Nottingham, UK
    1. Correspondence to Dr Joseph C Manning; joseph.manning{at}nuh.nhs.uk

    Abstract

    Introduction Currently, no standardised, evidence-based assessment tool for assessing immediate self-harm and suicide in acute paediatric inpatient settings exists.

    Aim The aim of this study is to develop and test the psychometric properties of an assessment tool that identifies immediate risk of self-harm and suicide in children and young people (10–19 years) in acute paediatric hospital settings.

    Methods and analysis Development phase: This phase involved a scoping review of the literature to identify and extract items from previously published suicide and self-harm risk assessment scales. Using a modified electronic Delphi approach, these items will then be rated according to their relevance for assessment of immediate suicide or self-harm risk by expert professionals. Inclusion of items will be determined by 65%–70% consensus between raters. Subsequently, a panel of expert members will convene to determine the face validity, appropriate phrasing, item order and response format for the finalised items.

    Psychometric testing phase: The finalised items will be tested for validity and reliability through a multicentre, psychometric evaluation. Psychometric testing will be undertaken to determine the following: internal consistency, inter-rater reliability, convergent, divergent validity and concurrent validity.

    Ethics and dissemination Ethical approval was provided by the National Health Service East Midlands—Derby Research Ethics Committee (17/EM/0347) and full governance clearance received by the Health Research Authority and local participating sites. Findings from this study will be disseminated to professionals and the public via peer-reviewed journal publications, popular social media and conference presentations.

    • risk management
    • paediatrics
    • mental health

    This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

    http://dx.doi.org/10.1136/bmjopen-2017-020964

    Statistics from Altmetric.com

      Request Permissions

      If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

      Strengths and limitations of this study

      • To our knowledge, this is the first UK-based study to develop an assessment tool to ascertain immediate risk of suicide and self-harm in children and young people presenting to acute paediatric hospital settings in mental health crisis.

      • This study seeks to address the limitations of established assessment tools of immediate risk of suicide and self-harm in children and young people, in adherence to published psychometric evaluation guidelines.

      • This protocol is tailored towards an English-speaking population, therefore, further national and international testing and adaptations may be required to achieve wider generalisability.

      • An opportunistic sample of self-selected experts will be used to inform the construction of the suicide and self-harm risk assessment.

      • They may however have more motivation to take part than other populations and/or possess preconceived investments leading them to respond in a particular way.

      Introduction 

      Mental health (MH) crisis has been defined as ‘an acute disruption of psychological homeostasis in which one’s usual coping mechanisms fail and there is distress and functional impairment’.1 This may include: extreme anxiety or panic attacks; psychotic episodes (including delusions, hallucinations, paranoia and/or hearing voices); hypomania or mania; other behaviours that feel out of control and acts of suicide or self-harm.2 MH crisis is recognised as a psychiatric emergency as it is ‘an acute disturbance of thought, mood, behaviour or social relationship that requires immediate intervention’.3 It is estimated to be the primary cause of around 5% of emergency department (ED) attendances4 with the most prevalent presenting conditions being self-harm and/or suicidal behaviours. For these conditions alone, data for children and young people (CYP) aged 10–19 years in England indicates the prevalence of suicide is 4.3 per 100 0005 and self-harm is 435.95 per 100 000.6

      Care pathway for CYP in MH crisis

      Nationally the care pathway for CYP in MH crisis varies considerably. The majority of acute and initial care is delivered by non-MH professionals in general healthcare settings,7 such as paediatricians and registered children’s nurses. Whereas specialist mental healthcare is delivered by professionals (including psychiatrists, nurses, social workers and psychologist) with specialist training, skills and knowledge in working with CYP with MH difficulties.8

      For CYP, initial triage and care is delivered by paediatricians and registered children’s nurses working in acute paediatric settings,9 including EDs and paediatric inpatient wards. The main focus of this initial part of the CYP care pathway is to ensure that they are appropriately assessed to direct a plan of care and to ensure that they are safe until definitive and expert MH assessment is undertaken.

      Professionals who provide care for CYP experiencing MH crisis identify, it can be one of the most complex and stressful duties undertaken in practice.10 This could be attributed to the acuity of the child’s status, as crisis is defined as a psychiatric emergency. However, health professionals’ responses could also be governed by the perception that poor experience and outcome of this acute phase of the CYP care pathway may negatively impact on their adherence with follow-up and future MH. In turn, there is accumulating evidence that non-adherence to follow-up is a predictor of poor outcomes, in terms of repeated self-harm and suicide but also in a variety of other psychosocial outcomes.

      The assessment of CYP experiencing a MH crisis should involve: the identification of the main clinical and demographic features known to be associated with their MH crisis and identification of the key psychological characteristics associated with risk, in particular depression, hopelessness and continuing suicidal intent.11 In addition, any immediate physical health needs (such as wound care, use of antidotes, gastric irrigation and psychotropic medication should be addressed)12; and their safety should be maintained until expert assessment is undertaken by specialist MH professionals. However, in order to implement a plan of care where attempts can be made to mitigate immediate risks, health professionals (such as paediatricians and children’s nurses) need to be supported to make an informed assessment. Invariably this will have to be made in time limited circumstances and with CYP with potentially changing MH status. Therefore, the assessment should focus on identifying the most pertinent risks (ie, immediate risk of self-harm or suicide) and take into account risk factors, coping abilities and assessment of lethality of previous suicidal and self-harm behaviour.13 14 Such factors can be useful to differentiate between high-risk and low-risk suicidal and self-harm behaviours.15

      Assessing risk of self-harm and suicidal intent

      Evidence suggests that risk assessments are no more accurate at predicting risk than expert specialist MH professional clinical judgement in non-acute psychiatric outpatients.16 However, acute paediatric care settings present specific differences in utility, focus and context that make the application of an assessment of suicide and self-harm unique.7 For example, the assessment is usually made by non-MH experts who may lack specialist knowledge and experience to inform clinical decisions. Furthermore, assessments are focused on immediate risks of self-harm or suicide during receipt of acute paediatric care; not projected to weeks, months or years. Additionally, assessments are performed in time limited circumstances with CYP with potentially dynamic and fluctuating MH. Therefore, to enable implementation of a plan of care where immediate risks can be mitigated, healthcare professionals require appropriate support and guidance to inform their assessment. In current practice, risk assessments of CYP suicidal or self-harm behaviour in paediatric inpatient departments are typically undertaken through a variety of non-validated tools that are often created by the specific department, and then generally completed by non-MH clinicians.17 However, some acute paediatric departments do use previously validated suicide risk assessment tools, but these tools have not been validated in populations presenting to acute paediatric care, very rarely assess immediate suicide risk and rarely include items on self-harm.

      As such, across the UK, there is currently no standardised, specifically designed risk assessment tool being used within paediatric inpatient settings that assesses the immediate risk of self-harm or suicide for CYP experiencing MH crisis.

      The aim of this study is to develop and provide an initial psychometric evaluation of an evidence-based tool to assess the immediate safety of CYP (aged 10–19 years) in MH crisis presenting to acute paediatric inpatient hospital settings.

      Primary objectives

      1. To identify relevant evidence-based self-harm and suicide risk assessment tools from published literature.

      2. To identify, through expert consensus, the most pertinent items for inclusion in a CYP-Mental Health Safety Assessment Tool (CYP-MH SAT) to assess immediate risk of self-harm and suicide in CYP experiencing MH crisis.

      3. To establish construct validity of a CYP-MH SAT through agreement from an expert panel.

      4. To establish the psychometric properties of CYP-MH SAT in CYP presenting in MH crisis to the ED and inpatient wards in acute paediatric hospital settings.

      Methods

      This study incorporates three phases as outlined in figure 1:

      Figure 1
      Figure 1

      Children and Young People (CYP)-Mental Health Safety Assessment Tool study flow chart. C-SSRS, Columbia suicide severity rating scale.

      Phase 1: scoping review of existing evidence

      The possible domains for the development of a CYP-MH SAT to be used by non-expert MH professionals in an acute paediatric setting were identified from clinician feedback and themes identified from the investigation and root cause analysis of the high-level incidents at a local National Health Service (NHS) Trust. From this, the immediate risk of (1) self-harm and (2) suicidal behaviour were identified as the most important factors to be assessed by the tool.

      Subsequently, we conducted a scoping review of published literature to identify evidence-based items used to assess risk of self-harm and/or suicide in CYP in acute inpatient settings. From this review, 22 assessment tools were identified, of which 12 were accessible and retrieved.18–29

      Importantly, none of 22 assessment tools identified in this scoping review were specifically developed for and tested on a population accessing paediatric inpatient care in MH crisis. Thus, all items from the 12 accessible assessment tools were collated and grouped into thematically similar subject headings using framework analysis.30 Following removal of duplicate items and collapsing of similar items, a total of 93 items were identified (themes, subthemes and items are outlined in online supplementary table 1). These items identified from the literature will now be used to inform the preliminary round of the Delphi survey, proposed to be undertaken in phase 2a.

      Supplementary file 1

      Phase 2a: Delphi survey

      The purpose of this phase is to gain consensus of the relevance of items identified from the scoping review (phase 1) using the Delphi method. The Delphi method involves successive questioning with a panel of experts and key stakeholders to develop consensus regarding a particular issue.31 For this study, experts and key stakeholders will be requested to rank the items according to their perceived relevance. Expertise will be defined as having experience of delivering care to CYP in MH crisis or having expertise in CYP MH. In order to canvass the views of a range of experts and key stakeholders, no stipulation will be made as to how long they had been working in this field. However, information pertaining to the characteristics of participants was collected to explore and contextualise the results.

      To form the panel of experts and key stakeholders, we will aim to recruit a purposive sample of 20–30 professionals including: Psychiatrists, Paediatricians, Nurses, Clinical Support Workers, Academics, Public and Patient Involvement representatives. Participants will be recruited from three NHS Trusts and two non-NHS organisations (Higher Education Institution and Local council). We would like opinions from a range of differing staff types/grades and aim to recruit equal proportions of each staff type to prevent over-representation from one particular view point possibly shared by a group of individuals sharing the same characteristic (such as job type).32

      Staff will be invited to take part by the site Local Collaborator via an email containing a participant information sheet, instructions for questionnaire completion and a link to a modified electronic Delphi survey (Bristol Online Survey: BOS software). Staff will be informed that completion of the survey indicates consent to take part.

      The survey will consist of two sections: (1) demographics: email address, job title, staff grade, employing organisation, gender, date of birth and ethnicity; and (2) the list of items generated from phase 1 (see online supplementary table 1). Participants will be asked to rate each item in relation to its relevance in assessing immediate risk of self-harm and/or suicide for CYP presenting in MH crisis to acute paediatric hospital settings. Relevance to the assessment of immediate risk of suicide and/or self-harm for CYP in MH crisis in acute paediatric inpatient settings will be reported on a 4-point Likert scale (1-not relevant, 2-somewhat relevant, 3-quite relevant, 4-highly relevant)33 using a closed questioning procedure. The survey also allows participants to type any items they think should be included using open-ended questioning. The survey will take no longer than 15 minutes to complete and participants have 2 weeks to complete it.34 Staff will receive an email reminder 1 week before the return due date. Newly identified items from open-ended questioning will be included in the second iteration of the questionnaire to decipher consensus on the new items. Items will be removed if they do not achieve the prespecified consensus margin (≥65%–70%; median ≥3.25).35 Only items within the prespecific consensus margin will be redistributed in successive Delphi survey rounds. The survey will be distributed in three rounds to achieve consensus.36

      Phase 2b: nominal group technique

      The nominal group technique is widely used for problem identification, solution generation and decision-making37 and can be used in groups of varying sizes.38 In this phase, respondents from phase 2a will be invited to attend a 2-hour meeting and will be asked to provide their opinions regarding face validity of the CYP-MH SAT.39 Group facilitators (JCM, GMW and TC) will facilitate discussions regarding any required changes to the tool including item phrasing, attribution of suitable rating scales and question order. When disagreements occur, opinions with the highest consensus in the group will be prioritised. The CYP-MH SAT will be considered finalised for further psychometric testing once the changes subject to expert opinion have been implemented.

      Phase 3: psychometric evaluative study

      The study is a multicentre, psychometric evaluation study.

      Eligibility and recruitment

      CYP ‘cases’ and ‘non-cases’ will be recruited from acute paediatric hospital settings (EDs and wards) across three NHS trusts in the East Midlands, UK. CYP will be considered eligible for recruitment based on the criteria outlined in box 1.

      Box 1

      Eligibility criteria for entry into phase 3

      Inclusion criteria

      • Aged 10–19 years.

      • Approved National Health Service Trust site.

      • Cases: Presenting to acute paediatric hospital setting in mental health crisis as defined as: having reached ‘breaking point’, likely to harm themselves or others and behaviours that feel out of control, including: extreme anxiety or panic attacks; psychotic episodes (including delusions, hallucinations, paranoia and/or hearing voices); hypomania or mania and acts of suicide or self-harm; those who have engaged in non-suicidal self-injury or low-lethality self-harm.

      • Non-cases: Presenting to acute paediatric setting not in mental health crisis but presenting with a primary physical medical illness or injury. This is defined as a health problem having physical origins, which can be short term (eg, accidental injury, influenza, migraine, infections) or long term (eg, diabetes, asthma, arthritis).

      • Mental capacity to consent.

      • Parental/legal guardian consent and child assent (under 16 years old).

      • Young person consent (over 16 years).

      Exclusion criteria

      • Unable to speak or comprehend English language.

      • Currently receiving active end-of-life care or considered too medically ill by clinicians.

      • Non-cases: Current mental health crisis.

      Registered clinical staff will make the initial approach to participants. The informed consent process will then be performed by trained registered clinical staff or research team members.

      Sample size and justification

      Based on needing 4–10 participants per questionnaire item in order to conduct the psychometric testing procedures,40 100–200 cases and 100–200 non-cases are required. We envisage the CYP-MH SAT that will be evaluated in phase 3 will not exceed 20 items.

      Measures/outcomes

      Two measures will be used in this study:

      1. The CYP-MH SAT: This measure is being developed as part of this study (phases 1 and 2) and aims to assess immediate risk of self-harm and immediate risk of suicidality for CYP (aged 10–19 years) presenting to acute paediatric care. The total number of items and face validity will be established on completion of phase 2b of this study. The CYP-MH SAT will be completed with both cases and non-cases.

      2. The Columbia Suicide Severity Rating Scale (C-SSRS): is a clinician rated, 19-item, measure of suicide risk.41 The C-SSRS is considered the current gold-standard assessment for suicide risk.42 The C-SSRS has good predictive validity, discriminant validity, convergent validity, and sensitivity and specificity.43 44 The C-SSRS will be completed with cases only to enable assessment of the level of agreement (convergent validity) with the CYP-MH SAT.

      In addition to the measures, demographic details (date of birth, gender, ethnicity, NHS site, site setting, hospital number), primary reason for presentation to hospital (International Classification of Diseases-10 categories), MH crisis hospital presentation in the past 6 months and treating physician contact details will also be collected. All data will be collected by registered clinical staff members located in EDs and paediatric wards across three sites. All registered clinical staff members who take part in the study will have received CYP-MH SAT training. The specific training required to administer the tool will be dependent on its content and format that will be determined during the development of this study. The study steering group (composed of clinical experts) will provide guidance on the development of training resources. It is anticipated that a minimum of five people per clinical area per site (n=circa 30) registered clinical staff members will administer the tool. We have processes in place to ensure that data will only be collected once from participants during their inpatient admission. However, if they are admitted again during the data collection period and consent to participate in the study then they will be counted as a new participant. Data will be collected from October 2017 to May 2018 (as shown in study timeline, figure 2), with recruitment ending earlier if sample size achieved.

      Figure 2
      Figure 2

      Study timeline.

      Patient and public involvement

      The inception and design of this study has been informed through meaningful engagement with patient, public and expert professionals. CYP, their parents and carers, and professionals from both acute paediatric care and child and adolescent mental health services have informed the topic and focus of the study through previous stakeholder engagement led by JCM and funded by NHS England (link to full report: http://eprints.nottingham.ac.uk/35284/1/Manningetal2015PCOMNHSEnglandFinal.pdf). In the development of this study, we have involved CYP to inform the focus, design and development of study information such as the participant information sheets.

      We will continually refer to guidance on involving the public in research can be on the INVOLVE website (see: http://www.invo.org.uk/). We have referred to the British Council ‘Learn English’ for advice regarding writing for lay audiences and specific topic areas (see: https://learnenglish.britishcouncil.org/en/writing) and the involve resource page (see: http://www.invo.org.uk/makeitclear/support-and-resources/). These resources have and will be fully used in the construction of study documents, procedures and dissemination activities.

      In addition, a group of clinical and research experts have been assembled to steer the direction of this study. This includes staff members who provide care for CYP experiencing MH problems and crisis events on a daily basis in a range of settings that include NHS, Local Government and charity organisations.

      Analyses

      Data will be analysed using IBM SPSS (version 22) using statistical techniques appropriate for cross-sectional research. For all study phases, counts (n) and proportions (%) will be calculated for categorical variables and descriptive statistics (mean, median, SD, range, IQR) for continuous variables.

      Phase 2a/b

      Mixed-methods analyses will be employed. Consensus agreement will be calculated using counts (n) and proportions (%) for a median relevance rating of 3.25 or above. Item content validity ratios (I-CVR)45 and Content Validity Indices (I/S-CVI)46 will be displayed as a decimal >0 and <1 (see figure 3). The CVR is a linear transformation of a proportional level of agreement on how many ‘experts’ within a panel rate an item as being ‘highly relevant’.47 A CVI demonstrates the proportion of staff ‘in agreement’ about the relevance (scoring either 3 or 4) of individual items.46 Field notes will be presented in narrative form48 to describe consensually agreed decision processes during the nominal group meeting.

      Figure 3
      Figure 3

      Content validity equations. I-CVR, Item content validity ratio; I/S-CVI, content validity indices.

      Phase 3

      A standard psychometric approach49 50 will be taken to test the psychometric properties (ie, the reliability and validity) of the CYP-MH SAT. The factor structure will be analysed through exploratory factor analysis to explore the most parsimonious factor structure of the scale. Equivalence reliability will be assessed through Cronbach alpha (α). Concurrent validity will be determined through Pearson’s correlation coefficients (or Spearman’s rank for non-parametric data) between the CYP-MH SAT and the C-SSRS. Inter-rater reliability will be established through Kappa coefficients (K) between CYP-MH SAT and inter-rater version scores from Case data (ie, two healthcare professionals will complete the assessment for each CYP to allow for a comparison between assessor scores). Convergent, divergent and discriminant validity will be determined by InterItem Correlations within and between assessment items and subscales. In order to assess convergent validity, we will correlate total scale scores to explore the relationship between the CYP-MH SAT and the C-SSRS. Discriminant/divergent validity will be ascertained through the correlation of case and non-case total scores where we would expect the scale to be able to differentiate between those presenting at the ED and children’s inpatient wards in MH crisis and those presenting at the ED and children’s inpatient wards not in MH crisis. In the case of non-normally distributed data, equivalent non-parametric analyses will be undertaken.

      Ethics and dissemination

      Ethical review

      The study is sponsored by the Nottingham University Hospitals NHS Trust, Research and Innovation department and funded by Nottinghamshire Clinical Commissioning Group. Neither the sponsor nor the funders will be involved in the analysis of study data or report writing.

      The study did not commence prior to any approval mechanisms from the Sponsor Health Research Authority/Research Ethics Committee and Research and Development authorities. All study procedures will be conducted in line with Good Clinical Practice and NHS research ethical guidance.

      Informed consent

      All participants will be fully informed about the study prior to any participation in research activities in the form of a patient information sheet and have the opportunity to ask questions throughout the study.

      For phases 2a and 2b of the study, staff consent will be implied through electronic completion of the modified e-Delphi questionnaire (phase 2a) and through staff attendance at the modified nominal group technique meeting (phase 2b).

      For phase 3, all participation begins after a consent form has been signed and dated. In compliance with the Mental Capacity Act,51 nominee parental consent will be provided for children under 16 years with the opportunity to provide written assent for themselves. Young people over age 16 can consent for themselves unless assessed by the clinician to be too medically ill or mentally incapacitated. Written informed consent will indicate consent to notify the CYP’s hospital treating physician that they are participating in the study. All information will remain confidential and anonymised within the study team and regulatory authorities, unless there is indication of any risk of harm posed to the participant or others around them whereby members of the care team will be informed and local NHS reporting procedures followed. Anonymity will be ensured through the allocation of a study identification number to corresponding case report forms for each participant and separate storage of any patient identifiable data. All data will be stored on a secure server in encrypted format, with limited access permissions.

      Potential participant distress and burden

      While there is evidence that indicates performing a self-harm/suicide safety assessment is not harmful,52 we recognise that there is the possibility that the administration of the CYP-MH SAT and C-SSRS may cause some distress to participants. Therefore, only registered clinical staff will be trained in study procedures and will administer the measures. In the event that participants become too distressed their participation will be withdrawn. All participants will be made aware of their right to withdraw at any time and voluntary participation at study conception. All participants will continue to receive usual care from their care team during and following their involvement in the study. No care will be withheld or adapted for any participants taking part.

      Local procedures will be adhered to in the case of participant disclosure throughout the course of the study. All staff will have received safeguarding training and know to share information regarding child protection as part of their professional practice with appropriate authorities (ie, social services).

      Furthermore, for the CYP-MH SAT, we have integrated within the development phase of the tool, mechanisms to ensure that the items are appropriate and phrased sensitively. The e-Delphi procedure will apply a stringent prespecified consensus margin of ≥65%–70% consensus and a median value of 3.25, it is expected this will substantially reduce the amount of question items to enable a rapid assessment and minimise participant exposure. Furthermore, the tool will be scrutinised by the expert panel to ensure appropriate and sensitive terminology and phrasing will be used for each item.

      Methods of disseminating findings

      This paper serves as an important step in the dissemination of the findings by outlining the project background and aims, details of methods used and the practical challenges that may be faced and how these will be overcome.

      We intend that the findings from this study will result in both the development and evaluation of a clinically appropriate and relevant MH SAT for use with CYP in MH crisis. This, in turn, will provide the NHS with an evidence based, assessment tool that could be employed to enhance the safety of CYP experiencing MH crisis admitted to an acute paediatric inpatient setting.

      The research findings will be disseminated to regional, national and international audiences, including service users, clinicians, academics, service commissioners and policy-makers. Study findings will be disseminated in peer-reviewed, high-quality academic and professional journals in accordance with publishing guidelines and provide the opportunity for participants to gain access to the final publication and a summary of findings in lay format. Findings will be presented at local, national and international conferences.

      Acknowledgments

      We would like to thank all the research sites who have supported the study and principal investigators/local collaborators. We would further like to thank the Study Steering Group for providing practice based and academic advice throughout the study. We would also like to acknowledge Ms Dorothy Bean (Divisional Head Nurse, Family Health), Dr Jonathon Evans (Divisional Clinical Director, Family Health), and Dr Stephanie Smith (Family Health and Medical Deputy Divisional Director) for their support in the development and delivery of this study.

      References

      1. 1.↵
        2. Lewis S ,
        3. Roberts AR
        . Crisis assessment tools: the good, the bad, and the available. Brief Treat Crisis Interv 2001;1:17–28.doi:10.1093/brief-treatment/1.1.17
        OpenUrlCrossRef
      2. 2.↵
      3. 3.↵
        2. Allen MH ,
        3. Forster P ,
        4. Zealberg J , et al
        . Report and recommendations regarding psychiatric emergency and crisis services: a review and model program descriptions. Washington DC: American Psychiatric Association Task Force on Psychiatric Emergency Services, 2002.
      4. 4.↵
        2. Parsonage M ,
        3. Fossey M ,
        4. Tutty C
        . Liason psychiatry in the modern NHS. London: Centre for Mental Health, 2012.
      5. 5.↵
        2. Hagell A ,
        3. Coleman J ,
        4. Brooks F
        . Key data on adolescence 2015. London, 2015.
      6. 6.↵
        Public Health England. Children and young people’s mental health and wellbeing: identification of need data: PHE. 2017 http://fingertips.phe.org.uk/profile-group/mental-health/profile/cypmh/data#page/0/gid/1938133090/pat/6/par/E12000004/ati/102/are/E06000015 (accessed 10 Oct 2017).
      7. 7.↵
        2. Crawford T ,
        3. Geraghty W ,
        4. Street K , et al
        . Staff knowledge and attitudes towards deliberate self-harm in adolescents. J Adolesc 2003;26:619–29.doi:10.1016/S0140-1971(03)00060-5
        OpenUrlCrossRef
      8. 8.↵
        YoungMinds. Who’s Who: find out what all the roles in Child and Adolescent Mental Health Services (CAMHS) mean and what they do. 2018 https://youngminds.org.uk/find-help/your-guide-to-support/whos-who/ (accessed 15 Jan 2018).
      9. 9.↵
        2. Anderson M ,
        3. Standen PJ
        . Attitudes towards suicide among nurses and doctors working with children and young people who self-harm. J Psychiatr Ment Health Nurs 2007;14:470–7.doi:10.1111/j.1365-2850.2007.01106.x
        OpenUrlPubMed
      10. 10.↵
        2. McLaughlin S ,
        3. Bailey D ,
        4. Bonner G , et al
        . Improving confidence in suicide risk assessment. Nurs Times 2014;110:16–18.
        OpenUrl
      11. 11.↵
        National Institute for Health and Clinical Excellence and National Collaborating Centre for Mental Health. Self-harm: The short-term physical and psychological management and secondary prevention of self-harm in primary and secondary care. Manchester: National Institute for Health and Clinical Excellence and National Collaborating Centre for Mental Health, 2004.
      12. 12.↵
        2. Olfson M ,
        3. Gameroff MJ ,
        4. Marcus SC , et al
        . Emergency treatment of young people following deliberate self-harm. Arch Gen Psychiatry 2005;62:1122–8.doi:10.1001/archpsyc.62.10.1122
        OpenUrlCrossRefPubMedWeb of Science
      13. 13.↵
        2. Pfeffer CR
        . In: King RA , Apter A , eds. Assessing suicidal behavior in children and adolescents. Suicide in children and adolescents. Cambridge: Cambridge University Press, 2003:320–20.
      14. 14.↵
        2. Hendren RL
        . In: Blumenthal SJ , Kupfer DJ , eds. Assessment and interviewing strategies for suicidal patients over the life cycle. Washington DC: American Psychiatric Press, 1990:799–99.
      15. 15.↵
        2. Shaffer D ,
        3. Pfeffer CR
        . Practice parameter for the assessment and treatment of children and adolescents with suicidal behavior. J Am Acad Child Adolesc Psychiatry 2001;40:24S–51.doi:10.1097/00004583-200107001-00003
        OpenUrlCrossRefPubMedWeb of Science
      16. 16.↵
        2. Quinlivan L ,
        3. Cooper J ,
        4. Meehan D , et al
        . Predictive accuracy of risk scales following self-harm: multicentre, prospective cohort study. Br J Psychiatry 2017;210:429–36.doi:10.1192/bjp.bp.116.189993
        OpenUrlAbstract/FREE Full Text
      17. 17.↵
        2. Horowitz LM ,
        3. Bridge JA ,
        4. Pao M , et al
        . Screening youth for suicide risk in medical settings: time to ask questions. Am J Prev Med 2014;47(Suppl 2):S170–75.doi:10.1016/j.amepre.2014.06.002
        OpenUrlCrossRefPubMed
      18. 18.↵
        2. Horowitz LM ,
        3. Bridge JA ,
        4. Teach SJ , et al
        . Ask Suicide-Screening Questions (ASQ). Arch Pediatr Adolesc Med 2012;166:1170–70.doi:10.1001/archpediatrics.2012.1276
        OpenUrlCrossRefPubMed
      19. 19.↵
        2. Pfeffer CR
        . Child suicide potential scales. New York: Guilford Press, 1986:173–203.
      20. 20.↵
        2. Orbach I ,
        3. Carlson G ,
        4. Feshbach S , et al
        . Attitudes toward life and death in suicidal, normal, and chronically ill children: an extended replication. J Consult Clin Psychol 1984;52:1020–7.doi:10.1037/0022-006X.52.6.1020
        OpenUrlCrossRefPubMedWeb of Science
      21. 21.↵
        2. Orbach I ,
        3. Milstein I ,
        4. Har-Even D , et al
        . A multi-attitude suicide tendency scale for adolescents. Psychol Assess 1991;3:398–404.doi:10.1037/1040-3590.3.3.398
        OpenUrlCrossRef
      22. 22.↵
        2. Osman A ,
        3. Downs WR ,
        4. Kopper BA , et al
        . The reasons for living inventory for adolescents (RFL-A): development and psychometric properties. J Clin Psychol 1998;54:1063–78.doi:10.1002/(SICI)1097-4679(199812)54:8<1063::AID-JCLP6>3.0.CO;2-Z
        OpenUrlCrossRefPubMedWeb of Science
      23. 23.↵
        2. Horowitz LM ,
        3. Wang PS ,
        4. Koocher GP , et al
        . Detecting suicide risk in a pediatric emergency department: development of a brief screening tool. Pediatrics 2001;107:1133–7.doi:10.1542/peds.107.5.1133
        OpenUrlAbstract/FREE Full Text
      24. 24.↵
        2. Miller IW ,
        3. Norman WH ,
        4. Bishop SB , et al
        . The modified scale for suicidal ideation: reliability and validity. J Consult Clin Psychol 1986;54:724–5.doi:10.1037/0022-006X.54.5.724
        OpenUrlCrossRefPubMedWeb of Science
      25. 25.↵
        2. Cotton CR ,
        3. Range LM
        . Suicidality, hopelessness, and attitudes toward life and death in children. Death Stud 1993;17:185–91.doi:10.1080/07481189308252615
        OpenUrl
      26. 26.↵
        2. Reynolds WM
        . Suicidal ideation questionnaire-junior. Odessa, FL: Psychological Assessment Resources, 1987.
      27. 27.↵
        2. Reynolds WM
        . Development of a semistructured clinical interview for suicidal behaviors in adolescents. Psychol Assess 1990;2:382–90.doi:10.1037/1040-3590.2.4.382
        OpenUrlCrossRef
      28. 28.↵
        2. Beck AT ,
        3. Schuyler D ,
        4. Herman I
        . In: Beck AT , Resnik HLP , Lettieri DJ , eds. Development of suicidal intent scales. Oxford: Charles Press Publishers, 1974:249–49.
      29. 29.↵
        2. Cull JG ,
        3. Gill WS ,
        4. Scale SP , et al
        . Suicide probability scale. Los Angeles, CA: Western Psychological Services, 1982.
      30. 30.↵
        2. Ward DJ ,
        3. Furber C ,
        4. Tierney S , et al
        . Using framework analysis in nursing research: a worked example. J Adv Nurs 2013;69:n/a–31.doi:10.1111/jan.12127
        OpenUrlCrossRef
      31. 31.↵
        Determining what could/should be: the delphi technique and its application. Meeting of the 2006 annual meeting of the Mid-Western educational research association, Columbus, Ohio 2006.
      32. 32.↵
        2. C-c H ,
        3. Sandford B
        . The delphi technique: making sense of consensus. Practical Assessment, Research & Evaluation 2007;12:1–8.
        OpenUrl
      33. 33.↵
        2. Davis LL
        . Instrument review: getting the most from a panel of experts. Applied Nursing Research 1992;5:194–7.doi:10.1016/S0897-1897(05)80008-4
        OpenUrlCrossRef
      34. 34.↵
        2. Delbecq AL ,
        3. Van de Ven AH ,
        4. Gustafson DH
        . Group tecniques for program planning: a guide to nominal group and Delphi processes. Glenview: Scott, Foresman, 1975.
      35. 35.↵
        2. Green P
        . The content of a college-level outdoor leadership course. 1982:26.
      36. 36.↵
        2. Diamond IR ,
        3. Grant RC ,
        4. Feldman BM , et al
        . Defining consensus: a systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol 2014;67:401–9.doi:10.1016/j.jclinepi.2013.12.002
        OpenUrlCrossRefPubMed
      37. 37.↵
        2. Delbecq AL ,
        3. Van de Ven AH
        . A group process model for problem identification and program planning. J Appl Behav Sci 1971;7:466–92.doi:10.1177/002188637100700404
        OpenUrlCrossRefWeb of Science
      38. 38.↵
        2. Dunnette MD ,
        3. Campbell J ,
        4. Jaastad K
        . The effect of group participation on brainstorming effectiveness for 2 industrial samples. J Appl Psychol 1963;47:30–7.doi:10.1037/h0049218
        OpenUrlCrossRefWeb of Science
      39. 39.↵
        2. Hardesty DM ,
        3. Bearden WO ,
        4. Hardesty DM , et al
        . The use of expert judges in scale development: Implications for improving face validity of measures of unobservable constructs Implications for improving face validity of measures of. J Business Resear 2004;2014:98–107.doi:10.1016/S0148-2963(01)00295-8
        OpenUrl
      40. 40.↵
        2. Terwee CB ,
        3. Bot SD ,
        4. de Boer MR , et al
        . Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol 2007;60:34–42.doi:10.1016/j.jclinepi.2006.03.012
        OpenUrlCrossRefPubMedWeb of Science
      41. 41.↵
        2. Posner K ,
        3. Brent D ,
        4. Lucas C , et al
        . Columbia-suicide severity rating scale (C-SSRS). New York, NY: Columbia University Medical Center, 2008.
      42. 42.↵
        US Food and Drug Administration. Guidance for industry: suicidality: prospective assessment of occurrence in clinical trials, draft guidance. Rockville, MD: US Department of Health and Human Services, 2012.
      43. 43.↵
        2. Posner K ,
        3. Brown GK ,
        4. Stanley B , et al
        . The Columbia-suicide severity rating scale: initial validity and internal consistency findings from three multisite studies with adolescents and adults. Am J Psychiatry 2011;168:1266–77.doi:10.1176/appi.ajp.2011.10111704
        OpenUrlCrossRefPubMedWeb of Science
      44. 44.↵
        2. Mundt JC ,
        3. Greist JH ,
        4. Gelenberg AJ , et al
        . Feasibility and validation of a computer-automated Columbia-suicide severity rating scale using interactive voice response technology. J Psychiatr Res 2010;44:1224–8.doi:10.1016/j.jpsychires.2010.04.025
        OpenUrlPubMed
      45. 45.↵
        2. Lawshe CH
        . A quantitative approach to content validity. Pers Psychol 1975;28:563–75.doi:10.1111/j.1744-6570.1975.tb01393.x
        OpenUrlCrossRef
      46. 46.↵
        2. Lynn MR
        . Determination and quantification of content validity. Nurs Res 1986;35:382???386.doi:10.1097/00006199-198611000-00017
        OpenUrl
      47. 47.↵
        2. Ayre C ,
        3. Scally AJ
        . Critical values for lawshe’s content validity ratio. Measurement and Evaluation in Counseling and Development 2014;47:79–86.doi:10.1177/0748175613513808
        OpenUrlCrossRef
      48. 48.↵
        2. Pentland BT
        . Building process theory with narrative: from description to explaination. Academy of Management Review 1999;24:711–24.
        OpenUrlAbstract/FREE Full Text
      49. 49.↵
        2. Tabachnick BG ,
        3. Fidell LS
        . Using multivariate statistics. Boston, MC: Pearson Education, 2007.
      50. 50.↵
        2. Haynes SN ,
        3. Nelson K ,
        4. Blaine DD
        . Psychometric issues in assessment research. Handbook of research methods in clinical psychology. 2nd edn. Hoboken, NJ, US: John Wiley & Sons Inc, 1999:125–54.
      51. 51.↵
        The Stationery Office. The mental capacity act: code of practice. England: The Stationery Office, 2007:1–92.
      52. 52.↵
        2. Gould MS ,
        3. Marrocco FA ,
        4. Kleinman M , et al
        . Evaluating Iatrogenic Risk of Youth Suicide Screening Programs. JAMA 2005;293:1635–35.doi:10.1001/jama.293.13.1635
        OpenUrlCrossRefPubMedWeb of Science

      Footnotes

      • Contributors JCM is chief investigator and was responsible for obtaining funding for the study and maintains overall responsibility for all study aspects. GMW drafted the initial manuscript. JCM, TC, AA, MW and JC developed the initial concepts and have contributed to the study design and critical appraisal of this paper. The CYP-MH SAT study group have advised on the study design and the revision of this manuscript.

      • Funding This study is funded by Nottinghamshire Clinical Commissioning Group. The study sponsor is Nottingham University Hospitals NHS Trust.

      • Disclaimer The views represented are the views of the authors alone and do not necessarily represent the views of the Nottinghamshire Clinical Commissioning Group or Nottingham University Hospitals NHS Trust.

      • Competing interests None declared.

      • Patient consent Not required.

      • Ethics approval Full ethical approval has been received from NHS East Midlands—Derby Research Ethics Committee (17/EM/0347).

      • Provenance and peer review Not commissioned; externally peer reviewed.

      • Collaborators The CYP-MH SAT study steering group have advised on the study design. Members of this group include: Rachel Barker (Sherwood Forest Hospitals NHS Foundation Trust), Marie Armstrong (Nottinghamshire Healthcare NHS Foundation Trust), Sharon O’Love (SHARP), Rebecca Green (SHARP), Dr Damian Wood (Nottingham University Hospitals NHS Trust), Tessa Jones (Nottingham University Hospitals NHS Trust), Rachel Dolby (Nottingham University Hospitals NHS Trust), David Clark (Nottingham University Hospitals NHS Trust) , Dr Lynda Walton (Nottingham University Hospitals NHS Trust), Elizabeth Byrne (Nottingham University Hospitals NHS Trust), Professor Joanne Cooper (Nottingham University Hospitals NHS Trust), Kate Rodgers (Sherwood Forest Hospitals NHS Foundation Trust), and Dr Rebecca Sands (Sherwood Forest Hospitals NHS Foundation Trust).