You are here

  • Home
  • Archive
  • Volume 8, Issue 6
  • Realist Evaluation of the Use of Patient Experience Data to Improve the Quality of Inpatient Mental Health Care (EURIPIDES) in England: study protocol

Article Text

Article menu

Download PDFPDF

XML
Mental health
Protocol
Realist Evaluation of the Use of Patient Experience Data to Improve the Quality of Inpatient Mental Health Care (EURIPIDES) in England: study protocol
  1. Scott Weich1,
  2. Sarah-Jane Hannah Fenton2,
  3. Kamaldeep Bhui3,
  4. Sophie Staniszewska4,
  5. Jason Madan4,
  6. Michael Larkin5,
  7. Elizabeth Newton6,
  8. David Crepaz-Keay7,
  9. Alastair Canaway4,
  10. Charlotte Croft8,
  11. Frances Griffiths4
  1. 1 School of Health and Related Research, University of Sheffield, Sheffield, UK
  2. 2 Institute for Mental Health, University of Birmingham, Birmingham, UK
  3. 3 Barts & the London Medical School, Queen Mary University, London, UK
  4. 4 Warwick Medical School, University of Warwick, Coventry, UK
  5. 5 School of Life and Health Sciences, Aston University, Birmingham, UK
  6. 6 School of Psychology, University of Birmingham, Birmingham, UK
  7. 7 Empowerment and Social Inclusion, Mental Health Foundation, London, UK
  8. 8 Entrepreneurship & Innovation, Organising Healthcare Research Network, Warwick Business School, Coventry, UK
  1. Correspondence to Professor Scott Weich; s.weich{at}sheffield.ac.uk

Abstract

Introduction Inpatient mental healthcare continues to be an area of high risk and where patients report negative experiences. To ensure the patient voice is heard, National Health Service (NHS) Trusts are required to collect feedback from patients routinely. We do not know what kinds of feedback are most important or what management processes are needed to translate this into effective action plans. Further, we do not know if this makes any difference to the patients themselves. This study seeks to explore which of the many different approaches to collecting and using patient experience data are the most useful for supporting improvements in inpatient mental healthcare. The overarching aim of the study is to arrive at recommendations for best practice in the collection and use of patient experience data in NHS England adult inpatient mental health settings. We present the protocol for Realist Evaluation of the Use of Patient Experience Data to Improve the Quality of Inpatient Mental Health Care study (EURIPIDES).

Methods and analysis The study is composed of five work packages (WPs), including a systematic review of patient experiences (WP1); a telephone survey to assist the selection of case sites (WP2); six indepth case studies involving interviews with service users, carers and staff to enable a realist evaluation of the use of patient experience to improve quality in adult inpatient mental health services (WP3); an economic evaluation of patient experience feedback activity (WP5); and a consensus conference (WP4). We discuss the methodological rationale for the five WPs.

Ethics and dissemination This study has received approval from West Midlands/South Birmingham NHS Research Ethics Committee. The outcome of the consensus conference meeting (WP4) will form the basis of the outputs to be disseminated to NHS providers. Dissemination will also take place through publications and presentations at relevant conferences.

  • patient experience
  • inpatient
  • acute care
  • mental health services
  • quality improvement
  • realist evaluation

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

https://doi.org/10.1136/bmjopen-2017-021013

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • This is the first national study to consider how best to collect and use patient experience feedback regarding the experience of inpatient mental health services.

    • The study uses a mixed-methods approach incorporating a national survey, a realist evaluation and economic modelling.

    • This study incorporates patient and public involvement at every level, through employing survivor researchers who have lived experience of mental health services and through having a lay service user and carer reference group.

    • This is an observational study, based on a relatively small number of case sites (six). While the use of a cross-comparative realist evaluation methodology will strengthen the rigour of this research, there will remain limitations in terms of generalisability from a small number of sites.

    • There will be no follow-up, and therefore we will not be able to discern evidence of prospective service improvement actions arising from the analysis and interpretation of patient experience data.

    Introduction 

    The National Health Service (NHS) is under pressure to deliver timely, effective and affordable care with increasingly constrained resources. There is also concern about care standards.1–5 As a result, the National Institute for Health and Care Excellence  (NICE), the NHS National Quality Board and others have restated core principles of patient-centred care including dignity, compassion, choice and autonomy2–7 and called for a strengthening of the patient voice. Healthcare providers are required to collect data to assess patients’ experiences of care.8 9 In particular, NHS providers are required to provide an opportunity for people who use NHS services to give feedback through the Friends and Family Test and report this data to NHS England each month.10 However, despite a surfeit of routinely collected patient experience data,8 9 most are of limited value11 either because of methodological problems (including poor or unknown psychometric properties or missing data) or because measures lack granular detail.12 There is a dearth of evidence about the processes required to analyse, interpret and translate these data into tangible actions, better outcomes for patients, and more efficient and cost-effective care.11 13

    It is not known whether reporting patients’ experiences is associated with improved outcomes (including clinical and functional outcomes and quality of life), reduced carer burden or reduced costs.14 15 Nor do we know how any effect might be mediated (eg, via better treatment adherence), or which types of patient experience data are of most use for improving services.11 14 16–19 We hypothesise that there will be discernible differences between organisations in commitment to, and capacity for, using patient experience data to improve services,4 and that these differences are manifested in processes that support innovation and quality improvement.20–22 These include commitment to service improvement among senior leaders, decentralised decision-making (through identifiable champions for change), role clarity within the organisation and support for risk-taking.20 21 We anticipate that organisations that use patient experience data most effectively will also have robust data-collection strategies. Finally, we believe that organisations that are more patient centred will demonstrate adoption of codesign approaches to service improvement, and will involve service users and carers.22 23

    Experiences of inpatient care

    Inpatient mental health services are currently overstretched, unpopular with service users,22 expensive and where serious incidents such as suicide continue to occur. Ethnic inequalities in mental health service experience are most pronounced in inpatient settings.23 Recent reports24–26 have highlighted adverse experiences, including lack of privacy and dignity, fear of assault, overcrowding, noise, lack of therapeutic activities and limited individual recovery-focused support, and an emphasis on coercion, control and restraint. Patients of black ethnicity (including both African-Caribbean and black African groups) are over-represented in inpatient settings,27 receive higher doses of medication and experience higher rates of seclusion, physical restraint and injury28 and suicide.29 Many people who receive inpatient mental healthcare do so while compulsorily detained.25

    Three initiatives that attempt to raise the standards of inpatient mental healthcare have been identified across NHS settings in England. These include: (1) Star Wards,30 a third sector initiative that uses patient experience information to develop and share best practice; (2) Productive Wards,31 an initiative led by the NHS Institute that focuses on the adoption and spread of a model of ‘lean working’; and (3) and the Royal College of Psychiatrists’ Accreditation for Inpatient Mental Health Services scheme which is based on evaluation against a quality standard and broadly focuses on raising general standards of care, timely and purposeful admission, safety, the environment and facilities, and therapies and activities.32 Although most NHS providers have participated in one or more of these, concerns remain about the quality of inpatient care.33 We will pay attention to the four domains identified by the NICE Quality Standard which apply to inpatient care: shared decision-making, contact with staff, meaningful activity and use of compulsion.6 34

    NHS Trusts are required to collect feedback from patients routinely. It is not known what kinds of feedback are most important or what management processes are needed to translate this into effective action plans. Realist Evaluation of the Use of Patient Experience Data to Improve the Quality of Inpatient Mental Health study (EURIPIDES)35 views the wide range of current approaches to this as a natural experiment from which we can learn about what is (and is not) working.

    Aims

    The research question is ‘Which approaches to collecting and using patient experience data are most useful for supporting improvements in inpatient mental healthcare?’

    The overarching aim of the study is to arrive at recommendations for best practice in the collection and use of patient experience data in NHS England adult inpatient mental health settings. The study has five aims linked to five work packages (figure 1):

    Figure 1
    Figure 1

    Overview of Realist Evaluation of the Use of Patient Experience Data to Improve the Quality of Inpatient Mental Health (EURIPIDES) work packages (WPs). NHS, National Health Service.

    Aim 1: A systematic review to identify patient experience themes relevant to mental healthcare (WP1).

    Aim 2: A national telephone survey of adult mental health inpatient providers to describe and classify approaches to collecting and using patient experience data to improve inpatient mental health services across England (WP2).

    Aim 3: We will use the telephone survey information to choose six Trusts for indepth case studies where we will carry out interviews to find out what works for whom, and where (WP3).

    Aim 4: We will present our findings to experts (including policy-makers, mental health professionals, people working in NHS England in roles in relation to improvement, service users and carers) at a ‘consensus conference’ to agree on recommendations about best practice (WP4). We will ensure that our results are anchored in what is acceptable, feasible and sustainable in real-world NHS settings.

    Aim 5: We will examine the costs associated with collecting patient experience data and model costs that would arise if ‘best practice’ in collecting and using patient experience data is widely adopted (WP5).

    Methods

    There are five work packages: systematic review (WP1, aim 1); survey of all NHS providers of inpatient mental healthcare responsible for at least 50 adult mental health beds in England, to populate a sampling frame for WP3 (WP2, aim 2); indepth case studies, analysed using a realist approach (WP3, aim 3); health economic modelling to estimate resource requirements and barriers to adoption of best practice (WP5, aim 5); and a consensus conference to agree recommendations about best practice (WP4, aim 4). The study started in December 2015 and will run until August 2018 (figure 2).

    Figure 2
    Figure 2

    Timeline for Realist Evaluation of the Use of Patient Experience Data to Improve the Quality of Inpatient Mental Health research activity.

    WP1: systematic review

    We will undertake a systematic review to identify evidence-based patient experience themes relevant to inpatient mental health settings. The literature review will help to identify what areas (from the perspective of patients who have lived experience in adult inpatient settings) may be relevant to focus on in relation to experience.

    A search strategy has been developed for key databases: MEDLINE (OVID), CINAHL (OVID), and PsycInfo (OVID). All study designs that include inpatients of mental health institutions, and where outcomes have included experiences of inpatient care, will be considered. No restrictions were applied to country but articles were only included if peer reviewed and published in English between 2000 and 2015. The starting date was chosen to coincide with publication of the National Framework for Mental Health (1999) for England, which led to significant service improvements. Papers will be excluded if they are: abstract or conference proceedings, editorials, letters, commentaries or case studies; based on pre-2000 data; included children and adolescents (aged under 18 years); or not in English language. Two reviewers will independently check 20% of abstracts before obtaining full-text articles. Due to the largely qualitative nature of the studies, quality and risk of bias will be guided by the Critical Appraisal Skills Programme (CASP) qualitative checklist which will be adapted for use across different study types.

    WP1 will provide a thematic framework to base the WP2 and WP3 data collection around, as these data will cross-refer to the themes identified in this systematic review.

    WP2: survey of patient experience leads

    The principal aim of WP2 is to create a sampling frame for selecting sites for WP3 case studies. These survey data will additionally be used to identify the variety of work taking place across Trusts and will represent the first survey of inpatient mental health patient experience data collection and management in the NHS in England. We will undertake semistructured telephone interviews with patient experience leads in NHS Trusts in England with 50 or more adult inpatient mental health beds. Patient experience leads are the persons designated within the NHS Trusts to manage the portfolio of work relating to collecting and using patient experience feedback. We will use local Trust Research and Development offices to identify the person who is the designated patient experience lead. The interview will explore the collection and use of patient experience data, and the organisational processes that facilitate translation of this into service improvement actions. We will create a provider typology based on organisational characteristics and approaches to collecting and using patient experience data.

    Participants and data collection

    We will identify the nominated patient experience lead from each eligible provider. Participant information and consent forms will be sent in advance of telephone interviews. Study aims and design will be explained in the participant information sheet and reiterated prior to telephone interview. Participants will be asked for permission to recontact them to discuss participation in subsequent work packages.

    Data will be collected by questionnaire and semistructured telephone interview. The former will be used to collect information about Trusts, including location and size (budget, number of staff), and inpatient services (number, location, ward type, bed numbers and length of stay). We will ask about methods for, and frequency of, collecting patient experience data from users of inpatient services, how these data are processed (and by whom) and whether these are fed into service improvement fora. We will ask for copies of questionnaires or other data-collection tools used, and examples of reports arising from the collection of patient experience data.

    We will ask about who is responsible for data collection, cleaning, analysis and reporting, and to whom reports are passed. The interview will explore how often data are collected and their results reviewed, and for information about any actions that have occurred as a result. Participants will be asked about perceived strengths and weakness, costs, benefits and sustainability of patient experience data-collection methods, about the organisational narrative and rationale for their chosen approach, service user involvement and integration with strategic priorities and impact. We will ask about organisational processes that facilitate or hinder translation of patient experience data into tangible service gains, and about participation in national mental health inpatient service improvement programmes. Analysis of WP2 data will populate a sampling frame for WP3.

    Data analysis

    Analysis of WP2 data will be largely descriptive and designed to populate a sampling frame for WP3. We will use the framework for classifying patient experience data-collection methods developed by the Health Foundation,36 based on the two dimensions of ‘descriptiveness’ and ‘generalisability’. We will classify participating providers into four categories using ratings on two dimensions: ‘patient experience data collection’ and ‘patient experience data use’, each of which will be dichotomised as ‘limited’ or ‘extensive’. Ratings will be made independently by two reviewers and any discrepancies discussed and resolved. By definition, no providers will be classified as ‘limited patient experience data collection, extensive use’, and all providers will therefore be allocated to the remaining three categories, namely1 ‘limited data collection, limited use’2; ‘extensive data collection, limited use’; and3 ‘extensive data collection, extensive use’. We anticipate that most providers will be allocated to categories 1 and 2. To sample for WP3, we will stratify the above classification by size of provider (small, medium and large) and location (urban or rural). The size classification will be based on numbers of inpatient beds in each Trust.

    WP3: indepth case studies in NHS Trusts

    WP3 is a comparative realist evaluation across six case sites. We will survey and evaluate current approaches to collecting and using patient experience data to improve inpatient mental health services. Sites will be purposively selected using a realist case study approach37–39 to explore generative mechanisms and identify using context-mechanism-outcome configurations: what works for whom, in what circumstances and why. We consider a case to be an NHS England Trust where there are over 50 adult inpatient mental health beds. Each case may contain a group of wards providing inpatient psychiatric care to adult patients. Ours is a deliberately bottom–up approach, in which we will treat the wide range of current approaches to collecting and using patient experience data as a natural experiment in which to learn about what is (and is not) working. We will search for evidence of service user and carer involvement and evidence of potential impacts in the form of service improvement activity.

    Participants and data collection

    Our sampling frame will be NHS service providers responding to the survey in WP2 and who consent to being recontacted. We will select inpatient units within provider organisations after familiarising ourselves with WP2 findings and publicly available information (eg, Trust websites and Care Quality Commission reports) and in discussion with WP2 participants. Final selection decisions will be made following initial site visits. These site visits will be to discuss the research with the Trust (postselection) and confirm that they have the capacity and capability to take part and that there is a designated principal investigator in place. Our estimate of six case studies is based on sampling at least one of each type of case (ie, based on the three-level provider classification developed in WP2). We will ensure diversity on provider size and location, including geographical spread, urban/rural setting and ethnic diversity (based on population of the region). We will first map and then select inpatient wards in each participating provider, in discussion with senior clinical and operational staff.

    We will interview those working on inpatient wards (including lead clinicians and team managers), operational managers and Trust leads (for patient and public involvement, inclusion, quality and patient experience, as appropriate). Interviews will cover how patient experience data are collected and used, enablers and barriers to their collection and use, and how the wider service context (deprivation, bed shortages, rates of complaints and serious incidents, and culture of service improvement) influences these processes. Interview schedule development will be informed by the results of WP1 (to permit evaluation of the content of patient experience data being collected) and by the literature on features of healthcare organisations associated with quality improvement.31 40 Sampling will be purposive to ensure those with relevant knowledge and experiences are interviewed. Interviews with senior operational managers will identify and explore evidence for organisational processes that support quality improvement, including decentralised decision-making, role clarity, senior support for risk-taking and existence of processes for planning, implementing and reflecting on service change.

    We will interview service users and a small number of carers to explore whether and how patient (and carer) experience data are gathered, and perceptions about how valid this is as a means of improving service delivery. Carers are defined a friends or family members who are identified as having a role in the life of an inpatient who involves in offering some pastoral support to the individual. Carers are different to individuals who may exist in a formal advocacy capacity in an inpatient’s life. Inpatients for whom discharge is planned during the data-collection period and who are considered by the clinical team to have capacity will be approached for interview by a member of staff (research nurse or member of the clinical team). If they are prepared to participate, interviews will be held in a private room and the confidential nature of the interview explained carefully as service users may be wary of making negative comments about their healthcare. We will follow an appropriate and culturally sensitive process for obtaining informed consent. Where required, the use of qualified and trained interpreters will be offered and provided during the interview. Service user interviews will be undertaken by a survivor researcher (SR) or a member of the research team. SRs will be trained to undertake interviews and will be mentored by experienced members of the research team.

    Participants will be asked about opportunities to tell providers about their experiences of inpatient care (or caring for someone who was an inpatient) and about their perceptions of these processes. We will ask whether those using services are aware of any channels through which service user feedback is shared and/or acted on. As complaints and compliments are methods for collecting information about patient experience, we will explore the ease or otherwise of making complaints and whether and how outcomes are fed back, and whether, why and how patients give compliments. Where an inpatient unit has a service user forum, we will seek to interview participants as members may have a longer term view of patient experience data collection and use. We will seek to gather information on how patients feed back their experiences to services in as many ways as possible, and to understand if and how patients have seen changes enacted based on their feedback. While the duration of the study precludes collecting evidence of service improvement, we are interested in identifying instances where patient experience data are used to inform service development. We will look for evidence of service user and carer involvement, and for service-improvement activity.

    We estimate that up to 30 interviews per case study site, split between staff (15), service users (10) and carers (5) will result in saturation. The interviews will take place over a 6-week period in each Trust. Each of these groups will be interviewed using an interview schedule tailored to their group type. These semistructured interviews will be audio-recorded (with participants’ permission) and transcribed; if a participant does not wish to be audio-recorded, field notes will be taken. Reflective field notes will also be taken by researchers about the inpatient unit and content of interviews.

    Data analysis

    Stage 1

    Initial analysis will be undertaken after the first week of data collection. We will examine our data, mapping processes described for collection and use of patient experience data, and service users’ and carers’ awareness of and perceptions of this process. The health economics team will consider whether the data provides them with the details they require for later modelling of the resource use associated with best practice in collecting, interpreting and acting on patient experience data. We will write a summary of each case study site in relation to the collection and use of patient experience data. We will then return to the site for a week to undertake further interviews. We will check our understanding of what happens at each site with staff and service users, seek to fill gaps in our understanding, probe further where appropriate and continue sampling and interviewing until we are confident of data saturation.

    Stage 2

    Data will then be analysed using a realist approach38 39 to identify what does (or does not) work well, where, when and for whom. This approach is appropriate for understanding whether and how a complex intervention (collecting and using patient experience data) is effective in achieving its desired aims (improving service quality), allowing for (and making use of) contextual variation in implementation and outcome. Data will first be analysed case by case. For each case we will: (a) undertake thematic analysis41 to identify the desired aims, outcomes and impacts of the intervention (collection and use of patient experience data) as perceived by staff and patients, and to explore variation within and between groups; (b) summarise the context for each inpatient service (eg, case mix, locality, service improvement record of the organisation), and we will iteratively develop a template for summarising contextual factors; and (c) refine Stage 1 mapping of processes used for collecting and processing patient experience data through further indepth analysis of the whole case dataset.42

    Thus for each case, we will identify contextual factors, processes for collecting and using patient experience data, including the type of data collected and the results of this collection including evidence of service improvement activity. We will undertake cross-case comparison to identify common and idiosyncratic configurations of what works, for whom, in what circumstances and why.37 38 42

    WP4: consensus conference

    The aim of WP4 is to achieve expert consensus about best practice in the collection and use of patient experience data to improve inpatient mental healthcare. We will undertake a consensus conference with stakeholders where the questions will be: What are the optimal ways of collecting patient experience data, and what processes (in terms of analysis, reporting, interpretation and action planning) are necessary to ensure that patient experience data are used to deliver improvements in inpatient mental healthcare? What recommendations (for collecting and using patient experience data) would those who commission and deliver specialist mental healthcare be willing and able to adopt? What contextual factors are important to consider? Are there any types of activity that Trusts should be advised to discontinue?

    Participants and process

    Participants will have expertise in inpatient mental healthcare and the use of patient experience data (clinicians, Trust Executive Directors and operational managers, commissioners, service users and carers). We will aim for 30–40 participants, to be recruited in part through our national survey of patient experience leads, alongside making approaches to relevant charities nationally (Health Foundation, Mind, Kings Fund and the Mental Health Foundation (MHF) who are partners in this research), NHS Confederation, NHS England, Royal College of Psychiatrists, Picker Institute and (other) experts who have published in this field.

    Conference participants will be provided with information in advance, including the results of prior WPs and the consensus questions in a form suitable for nominal group technique (NGT).43 Mini presentations at the start of the day will act as a reminder. A skilled facilitator will lead initial discussion. For each question, three NGTs will then be run separately, led by trained facilitators with an assistant, with different participants (9–12 participants per NGT group) to reduce the effect of variation in group composition. Participants will be allocated to mixed groups including policy leads, NHS Trust staff, service users and carers. Participants will each take part in up to three NGT exercises. Each NGT will involve an initial round of input from the participants followed by private ranking. Rankings will be tabulated and re-presented for discussion and further ranking. Final rankings will then be tabulated. Results from all of the NGTs will be presented to the whole conference for final discussion until consensus is reached. If we are unable to achieve final consensus on the day, we will augment WP4 by employing a Delphi process after the consensus conference.

    WP5: economic modelling of costs associated with the different ways of collecting and using patient experience data to improve inpatient mental health services

    The aim of WP5 is to estimate the costs associated with different ways of collecting and using patient experience data to improve inpatient mental health services, as identified in WP3 and discussed in WP4. We will identify, for each of the case studies undertaken in WP3, the impact of the processes for the collection, analysis and use of patient experience data on health service resource use. Data from the WP3 will be used to estimate costs associated with implementation of these processes, using standard sources of unit cost data (eg, NHS reference costs). Resource use data will be combined with unit costs to estimate total costs associated with different processes. We will identify the drivers of variation in costs across case studies, and provide these findings to attendees of the consensus conference to assist their discussion of optimal approaches to patient experience data collection.

    We will not have sufficient data on the long-term impact of such changes on relevant outcomes to support a formal cost-effectiveness analysis of implementing best practice in the collection and use of patient experience data. Instead, we will develop conceptual models to represent the proximal impact of improved collection and use of patient experience data, and how these might result in long-term changes in patient health and their utilisation of health and social care.

    Data analysis

    The conceptual modelling exercise will draw on data and insights gathered from the six organisational case studies analysed in WP3. Conceptual modelling involves identifying a structured set of activities, and relations between them, that describe a system.44 This will include description of changes that have been, or might be, implemented, impacts on patients or the organisation, and the implications of these impacts for patient health and service organisation. Conceptual models will provide diagrammatic representation of the causal chains linking patient data collection with changes to patient experience and health outcomes, and service resource use. We will speak to senior NHS mental health professionals to explore the obstacles to adopting best practice. These models will therefore be further refined following the consensus conference though discussion with operational and clinical decision-makers, to inform selection of outcomes and data gathering activities for subsequent economic evaluations. They will also be used as the basis of discussions with operational and clinical decision-makers at the consensus conference to capture their views on the evidence required to justify adoption of best practice processes.

    Patient and public involvement

    Patient and public involvement is led by the MHF. Informal discussions about this research were held with SRs and the National Survivor User Network (NSUN).45 The response has been positive since this work resonates with MHF and NSUN priorities. Both organisations have lobbied for better understanding of the purpose, process and impact of patient experience feedback and both are active in articulating the patient view in mental healthcare and research.

    Mental health service users (and carers) will be involved in all aspects of this research. We will use standards for service user involvement in research developed by NSUN and the MHF. SRs, all of whom have experience of working for or with the MHF, will be involved in WP2 (survey of mental health inpatient providers), WP3 (case studies) and WP4 (consensus conference). In WP2, SRs will advise on the questionnaire and topic guide and on the development of the WP3 sampling frame. In WP3, SRs will conduct interviews with service users. In WP4, SRs will help in recruiting service user and carer participants and will act as facilitators at the consensus conference itself.

    In addition to recruiting and supporting SRs, the MHF will also co-ordinate and facilitate an advisory panel of ‘ordinary’ mental health service users and carers composed of individuals with experience of using (or caring for people who use) services but not of research per se. This advisory group will run parallel to the study but will not include the SRs, instead providing the voice of the active service user to help guide and steer the research activity. This panel will meet at least four times during the project lifetime and will advise on key decisions as the project develops, for example, design, methods and questions. We will recruit 8–10 members with recent experience of using or caring for someone who has used inpatient services. We will ensure diversity within the advisory group with respect to gender, ethnicity, age and geography. All service users and carers taking part in advisory group meetings or the consensus conference will be paid honoraria reflecting the value of their contribution in addition to being reimbursed for travel expenses.

    Ethical considerations

    We are aware that all research activity impacts upon those who are approached to participate and those who do participate in it. This study is not designed to explore patients or carers’ experiences of inpatient services, rather it seeks to understand their experience of being asked about how they found inpatient services. It is recognised that in order to demonstrate their understanding they may draw on wider themes related to the inpatient setting and describe their experiences; however, it is not the explicit aim of the research to ask any questions of a personal or sensitive nature. The research poses ethical issues in relation to: obtaining adequate consent, ensuring anonymity, working with participants who are vulnerable due to their mental health issues, and minimising possible distress of participants when interviewing. These issues will be addressed through having a clear and robust consent procedure that explains how disclosure by participants will be managed, by de-identifying recordings and transcripts through anonymising them using a unique code identifier and by offering participants the opportunity to be accompanied or to leave the interview or terminate at any point should they become distressed during the process, and through giving them the opportunity to withdraw from the study within a reasonable time period.

    Data management

    Interview data will be audio-recorded (with participants’ permission), transcribed and anonymised. If a participant does not wish to be audio-recorded, field notes will be taken of the interview. Reflective field notes will be taken by researchers about the inpatient unit and content of interviews. Field notes will be transcribed and anonymised. Data from each case study site will form a dataset for analysis. Data will be uploaded into MAXQDA 12 software to aid with thematic analysis. Study data will be held securely at the University of Warwick.

    Discussion

    Our results will provide the first comprehensive overview of current approaches to collecting and using patient experience data to improve inpatient mental healthcare in England. Our data will enable us to understand how organisations collect and use these data, and how they mobilise and use this potential source of new knowledge to improve services. Information relating to how data are prioritised and what collection strategies are used will inform how patient experience data underpins quality improvement in adult inpatient settings. Study findings (including the results of the WP2 survey) will be shared with the Trusts so that they may understand the range of ways in which this work is being undertaken nationally. This information will assist them in appraising their own approach in the wider national context. In depth at case study research undertaken in six sites that will be chosen to reflect the diversity of approaches, will enable us to identify the organisational characteristics that enable and constrain these processes. We will seek evidence of changes to services and to patient outcomes, and we will identify what is and is not working, where and why. At our consensus conference of experts (including service users and carers), we will agree recommendations based on what is judged feasible, acceptable and sustainable in NHS settings according to commissioners, service providers and service users and carers (WP4). We will extend this by modelling variation in resources (costs) associated with adopting new ways of collecting and using patient experience data and associated service improvements, the obstacles to this and the value (ie, cost) of evidence required to convince NHS commissioners and providers to substantially alter the way they deliver inpatient mental healthcare (WP5).

    We recognise three principal limitations in this study: Firstly, the finite case study sample size (six), means that there are limits to the variation we can observe in the phenomena of interest. Secondly, and we will not be able to assess service changes themeselves, hence we can only look for evidence that patient experience data are used to inform service change. Finally, another limitation/risk that was highlighted at the outset of the study was associated with potential non-response to WP2, which might in turn lead to bias in selection of WP3 sites.

    It is anticipated that the findings from this research will be disseminated and used to inform different parts of the patient experience feedback cycle, including data collection and analysis, and service improvement implementation, change and feedback. Further research will be developed to evaluate the impacts of this dissemination of the study findings on service delivery and outcome. Study findings will be made as widely available as possible for adoption where appropriate. The results of the consensus conference will be circulated to all participating Trusts. The project website will provide links to the published project outputs. The project will send a summary of the results of the study to all mental health patient experience leads across England. Further academic dissemination will take place through publications and presentations at relevant conferences. Any new data generated by this study and the intellectual property within it will be owned by the University of Warwick.

    Acknowledgments

    None

    References

    1. 1.↵
      2. Francis R
      . Report of the Mid Staffordshire NHS Foundation Trust Public Inquiry [Internet]. London, 2013:1–125. Report No: HC 947. https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/279124/0947.pdf.
    2. 2.↵
      2. Staniszewska S ,
      3. Bullock I
      . Can we help patients have a better experience? Implementing NICE guidance on patient experience. Evid Based Nurs 2012;15:99.doi:10.1136/eb-2012-100988
      OpenUrlFREE Full Text
    3. 3.↵
      National Institute for Health and Clinical Excellence. Patient experience in adult NHS services: improving the experience of care for people using adult NHS services. [Internet]. 2012:1–31. Report No: Clinical Guideline 138 https://www.nice.org.uk/guidance/cg138.
    4. 4.↵
      NHS Confederation. Feeling better? Improving patient experience in hospital. London: NHS Confederation, 2010:1–20. http://www.nhsconfed.org/~/media/Confederation/Files/Publications/Documents/Feeling_better_Improving_patient_experience_in_hospital_Report.pdf
    5. 5.↵
      Department of Health. NHS Patient Experience Framework: NHS, 2012. Report No: Gateway reference number 17273. https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/215159/dh_132788.pdf (accessed 19 May 2017).
    6. 6.↵
      National Institute for Health and Care Excellence. Service user experience in adult mental health: improving the experience of care for people using adult NHS mental health services [Internet]: NICE, 2011:1–42. Report No: CG 136. https://www.nice.org.uk/guidance/cg136/resources/service-user-experience-in-adult-mental-health-improving-the-experience-of-care-for-people-using-adult-nhs-mental-health-services-35109513728197
    7. 7.↵
      2. Staniszewska S ,
      3. Boardman F ,
      4. Gunn L , et al
      . The Warwick Patient Experiences Framework: patient-based evidence in clinical guidelines. Int J Qual Health Care 2014;26:151–7.doi:10.1093/intqhc/mzu003
      OpenUrlCrossRefPubMed
    8. 8.↵
      Care Quality Commission. Community mental health survey. 2015 http://www.cqc.org.uk/content/community-mental-health-survey-2015 (accessed 23 Aug 2016).
    9. 9.↵
      NHS England. Friends and Family Test. https://www.england.nhs.uk/ourwork/pe/fft/ (accessed 19 May 2017).
    10. 10.↵
      NHS England. The Friends and Family Test [Internet]. Leeds: NHS England, 2014. Report No: 03548. https://www.england.nhs.uk/wp-content/uploads/2015/07/fft-guidance-160615.pdf (accessed 3 Mar 2017).
    11. 11.↵
      2. Coulter A ,
      3. Locock L ,
      4. Ziebland S , et al
      . Collecting data on patient experience is not enough: they must be used to improve care. BMJ 2014;348:g2225–4.doi:10.1136/bmj.g2225
      OpenUrlFREE Full Text
    12. 12.↵
      NHS England. Review of the Family and Friends Test [Internet]. 2014 https://www.england.nhs.uk/wp-content/uploads/2014/07/fft-rev1.pdf (accessed 21 Feb 2017).
    13. 13.↵
      2. Boiko O ,
      3. Campbell JL ,
      4. Elmore N , et al
      . The role of patient experience surveys in quality assurance and improvement: a focus group study in English general practice. Health Expect 2015;18:1982–94.doi:10.1111/hex.12298
      OpenUrlCrossRefPubMed
    14. 14.↵
      2. Doyle C ,
      3. Lennox L ,
      4. Bell D
      . A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open 2013;3:e001570.doi:10.1136/bmjopen-2012-001570
    15. 15.↵
      2. Boulding W ,
      3. Glickman SW ,
      4. Manary MP , et al
      . Relationship between patient satisfaction with inpatient care and hospital readmission within 30 days. Am J Manag Care 2011;17:41–8.
      OpenUrlPubMedWeb of Science
    16. 16.↵
      2. Black NVM ,
      3. Hutchings A
      . Relationships between patient reported experience (PREMs) and patient reported outcomes (PROMs) in elective surgery. BMJ Qual Saf First 2014:1–9.
    17. 17.↵
      2. Fenton JJ ,
      3. Jerant AF ,
      4. Bertakis KD , et al
      . The cost of satisfaction: a national study of patient satisfaction, health care utilization, expenditures, and mortality. Arch Intern Med 2012;172:405.doi:10.1001/archinternmed.2011.1662
      OpenUrlCrossRefPubMedWeb of Science
    18. 18.↵
      2. Manary MP ,
      3. Boulding W ,
      4. Staelin R , et al
      . The patient experience and health outcomes. N Engl J Med 2013;368:201–3.doi:10.1056/NEJMp1211775
      OpenUrlCrossRefPubMedWeb of Science
    19. 19.↵
      2. Rose G ,
      3. Beale I ,
      4. Malone J , et al
      . Validity of a scale for consumer evaluation of mental health service delivery. Int J Pers Centered Med 2011;1:733–40.
      OpenUrl
    20. 20.↵
      2. Keown OP ,
      3. Parston G ,
      4. Patel H , et al
      . Lessons from eight countries on diffusing innovation in health care. Health Aff 2014;33:1516–22.doi:10.1377/hlthaff.2014.0382
      OpenUrlAbstract/FREE Full Text
    21. 21.↵
      2. Greenhalgh T ,
      3. Robert G ,
      4. Macfarlane F , et al
      . Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q 2004;82:581–629.doi:10.1111/j.0887-378X.2004.00325.x
      OpenUrlCrossRefPubMedWeb of Science
    22. 22.↵
      Rethink. Future Perfect: Mental health service users set out a vision for hte 21st century. Surrey: Rethink, 2005:1–20. file:///C:/Users/hww774/Downloads/future-perfect3.pdf (accessed 19 May 2017).
    23. 23.↵
      Royal College of Psychiatrists. Improving in-patient mental health services for Black and minority ethnic patients: Recommendations to inform accreditation standards. London: The Royal College of Psychiatrists, 2010:1–33. Report No: OP71. http://www.rcpsych.ac.uk/files/pdfversion/OP71.pdf
    24. 24.↵
      Mind. Listening to experience: An Independent Inquiry into Acute and Crisis Mental Healthcare. London: Mind, 2011. https://www.mind.org.uk/media/211306/listening_to_experience_web.pdf (accessed 22 Feb 2017).
    25. 25.↵
      Commission for Healthcare Audit and Inspection. The pathway to recovery: A review of NHS acute inpatient mental health services. London: Healthcare Comission, 2008:1–96. http://www.ucl.ac.uk/core-resource-pack/resources/papers/healthcare_commission_2008_The_pathway_to_recovery.pdf (accessed 22 Feb 2017).
    26. 26.↵
      Care Quality Commission. Monitoring the Mental Health Act in 2012/13. Newcastle upon Tyne: Care Quality Commission, 2014:1–94. https://www.cqc.org.uk/sites/default/files/documents/cqc_mentalhealth_2012_13_07_update.pdf (accessed 22 Feb 2001).
    27. 27.↵
      2. Bhui K ,
      3. Stansfeld S ,
      4. Hull S , et al
      . Ethnic variations in pathways to and use of specialist mental health services in the UK. Systematic review. Br J Psychiatry 2003;182:105–16.doi:10.1192/bjp.182.2.105
      OpenUrlAbstract/FREE Full Text
    28. 28.↵
      Commission for Healthcare Audit and Inspection. Count me in 2008: Results of the 2008 national census of inpatients in mental health and learning disability services in England and Wales. London: Commission for Healthcare Audit and Inspection, 2008:1–90. Report No: 146. https://www.diversecymru.org.uk/wp-content/uploads/count-me-in-2008.pdf (accessed 22 Feb 2017).
    29. 29.↵
      2. Bhui KS ,
      3. Dinos S ,
      4. McKenzie K
      . Ethnicity and its influence on suicide rates and risk. Ethn Health 2012;17:141–8.doi:10.1080/13557858.2011.645151
      OpenUrlPubMed
    30. 30.↵
      Star Wards [Internet]. Star Wards. http://www.starwards.org.uk/ (accessed 1 Dec 2017).
    31. 31.↵
      2. Morrow E ,
      3. Robert G ,
      4. Maben J , et al
      . Improving healthcare quality at scale and pace. Lessons from The Productive Ward: Releasing time to careTM programme. Full report. Int J Qual Health Care 2011;25:237–53.
      OpenUrl
    32. 32.↵
      2. Cresswell J ,
      3. Beavon M
      . Accreditation for Inpatient Mental Health Services (AIMS): Standards for Inpatient Wards - Working-Age Adults. London: Royal College of Psychiatrists’ Centre for Quality Improvement, 2010:1–40. Report No: CRTU040. http://www.rcpsych.ac.uk/pdf/Standards%20for%20Inpatient%20Wards%20-%20Working%20Age%20Adults%20-%20Fourth%20Edition.pdf (accessed 22 Feb 2017).
    33. 33.↵
      Hill A, correspondent social affairs. Mental health services in crisis over staff shortages: The Guardian. https://www.theguardian.com/society/2011/jun/20/mental-health-services-in-crisis-over-staff-shortages (accessed 23 Aug 2016).
    34. 34.↵
      National Institute for Health and Care Excellence. Patient experience in adult NHS services. Evidence Update February 2014: Evidence Update 52. London: NICE, 2014:1–26. Report No: Evidence Update 52.
    35. 35.↵
      University of Warwick. EURIPIDES [Internet]. 2017 http://www2.warwick.ac.uk/fac/med/research/mhwellbeing/euripides/ (accessed 18 May 2017).
    36. 36.↵
      2. De Silva D
      . Measuring Patient Experience [Internet]. London: The Health Foundation, 2013:1–50. Report No: No. 18. http://www.health.org.uk/sites/health/files/MeasuringPatientExperience.pdf (accessed 22 Feb 2017).
    37. 37.↵
      2. Yin RK
      . Case study research: design and methods. 4th ed. Los Angeles, Calif: Sage Publications, 2009:219. (Applied social research methods).
    38. 38.↵
      2. Pawson R ,
      3. Tilley N
      . Realistic evaluation. London; Thousand Oaks, Calif. Sage 1997.
    39. 39.↵
      2. Wong G ,
      3. Westhorp G ,
      4. Manzano A , et al
      . RAMESES II reporting standards for realist evaluations. BMC Med 2016;14:96.doi:10.1186/s12916-016-0643-1
      OpenUrl
    40. 40.↵
      2. Grol RP ,
      3. Bosch MC ,
      4. Hulscher ME , et al
      . Planning and studying improvement in patient care: the use of theoretical perspectives. Milbank Q 2007;85:93–138.doi:10.1111/j.1468-0009.2007.00478.x
      OpenUrlCrossRefPubMedWeb of Science
    41. 41.↵
      2. Boyatzis RE
      . Transforming qualitative information: thematic analysis and code development. Thousand Oaks, CA: Sage Publications, 1998:184.
    42. 42.↵
      2. Miles MB ,
      3. Huberman AM ,
      4. Saldaña J
      . Qualitative data analysis: a methods sourcebook. Third edition. Thousand Oaks, Califorinia: SAGE Publications, Inc, 2014:381.
    43. 43.↵
      2. Carney O ,
      3. McIntosh J ,
      4. Worth A
      . The use of the Nominal Group Technique in research with community nurses. J Adv Nurs 1996;23:1024–9.doi:10.1046/j.1365-2648.1996.09623.x
      OpenUrlCrossRefPubMedWeb of Science
    44. 44.↵
      2. Tappenden P
      . Conceptual Modelling For Health Economic Model Development [Internet]. HEDS Discussion Paper 12/5 (Unpublished). 2012 http://eprints.whiterose.ac.uk/74464
    45. 45.↵
      National Survivor User Network [Internet]. National Survivor User Network. https://www.nsun.org.uk/ (accessed 1 Dec 2017).

    Footnotes

    • Contributors This project is a collaboration between the University of Warwick, The University of Sheffield, The University of Birmingham, Aston University, Queen Mary University London and the Mental Health Foundation. SW is the chief investigator and work package (WP) lead for WP2 and line manages the Research Fellow (RF) and a Research Associate (RA). SS is the WP lead for WP1. FG is the WP lead for WPs 3 and 4. JM is the WP lead for WP5. KB is a coapplicant, a member of the Project Oversight Group (POG) and line manages an RA. CC, ML and EN are coapplicants and members of the Project Oversight Group. DC-K is a coapplicant and co-ordinates the project Patient and Public Involvement (PPI). S-JHF and AC are the RFs who work on WP2-4 and WP5, respectively.

    • Funding The EURIPIDES study is funded by the National Institute of Health Research’s Health Services and Delivery Research Programme.

    • Competing interests None declared.

    • Patient consent Not required.

    • Ethics approval HRA, West Midlands - South Birmingham Research Ethics Committee.

    • Provenance and peer review Not commissioned; externally peer reviewed.