Background

Complex interventions have been increasingly employed in an attempt to enhance health service delivery as well as other societal issues.1 Randomised controlled trials (RCTs) are traditionally considered as the reference standard for establishing the effectiveness of interventions.1 2 Recent efforts have been made to include process evaluations as a core component of investigations of effectiveness, as stated in a recent Medical Research Council (MRC) guidance document.1 Conducting a process evaluation of an intervention, particularly in the case of complex quality improvement interventions, is important to gain a deeper understanding of the mechanisms influencing effectiveness (or lack of it), to explain discrepancies between expected and observed outcomes, to highlight the complexities of an intervention and the impact of contextual factors on outcomes and thus to better inform implementation.2–4

The MRC framework highlights three key functions of process evaluations: (1) examining the implementation process and its content (fidelity adaptation, dose and reach); (2) understanding the mechanisms of impact (participants’ response to the intervention; mediators; unexpected pathways and consequences) and (3) investigating the influence of the context of the intervention. Such a framework enables to capture the complexities of developing and implementing a health service intervention, so to offer useful insights for future quality improvement. In this process evaluation, we aim to use the MRC framework to evaluate the process, delivery and impact of the implementation of an allied health team-based intervention within an emergency setting.

Intervention characteristics

The present process evaluation will explore the process of implementation of the OPTIMEND intervention (ie, ‘Optimising early assessment and intervention by Health and Social Care Professionals in the Emergency Department’). OPTIMEND is the first RCT aimed to measure the impact of early assessment and intervention by a team of health and social care professionals (HSCPs) working in the emergency department (ED) on the quality, safety and cost-effectiveness of care for older adults, as compared with usual ED care. The HSCP team comprises of a senior physiotherapist, a senior occupational therapist, and a senior medical social worker providing functional assessment, early interventions and discharge plans to adults aged ≥65 years. A total of 354 participants were recruited in the study from December 2018 until May 2019 and randomly allocated to the HSCP intervention or ED usual care (ie, medical team). Participants in both intervention and control groups are followed-up through telephone assessment at 30 days, 4 and 6 months after the ED index visit (ongoing until November 2019). Primary outcomes of the trial include ED length of stay and rates of hospital admissions. Secondary outcomes include function and quality of life (baseline and follow-up), satisfaction with care, ED revisits and healthcare utilisation (follow-up) and cost-effectiveness.

Following the MRC framework for complex interventions,1 the design of the trial was informed by a systematic review of the existing international literature regarding the effectiveness of HSCP interventions in the ED.5 A qualitative study was also conducted with a range of stakeholders including ED patients and their families, ED staff, HSCPs and prehospital staff to explore their views on the role and impact of HSCPs working in teams in the ED. A paper reporting the findings of this phase is currently in submission. We also carried out an analysis of routine observational data to describe the flow of patients who attend a large Irish ED without a dedicated HSCP team in the ED. Allied health team services in the ED are routine practice in certain areas, such as in Australia6; however, the evidence on the impact HSCP teams on the quality, safety and cost-effectiveness of care is limited and heterogeneous. For this reason, there is a dearth of evaluations available on the implementation, delivery and impact of this model of care, often limited to investigations of acceptability or patient/staff satisfaction.7–9 The OPTIMEND is the first study internationally to test the effectiveness an ED-based HSCP team intervention by adopting a robust methodology, thus offering the opportunity to evaluate its implementation.

Theoretical framework

The causal assumptions of the intervention and theoretical framework guiding this evaluation are outlined in the logic model presented in figure 1, based on logic models recommended elsewhere.2 10 A key input for the intervention came from the emergency care national priorities set by the ED taskforce within the Health Service Executive (HSE) in Ireland, which included improving workforce and interdisciplinary care in emergency settings in order to enhance patient and process11 outcomes; following this, funding was secured for the design and implementation of an HSCP team intervention in the ED of a regional hospital in Ireland with a large catchment area, using the MRC framework for complex interventions. A synthesis of the evidence on this model of care and consultations with relevant stakeholders, as described in the previous section, informed the development of the intervention. Key assumptions of this HSCP intervention were that having a multidisciplinary team of professionals with specialised skills in the care of the older person would enhance the quality and timeliness of decision-making (ED processes), and that this would result in shorter stay for older adults as well as reduced rates of unnecessary hospital admissions (ED performance). Ultimately, it is expected that the intervention will benefit patient’s health outcomes by promoting better functioning and quality of life than usual ED care, higher satisfaction with the care received, and a better use of primary and community care.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

HSCP intervention logic model. ED, emergency department; HSCP, health and social care professionals; MSW, medical social worker; PT, physiotherapist; OT, occupational therapist.

Objectives

Based on the characteristics and assumptions of the OPTIMEND trial, the aim of this process evaluation is to understand the functioning and effects of the OPTIMEND intervention by examining how the intervention was delivered and received in practice. In line with the MRC guidelines for process evaluations of complex interventions,2 the study has the following objectives to achieve this aim:

1. To describe and analyse the implementation of the OPTIMEND trial (what was delivered and how), including an exploration of the intervention fidelity, dose and reach.

2. To explore the mechanisms of impact within the intervention (ie, barriers and facilitators of implementation in relation to participants’ responses, potential mediators and unexpected pathways).

3. To highlight contextual influences on impact, delivery and acceptability (ie, individuals, physical environment, ED processes and relations, hospital and healthcare system).

Design

The process evaluation will employ a mixed-methods approach to address the above objectives in relation to a HSCP intervention in the ED tested within a randomised controlled trial; the trial for which this process evaluation will be conducted is registered on ClinicalTrials.gov, NCT03739515; registered on 12 November 2018.

The reporting of this protocol aligns with the Standard Protocol Items for Clinical Trials (SPIRIT) guidelines,12 and a full reporting SPIRIT checklist is presented in the online supplementary file 1. However, given the nature of the study (ie, not a trial but a process evaluation), the protocol has been written by incorporating appropriate elements of the Criteria for Reporting the Development and Evaluation of Complex Interventions (CReDECI 2) in healthcare (revised guideline),13 particularly in relation to reporting the development and evaluation of the intervention. Key considerations suggested by the MRC2 will be made in relation to the relations between the quantitative and qualitative components of the evaluation and the relation of the process evaluation to other evaluation components (trial outcomes on clinical and cost-effectiveness).

Participants

The evaluation will involve key staff members working in the hospital where the OPTIMEND intervention was carried out (University Hospital Limerick, Ireland), including the HSCPs who implemented the intervention and other staff members who worked in the ED during the OPTIMEND trial and/or contributed to the development and implementation of the intervention. Given the characteristics of the setting and the fact that the intervention was conducted at one site only, it is anticipated that around 20–25 participants will complete the study. Specifically, the following participant categories will be included in the study:

• The clinical team involved in the intervention (senior physiotherapist, senior occupational therapist, senior medical social worker, research nurse).

• ED doctors (4–5 participants).

• ED nurses (4–5 participants).

• Other hospital staff members who contributed to the development and implementation of the intervention (eg, Informatics, Planning and Performance Department; Departments managers; other HSCPs).

Participant recruitment will be conducted through convenience and snowball sampling, with prospective participants being identified by the research team and the clinical team involved in the intervention. The clinical team will also act as gatekeepers linking potential participants with the researcher managing enrolment (MC); furthermore, study leaflets will be distributed at UHL. Prospective participants will be provided with an information sheet outlining the evaluation aim and procedure; written informed consent will be sought prior to participation. At the time of submission of this protocol, participants recruitment is ongoing and expected to be completed by the end of July 2019.

Outcomes and measures

Using the MRC process evaluation framework, the study will focus on the measures and research questions outlined in table 1. The process of implementation will be described in terms of activities and processes put in place for the development and delivery of the implementation, the fidelity of the intervention (adherence to protocol and evidence as well as adaptations), its dose and reach. Mechanisms internal to the intervention will be investigated in relation to the participants’ interaction with the intervention, potential mediators and unexpected pathways. Lastly, using a system approach, potential facilitators and barriers to implementation outside of the intervention will be explored at the level of individuals, the ED physical environment, procedures, communication and the broader healthcare system.

Data collection and analysis

As described in table 1, a mix of quantitative and qualitative methods will be used to address the objectives of this process evaluation.

The content and process of delivery will be evaluated quantitatively through the intervention activity logs. The implementation will also be investigated in terms of fidelity, dose and reach. Fidelity is a central measure in process evaluations4 which provides information on the extent to which the intervention was delivered as planned or adapted to a specific context. Although maintaining appropriate levels of fidelity has been suggested to enhance the impact of intervention,14 debates on the tension between intervention fidelity and adaptation are ongoing, translating into a variety of frameworks attempting to conceptualise fidelity.2 For the purpose of this process evaluation, we will use the framework proposed by Carroll and colleagues4 in relation to implementation fidelity for health services interventions and the MRC guidelines2 to integrate a quantification of adherence, dose and reach with a qualitative exploration of mechanisms of impact within and beyond the intervention. The trial activity logs and recruitment logs will be analysed to quantify and describe the intervention delivery, comparisons will be made with the trial protocol and evidence base (ie, systematic review) to evaluate adherence and dose; descriptive quantitative analyses of participants lost at follow-up will be carried to quantify attrition. Potential modifications will be quantified and described with the help of the clinical team using Stirman’s framework for interventions adaptations15; a detailed evaluation form is included in the online supplementary file 2, which focuses on what was modified and at what level of delivery, the nature of the modification and the agents of the modification.

The qualitative elements of the implementation will be explored via semistructured interviews and focus groups. An interview schedule is presented in the online supplementary file 3, with questions tailored to the trial clinical team and to other staff members. The members of the trial clinical team will be interviewed as a group to describe the process of implementation and delivery, as well as discuss its acceptability and impact; group interviews will also be organised with other members of staff, paying attention to capture the different perspectives of multiple professionals. In addition, prospective participants who do not wish or are not able to take part in the focus groups will be invited to participate in 1:1 semistructured interviews. Group and individual interviews have a number of strengths and weakness which make it preferable to adopt a flexible approach.16 On one hand, working with a group facilitates participants who might have time restrictions or feel at ease contributing as a member of a group; on the other hand, individual interviews provide space to individuals who may be unwilling to contribute within a group and can help to elicit more personal and truthful responses because removing potential biases related to group dynamics and social desirability. The interviews and focus groups will be audio recorded and transcribed. The data will be inputted in the software NVivo V.11 Plus (QSR International Pty Ltd) and analysed using the six steps of thematic analysis,17 18 with the aim to highlight the central themes related to the research questions above. While the analysis will be data driven, the evaluation is informed by an existing framework, thus emerging themes will be compared with the framework to evaluate fit.

By integrating the data collected quantitatively and qualitatively, our analysis will focus on providing a description of the process of implementation as well as considerations of the feasibility and acceptability of the intervention as perceived by key stakeholders involved.

All electronic and hardcopy data will be stored safely by the research team and retained in accordance to the data management policies and procedures of the University of Limerick, Ireland. Access to the data will be limited to the research team members involved in data analysis (MC, KR and RG).

Patient and public involvement statement

This process evaluation will not involve patients directly, as their perceptions of the intervention are investigated as part of the effectiveness study (currently in progress) and it was felt that involving patients in the process evaluation as well may cause a burden without providing novel information. The research questions of this study were informed by the need for quality and timeliness of assessment and intervention in the ED expressed by health service users at a Patient and Public Involvement initiative organised by the Health Service Executive’s Advocacy Unit in Ireland (https://www.hse.ie/eng/about/who/qid/person-family-engagement/listening-reports/listening-report-16.pdf).

Study status

At the time of submission, the status of this study is currently ‘recruiting’. Recruitment for the study commenced in June 2019 and it is anticipated to be completed by the end of July 2019.