Can implementation failure or intervention failure explain the result of the 3D multimorbidity trial in general practice: mixed-methods process evaluation

Objectives During a cluster randomised trial, (the 3D study) of an intervention enacting recommended care for people with multimorbidity, including continuity of care and comprehensive biennial reviews, we examined implementation fidelity to interpret the trial outcome and inform future implementation decisions. Design Mixed-methods process evaluation using cross-trial data and a sample of practices, clinicians, administrators and patients. Interviews, focus groups and review observations were analysed thematically and integrated with quantitative data about implementation. Analysis was blind to trial outcomes and examined context, intervention adoption, reach and maintenance, and delivery of reviews to patients. Setting Thirty-three UK general practices in three areas. Participants The trial included 1546 people with multimorbidity. 11 general practitioners, 14 nurses, 7 administrators and 38 patients from 9 of 16 intervention practices were sampled for an interview. Results Staff loss, practice size and different administrative strategies influenced implementation fidelity. Practices with whole administrative team involvement and good alignment between the intervention and usual care generally implemented better. Fewer reviews than intended were delivered (49% of patients receiving both intended reviews, 30% partially reviewed). In completed reviews >90% of intended components were delivered, but review observations and interviews with patients and clinicians found variation in style of component delivery, from ‘tick-box’ to patient-centred approaches. Implementation barriers included inadequate skills training to implement patient-centred care planning, but patients reported increased patient-centredness due to comprehensive reviews, extra time and being asked about their health concerns. Conclusions Implementation failure contributed to lack of impact of the 3D intervention on the trial primary outcome (quality of life), but so did intervention failure since modifiable elements of intervention design were partially responsible. When a decisive distinction between implementation failure and intervention failure cannot be made, identifying potentially modifiable reasons for suboptimal implementation is important to enhance potential for impact and effectiveness of a redesigned intervention. Trial registration number ISRCTN06180958


Gender
Both female

Experience and Training
CM has training and over 5 years experience in qualitative research and research methods. Experience in various environments (primary care and secondary care) as a researcher, research nurse and clinical nurse and experience as a counsellor and group facilitator. PD is a qualified GP with additional academic experience of research.

Relationship established
Prior to study commencement, the interviewer and the participants had no previous contacts. Rapport was built before interview, focus groups or observations by answering questions from participants and taking informed consent.

Participant knowledge of the interviewer
The participants did not have prior knowledge of the interviewer before the study. When participants were recruited, they were provided with an information leaflet about the study and purpose of the interview/focus group/observation which was repeated prior to data collection beginning. Information about the researcher was not provided other than her role in the research team.

Interviewer characteristics
The principle qualitative researcher (CM) is a white, university-educated British woman with nursing, counselling and research qualifications. Qualitative research is always influenced by the perspective of the researcher, and in this case the nursing perspective and primary care clinical experience may have fed into the way some clinical participants were interviewed.

Methodological orientation and Theory
The key methodological framework used was a framework for process evaluation for cluster randomised trials and the MRC guidance for the process evaluation for complex interventions framework. Mixed methods were used, and thematic analysis was used for the qualitative data.

Sample
Intervention practices taking part in the 3D trial were purposively sampled for the process evaluation based on their characteristics. Individual staff members and clinicians of those practices that agreed to take part in the process evaluation were separately invited to take part in the process evaluation, based on their roles. Patient participants were sampled based on their responses to a baseline questionnaire.
11. Method of approach Patient participants were approached by invitation letter including information sheet and staff and clinicians by email with invitation letter and information attached. In both cases follow up contact was made to discuss possible participation and to arrange the details.

Sample Size
The total number of interviews with staff, including informal debriefs after 3D reviews, was 32 (18 GPs, 20 nurses and 9 administrator interviews). Some individuals were interviewed twice so the actual number of those interviewed was 11 GPs, 14 nurses, 7 administrators and 38 patients (including the 22 patients who attended a focus group). 28 intervention review observations were carried out.
13. Non-participation Some patients refused interviews or focus group and 1 nurse refused review observation

Setting of Data Collection
Interviews were conducted in GP practices, patients' homes or, in the case of focus groups, local halls, depending on convenience and patient preference.
Observations were all carried out at the GP practice.

Presence of nonparticipants
Patients' carers were sometimes present at review observations, interviews or focus groups but all of them also provided consent. The researcher was present in a non-participatory role at observations 16. Description of the sample GPs, administrators, practice nurses and patients from 9 different GP practices 17. Interview guide Interview guides, a focus group schedule and an observation guide were used to act as a checklist but without imposing a set structure 18. Repeat interviews Repeat interviews were carried out with some nurses, GPs and administrators who were interviewed both at beginning and end of the trial

Audio-/visual recording
We used audio recording to collect all data.
20. Field notes Field notes were made during the observations to note participant expression, or other non-verbal cues and in all instances of data collection to describe the ambience of the GP practice and reception and aspects of the environment and interaction.

Duration
Pre-arranged interviews lasted 15-50 minutes and followup interviews lasted 5-24 minutes. Focus groups lasted an hour. Review observations lasted between 20 and 60 minutes.

Data Saturation
The concept of information power was used, rather than data saturation, since it is more in keeping with the process evaluation focus.

Transcripts returned
Transcripts were not returned to participants for comment or correction.

Number of data coders
One (

Data and findings consistent
Yes.

Clarity of major themes
Major themes are based around intervention components as the purpose of the paper is to assess implementation fidelity 32. Clarity of minor themes Not applicable