Article Text
PDF
Abstract
Objectives Assess feasibility of a cluster randomised controlled trial (RCT) to measure clinical and cost-effectiveness of an enhanced recovery pathway for people with hip fracture and cognitive impairment (CI).
Design Feasibility trial undertaken between 2016 and 2018.
Setting Eleven acute hospitals from three UK regions.
Participants 284 participants (208 female:69 male). Inclusion criteria: aged >60 years, confirmed proximal hip fracture requiring surgical fixation and CI; preoperative AMTS ≤8 in England or a 4AT score ≥1 in Scotland; minimum of 5 days on study ward; a ‘suitable informant’ able to provide proxy measures, recruited within 7 days of hip fracture surgery. Exclusion criteria: no hip surgery; not expected to survive beyond 4 weeks; already enrolled in a clinical trial.
Intervention PERFECT-ER, an enhanced recovery pathway with 15 quality targets supported by a checklist and manual, a service improvement lead a process lead and implemented using a plan–do–study–act model.
Primary and secondary outcome measures Feasibility outcomes: recruitment and attrition, intervention acceptability, completion of participant reported outcome measures, preliminary estimates of potential effectiveness using mortality, EQ-5D-5L, economic and clinical outcome scores.
Results 282 participants were consented and recruited (132, intervention) from a target of 400. Mean recruitment rates were the same in intervention and control sites, (range: 1.2 and 2.7 participants/month). Retention was 230 (86%) at 1 month and 54%(144) at 6 months. At 3 months a relatively small effect (one quarter of an SD) was observed on health-related quality of life of the patient measured with EQ-5D-5L proxy in the intervention group.
Conclusion This trial design was feasible with modifications to recruitment. Mechanisms for delivering consistency in the PERFECT-ER intervention and participant retention need to be addressed. However, an RCT may be a suboptimal research design to evaluate this intervention due to the complexity of caring for people with CI after hip fracture.
Trial registration number ISRCTN99336264.
- geriatric medicine
- rehabilitation medicine
- Hip
- delirium & cognitive disorders
- rehabilitation medicine
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information. Data requests should be directed to: Professor Chris Fox Clinical Professor, University of Exeter lhttps://orcid.org/0000-0001-9480-5704Email:Christopher.Fox@exeter.ac.uk.
This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This feasibility randomised controlled trial provides valuable evidence that the intervention and trial design can be delivered but would require a substantially larger number of trial sites and larger sample size.
As only a small proportion of people of non-white ethnicity were recruited (patients and suitable informants) it is unclear how successful recruitment and retention of participants from wider ethnic backgrounds would be.
The duration and type of cognitive impairment, that is, established dementia versus temporary delirium, was not controlled for within the analysis.
Health economic data collection should be simplified and data extracted from hospital records to reduce burden on suitable informants.
Introduction
Hip fracture is associated with advancing frailty and has substantial impact on the health, well-being and independence of older people and their families.1 2 Acute hip fracture care costs an estimated £1.1 billion per annum in the UK.3 In the 12 months after fracture, patients are at increased risk of cognitive and functional decline, admission to long-term care institutions and higher mortality.4 People with cognitive impairment (CI) are among the most vulnerable in acute hospital settings,5 with lower short-term survival and 15% mortality during admission.4 They are susceptible to suboptimal and inconsistent care standards that contribute to cognitive deterioration, increase risk of postoperative complications, prolong length of stay and cause loss of independence.6
In older adults with hip fracture, approximately 19% have dementia and up to 42% some degree of CI that may not meet criteria for a dementia diagnosis.7 People with hip fracture and CI are frequently cared for in environments which deliver excellent hip fracture care but are less skilled managing people with CI.8 9 Hospital care of patients with CI remains an ongoing area of concern5 with systemic failures in the care of older people repeatedly identified.10 Hospital staff may lack the knowledge and skills necessary to identify and assess CI, leading to underidentification which negatively affects access to rehabilitation services, supported discharge planning, person-centred care plans and involvement of families and carers.11–14
This study assessed the feasibility of a cluster design randomised controlled trial (RCT) to measure the clinical and cost-effectiveness of an enhanced recovery pathway versus standard care in acute hospitals for people after hip fracture surgery who demonstrate CI. Feasibility objectives included recruitment, retention, outcome selection, sample size estimation and acceptability of intervention training and delivery in National Health Service (NHS) services.
Methods
This paper has been prepared in accordance with the Consolidated Standards of Reporting Trials (CONSORT) Extension for Pilot and Feasibility Studies15 reporting guideline. The study methods are summarised below and previously reported in detail.16
Public and patient involvement
Patients and the pubic were involved from the conception of this study, through the review and funding process, the study, analysis and writing the findings. They were part of the steering, oversight and data monitoring groups.
Design and setting
A multicentre, feasibility, cluster RCT was undertaken between 2016 and 2018. In line with MRC guidance for complex interventions, an integrated process evaluation was conducted17; this is currently under review.
Randomisation
Randomisation was stratified by geographical area, with one intervention and one control hospital in UK region. Ten NHS hospitals were randomised to deliver experimental (PERFECT-ER) or control interventions. An additional site was recruited as a control group in July 2017 when another control site failed to recruit, and recruitment was extended from 10 months to 15 due to difficulties recruiting suitable informants. Recruitment was between November 2016 to February 2018.
Participants
Inclusion criteria
Participants were included if:
Confirmed proximal hip fracture requiring surgery.
Aged 60 years or over at the time of surgery.
Preoperative Abbreviated Mental Test Score (AMTS ≤8 in England (including those with zero because of an inability to answer questions) or a 4AT score ≥1 in Scotland.
Minimum of 5 days on the study ward.
Patient had a ‘suitable informant’ (eg, relative, unpaid or paid carer, care home manager) with a minimum of once a month face-to-face or telephone contact with the patient and able to provide proxy measures where required.
Both patient and suitable informant to be recruited into the trial within 7 days of the hip fracture surgery.
Exclusion criteria
Participants were excluded if:
Did not undergo hip surgery.
Patient not expected to survive beyond 4 weeks.
Patient already enrolled in a clinical trial of an investigational medicinal product.
Sample size
The target sample was 400 patient participants (200 per arm) from 10 centres (40 patient participants per site), based on the degree of precision for the estimated intraclass correlation coefficients (ICC). This was expected to provide a SE for the ICC of between 0.033 and 0.041, for a true ICC value of between 0.05 and 0.10 for any endpoint. A priori, it was expected that four participants would be recruited per site, per month, over 10 months recruitment period.
Participant recruitment and consent
A three-step recruitment process was implemented, guided by previous phases of the PERFECTED programme, previous studies18 19 and input from clinical and academic collaborators:
Research nurses identified all new hip fracture admissions and screened for prerecruitment eligibility in collaboration with clinical staff.
Patients (and where possible their potential suitable informant) were approached by the research nurse who provided study information as soon as clinically appropriate. Mental capacity was assessed by the research nurse, according to the appropriate legislative frameworks. In those lacking capacity to consent, consultee agreement from a relative or professional caregiver was sought, following the requirements of UK capacity legislation.20 21
The research nurse approached the patient and suitable informant to obtain written informed consent.
Intervention
Experimental intervention: PERFECT-ER
The PERFECT-ER is a multicomponent intervention, implemented using service improvement principles, comprising:
The PERFECT-ER checklist and manual.
A Service improvement lead (SIL) and PERFECTED process lead (PPL).
A model for change (plan–do–study–act).22
The checklist has 15 organisational items, and 68 individual patient items grouped into three stages (admission and preoperative; postoperative and rehabilitation; and discharge), reflecting the patient journey through acute care settings. It was designed to identify areas of strength, and potential for improvement in practice, and overarches current hip fracture guidance. A comprehensive handbook explaining how to implement and use the intervention (the PERFECT-ER manual) was provided.
In the 3 months prior to recruitment commencing, the intervention was implemented in intervention sites by the SIL working 0.50 FTE, following the handbook and adherence assessed. When sites commenced recruitment, SIL resource was reduced to 0.2 FTE for the study period. A senior clinician (PPL) assisted the SILs for an hour a week to implement PERFECT-ER then an hour per month during recruitment.
Comparator group
The control group received treatment as usual. What this consisted of was recorded to determine local practice which followed National Institute for Clinical Excellence (NICE) guidance for hip fracture care23
Outcomes
Data were collected from medical records of participating hospitals, the National Hip Fracture Database (NHFD)24 and participants and suitable informants (summarised in online supplemental table 1). Study feasibility outcome measures included: recruitment and attrition, intervention acceptability and fidelity, completion of participant reported outcome measures. The delivery of the intervention was monitored by auditing the patients notes against the PERFECT-ER checklist. Five patients per site were audited at the beginning of each implementation cycle and at the end of the trial: at 3 months pretrial, 1.5 months pretrial, trial baseline, 4 months, 7 months, 10 months, 13 months and 15 months. Clinical outcomes: mortality rate at 30 and 120 days; Bristol Activities of Daily Living Scale; hospital admissions (number, length of stay and time to first admission); falls and mortality during previous 6 months and the number of medications. Economic measures: quality-adjusted life-years (QALY) of the participant1 computed from DEMQOL-U and DEMQOL-PROXY-U) and2 computed from EQ-5D-5L completed by participants and again by proxy, QALY of the suitable informant (unpaid carer), use of health, social and unpaid care collected via the client services receipt inventory (CSRI)25 and hospital service use abstracted from hospital records. Costs of the intervention were assembled from time inputs of personnel providing PERFECT-ER, including time spent championing the ERP in study setup (online supplemental table 2). Costs of inputs per site were calculated by dividing the costs of each role by the number of potentially affected patients on each study ward over the intervention period. Unit costs for other services were from published sources.26–29
Supplemental material
Supplemental material
Statistical analysis
Clinical outcome analysis
The data analyses summarise study process information including recruitment, participant ‘flow’ and retention, sample characteristics and completeness of baseline and follow-up outcome measures. To assess fidelity of the intervention the mean ‘PERFECT-ER’ score of enacted checklist items was determined.
For each outcome measure, at each follow-up point, an ICC was calculated together with a 95% CIs. These were calculated to assist the choice of primary outcome measure and inform potential sample size calculations for a definitive trial.
A precise estimate of intervention efficacy was not a primary objective of the data analyses. However, all efficacy outcome measures were modelled using a general linear model including the baseline value of the outcome (where available) and the treatment arm. Generalised estimating equations were used to account for ‘clustering’ created by the hospital-level randomisation, thus accounting for the lack of independence of patient-level data within individual hospitals. The estimates of between arm difference are provided with 95% CIs. The relationship between the individual ‘PERFECT-ER’ score and outcomes was considered and a Pearson correlation coefficient calculated to assess the strength of the linear relationship. The difference in mean ‘PERFECT-ER’ score between those known to have died during the study and those known to have survived was also calculated.
Economic analysis
The economic evaluation took an NHS and Personal Social Services (social care) perspective and a societal perspective, incorporating costs of unpaid care and out-of-pocket expenses (for equipment, adaptations, travel to healthcare appointments).
We computed utilities (to subsequently calculate QALYs) using societal weights (DEMQOL-U from the DEMQOL; DEMQOL-Proxy-U from the DEMQOL-Proxy; and EuroQuo 5D 5L (EQ-5D-5L).30 31 QALYs over the intervention period were derived using the trapezoid method to approximate the area under the quality of life curve, with linear interpolation between time points.
We examined the ICC of QALY and total costs at 6-month follow-up, with Searle’s confidence intervals (using the arithmetic mean cluster size for unbalanced data) derived from one-way analysis of variance.32
We examined the extent to which hospital services use extracted from hospital records gave the same estimates as data collected by suitable informant report. We examined the level of agreement on frequency of service use (counts) and total hospital costs between the two sources as estimated by Lin’s concordance correlation coefficient.33 We also examined agreement between sources using the 95% limits of agreement approach,34 which calculates means and SD of paired differences and the CI for the difference, conditional on those differences being normally distributed and independent of the measures’ magnitudes.35 Research nurses recorded the time taken to complete sections of the PERFECT-ER case report forms, covering multiple instruments/questions. To calculate a time-per-question estimate, the time taken to complete the CSRI, hospital use and medications review questions was divided by the number of items in the respective sections. Time taken to complete the measures was calculated by multiplying the total number of questions by the time-per-question.
Indicative cost-effectiveness analyses were conducted but are not reported here; details are available from the corresponding author.
Results
Participant recruitment and retention
Figure 1 illustrates patient flow. Recruitment rate by centre is presented in table 1. Hospital characteristics at baseline are described in online supplemental table 3, which shows sites in both intervention and control groups are broadly similar. 282 participants, 132 from intervention sites and 150 from control, were recruited. There were 151 months of site recruitment, 70 in intervention and 81 in control sites. Average recruitment rates did not differ between intervention and control sites, ranging from 1.2 to 2.7 participants/month. Mean recruitment rate was 1.87 per site/month. This contrasts with the expected four per site/month. The demographic characteristics of the 282 study participants and suitable informant characteristics are shown in table 2.
Supplemental material
Patient flow diagram. SI; Significant Informat, AMTS; Abbreviated Mental Test Score.
Recruitment rates by centre
Participant and suitable informant baseline characteristics
Overall, the attrition rate was 50.7% (143/282). For the PERFECT-ER intervention attrition was 48.5% (64/132) and for control 52.7% (79/150).
Intervention delivery
Although implementation was standardised across sites overall compliance with the intervention fluctuated over time and between sites. This is explored fully in the process evaluation (under review).
Missing data
The degree of missing data varied across measures and across time points. For example, baseline data collection consistently demonstrated high missingness for all outcomes (online supplemental table 4). In contract, at discharge onwards, there were low missingness with the exception of the HowRwe at discharge EQ-5D-5L. Patient at 1, 3 and 6 months, and the Timed Up and Go at 3 months. The EQ-5D-5L for the suitable informant and proxy both demonstrated high missingness at 6 months in the intervention group (online supplemental table 4).
Supplemental material
Economic outcomes
For economic data collection, there was relatively low occurrence of missing data for all health utilisation variables in primary care (6%–8%) and hospital care, including both suitable informant-reported and hospital records-extracted use of emergency department, inpatient and outpatient services (4%–13%). Of a maximum of 23 medications reported, 3–4 costs were missing per case across the time points. More data were missing for suitable informant-reported unpaid care and lost working time. This was primarily because research nurses did not indicate whether the suitable informant was an unpaid or paid carer in 25% of cases at baseline and 17%, 15% and 13% of cases at 1, 3 and 6 months follow-up, respectively. Where the suitable informant was identified as an unpaid carer, rates of missingness in the unpaid carer questions were between 2% and 8% at the first three time points and 2%–11% at 6-month follow-up.
Clinical outcome feasibility
The baseline characteristics and outcomes are presented in tables 3 and 4.
Estimates of outcome
Mortality and discharge destination outcomes
Mortality
Over the duration of the trial, 57 participants (20%) died. A slightly higher rate was observed in the intervention group than in the control group, (23% vs 18%). Death in hospital was determined from NHFD data and only available for participants in England, thus excluding 59 Scottish participants. Eleven participants (5% of those with NHFD data) died in hospital with more in the control group (6% vs 4%). There were 17 (6%) patients who died within 30 days of surgery and 52 (18.4%) within 6 months.
Discharge destination
Place of discharge from hospital was identified from the NHFD data, thus unavailable for 59 Scottish participants. The largest proportion of participants returned to their own home or moved into sheltered housing (42%). This destination was more likely in the control group (47%) than the intervention group (36%).
Quality of life
No differences were seen in health-related quality of life (HRQOL) between the control group and intervention group at discharge or 1-month follow-up. At 3 months, a potential beneficial effect of the intervention over control was evidenced for patient HRQOL based on the EQ-5D-5L by proxy: those in the intervention group had a mean EQ-5D utility score 0.071 higher than control (95% CIs: (0.018 to 0.124), p=0.009), a relatively small effect of around one quarter of an SD. A difference of 0.099, in favour of the intervention group, was also seen at the 6 months follow-up (95% CIs: (0.001 to 0.198), p=0.047).
Economic outcome feasibility
Intervention costs across the five study wards ranged from £131 to £485 per patient over the study period (online supplemental table 5). There were no significant differences in total costs between groups at any time point except in total health and social care (HSC) costs (including intervention costs) at 3 months using suitable informant reported data (£4004, 95% CIs: £30 to £7979, p=0.049). Total costs (including intervention costs) at each time point are summarised in online supplemental table 6.
Supplemental material
Supplemental material
Total costs over the intervention period (online supplemental table 7) differed depending on the perspective and the source of data on hospital utilisation. HSC costs based on suitable-informant-reported data, including or excluding intervention costs, were significantly higher in the intervention than control group. However groups did not differ on total societal costs, including or excluding intervention costs, regardless of source. Suitable informant data differed from the hospital records-extracted data in that it could include hospital stays from trusts other thans those providing the hospital records, which may partly explain discrepancies between costs from different sources.
Supplemental material
Group ICCs for 6 months costs and QALY are given in online supplemental table 8 . In the costs data, a pattern of negative ICC estimates indicated little clustering in the intervention group but some degree of clustering in the control group data. ICC for QALY ranged from 0.004 to 0.268 in the intervention and from −0.04 to 0.263 in the control group.
Supplemental material
Concordance between hospital records-extracted and suitable-informant-reported sources on frequency of hospital service use and costs was generally weak, although Lin coefficients ranged between 
= 0.099 and 
= 0.813 for service use across time points (online supplemental table 9). Concordance on hospital costs was high at the baseline (
=0.660) but was 
=0.379 at 1 month and 
<0.3 at three and 6 months. Limits of agreement showed that the two measures yielded estimates within £3400 of each other at baseline, £7000 at 1 month and similar at 6 months, but at 3 months the limits of agreement were much wider (£8020 to £10 693).
Supplemental material
Sample size calculation
ICCs were estimated, with 95% CIs to inform a sample size calculation. The highest value was estimated for the PERFECT-ER score, 0.748, indicating a substantial degree of between-hospital variation compared with variation between-individuals within hospitals. This is not surprising given the intervention aimed to standardise practice within intervention hospitals thereby inflating the ICC. At follow-up time points, the ICCs typically ranged between 0.05 and 0.1. At 6 months, estimates for the MMSE and EQ-5D-5L by proxy were negative and, since a negative value is theoretically not possible and results from estimation error, these were interpreted as being a ‘small’, positive value, near to zero.
Discussion
The findings indicate that modifications are necessary to the trial design for a viable definitive trial. While this study successfully demonstrated the ability to recruit from a variety of different UK sites, the rate was lower than anticipated. There was a lot of missing data for some measures, therefore, steps to improve retention of participants at follow-up time points is warranted, and a sufficiently large inflation of the sample size is required to compensate for missingness. Mortality has been suggested as an appropriate primary outcome. Economic data collection proved burdensome to suitable informants. A definitive trial should reduce this burden for example, by extracting hospital services use data from hospital records.
We hypothesise short-term mortality (30 days) may be reduced by the PERFECT-ER intervention due to the cumulative effect of increased good practices across the range of care domains. This builds on previous work10 36–38 which recognises complex associations between hospitalisation, pre-admission CI, postadmission CI, functional decline and mortality. Through this, we would recommend mortality be a proposed primary outcome if a future definitive trial is undertaken.
Complex interventions that focus on staff quality improvement and associated implementation methods such as plan–do–study–act methods22 present challenges for investigation using RCTs.39 The management and care of people with dementia and CI with hip fracture is complex. This is an example of a ‘wicked problem’, defined as complex, messy and stubborn challenges which continually evolve and has, at its core, many reasons for being, with no single solution which can be applied in all circumstances. Ultimately ‘wicked problems’ are those which cannot be reduced to a set of fixable problems and are often impossible to ‘solve’ because of incomplete, competing and changing requirements and where the solutions needed are ‘better or worse’ rather than ‘right or wrong’.40–42 While pragmatic RCTs, which offer tailoring and flexibility in experimental interventions, are one approach to testing management strategies for such healthcare challenges, other research methodologies may provide important insights. Further consideration of a range of methodological approaches may be more appropriate to answer this research question before automatically embarking on a clinical trial pathway.
Conclusion
This study has demonstrated that PERFECT-ER can be implemented and widely accepted across a number of different health services in the UK’s NHS. We have shown it is feasible, with modifications, to undertake a definitive trial and economic evaluation using the developed and refined recruitment and consenting practices. However, care of people with CI and hip fracture poses a ‘wicked problem’ and further definitive research using an RCT approach should be deliberated against other methods of evaluation.
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information. Data requests should be directed to: Professor Chris Fox Clinical Professor, University of Exeter lhttps://orcid.org/0000-0001-9480-5704Email:Christopher.Fox@exeter.ac.uk.
Ethics statements
Patient consent for publication
Ethics approval
Ethical approval for the trial was given by Camden and Kings Research Ethics Committee (reference number: 16/LO/0621) and Scotland Research Ethics Committee A (reference number: 16/SS/0086). Participants gave informed consent to participate in the study before taking part.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @JaneCross12, @DrSiHammond
Contributors JLC, SPH, LS, FP, CH, TB, BP, SD, DL, MK, AM, MP, OS, TOS, JW, RH, CB and CF made substantial contributions to the conception or design of the work; DL provided PPI input throughout the study. SPH and TB led on the acquisition of the data, LS, CF, SPH and JLC led the statistical analysis and interpretation of data for the work; JLC led the drafting of the paper. All authors were involved in revising it critically for important intellectual content; All authors reviewed the paper and gave their final approval of the version to be published; all authors give their agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. CF accepts full responsibility for the work and/or the conduct of the study, had access to the data, and controlled the decision to publish.
Funding This work was funded by NIHR Programme Grants for Applied Research (PGfAR) Programme grant number (ref: DTC-RP-PG-0311–12004). the views expressed are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.