Article Text
PDF
Abstract
Objective A new prognostic model has been developed and externally validated, the aim of which is to assist in the management of the blunt chest wall trauma patient in the emergency department (ED). The aim of this trial is to assess the feasibility and acceptability of a definitive impact trial investigating the clinical and cost-effectiveness of a new prognostic model for the management of patients with blunt chest wall trauma in the ED.
Design Stepped wedge feasibility trial.
Setting Four EDs in England and Wales.
Participants Adult blunt chest wall trauma patients presenting to the ED with no concurrent, life-threatening injuries.
Intervention A prognostic model (the STUMBL score) to guide clinical decision-making.
Outcome measures Primary: participant recruitment rate and clinicians’ use of the STUMBL score. Secondary: composite outcome measure (mortality, pulmonary complications, delayed upgrade in care, unplanned representations to the ED), physical and mental components of quality of life, clinician feedback and health economic data gathering methodology for healthcare resource utilisation.
Results Quantitative data were analysed using the intention-to-treat principle. 176 patients were recruited; recruitment targets were achieved at all sites. Clinicians used the model in 96% of intervention cases. All feasibility criteria were fully or partially met. After adjusting for predefined covariates, there were no statistically significant differences between the control and intervention periods. Qualitative analysis highlighted that STUMBL was well-received and clinicians would support a definitive trial. Collecting data on intervention costs, health-related quality of life and healthcare resource use was feasible.
Discussion We have demonstrated that a fully powered randomised clinical trial of the STUMBL score is feasible and desirable to clinicians. Minor methodological modifications will be made for the full trial.
Trial registration number ISRCTN95571506; Post-results.
- trauma management
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
The main strength of this feasibility trial is that it has tested all of the methodological components of the future definitive trial, including the trial infrastructure, randomisation process, data management system, both qualitative and quantitative analysis, follow-up procedures and a health economic analysis.
The health economic analysis undertaken will ensure accurate future funding applications for the full definitive trial.
The qualitative analysis has informed the researcher teams’ understanding of the use of the risk score by clinicians in daily clinical practice.
The cluster randomised trial design that needed to be tested in this feasibility trial was limited by the small number of clusters (emergency departments participating). In the future definitive trial, more clusters will be possible, due to the greater number of participating sites.
Introduction
Blunt chest wall trauma accounts for over 15% of all trauma admissions to emergency departments (EDs) worldwide, with reported mortality ranging between 4% and 60%.1–3 Difficulties in the management of blunt chest wall trauma patients in the ED are becoming increasingly well recognised in the literature.4 5 The patient commonly presents to the ED with no respiratory difficulties, but can develop complications up to approximately 72 hours after initial presentation.1–4 Clinical symptoms are not considered by most clinicians to be an accurate predictor of outcome following non-life-threatening blunt chest wall trauma.1–7 Decisions regarding the appropriate level of care required by the patient following discharge from the ED are therefore difficult and are further compounded by the lack of current national guidelines. Improvements in identification of high-risk patients on initial assessment in the ED could lead to a reduction in these poor outcomes.8 9
A new prognostic model (STUMBL Score) to assist the management of blunt chest wall trauma patients in the ED has been developed and externally validated in a large multicentre study.1 5–7 Guidelines recommend that prognostic models should not be used in clinical practice until an impact trial has been completed, in which the clinical application of the model has been tested.10 11 The aim of this trial was to establish the feasibility and acceptability of a future definitive trial, which will determine whether the prognostic model can be used safely and effectively in clinical practice in the UK.
Methods
Trial design and randomisation
This was a multicentre, prospective, cluster randomised (stepped wedge) trial12 with a qualitative component. The unit of randomisation was the ED, rather than the patient, since the intervention was used by ED clinicians with the aim of studying effects on patient outcomes.13 All EDs began as controls testing conventional management without the model for a period of 1 month. Every month, one hospital was randomly assigned to the intervention period (using the prognostic model), until all hospitals were intervention sites. Online supplementary file 1 outlines the trial procedure, and the protocol describes the trial in full.14
Supplemental material
Population
Patients presenting to the ED with isolated blunt chest wall trauma were included in the trial if they were aged 18 years or over and capable of giving consent to participation. Exclusion criteria were under the age of 18 years, lacked capacity to give informed consent, presented with immediately life-threatening injuries or any concurrent injury that would determine the patient’s management.
Patient and public involvement
Two ex-trauma patients were members of the STUMBL Trial Development Group that developed the protocol and designed the study. They continued to sit on the Trial Management Group, attending all research meetings and contributing to the overall running of the trial. A further two ex-trauma patients sat on the Trial Steering Committee (TSC). Trial participants were instructed to contact the research team if they were interested in the trial results.
Setting and recruitment
This feasibility trial ran in four EDs in the UK: Royal Gwent Hospital, Newport; Musgrove Park Hospital, Taunton (both non-major trauma centres); Salford Royal Hospital and Manchester Royal Infirmary (both major trauma centres). ED clinicians and research nurses were responsible for screening, recruiting and consenting eligible patients to the trial.
Sample size
The trial had a 5-month recruitment period, and each hospital aimed to recruit 30–80 patients. This was the minimum number considered necessary to test data collection processes based on existing recommendations for meaningful estimates of parameters of interest.15 This recruitment period also allowed for loss to follow-up, low response rates with follow-up surveys and difficulties with recruiting at weekends if research nurses only worked Monday to Friday.
Intervention
The intervention was a prognostic model to guide clinicians' decision-making. The model was used during initial patient assessment and provided a suggestion of the appropriate management (whether the patient could be safely discharged home or needed admission to either a ward or a critical care). Participants were asked to complete one survey (SF-12v1)16 on initial presentation and two more surveys at 6 weeks (SF-12v1 and a Client Services Receipt Inventory (CSRI)). Online supplementary file 2 illustrates the patients’ journey.
Supplemental material
Outcome measures
Primary outcome measures
1. Patient recruitment rate.
2. Clinicians’ use of the model in the intervention period.
Secondary outcome measures
Patient outcomes: (1) composite outcome measure: which includes in-hospital mortality, pulmonary complications (chest infection, pneumonia, pneumothorax, haemothorax, pleural effusion or empyema) diagnosed during hospital admission, delayed upgrade in care (defined as a deterioration in a patient’s condition on the ward, requiring admission to critical care) and unplanned representations to the ED, (2) overall mean quality of life (as reported by SF-12v1 survey).
Organisational outcomes: (1) clinician recruitment rate, (2) participant questionnaire response rate and (3) clinicians training attendance rate.
Qualitative outcomes: (1) clinician feedback regarding training quality and use of the prognostic model in clinical practice, (2) clinician and research nurse feedback regarding trial methods.
Health economic outcomes: (1) trainer/trainee hourly rates/training time/travelling time, (2) equipment costs and (3) resource usage by patients during the trial period.
Criteria for establishing feasibility
In order to evaluate the feasibility of a full definitive trial, the TSC assessed the findings against predetermined success criteria (box 1) using a traffic light system.17
Trial feasibility criteria
Primary outcomes
80% or more of eligible patients consent to data collection and follow-up.
90% or more use of the model by clinicians during intervention period.
Secondary outcomes
95% or more of clinicians working within the participating emergency department agree to take part in the trial.
Follow-up data for patient secondary outcomes can be collected for 80% or more of patients.
All clinicians involved in the trial receive formal training in the use of the model.
Patient secondary outcome measures reported in the intervention period are equal to, or better than, those reported during the conventional management period.
Mean quality of life reported in intervention arm is not less than 80% of that reported in control arm.
Traffic light assessment: Green means the target was achieved, amber means the target was not achieved but progression is possible with some minor protocol modifications and red means progression to a full trial is not possible.
Data collection and management
Research Electronic Data Capture18 was used for data capture, management and completion of the electronic case report forms.
Data analysis
Quantitative analyses were performed on Stata V.15.1 SE using the intention-to-treat principle and were not powered to detect clinically important effects. Results are presented as numbers (percentages), means (SD) and medians (interquartile ranges) where non-normally distributed. Baseline characteristics between groups were completed using the Wilcoxon rank-sum test and Fisher’s exact test. The risk score was also calculated retrospectively for control cases for baseline comparisons purposes only.
A binary composite outcome measure was calculated, indicating if the participant had experienced at least one of the contributing factors (mortality, unplanned representation to the ED, delayed upgrade in care or onset of pulmonary complications). Intervention/control groups were compared using logistic regression analysis (using exact logistic regression where there were zero-count cells). Covariates included age at the time of presentation, the number of clinically suspected rib fractures, preinjury use of anticoagulants, the presence of pre-existing chronic lung disease and oxygen saturation on initial assessment in the ED. Analysis of individual components of the composite outcome measure was completed using the same method.
Sensitivity analyses were undertaken following clinical review of the data. The following factors were considered: the type of logistic regression analysis used, inclusion of additional covariates, representations leading to immediate discharge and unrelated representations/deaths. Each was analysed individually, on the composite outcome measure, with all covariates included.
A purposive sample of the clinicians based on seniority, location and trial participation were invited to take part in a short interview in which they were asked to discuss the model in relation to their clinical practice. Two research nurses (where available, the Royal Gwent Hospital only had one research nurse) from each site were asked to attend a focus group in which the trial’s methodology was discussed. One site was unable to attend. Focus group and interviews were audio-recorded and transcribed verbatim by a professional transcription service. The qualitative researcher (CO’N) monitored this process; transcripts were checked for accuracy against the original recording and were anonymised in accordance with best practice standards. The cleaned and anonymised transcripts were uploaded into NVIVO V.11.19
Thematic analysis was used to analyse the transcripts using a code book, which developed initially from the background literature and feedback on the training as well as issues that emerged through the process of constant comparison, which underpins qualitative data analysis. Due to the small numbers of transcripts, the coded transcripts were checked by a second qualitative researcher, using the coder comparison query tool.
Health economic analysis focused on establishing the most appropriate framework for a future health economic analysis. The feasibility of collecting data on quality of life and resource use was assessed, along with a provisional assessment of cost categories associated with the intervention (eg, staff training time). To capture resource use by patients from an NHS/PSS (National Health Service/Personal Social Service) perspective, an adapted resource usage questionnaire was used. Data were assessed to examine the completeness of data captured, such as response rate and missing items.
Results
All sites were able to recruit over the 5-month period, and there were no difficulties implementing the stepped wedge design (online supplementary file 1). There were no obvious differences between numbers recruited at each site when considering major trauma centre status, with the Royal Gwent Hospital recruiting 31 patients, Musgrove Park Hospital recruiting 47 patients (plus one withdrawal of consent), Salford Royal Hospital recruiting 38 patients and Manchester Royal Infirmary recruiting 60 patients (figure 1). There was some inconsistency around screening/consenting between sites. For example, research nurse availability affected the recruitment and consent process in smaller sites. Another inconsistency occurred as one site did not keep a screening log throughout the trial.
STUMBL CONSORT diagram. CONSORT, Consolidated Standards of Reporting Trials; CSRI, Client Services Receipt Inventory .
Feasibility criteria
All feasibility criteria were green or amber (table 1). The primary outcomes were both green. It proved impractical to obtain precise figures for the number of clinicians working in EDs over the trial period, but the information available suggests that no one dissented from using the tool and all were appropriately trained. There were initial difficulties in obtaining follow-up questionnaire data; however, response rates improved once telephone and email follow-ups were introduced, justifying the amber status. Clinical follow-up data were only from hospital records (ie, participants who were admitted or subsequently represented to hospital).
Feasibility outcomes
Baseline characteristics
The intervention group was significantly older, with more severe injuries and more likely to sustain a low velocity fall, when compared with the control group. There were a significantly higher number of road traffic accidents in the control group (table 2). To understand the differences between groups, we retrospectively calculated the risk score for participants in the control group using routinely collected data. We found that the intervention group scored significantly higher on the prognostic model than the control group.
Baseline subject characteristics
Clinical outcomes
There was a complication rate of 8%, when considering the composite outcome measure (table 3). The ORs for mortality and delayed upgrade in care were calculated using exact logistic regression with no covariates due to 0-count cells.
Secondary outcome measure (composite measure)
Results of the quality of life analysis (SF-12v1) found no statistically significant difference between the intervention and control groups, in either the physical or mental components (table 4).
Health-related quality of life
Sensitivity analysis—composite outcome measure only
The type of regression analysis undertaken had the largest effect on the results. Due to the low number of events seen, we considered the impact of using an exact logistic regression (using the Stata exlogistic command) and a Penalised Maximum Likelihood Estimation (the ‘Firth method’, using the Stata firthlogit command). Exact logistic regression produced results consistent with our initial calculation; the Firth method produced results that were more favourable to the intervention. Further analysis that considered other covariates made little difference to the overall findings, nor did whether the prognostic model’s management guidance was followed, accounting for the unrelated deaths/representations, or our assumption that all pulmonary complications diagnosed on the day of randomisation were present at admission. Excluding representations leading to a discharge had a large impact on the size of the 95% CI, but this may be due to very low number of events remaining for analysis (online supplementary file 3).
Supplemental material
Missing data
Some data did not appear to be missing entirely at random, with one site missing considerably more data than the others. Exploration suggested that this was due to lower research nurse levels and unforeseeable circumstances. There was difficulty in obtaining completed CSRI forms compared with SF-12s. Some CSRIs were completed by research nurses using available NHS records and contacting General Practitioner (GPs), particularly in one of the early intervention sites, leading to a higher proportion completed than would otherwise be the case.
Serious adverse events
Two deaths were reported. One was unrelated to the chest injury, the other patient was placed on a palliative care pathway following admission to hospital.
Qualitative results
Four research nurses attended the focus group and nine clinician interviews were completed. Thirty-six training feedback questionnaires were completed. Following coding, three main themes covering issues of process, impact and mainstreaming were identified.
Trial processes refer to informants’ perspectives on the running of the trial from planning and design issues through to data collection, publicity and training. Staff were positive about the trial, they liked the stepped wedge design and data collection was largely unproblematic. The use of social media was commended and staff were keen for this to be retained. Using Twitter to drive patient recruitment was highlighted by everyone as a great motivator and a way to encourage team involvement and to drive recruitment.
The consent model used in this trial, and the alternative of presumed consent, was not considered contentious by the teams. The training was well received and could be offered in different formats. Data collection tools and improvements made to them during the trial were considered appropriate. Some improvements such as telephone follow-ups worked well. Otherwise, there were no other suggested changes regarding trial processes. It was clear that staff appreciated the value of the STUMBL Score and would support a definitive trial to ensure its wider implementation.
Health economic evaluation
There were no significant differences in quality of life between control and intervention groups at either baseline or follow-up. However, analysis only included complete cases (n=126 of 176), with no imputation undertaken, and should be viewed with caution. The mean training time was 31 min (SD: 24 min), and it usually took place at a trainee’s place of work. As the pay grades for clinicians varied, training costs varied substantially between £6.45 and £108 per session (mean = £31.82, SD = £20.6). When training could be carried out in a trainee’s place of work, the cost per person was minimised.
The CSRI and SF-12 captured the data required for a full economic evaluation in the main trial. Measures can be taken to adapt the instruments to capture key data that are likely to have a greater impact on costs.
Discussion
This is the first trial to test the feasibility of conducting a cluster-randomised controlled trial into the effectiveness of a prognostic model for the management of blunt chest wall trauma patients presenting to the ED. All feasibility criteria were met, suggesting that progression to a full impact trial is possible.
This trial has highlighted some minor modifications that need to be made to the methods prior to moving forward to the full trial. Research nurse availability varied substantially between sites. In one of the smaller (trauma unit) sites, difficulties in patient recruitment were noted due to the fact that there was only one research nurse working on the trial, so out-of-hours recruitment was not possible. This suggests that capacity is something to consider carefully when recruiting sites for any future definitive trial. Reliance on research nurses to recruit participants, rather clinicians recruiting as part of patient management, will need consideration when planning the future trial. A retrospective consent model was discussed in some clinician interviews, which may need consideration as one strategy to improve recruitment numbers.
Participant outcomes were compared to ensure that there were no large differences between groups and that we remain in equipoise regarding use of the model in EDs. Although analysis of the composite outcome measure suggests a difference between groups, this may be because the two major trauma centres were the last sites to transition to the intervention (and therefore had longer in the control phases) and collectively, they recruited most patients to the trial. The intervention group was significantly older, more severely injured and scored higher on the prognostic model than the control group (using retrospective scores for the control group). A larger number of sites will be used in the future trial, which should reduce baseline variation. Overall, the trial population was younger than previous research1 6 by the group which may be due to the inclusion of a higher proportion of major trauma centres. Due to a low number of adverse events compared with previous research, the regression analysis produced wide CIs. This trial is not powered to detect differences in outcomes, and therefore any differences between groups should be considered with caution.
The complication rate was lower than that reported in our development and validation work.1 6 This could be explained by the proportion of major trauma centres included in this trial, when compared with the previous work, although this would need further investigation. This information will prove useful when calculating the sample for a full trial.
There was difficulty in obtaining completed CSRI surveys. As a result, some CSRIs were completed by research nurses using available NHS records and contacting GPs. The most costly data gathered from the CSRI questionnaire would normally be inpatient stays and outpatient visits, especially when there are multiple stays/visits following the initial visit to the ED. Unfortunately, an oversight of the timing of the CSRI meant questions only asked about the 6 weeks following discharge, resulting in a failure to capture costs involved in the initial ED treatment. This is a key cost driver, and in a full trial, the CSRI questions would be amended to capture recall since arrival at ED.
In conclusion, this work has demonstrated that progression to the full definitive impact trial is feasible, with some minor modifications.
Acknowledgments
The authors acknowledge the contribution of all research nurses, clinicians, R&D Department teams, TMG and TSC members, and Swansea Trials Unit staff at Swansea University. The authors also acknowledge the work of the patient representatives Mrs Hazel Spencer, Mr David Roberts, Mr Joe Waters and Ms Joanne Prosser.
References
Footnotes
Contributors All authors (CB, PAE, HAH, CO’N, TD, AW, SG, FEL, RB, JG, SJ, ZA) contributed to the conception and design of the trial. CB, HAH, ZA, FEL, RB, PAE, SJ and JG contributed to the running of the trial. TD, AW, HAH, CO’N and SG completed the data analysis. CB, HAH, TD, CO’N and SG wrote the initial draft and all authors (CB, PAE, HAH, CO’N, TD, AW, SG, FEL, RB, JG, SJ, ZA) contributed to its revision.
Funding This trial is supported by a Research for Patient and Public Benefit (RfPPB) Grant from Health and Care Research Wales. Project reference: 1193.
Competing interests None declared.
Ethics approval This trial received ethics approval by the WalesResearch Ethics Committee 6 (Ref: 16/WA/0290).
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Data are available upon reasonable request.
Patient consent for publication Not required.