Recruitment, consent and retention of participants in randomised controlled trials: a review of trials published in the National Institute for Health Research (NIHR) Journals Library (1997–2020)

Objectives To review the consent, recruitment and retention rates for randomised controlled trials (RCTs) funded by the UK’s National Institute for Health Research (NIHR) and published in the online NIHR Journals Library between January 1997 and December 2020. Design Comprehensive review. Setting RCTs funded by the NIHR and published in the NIHR Journals Library. Data extraction Information relating to the trial characteristics, sample size, recruitment and retention. Primary and secondary outcome measures The primary outcome was the recruitment rate (number of participants recruited per centre per month). Secondary outcomes were the target sample size and whether it was achieved; consent rates (percentage of eligible participants who consented and were randomised) and retention rates (percentage of randomised participants retained and assessed with valid primary outcome data). Results This review identified 388 individual RCTs from 379 reports in the NIHR Journals Library. The final recruitment target sample size was achieved in 63% (245/388) of the RCTs. The original recruitment target was revised in 30% (118/388) of trials (downwards in 67% (79/118)). The median recruitment rate (participants per centre per month) was found to be 0.95 (IQR: 0.42–2.60); the median consent rate was 72% (IQR: 50%–88%) and the median retention rate was estimated at 88% (IQR: 80%–97%). Conclusions There is considerable variation in the consent, recruitment and retention rates in publicly funded RCTs. Although the majority of (6 out of 10) trials in this review achieved their final target sample; 3 out of 10 trials revised their original target sample size (downwards in 7 out of 10 trials). Investigators should bear this in mind at the planning stage of their study and not be overly optimistic about their recruitment projections.


INTRODUCTION
Randomised controlled trials (RCTs) are the 'gold-standard' research design for evaluating the effectiveness of interventions in health, education and policy. 1 Conducting an RCT requires major financial investment and substantial amounts of public funding is spent in this area each year.In 2019/2020, the National Institute for Health Research (NIHR) in England awarded over £250 million of funding to 310 research projects with a substantial proportion of this invested in RCTs. 2 There are many practical challenges associated with conducting clinical trials.The leading reason for premature discontinuation of RCTs is poor recruitment of participants 3 4 with accrual often taking longer or being more difficult than expected.Poor recruitment can have a number of consequences including the study being underpowered if the target Strengths and limitations of this study sample size is not met and increased costs if an extension is required. 5Furthermore, discontinued RCTs are less likely to be published in medical journals 3 which has ethical implications around research waste. 6here have been a number of previous studies in the UK investigating recruitment and retention in publicly funded RCTs.The earliest review, a cohort of trials funded by the Medical Research Council and NIHR Health Technology Assessment (HTA) between 1994 and 2002, reported that 31% (38/122) of the trials successfully recruited to their original recruitment target, with 54% (65/122) of trials awarded a grant extension. 7There is evidence of a marginal improvement in these figures over time, with results from a cohort of 151 RCTs funded by the NIHR HTA programme between 2004 and 2016 finding that 40% (61/151) of trials successfully recruited to their original sample size, and 32% (49/151) of trials extended their recruitment. 8In the same study, the median recruitment rate was found to be 0.92 (IQR: 0.43-2.79)participants per centre per month.
Following the publication of the review by Walters et al 8 in 2017, there have been several Cochrane systematic reviews looking at strategies for improving recruitment 9 and retention 10 of participants in RCTs.Two strategies for improving recruitment were identified with highcertainty evidence: using open trials rather than blinded, placebo controlled trials, and telephone reminders to people who did not respond to postal invitations.There has also been a systematic review of statistical models for predicting recruitment at the design stage of a clinical trial 11 but a survey of statisticians in UK and European clinical trial networks found that 90% (62/69) did not use statistical models for recruitment prediction. 12In 2014, a trials methodology research priority setting exercise was conducted using a Delphi survey of directors of UK Clinical Research Collaboration registered Clinical Trials Units (CTUs). 13Two of the three highest priority areas were 'Research methods to boost recruitment in trials' and 'Methods to minimise attrition'.
The Consolidated Standards of Reporting Trials (CONSORT) statement was first published in 1996, 14 and revised in 2001 15 and 2010. 16It is a checklist of standards for reporting how a trial was designed, analysed and interpreted, and it has been endorsed both by prominent general medical journals and many specialist medical journals. 17However, reporting guidelines such as CONSORT are not adopted and adhered to as much as they should be 18 with the previous review of recruitment and retention in RCTs by Walters et al 8 finding that 63% (95/151) of trials demonstrated complete compliance with CONSORT statement and reported each of the number: screened, eligible, declined consent, recruited and assessed for their primary outcome.
This review aims to update previous research on how well recruitment and retention figures are reported, and the rates of recruitment and retention in trials published in the NIHR HTA Journal between January 2004 and April 2016. 8In this study, we update and extend this review to look at trials published in the NIHR Journals library from January 1997 to December 2020.study. 19Reports of internal pilot trials that either went on to contribute outcome data to a full trial or were terminated prior to the full trial because of recruitment issues were included in the review.

Data extraction
After the NIHR reports had been selected for inclusion, information was extracted using a standardised data extraction form.For each of the included trials the following information was extracted.
► Trial characteristics, including the trial design, clinical area, type of intervention, type of control, number of arms, use of blinding of trial participant, geographical region, number of centres, any support provided by a CTU and whether there was any description of pilot or feasibility work done prior to the start of the trial.► Sample size, recruitment and retention information, including the target and actual sample size, the overall and centre-specific recruitment period and CONSORT information on the numbers screened, consented, randomised and analysed for the primary outcome. 16he selection of RCTs and data extraction was conducted by a team of reviewers (RMJ, RA, JH, AR and IS).Three reviewers (RMJ, RMS and SJW) conducted quality assurance checks on 30% of the included trials after the data extraction was completed, and disagreements were discussed to achieve consensus.

Analysis
The primary outcome for the review was the recruitment rate for each trial.This was defined as the number of participants recruited and randomised per centre per month.Where explicit dates were reported the recruitment rate was calculated as the time between the date of recruitment start and the date of recruitment completion.In cases where only the months of recruitment were reported the recruitment period was estimated as the time between the first of the month and the end of the final month.If the date of the first participant recruited was reported instead of the start date of recruitment then the start of recruitment was taken as the first of the month of the first participant recruited.When the start of recruitment was not reported the start of screening was used to calculate the recruitment period.The recruitment period was estimated by subtracting the length of the follow-up period from the length of the study period when explicit information on the start and end of recruitment was not reported.
The recruitment rate was calculated in two different ways.The overall recruitment rate was calculated as the total number of participants recruited divided by the maximum number of recruiting sites, then divided by the total number of months that the trial recruited for.This overall recruitment rate is likely to be an underestimate for multicentre trials because each trial site is unlikely to open for recruitment at the same time and will not recruit for the entire recruitment period.To allow for the difference in start-up times and recruitment periods between sites, where available, the site-specific recruitment periods were extracted.These were averaged over the number of sites to give an average site-specific recruitment period.The average recruitment rate was calculated as the total number of participants recruited divided by the maximum number of sites, then divided by the average number of months that the trial recruited for.
The secondary outcomes for the review were the target sample size and whether it was achieved, the consent rate and the retention rate.The consent rate was calculated as the percentage of eligible participants that consented and were randomised (ie, the total number of participants recruited and randomised divided by the number of eligible participants).The retention rate was calculated as the percentage of randomised participants that were assessed for the primary outcome and included in the analysis of the primary outcome (ie, the number of participants included in the analysis of the primary outcome divided by the number of participants recruited and randomised).
Recruitment rates were summarised using the median and IQR due to the skewed distribution of the data. 20he median and IQR were also used to summarise the secondary outcomes of the consent and retention rates.Comparisons of recruitment and retention rates were made between different trial characteristics using appropriate non-parametric tests; Mann-Whitney U test (for characteristics with two levels), Kruskal-Wallis test (three or more nominal levels) and Jonckheere-Terpstra test (three or more ordered levels).Analysis was conducted on a complete case basis so where the characteristics information, recruitment rate or retention rate were missing these were excluded.All statistical analyses were conducted in R V.4.1.0, 21figures were produced using the package ggplot2 22 and the Jonckheere-Terpstra test conducted using the package clinfun. 23tients and public involvement Patients and/or the public were not involved in the design, conduct, reporting or dissemination plans of this research.

RESULTS
Between 1 January 1997 and 31 December 2020, 1899 reports were published in the five NIHR journals.Following screening, 1299 of these were excluded as reports of non-RCTs.The search identified 600 reports of RCTs of which 221 were excluded after applying the exclusion criteria (101 cluster RCTs; 95 pilot/feasibility RCTs; 14 follow-on studies; 6 adaptive designs; 3 influenza vaccination trials and 2 ongoing trials).Eight NIHR reports described the results of multiple independent trials (7 reports described 2 RCTs and 1 described 3 RCTS), therefore in total, 388 individual RCTs from 379 reports were included in the review and analysed as shown in figure 1.This includes 151 RCTs from the review by Walters et al. 8

Trial characteristics
The characteristics of the 388 trials included in the review are summarised in The recruitment and sample size characteristics of the RCTs included in the review are summarised in table 2. The majority of trials (354/388) were multicentre with a median of 17 centres (IQR: 7-37).The final recruitment target (sample size) ranged from 44 participants to 46 000 participants and the final number recruited ranged from 2 participants to 47 062.The RCT with the highest final recruitment target and highest number recruited was an obstetrics trial investigating computerised interpretation of fetal heart rate during labour. 24There were four trials that recruited less than 10 participants, two were discontinued at the end of an internal pilot phase due to low recruitment 25 26 and the remaining two had no pilot

Open access
phase. 27 28Overall, 63% (245/388) of trials recruited to their final recruitment target but 32% (79/245) of these trials required an extension to their recruitment period to meet the target.A further 22% (86/388) of trials recruited to within 80% of their final recruitment target with 36% (31/86) of these trials having an extension to their recruitment period.The original recruitment target was revised in 30% (118/388) of trials (downwards in 67% (79/118)).For the majority of trials the primary outcome was collected at between 1 and 18 months postrandomisation.Recruitment, consent and retention rates From the 365 trials with sufficient information to calculate the recruitment rate, the median was found to be 0.95 participants recruited per centre per month.The highest recruitment rate (57.75 participants per centre per month) was in a trial comparing medical to surgical termination of pregnancy with a target sample size of 2232 women 29 and the lowest (0.01 participants per centre per month) was in a trial treatment for transverse myelitis. 27The 80th and 90th percentiles were found to be 3.70 and 9.47 participants recruited per centre per month, respectively.From the 22 single centre trials with sufficient information, the median recruitment rate was found to be 14.However, there are only six trials from the PHR journal included in this review and three of these trials [30][31][32] have a recruitment rate of 10 participants per centre per month or greater.Figure 2 shows the distribution of recruitment rates by clinical area.The highest median recruitment rate was for dentistry (1.95 participants recruited per centre per month) but this was only from five trials.][38] The median consent rate (percentage of eligible participants consented and randomised) was found to be 72% (IQR: 50%-88%).Table 4 shows some variability in consent rates between the journals with the HS&DR journal having the largest median rate (81%, IQR: 60%-97%) and the PHR journal the lowest (57%, IQR: 40%-68%).However, there is not an overall statistically significant difference in consent rates between the five NIHR journals (p=0.225).The median retention rate (per cent of randomised participants retained and assessed in the analysis of the primary outcome) was found to be 88% (IQR: 80%-97%).There were four trials 25 26 28 39 with a retention rate of 0%, these trials were all stopped early due to problems with recruitment and the planned statistical analysis for the primary outcome was not performed.Retention rates do not differ greatly between the five NIHR journals (p=0.118)(table 4).

CONSORT and recruitment data
The trial recruitment and retention rates are summarised by trial characteristics in tables 5 and 6, respectively.There is some statistical evidence of an association between the setting of the trial, final recruitment target and the total number of participants recruited but the median rates show no clear patterns to these associations.
The results of the current review, in terms of successful recruitment to target sample size, have been compared Open access with three previous reviews 5 7 8 in table 7.As this review updates the findings of Walters et al 8 and due to there being some overlap with the trials included in Sully et al 5 ; a column has been included for the non-overlapping time interval (2017-2020) in addition to the full time interval (1997-2020).Figure 3 shows the percentage of trials recruiting 100% of the final target and 80% or more of the final target by publication year.There is no clear trend in the to 2020.However, there is some evidence of a difference in recruitment rates between the five NIHR journals, and therefore, any improvement may be due to the inclusion of trials from the journals (EME, PGfAR, HS&DR and PHR) that were not included in the review by Walters et al. 8 There is still cause for some concern with 30% (118/388) of trials revising their original recruitment target with the majority (67% (79/118)) revising the target downwards, and a third (128/388) of trials having an extension to their recruitment period.These findings remain consistent with the concerns expressed by CTU directors. 13he median consent and retention rate are consistent with the result of Walters et al. 8 The retention figure may be an overestimate as it will be affected by trials using time to event outcomes, where missing outcomes are censored at the time of loss to follow-up but included in analyses using survival models.The target sample size for any trial should allow for participant withdrawals and loss to follow-up 40 with the expected withdrawal proportion obtained from reports of studies conducted in the same clinical area.20 However, if no such information is available then a pragmatic approach would be to use the median retention rate from this review (88%) and assume an expected withdrawal proportion of at least 10%.
This study has the following limitations.First, the review was restricted to publicly funded trials published in the NIHR Journals Library, which may limit the generalisability of the findings.It is possible that that problems with recruitment and retention of participants in NHIR funded trials will be less pronounced than in other trials due to the rigorous appraisal of feasibility prior to funding and the ongoing monitoring during the conduct of the trial.However, as the NIHR Journals Library intends to publish all research from EME, HS&DR, HTA, PGfAR and PHR funded projects, it has less chance of publication bias compared with a review of other journals where publishing is more selective and information related to recruitment is published in less detail.Second, the data extraction was conducted by several independent reviewers and although reviewers conferred to try and ensure consistency and quality assurance checks were completed on a sample of reports, it is possible that errors have occurred.Third, the calculation of recruitment rates was limited by the information reported.For some trials, centre-specific recruitment information was not available meaning that crude recruitment rates, assuming all centres were recruiting for the same time period, were calculated.In these cases, the calculated recruitment rate may be an underestimate of the true recruitment rate.This review found considerable variation in the consent, recruitment and retention rates in publicly funded RCTs.Although the majority of (6 out of 10) trials in this review achieved their final target sample; 3 out of 10 trials published in NIHR Journals Library revised their original target sample size (downwards in 7 out of 10 trials).Investigators should bear this in mind at the planning stage of their study and not be overly optimistic about their recruitment projections.
Contributors RMJ is the guarantor of the study, had full access to all the data in the study, and is responsible for the integrity of the data and the accuracy of the data analysis.RMJ and SJW contributed to the study concept and design.RMJ, RA, JH, AR and IS contributed to the selection of data and conducted the data extraction.RMJ conducted the data analysis and drafted the manuscript.RMJ, RMS and SJW contributed to the quality assurance check of the data.All authors critically revised the manuscript and approved the final manuscript.
RCTs published in the NIHR Journals Library from January 1997 to December 2020 were reviewed.Established in 2006, the NIHR is now the largest funder of health and social research in England. 2 The NIHR Journals Library publishes five peer reviewed journals reporting the results from a range of health research areas: Efficacy and Mechanism Evaluation (EME), Health Services and Delivery Research (HS&DR), HTA, Programme Grants for Applied Research (PGfAR) and Public Health Research (PHR) (https://www.journalslibrary.nihr.ac.uk/journals/).The first volume of the HTA journal was published in 1997 whereas the other four journals are more recent with the first volumes of the HS&DR, PGfAR and PHR journals published in 2013 and the first volume of the EME journal published in 2014.

Table 1
Characteristics of the trials included in the review ContinuedFigure 1 Flow diagram of search and selection process of individually RCTs from the five NIHR journals between 1 January 1997 and 31 December 2020.EME, Efficacy and Mechanism Evaluation; HS&DR, Health Services and Delivery Research; HTA, Health Technology Assessment; NIHR, National Institute for Health Research; PGfAR, Programme Grants for Applied Research; PHR, Public Health Research; RCTs, randomised controlled trials.

Table 2
Recruitment and sample size characteristics of the trials included in the review

Table 5
Association between recruitment rate (number of participants/centre/month) and trial characteristics