Eligibility criteria

Studies met our inclusion criteria if they evaluated a strategy or strategies intended to improve the recruitment of men aged 50 years or older to an RCT. Studies must have reported at least one of the defined, quantitative, recruitment outcome measures. Studies were eligible irrespective of whether they recruited patients or healthy volunteers.

An initial scoping of the literature revealed that recruitment studies set within RCTs of both men and women often failed to provide adequate detail to determine the effectiveness of recruitment strategies on male participants alone. Therefore, to assess the impact of recruitment strategies on men, studies were only eligible for inclusion if set within an RCT recruiting men only.

The review included RCTs recruiting participants aged 50 years and older. Where the age range was not specified, studies were included where the mean/median age was 60 years or older, or where the disease of interest was prevalent in older men (eg, prostate cancer).

Included studies needed to evaluate a specific recruitment strategy or strategies; papers describing barriers and facilitators to recruitment or discussing informed consent but not presenting a specific strategy or approach to recruitment were excluded. Similarly, papers providing a brief account of recruitment without describing or evaluating specific strategies or approaches were excluded.

The search strategy was restricted to papers published since 2000. Communication channels and data management practices are central to recruitment research. Both of these areas have been transformed in the past 18 years by the growth of internet access. Evaluations published before 2000 are therefore less likely to be relevant to current trial practices, particularly those reporting advertising and media-related strategies.

Search strategy

A search of four databases (Medline, Embase, CINAHL and ORRCA) was performed in July 2017 and updated in December 2017. Studies published in English from 2000 onwards were considered for inclusion. Individualised search strategies (available in online supplementary file 1) were developed for each database using a combination of keywords relating to recruitment, enrolment, men and RCTs. In addition, the reference lists of all included articles and other recruitment-related systematic reviews were searched by hand to identify other potentially relevant papers.

Study selection and data extraction

Citations and abstracts were exported to Endnote Version X8.2 and duplicates were removed. A 10% random sample of citations was selected for independent screening for eligibility by two reviewers (KB and GW), with disagreement resolved by discussion. The Kappa statistic for double-screened citations indicated substantial agreement (Kappa=0.66) and the remaining 90% of articles were screened by KB alone.

Data from the included studies were extracted by KB using a prepiloted data extraction form. Studies were categorised according to disease area of the host RCT, type of host RCT (treatment, prevention or screening), number of participants in the recruitment study and recruitment study design. Where reported, the number of prospective participants who received the recruitment intervention and the number of those participants who went on to be screened and randomised to the host RCT were extracted. The costs incurred were also extracted.

Categorisation of studies

The Qualitative Research Integrated within Trials (QuinteT) group’s Screened, Eligible, Approached, Randomised (SEAR) framework was developed to map each stage of the recruitment pathway.32 We adapted this framework to categorise the included studies according to the stage or stages of the recruitment process they addressed: identification of participants (‘Screened’ in the SEAR framework), assessment of eligibility (‘Eligible’ in the SEAR framework) and patient information and consent (‘Approached’ in the SEAR framework).

Outcome measures

Our primary outcomes were: strategy uptake (defined as the percentage of people receiving the recruitment intervention who went on to be randomised to the host RCT), strategy contribution (defined as the percentage of all participants randomised to the host RCT who were randomised as a result of a particular strategy) and strategy cost (defined as direct or indirect cost per participant randomised).

Assessment of study quality

Six tools to assess study quality or risk of bias were identified from recent systematic reviews of recruitment strategies and were piloted for suitability and usability. After piloting, the National Heart, Lung and Blood Institute Quality Assessment Tools33 were selected as they addressed all included quantitative study designs, assessed key quality components and could be easily adapted for the assessment of non-clinical data. The tools listed criteria for judging study quality including study design, description of recruitment interventions, description and measurement of recruitment outcomes, completeness of outcome reporting, the performance of statistical testing and consideration of confounders. Based on these criteria, studies were subjectively judged by KB, in consultation with LA, as being of good (least risk of bias), fair (susceptible to bias) or poor (significant risk of bias) quality. Since this review addresses a methodological rather than a clinical question, the fair quality category was broadly defined to include studies that provided useful evaluation data even where some flaws were noted in the quality assessment. Quality assessments were performed with respect to the quantitative, recruitment-related outcomes of interest in this review only. The qualitative components of included mixed methods papers were not assessed as they were outside the scope of this review.

Methods of analysis

All studies, irrespective of quality, were included in the descriptive analysis in order to describe the full range of strategies evaluated and to assist with hypothesis generation for future research. Outcome measures were only analysed for studies of fair or good quality. Estimates from poor studies were excluded except where no estimates were available from studies of good or fair quality. In this case, the estimate is presented with a caveat that the study is of poor quality. We had planned to perform a meta-analysis if studies were sufficiently homogeneous in the target population and delivery of the intervention to do so.

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses checklist was completed and can be found in online supplementary file 2.

Patient and public involvement

Patients were not involved in the design or conduct of this systematic review. It is not possible to disseminate the results of this review to the participants of the included studies.

Study selection

Nine hundred and fifty-three unique papers were extracted. Of these, 16 recruitment studies were eligible for inclusion (figure 1). These 16 recruitment studies (listed in table 1) were conducted in the context of 12 RCTs since two RCTs hosted more than one recruitment study.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Search and screening results. RCT, randomised controlled trial.

Study characteristics

The characteristics of included studies are described in table 2. As one might expect in trials recruiting exclusively older men, most selected studies reported recruitment to prostate cancer trials (11 studies plus one additional study in various cancers including prostate), with other studies reporting recruitment to trials in benign prostatic hyperplasia, low testosterone and suicide prevention. Three studies focused on the recruitment of men from minority ethnic groups. Recruitment studies ranged in size from 155 to 51 085 screened participants and most commonly used a quantitative descriptive design.

Quality assessment

Most (10 of 16) studies were assessed as being of fair quality in relation to the recruitment outcomes of interest in this review. One study was evaluated as good, and five were evaluated as poor (tables 3–5). In general, all studies addressed a clear study question and enrolled a representative sample of participants. Recruitment outcomes were reliably measured and clearly reported, although few studies reported recruitment cost. However, the description and measurement of intervention delivery were often incomplete or missing. Some studies reported that interventions were delivered inconsistently across study sites, but this inconsistency was not accounted for in the reporting of outcomes. This limitation made comparisons within and between studies problematic. Possible confounding was also a common issue. Of the 16 recruitment studies, only three had a randomised design, and one additional, non-randomised study reported baseline demographic data by intervention group. In the remaining 12 studies, differences between the intervention groups in baseline characteristics could not be assessed. Therefore, differences in recruitment between groups may have been influenced by the characteristics of the individuals studied rather than the interventions evaluated. Furthermore, in some studies, several recruitment activities were implemented concurrently, but no study discussed the possible impact of this on the observed recruitment outcomes. Another common limitation was the lack of prospective study design. Three studies reported a prospective design; six reported a retrospective design and the remaining seven did not specify.

One study, which was otherwise of good quality, was assessed as fair due to inadequate sample size.34 Several studies incorporated both quantitative and qualitative designs, but were assessed for quality based on only quantitative analysis and outcomes.35–37

Stages of recruitment and associated outcomes

The included studies addressed three recruitment stages: (1) identification of prospective participants, (2) assessment of eligibility and (3) provision of participant information combined with seeking of consent. The strategies addressing each stage of recruitment are summarised below along with their reported recruitment outcomes. Outcomes are shown in table 6 (studies that reported strategy uptake), table 7 (studies that reported strategy contribution) and table 8 (studies that reported strategy cost).

Identification of prospective participants

Participant identification strategies were evaluated in nine studies.38–46 Excluding poor quality studies, all studies reported the contribution of participant identification strategies to enrolment (shown in table 7) while only four studies reported strategy uptake (shown in table 6) and two studies reported strategy cost (table 8). Within the participant identification category, we further grouped strategies as mass mailings, media coverage and advertising, health service referrals or community outreach activities. This categorisation was adapted from previous recruitment research.47 48 The data from table 7 have been summarised in table 9 to aid comparison between studies. The most frequently evaluated strategy were mass mailings and community outreach strategies (seven studies). Media strategies were evaluated in six studies and health service referrals in five studies.

Patient information and consent

Five studies evaluated strategies to improve the patient information and consent process.34–36 49 50 The strategy uptake reported in each of these studies is shown in table 6. The studies aimed to improve either the content of the information provided to the participant at the recruitment visit or the mechanism by which that information was provided. All studies in this category were hosted within prostate cancer treatment trials, perhaps reflecting the challenges inherent in recruiting participants to prostate cancer trials where treatment options may be diverse, for example, surgery, radiation and watchful waiting.

Two papers evaluated a trial-specific recruitment training intervention delivered to site-based recruitment staff and implemented within the feasibility and main phases of the PROTECT trial.35 36 The intervention involved audio-taping recruitment interviews and performing qualitative to investigate how trial information was delivered to participants at the recruitment visit and how this may impact consent rates. Results of this analysis then guided the development of the training intervention. After the implementation of the intervention, the recruitment rate was observed to increase from 30%–40% to 70% during the feasibility stage and to remain between 69% and 65% during the main study. An evaluation of a secondary, intensive training process for underperforming sites found that recruitment rates increased from 45% to 86% (p=0.020) at one site and from 50% to 78% (p=0.013) at another site but numbers at these two sites were small.36

Other participant information and consent interventions were evaluated in one study each. Recruitment by nurses was found to be more cost-effective than recruitment by urologists (£36.40 vs £43.29 per screening) and resulted in similar rates of consent (67% vs 71% p=0.60.)49 Multidisciplinary, group information sessions increased the consent rate from 0% to 16% when compared with a one-on-one recruitment consultation.50 A 30 min decision aid video presented at the recruitment visit may have reduced the consent rate in one study although the study was underpowered.34

Strategies addressing multiple stages of the recruitment process

Two studies evaluated strategies that addressed the recruitment process as a whole (identification of participants, assessment of eligibility, provision of participant information and seeking of consent) rather than one specific stage of recruitment. One study evaluated the impact of peer-conducted site monitoring visits on recruitment.37 The study reported that of the eight sites visited monitoring identified specific recruitment process issues at two sites. After monitoring, one site altered their process for participant reminders and subsequently consent form return rates increased by 5%. At another site, monitoring uncovered that eligibility criteria were being incorrectly applied and subsequently incorrect exclusion of prospective participants decreased by 5%. While these improvements to site processes are likely to have improved recruitment, the study did not report the impact on overall recruitment.

Another study evaluated four different approaches to recruit African-American men to a cancer screening trial.51 The study found that the most intensive intervention (mailing invitation endorsed by African-American community leader, phone screening by African-American interviewer and gathering baseline information at a church-based group session with transport provided) increased recruitment uptake from 2.9% to 3.9% compared with control (standard mailed invitation, phone screening by African-American or non-African-American interviewer and collection of baseline forms by mail). While this difference was statistically significant (p<0.01), it was small in absolute magnitude, and the cost of the most intensive intervention is likely to have been high although cost data were not reported. Other approaches that included less intensive combinations of mailing invitation endorsed by an African-American community leader, phone screening by African-American interviewer and gathering baseline information by phone did not result in a statistically significant increase in recruitment uptake compared with control.

Principal findings

In this review, we aimed to evaluate recruitment strategies in RCTs of men aged 50 years and older. We found that the best approaches for identifying participants were referral through an affiliated health service provider, media coverage and mass mailings. Community outreach activities and referrals from unaffiliated health service providers were not effective strategies for improving recruitment. Recruitment was improved by trial-specific training informed by qualitative analysis of the recruitment visit and delivered to site-based recruitment staff.

Context within the existing literature

This review included only recruitment evaluations in RCTs of men aged 50 years and over and was dominated by RCTs in prostate cancer. However, our findings were broadly consistent with recruitment studies in both men and women, ranging in age from young adults14 to the elderly,12 and across primary care,52 disease prevention,25 health screening,53 54 and cancer and surgical55 research. Nonetheless, some previous studies have reported differences in strategy effectiveness based on age and gender. Mass mailing strategies were more effective in men than women, and effectiveness increased with age.25 By contrast, online advertising strategies were more effective in women than men,29 56 and most recruited participants were adolescents and young adults.28 Community outreach activities appeared to have limited effectiveness in the general population,26 30 57 with some suggestion that they were more effective in women than in men.48 Elsewhere, two reviews reported that community outreach activities might be effective when recruiting hard-to-reach participants such as vulnerable58 and elderly12 populations. These reviews reported community outreach activities tailored to the specific target populations. Some tailoring was evident in the studies included in the current review (for example, holding screening sessions at men’s health events43 44 and producing brochures in colours expected to appeal to men42) but the context and content of community outreach activities were not described in detail. It is unknown whether further tailoring could have improved the effectiveness of community outreach activities in the recruitment of men aged 50 years and over. An upcoming Cochrane review may elucidate how age and gender modify the effect of specific recruitment strategies.59

Strengths and limitations

This review is strengthened by the adaption of the SEAR framework to categorise the included studies. Research into recruitment strategies is fragmented,5 and researchers seeking evidence-based solutions to their recruitment challenges may find the current evidence difficult to digest. Categorising studies according to the stage of the recruitment process rather than categorising by intervention characteristics has a number of advantages. First, it is intuitive to use and understand since it mirrors real-world trial processes. Second, for researchers using the SEAR framework to collect recruitment data and identify recruitment challenges, our review provides a roadmap for navigating the available evidence and selecting the most promising interventions to address these challenges.

By grouping studies according to the SEAR framework, our review uncovered inconsistencies in how strategies to identify prospective participants were evaluated. All studies in this category reported strategy contributions to overall recruitment but only four studies reported strategy uptake and two reported strategy costs. There was a lack of consensus across studies on which of these outcomes was most appropriate and it was unclear how studies decided whether strategies were effective or not. Intuitively, these three possible outcomes (contribution, uptake and cost) are, individually, insufficient to evaluate overall strategy effectiveness. For example, if a strategy contributed 80% of study participants does this indicate that the strategy was effective or simply that few other strategies were used? Likewise, if a strategy was low cost but resulted in few participants being randomised was it more or less effective than an expensive strategy that delivered large numbers of participants? Greater transparency in how strategies to identify participants are selected and assessed, and the costs involved would assist with the interpretation of study results in this area.

Of the estimated large number of men’s health RCTs conducted worldwide since 2000, only 16 studies of recruitment strategy evaluation were found, and 12 of these studies were related to prostate cancer (four from a single prostate cancer trial). Consequently, this review likely describes only a small fraction of the recruitment practices used to recruit men aged 50 years and older to RCTs and may be subject to publication bias. We included only single-gender, men’s RCTs in order to focus on gender-specific recruitment strategies. Several included studies described recruitment strategies that appeared to be male-focused (identifying participants through veteran’s groups and health services,39 44 holding screening sessions at men’s health events,43 44 offering screening outside normal working hours41 and producing brochures in colours expected to appeal to men42). However, no study explicitly presented a gender-sensitised approach to recruitment or addressed the literature on men’s health preferences.60 Since studies evaluating recruitment to RCTs of both men and women were excluded, the results presented in this review are likely to be most relevant to the small but growing number of RCTs in men only.61 However, our approach to synthesising recruitment evidence and evaluating strategy effectiveness by recruitment stage may be relevant to recruitment to RCTs more broadly.

This review considered only quantitative evidence from recruitment evaluation studies, a common approach in systematic reviews of recruitment strategies.7 8 13 14 However, qualitative research methods also have the potential to address recruitment challenges62 and several included studies presented both quantitative and qualitative evidence. Future systematic reviews of recruitment strategies may be strengthened by synthesising all available evidence using a mixed methods approach.63

We recommend caution when implementing recruitment strategies based on our findings since generalisability is hampered by weak recruitment study design, and insufficient reporting of the intervention content, context, delivery and cost in many of the included studies. Based on recently proposed criteria,64 additional evaluations of all potentially effective strategies identified in this review are likely to be of merit.

Implications for research

Our review uncovered areas of uncertainty across all stages of the recruitment process. In particular, further research is needed to assess whether gender-sensitised strategies can enhance recruitment of men aged 50 years and over to RCTs, and to assess the effectiveness of online advertising and promotions to recruit this demographic group. Future research may benefit from being conducted as a prospectively designed Study Within a Trial, following the recent guidance provided by Trial Forge.65 Researchers are encouraged to reveal how strategy effectiveness was assessed and to report cost outcomes. Since there are many uncertainties in recruitment methods, research should address one or more of the priority recruitment questions recently identified by the Prioritising Recruitment in Randomised Trials study.66 This will not only improve the impact of individual studies but also deepen the body of recruitment evidence in general.8