Objectives To systematically review published pretrial qualitative research studies and explore how their findings were used to inform recruitment and retention processes in full-scale trials.
Design Qualitative evidence synthesis using thematic analysis.
Data sources and eligibility criteria We conducted a comprehensive search of databases; Dissertation Abstracts International, CINAHL, Embase, MEDLINE, Sociological Abstracts and PsycINFO. We included all reports of pretrial qualitative data on recruitment and retention in clinical trials up to March 2018.
Data extraction and synthesis Two authors independently extracted data using a predefined data extraction form that captured study aims, design, methodological approach and main findings, including barriers and facilitators to recruitment and or retention. The synthesis was undertaken using Thomas and Harden’s thematic synthesis method and reported following the Enhancing Transparency in Reporting the Synthesis of Qualitative Research guidelines. Confidence was assessed using Grading of Recommendations Assessment, Development and Evaluation-Confidence in the Evidence from Reviews of Qualitative research approach.
Results Thirty-five papers (connected to 31 feasibility studies) from three different countries, published between 2010 and 2017 were included. All studies were embedded in pilot or feasibility studies to inform design aspects in preparation for a subsequent full-scale trial. Twelve themes were identified as recruitment barriers and three as recruitment facilitators. Two themes were identified as barriers for retention and none as retention facilitators. The findings from qualitative research in feasibility or pilot trials are often not explicitly linked to proposed changes to the recruitment and retention strategies to be used in the future or planned full-scale trial.
Conclusions Many trial teams do pretrial qualitative work with the aim of improving recruitment and retention in future full-scale trials. Just over half of all reports of such work do not clearly show how their findings will change the recruitment and retention strategy of the future trial. The scope of pretrial work needs to expand beyond looking for problems and also look for what might help and spend more time on retention.
- QUALITATIVE RESEARCH
- Clinical trials
- STATISTICS & RESEARCH METHODS
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Strengths and limitations of this study
Our comprehensive search strategy optimises the likelihood that we have identified relevant studies published in the time period in principal journals.
Although we did not apply a quality assessment checklist to individual included studies to consider the relationship between quality and maximising the value of pretrial qualitative research, the systematic methodology and the use of Grading of Recommendations Assessment, Development and Evaluation-Confidence in the Evidence from Reviews of Qualitative research to assess confidence in the findings is a strength of the review.
The review was based on what was written in published research and this may not reflect the breadth of qualitative research that is undertaken in practice.
Most of the included studies were UK based. This means it is uncertain whether and to what extent the findings apply to the trial environment outside the UK.
Recruitment of participants to, and their retention in, randomised controlled trials (RCTs) is a key determinant of research efficiency, but both can be challenging.1 Reviews of clinical trials funded by the UK Medical Research Council (MRC) and the National Institute for Health Research (NIHR) Health Technology Assessment (HTA) programme have shown that the proportion of trials achieving their original recruitment target was in the range of 31%–56%, and some suffered loss to follow-up of up to 77%.2–4 Despite a substantial body of literature on strategies to improve recruitment and retention in clinical trials, the quality of this evidence is lacking.5–9 The Cochrane Review on strategies to improve recruitment to RCTs found only three interventions with a high Grading of Recommendations Assessment, Development and Evaluation (GRADE) rated evidence and the corresponding review on interventions to improve retention found no high certainty evidence.5 10
Given the lack of certainty around effective strategies to improve recruitment and retention, trialists are increasingly integrating qualitative methods within randomised trials to unpack the complex processes involved.11 12 However, much of the qualitative work to date has been on intervention development and often done when the full trial is ongoing,13 which means it can sometimes be too late to prevent or rectify a problem that has already happened. In its framework for the evaluation of complex interventions the UK MRC strongly recommended that trialists use qualitative methods prior to running a full-scale trial to understand barriers to participation and to estimate response rates.14 Briel et al suggested that 89% of obstacles leading to the discontinuation of RCTs could be avoided if issues were identified and addressed during the trial planning stages.15 Likewise, a recent thematic synthesis of 45 qualitative studies16 exploring adult patients’ experiences with RCT participation identified the diverse psychological, physical and financial burdens experienced by patients across the whole process of the trial. The consideration of these modifiable factors at the pretrial stage (ie, research conducted or embedded with feasibility or pilot trials to inform trial design and conduct before recruitment to the full-scale trial starts, such as the volume, timing, complexity or format of trial information or the organisation of participants’ follow-up, could help to deliver more efficient RCTs and timely delivery of trial results.16 17
Qualitative research conducted during the pretrial stage could have a role in improving efficiency by identifying problems with recruitment or retention early and then suggesting solutions for the full-scale trial.18 19 O’Cathain et al noted, however, that pretrial qualitative research is underused, despite its potential to optimise trial design and recruitment.20 A recent meta-epidemiological study conducted to determine how often pilot studies planned to use qualitative data to inform the design and feasibility of a larger trial also highlighted that qualitative data collection was planned for in less than half of the protocols of pilot trials (92/227) in PubMed between 2013 and 2017.21 A recent methodological review of 160 publications (123 protocols and 37 completed trials) on the reporting of progression criteria from external pilot trials to definitive RCTs reported that recruitment and retention were the most frequent indicators contributing to progression criteria.22 However, progression criteria were mostly reported as distinct thresholds (eg, achieving a specific target; 133/160, 83%) with less than a third of the planned and completed pilot trials that included qualitative research reported how these findings would contribute towards progression criteria (34/108, 31%).
The aim of this qualitative evidence synthesis (QES) was to explore how pretrial qualitative research with trial participants, recruiters, clinicians, chief investigators and trial managers was used to inform recruitment and retention processes in full-scale randomised trials. Understanding how existing studies have employed qualitative methods at the pretrial stage to inform recruitment and retention in future full-scale trials has the potential to identify how the value of pretrial work could be maximised and highlight key aspects for others to focus on when considering this type of work.
This systematic evidence synthesis is reported in accordance with the Enhancing Transparency in Reporting the Synthesis of Qualitative Research statement.23 The protocol was developed but was considered outside of scope by International Prospective Register of Systematic Reviews as it does not address health outcomes.
Searches were conducted on key electronic databases from inception to 4 March 2018: Dissertation Abstracts International, CINAHL, Embase, MEDLINE, Sociological Abstracts, PsycINFO, SSCI (Social Science Citation Index), the Cochrane Library and HTA. There were no language, date or geographic restrictions. The MEDLINE search strategy is included in online supplemental document 1.
Different search strategies were used alongside electronic databases as using multiple search methods is more likely to locate relevant qualitative studies than relying solely on bibliographic databases.24 Methods applied included following up reference lists, hand searching and contacting experts or authors.
Types of studies
We included all primary qualitative studies embedded in health-related feasibility or pilot studies. We also included studies using mixed methods if a clearly identifiable qualitative component was present. Qualitative studies that explored recruitment and/or retention issues in a feasibility or pilot study to inform a subsequent, fully powered, Phase III randomised trial were included. Pretrial qualitative studies that indicated progress to a full-scale trial was not feasible due to poor recruitment were also included.
All studies focusing on the perceptions and experiences of trial participants (eg, patients, carers or parents) who took part in a healthcare related pilot or feasibility RCT were included.
We also included studies reporting on the perceptions of stakeholders directly or indirectly involved in recruiting or retaining participants to RCTs (including chief investigators, trial managers, clinicians, research nurses, funders and research ethics committees).
Intervention/phenomena of interest
The body of research for which qualitative research was used to explore ways of optimising recruitment and or retention in RCTs at the pretrial stage. All studies focusing on the perceptions and experiences of trial participants, recruiters, chief investigators and other trial stakeholders were included.
To identify perceived barriers and facilitators to recruitment and or retention and the changes made to inform the design of a definitive trial.
Titles and abstracts were screened by two reviewers independently (AE reviewed all studies along with either ST or KG) and disagreements were resolved by discussion. The full texts of potentially eligible studies were obtained and screened by two reviewers independently to confirm inclusion. Disagreements were resolved by discussion with a third opinion being sought if necessary.
Two reviewers independently (AE along with either ST, KG or HB) extracted data from eligible full-text papers using a prespecified data extraction form that included study aims, design, methodological approach adopted and main findings, including barriers and facilitators to recruitment and or retention. This was piloted on a subset of relevant studies and modified where necessary. All qualitative findings from the primary studies relevant to the research question were extracted. Findings were defined as any qualitative data describing a new concept, theme, subtheme or finding statement, presented in forms including, but not limited to, text, tables, diagrams, online supplemental files located anywhere in the paper. Participant quotations (first order constructs) and authors' interpretations (second order constructs) reported in the results/findings sections of included papers were extracted.
Quality appraisal of included studies
The application of quality criteria to qualitative research is widely debated.25 In this QES, we are not concerned with the methodological quality of the included qualitative work per se but its contribution to planning the future full-scale trial. We therefore defined quality as the contribution of the pretrial qualitative research to the full-scale trial endeavour (recruitment and retention) and whether the findings were used explicitly (as reported in the publications) to inform the plan of action before moving onto a full-scale trial. Quality assessment of the included studies against a specific checklist was not applied.
We followed the detailed methods for thematic synthesis outlined by Thomas and Harden.26 Coding and analysis were limited to the qualitative findings extracted from the primary studies; we did not code the whole of each included study because most of it was not relevant to our research question (see ‘Data extraction’). First, we inductively line-by-line coded the results/findings and discussion sections covering any text reported as direct/verbatim participant quotes as well as the authors’ interpretation of their data. Second, after extracting the reported barriers and facilitators to recruitment and retention, we created a codebook that was grouped into common themes. Team members (AE, KG and KH) then independently coded each extracted barrier and facilitator with the themes from the codebook. If new codes emerged, they were added iteratively to the codebook and the barriers and facilitators were rethemed accordingly. Third, the three reviewers (AE, KG and KH) met to reach consensus on the codes and themes, with further interpretative discussion focused on the research question to generate analytical themes. Throughout the coding process, the review authors met regularly to cross-check newly generated codes and themes against the data, discuss interpretation and synthesise the analytical themes.
As our primary aim was to assess the practical significance of pretrial qualitative research, we looked at each paper to identify whether qualitative findings were linked to any proposed changes to the recruitment and retention plan of action for subsequent full-scale trials.
Assessment of the certainty in evidence
The Confidence in the Evidence from Reviews of Qualitative research (CERQual) approach was used to to assess our confidence in the review findings.27 The CERQual approach is based on four components which include: the methodological limitations of included studies, the coherence of the review findings, the adequacy of data contributing to the review findings and the relevance of the included studies to the review question.
Each review finding was assessed by two reviewers (AE and KG) and concerns regarding any of the four components were noted. Four levels were used to describe the overall assessment of confidence in a review finding- high, moderate, low or very low. All review findings started off by default as ‘high confidence’ and were then ‘rated down’ by one or more levels if there were concerns regarding any of the CERQual components.
For CERQuaL assessment, we had no concerns regarding methodological limitations and relevance for the body of data contributing to each review finding. Our goal was not to judge whether some absolute standard of methodological quality had been achieved, but rather to indicate how and if findings from the qualitative research were transformed into an action plan to inform recruitment or retention processes for the full-scale trial. Considering that, a specific methodological quality checklist was deemed unnecessary as high or low scores would not affect our confidence in how and if qualitative findings informed the design of a subsequent full-scale trial. For the sake of brevity these two components were not included in the CERQual evidence profile.
Patient and public involvement statement
Patients and the public were not involved in the design, conduct, reporting or dissemination of our research.
Thirty-five studies (connected to 31 feasibility studies) met the prespecified inclusion criteria and were included in this QES.; For some feasibility studies, there was more than one paper reporting findings from qualitative investigations. We included all relevant studies for comprehensiveness and to make sure we captured all perspectives from stakeholders involved.
No additional papers were identified from reference searches, review papers or reports. Figure 1 shows details of studies screened, excluded and included.
Characteristics of the included studies
All the included studies were published in English19 28–61 and were conducted in three high-income countries: the UK (n=33), Canada (n=1) and Norway (n=1). The majority of included studies (n=33/94%) were funded by UK organisations with two non-UK funded studies. Of the UK studies, %70 (n=23) were funded by the NIHR.
Each study included between 10 and 69 participants, with findings from 917 people in total reported across the papers. Contributing to the sample were: trial participants (629, 69%), clinicians and recruiters (234, 26%), family carers (26, 3%) and members of the Trial Management Group (19, 2%). Online supplemental document 2 details the characteristics of the studies included in the review.
The setting of the feasibility studies in which the qualitative research was embedded included a range of clinical contexts such as; cancer (n=11), mental health (n=5), obesity (n=3), sexual and reproductive health (n=3), chronic fatigue (n=2), musculoskeletal conditions (n=2), pain (n=2), incontinence (n=2), tooth decay (n=1), childhood intermittent exotropia (n=1), renal disease (n=1), non-adherence to medications (n=1) and appearance-related distress (n=1). As expected, the clinical context differed as did the interventions under investigation; two studies28 38 were Clinical Trials of an Investigational Medicinal Product (CTIMP) and 29 were non-CTIMP studies. These interventions were also broadly categorised as: surgical (n=6) and non-surgical (n=25).
All the included studies were embedded in pilot or feasibility trials to inform design aspects in preparation for a subsequent full-scale trial. The main data collection and analysis methods used were interviews (n = 31; 88%) and thematic analysis (n = 25; 71%). Audio-recording of recruitment consultations and non-participant observations of consultations were used in six of the included studies.31 45 46 50 54 55
Twelve themes were identified as recruitment barriers and three as recruitment facilitators, whereas only two themes were identified as barriers for retention and none as retention facilitators (table 1). The findings from the included studies focused more on recruitment than retention and researchers tended to focus on problems (barriers) rather than what might help (facilitators). The link between pretrial qualitative findings and proposed changes to the recruitment and retention strategies to be used in any future full-scale trial was not always clear (online supplemental document 3).
The findings that led to the identification of the barriers and facilitators highlighted in table 1 and their link to the proposed changes for the full-scale trial are presented below in more detail.
Barriers to recruitment
A total of 12 recruitment barriers were identified. Online supplemental document 4 outlines the findings associated with each theme and their link to the proposed changes for the full-scale trial.
Participant level factors
Lack of clarity or understanding of randomisation
Six studies19 52 54 55 57 60 outlined the influence of randomisation as a major barrier to recruitment. Trial participants believed the concept of randomisation was often not clear or perceived haphazardly and some struggled to understand the need for randomisation.19 52 Despite explaining random allocation, some participants were still uncertain whether they would be selected based on some personal or illness characteristics.19 60
How do they choose? Say, likes of five will go for the test and five will’nae, how do they actually choose? (Patient)19
Link between randomisation findings and changes proposed for the full-scale trial
The changes planned before the full trial to deal with issues around clarity of the randomisation process were clearly linked to coded data in three of the six studies.19 54 55 To clarify the concept of randomisation, one study reported that randomisation will be explained to participants in the following way: ‘‘To try and make sure both groups are the same, each person is put into a group at random. This is the fairest way of deciding who gets the test and means everyone will have a 50/50 chance of being put in either group’’.19 In other cases, randomisation period was simplified and clarified and recruiters were encouraged to elicit patients’ lay views and explain that randomisation offered a way of resolving the dilemma of treatment choice.54 55
Two studies reported changes that were not explicitly linked to the qualitative findings.52 60 In one study, authors suggested that the focus would be on training trialists who are involved in recruitment to complicated trials, both in terms of communication processes and on the assimilation of complex trial pathways.52 To resolve misunderstanding about the process of random allocation, one study reported that the study team needs to spend more time at participating practices training them in the recruitment process; patients should be supported to take the necessary time to ensure understanding of patient information sheets before signing consent.60 In one study, no changes to address the lack of understanding of randomisation were reported.57
Strong patient treatment preferences
Patient treatment preferences was a theme in nine studies.29 31 32 35 45 49 54 55 57 Recruitment was hampered by strong preferences with patients often wanting the intervention and then expressing disappointment at being allocated to the control group.29 31 32 35 49 54 57
Recruiters’ perception of unequal treatment processes was also common, and they believed that many patients opted for one treatment because it was perceived as more convenient.45 In two studies,45 54 recruiters assumed that patients came with media information that was biased in favour of the intervention (radical treatment) and often expressed lay views that cancer should be surgically removed.
I still think to leave everyone, if you told in that group ‘right half of you are going to go to physio [therapy] and half advice.’ I think wouldn’t you feel a little bit jipped, knowing ‘wait a minute how come I’m not going to get anything’? (Patient)29
Link between treatment preferences findings and changes proposed for the full-scale trial
The changes proposed before the full trial to address patient treatment preferences were clearly linked to qualitative data in four studies.31 32 45 49 Changes reported were: recruiters were asked to move beyond initial probing questions in relation to patient preferences toward rectifying any erroneous views and to ask patients who appear to have a preference to ‘keep an open mind’ until they had heard all the relevant information,31 the need to gently challenge preferences that are based on inaccurate information and training recruiters to enable them to explain the need for randomisation and the rationale for the RCT to patients45 and the incorporation of a preference arm in a future trial to account for parental preferences.49
In five studies, no specific changes were reported to account for strong patient treatment preferences.29 35 54 55 57
Issues related to the control group
Participants’ lack of understanding the rationale for having a control group was a dominant theme in four studies.19 29 54 60 Some participants struggled with understanding the need for a control group and said that allocation to the control arm of the study would put them off from participating.19 The perceived inequity in the content of the control arm was a major barrier to recruitment as some patients felt that they would not receive the best treatment if they were allocated to standard care.29 60 In one study, the presentation of the control arm caused difficulties for both patients and recruiters with the potential for interpretation as ‘no treatment’.54
Participant: Aye. If I was one of the 50% when they said, “Right, we’re gonna take a sample from you and test it”, then yeh, but if I was one of the 50% that didn’t get picked (the control group), then no. I would rather not know, actually. No. (Patient)19
Link between control group findings and changes proposed for the full-scale trial
The changes proposed before the full trial to address the issues related to the control group were clearly linked to qualitative data in all four studies.19 29 54 The changes reported were: modification of the participant information leaflet (PIL) where the control group will be changed to non-test group, which is what participants were most comfortable with,19 giving participants the necessary time to ensure understanding of patient information sheets before signing consent, especially with regard to clinical equipoise and that they will not necessarily benefit from participation60 and augmenting the content of the control arm so that the trial arms could be perceived as more equitable.29
The burden imposed by participation in the trial was a prominent theme in four studies.19 38 49 52 The experience of completing and signing a consent form at the time of enrolment was burdensome in one study.38 In two studies, limited appointment time for the initial screening and the need for flexible appointments presented a challenge for participants to fully consider participation in the trial.19 49 In the study by Moynihan et al, patients commented on how poor administration and the need to ‘work’ their way around National Health Service waiting times prevented them from being fully included in the trial enterprise.52
Well, your appointments would have to be flexible, because people are still working. Not myself, I’m retired, but there are always people working who might not be able to get time off work (Patient)19
Link between participation burden findings and changes proposed for the full-scale trial
The changes proposed before the full trial to account for participation burden were not clearly linked to qualitative data in three studies.19 49 52 The changes proposed included facilitating a context in which patients feel fully included in the trial enterprise,52 separation of the role of the treating clinician from the main recruiter to the trial49 and providing a phone call to potential participants to discuss the study after anticipated receipt of the full PIL.19
In one study, no specific changes were reported to address this barrier.38
Beliefs and expectations about trial participation
Pre-existing beliefs and expectations among study participants hindered recruitment efforts in ten studies.19 30 33 36 39 42 45 52 59 60
Participants’ beliefs that undermined involvement in the trial process were: feelings of anxiety about a poor medical outcome and scepticism about being experimented on,36 60 negative image about the hospital ‘a place to die’,45 social desirability perception that the trial was designed to encourage people to stop smoking,19 60 feelings of isolation and powerlessness52 and a sense of denial (participants tended to deny their symptoms and therefore were ineligible).59 In other cases, nurses believed they needed to protect patients from additional burden (which implicitly they believed the trial would cause) and this was cited as a main recruitment barrier.30
You’ve got to explain everything and they don’t want to go to X hospital because they think once they go to—that’s where the oncology centre is -so they think when they go there, they die, because that’s where you go to die (Recruiter).45
Link between beliefs and expectations findings and changes proposed for the full-scale trial
The changes proposed before the full trial to address pre-existing beliefs and expectations were clearly linked to qualitative data in six studies.19 33 36 39 42 60 The changes proposed included asking recruiters to gently challenge patients’ preconceptions42 and to wait until the patient’s condition is more settled before providing appropriate written informed consent.36
One study reported changes which were not explicitly linked to coded data.52 In three studies, no specific changes were planned to address these issues.30 45 59
Lack of clinical equipoise
Twelve studies outlined the influence of lack of clinical equipoise as a major barrier to recruitment.29 31 32 35 42 45 48–50 52 54 55 Recruiters and clinical staff found it difficult to maintain equipoise as interviews revealed treatment preferences for certain subgroups of patients and this affected not only the number of individuals approached and invited but also the number of randomised participants.31 35 42 45 48 In many cases the explanation of the lack of evidence underlying the effectiveness and timing of intervention served to undermine the participant’s confidence in the treating clinician, and by extension, the trial.32 49
Audio recording of recruitment consultations revealed that the terminology used by recruiters created unbalanced presentations of treatment options for which one treatment was presented at greater length and more favourably than the other and this was a strong indicator for the lack of trial equipoise.31 32 45 50 54 55
I share the concerns and doubts that many of the patients do, i.e. that it won’t work and it’s difficult to sell a treatment when you yourself don’t really believe it’s going to make any difference (Principal investigator)32
Link between clinical equipoise findings and changes proposed for the full-scale trial
Changes planned before the full trial to maintain clinical equipoise were explicitly linked to qualitative data in six studies.29 31 42 45 49 54 Changes reported were: feedback sessions to be used to make recruiters aware of instances where they inadvertently used loaded terminology,31 asking recruiters to gently challenge and acknowledge their own bias in device preference,42 highlighting the need for principal investigators and recruiters to think more critically about the concept of scientific equipoise and how that should underpin the RCT,45 separation of the role of the treating clinician from the main recruiter to the trial,49 changing the order in which the treatments were presented and to describe their respective advantages and disadvantages in equivalent detail,54 training and monitoring of trial personnel to ensure notions of equipoise are delivered and reinforced consistently.29
Three studies suggested changes to maintain clinical equipoise but were not clearly linked to qualitative data.32 48 52 These changes involved providing frequent and comprehensive training to recruiters36 39 and finding ways of enabling practitioners to engage with study procedures.41 In three studies, no specific changes to maintain clinical equipoise were reported.35 50 55
Communicating study information and associated terminology
Presentation of trial information was a major barrier to recruitment and this was evident in eight studies.32 34 50 52–55 59 In many cases, patients failed to understand the language of trial procedures or interpreted trial and clinical terminology quite differently than as intended by practitioners (eg, ‘trial’ was interpreted as ‘try and see’).31 52 54 In other cases, recruiters and investigators agreed that the trial was difficult to explain and indicated that they found the quantity and content of trial information problematic.31 53 There were also cases where study documentation was perceived as long, difficult to understand or repetitive in places and this affected decision making.34 50 In the study by Griffin et al, graphic description of surgery was thought to have put patients off randomisation and surgeons tended to go beyond their protocol brief, to explain the trial rather than referring patients on to the trial recruiter for this information.32
There’s always a risk from the traction that it may stretch the nerves down the leg, so that could leave you with some numbness. If you’re very unlucky it could leave you with a little bit of weakness there (Principal investigator)32
Link between communication findings and changes proposed for the full-scale trial
The changes proposed before the full trial to address the problems related to the communication of study information and associated terminology were explicitly linked to qualitative data in five studies.34 50 54 55 59 The changes reported were: changing the order in which the treatments were presented and describing their respective advantages and disadvantages in equivalent detail,32 construction of a simpler version of the study flowchart and drafting a new, shorter and clearer participant information sheets which removed the ‘loaded’ terminology.50 55
Two studies suggested changes to improve trial presentation but were not clearly linked to qualitative data.32 52 These changes involved providing frequent and comprehensive training to recruiters on the assimilation of complex trial pathways.32 52 In one study, no specific changes were reported to address this barrier.53
Issues around the eligibility criteria
Another recurring theme that hampered recruitment efforts was the complexity trial staff faced in applying the eligibility criteria, which appeared in six studies.35 41 45 49 55 59 In some cases, interpretation of the eligibility criteria differed between centres; there was less clarity over the minimum age for recruiting participants to the study and recruiters thought there was leeway for interpretation of the inclusion/exclusion criteria in partnership with the trial team.35 41 45 55 In other cases, highly restrictive eligibility criteria and the difficulty to confirm eligibility for the trial at the initial screening visits hindered recruitment efforts.49 59
‘I personally don’t have a problem (with applying the eligibility criteria), but that’s because I deal with trials all the time (…), but I think with some of my colleagues, both juniors within oncology and colleagues in surgery are not as familiar with trials, maybe have a little more difficulty in interpretation’ (Recruiter).55
Link between eligibility findings and changes proposed for the full-scale trial
The changes proposed before the full trial to address the problems related the complexity of applying the eligibility criteria were clearly linked to qualitative data in four studies.35 41 45 49 The changes reported were: running screening training exercises to ensure similar screening standards and practices and an ‘assumed eligibility’ approach in all centres,35 close examination and regular meetings to discuss and resolve evolving issues45 and considering a limit on the upper age at which participants would be included.49 Two studies reported no changes to address this issue.55 59
Commitment to the trial
Variable staff commitment to the trial was a major barrier to recruitment in two studies.30 55 Recruiters believed that some trial members were very committed to the trial but others were less dedicated or even antagonistic to it, and this contributed to the development of strong patient treatment preferences to one arm or the other.55 In other cases, recruitment of fewer than anticipated dyads affected nurses’ commitment and the priority given to the trial.30
when we were doing the training it’s just right there. And then it slips to tenth place. And if you haven’t recruited, it’s twentieth place because you’re doing this, this and this (Recruiter).30
Link between staff commitment findings and changes proposed for the full-scale trial
The changes proposed before the full trial to address variable commitment by staff were clearly linked to qualitative data in one study55 where clinical centres were asked to identify two Lead Recruiters per site whose responsibilities would be to act as the focus for trial recruitment activity. The remaining study reported no changes to account for this barrier.30
Lack of confidence in approaching study participants
Lack of confidence in approaching study participants or the topic of interest hindered recruitment in two studies.32 33 In one study,32 time lag between recruitment clinics posed a challenge for research staff to preserve confidence and knowledge about the study. Research staff also showed their concerns about not being able to respond to patients’ questions and ask for consent without a senior clinician or surgeon signing the form for them.33
The gaps can be quite big between the patients, so I go back to my notes and reread everything again just before I’m going to see them so it’s fresh in my mind because otherwise you’re likely to forget (Recruiter).32
Link between ‘lack of confidence in approaching participants’ findings and changes proposed for the full-scale trial
The changes proposed before the full trial to account for the lack of confidence in approaching study participants were clearly linked to qualitative data in one study.33 The study highlighted the need for training primary care staff to address the lack of confidence in raising the sensitive issue of appearance-altering conditions.
For the remaining study, reported changes were not clearly linked to qualitative data.32 The study proposed providing frequent and comprehensive training to recruiters and modifying the support to teams in other centres according to their research experience.
Practical barriers to recruitment was a major recurring theme in 12 studies.30 32–34 37–39 43 48 49 53 59 Commonly cited barriers were: difficulty in implementing procedures owing to the multicentre nature of the pilot,32 barriers of the primary care environment33 37 (time-limited consultations, high workload and competing studies), widespread reluctance in practice to forgo written consent procedures at the time of trial enrolment,62 staffing issues (staff attrition, insufficient time, suboptimal use of skill-mix)30 39 43 48 and delay in recruitment appointments.49
I then had a full caseload, so I wasn’t taking on any new patients for quite a long time. […] We’ve had the consultants doing first visits and I would follow on afterwards because we’ve been so short staffed (Recruiter)30
Link between practical barriers findings and changes proposed for the full-scale trial
The changes proposed before the full trial to address practical barriers were clearly linked to qualitative data in five studies.34 38 39 53 59 The proposed changes included allowing flexibility in terms of how and when the research was conducted,34 ensuring that future trial centres are allocated adequate time and personnel,39 advising practitioners that patients will require longer appointments than normal for involvement in the trial.53
Four studies reported changes to address this barrier but these were not clearly linked to qualitative data.32 43 48 49 In three studies, no changes to address practical barriers were reported.30 33 37
Mismatch between the trial protocol and clinical care pathways
Integrating the trial into clinical practice was considered a particular challenge hindering recruitment in four studies.31 32 42 55 In some cases, the trial was presented as an ‘add-on’ rather than an integral part of existing clinical services.31 32 In other cases, the pathway that potential participants had to follow from diagnosis to being recruited to the trial proved extremely complex.55
I think what we didn’t appreciate was the number of the different pathways with which people actually come into that system, and the complexity (…) in terms of the treating centres and the randomising centres and all the different centres that are involved in an individual patient’s care (Principal Investigator).35
Link between integration findings and changes proposed for the full-scale trial
The changes proposed before the full trial to account for poor trial integration into clinical care pathways were clearly linked to qualitative data in two studies.31 55 Clinicians were asked to mention the study in the opening statements of the surgical consultations and to express enthusiasm for the study.31 Two studies proposed changes that were not explicitly linked to coded data.32 42 These involved providing frequent and comprehensive training to recruiters32 and recruiting a trial champion to encompass coordination and facilitation of appointments and communication.42
Facilitators of recruitment
A total of three recruitment facilitators were identified. Online supplemental document 5 outlines the findings associated with each theme and their link to the proposed changes for the full-scale trial.
Personal gain and making a difference
Potential participants’ sense of obligation and altruism was a major factor that impacted positively on their decisions to participate in five studies.33 35 36 41 44 Altruism was often cited as an important motivating factor, contributing to improved care for others in the future.35 36 41 In other cases, participants were motivated by having a personal interest in the topic and perceived that research may bring direct personal benefit.33 36 41
I know that’s sort of a I’ thing to say, but it’s true, I mean I’m not try’…, for sympathy, but I have had a terrible time, and I don’t want other people to have it like, if you know, if I have children I wouldn’t want them to have go through that I went through, and um, in generally I just, you know, want to take part in it for other people (Patient)44
Link between altruism findings and changes proposed for the full-scale trial
No changes were reported in the five studies to take advantage of the conditional altruism expressed by participants and its potential impact on recruitment before the full-scale trial starts.
Communicating study information
Providing clear and informative study information to potential participants was an important facilitator for recruitment in six studies.34–36 44 46 50 In many cases, providing clear and informative study information and ensuring study participants had a thorough understanding of the study were important factors to facilitate a decision about taking part.34 34–36 44 47 50 50 61 62 In the study by Realpe et al, a logical sequence for information sharing (six step recruitment model) emerged after analysis of recruitment consultations and this seemed to facilitate recruitment.46
So everything was really well explained you know, so yeah I mean I can’t fault it really, no I was well impressed with it all (Patient)35
Link between information communication findings and changes proposed for the full-scale trial
The changes planned before the full-scale to take advantage of providing clear study information were reported in only one study.46 The study proposed a six-step recruitment model (specifying: explain the condition, reassure patients about receiving treatment, establish uncertainty, explain the study purpose, give a balanced view of treatments, and explain study procedures) to train and support recruiters in the large number of new centres in the full-scale trial.
Social networks and experience of research
Patients’ social networks and positive experience of research helped to promote study participation in two studies.36 40
So, I think because a lot of them are friends here, so they talk, and, you know, if you’re doing that, “What do you think about it?” So, they ask each other.…Cause a lot of things happen that way here, cause they listen to what other patients talk to nurses about, then they think, “Oh, okay, I’ll try that, too” (patient)40
Link between networks and experiences findings and changes proposed for the full-scale trial
No changes were reported in the two studies that identified social networks as influential for recruitment before the full-scale trial starts.
Barriers to retention
Two retention barriers were identified. Online supplemental document 6 outlines the findings associated with each theme and their link to the proposed changes for the full-scale trial.
Burden of follow-up questionnaires
Nine studies outlined that the burden of follow-up questionnaires was a major barrier to retention.35 37 47 50 51 57–60 Across a variety of contexts, questionnaire structure was perceived to be burdensome and this encompassed many forms: forced choice responses of questionnaires which did not capture the reality of patients’ experiences,37 lack of clarity and difficulties with some of the wording in the questionnaires,51 60 repetitive and difficult-to-complete questionnaires.47 58 In two studies, the timing of questionnaires was perceived to be burdensome and irrelevant because it did not allow time for change when many patients had few, if any symptoms to report.35 50
I didn’t understand a lot of the questions so she [researcher] was having to interpret them… and that probably it probably went longer than what it should have done (patient)37
Link between questionnaire burden findings and changes proposed for the full-scale trial
The changes proposed before the full trial to address the burden of follow-up questionnaires were clearly linked to qualitative data in five studies.35 51 57–59 The changes reported involved modifying questionnaires to allow ‘short-cutting’ of irrelevant areas to reduce respondent burden,35 reducing the number of questionnaires in the subsequent trial59 and training fieldworkers in assisting participants with questionnaire completion if required.51
In two studies, changes reported were not clearly linked to coded data.47 50 These involved identifying measures to improve outcome data collection using a variety of strategies. Two studies reported no changes to address this barrier.37 60
Practical issues appeared to hinder participant retention in two studies.57 60 Some participants reported that making journeys to the site required considerable effort.57 60 A small minority of patients found the process of getting a chest X-ray difficult. Some participants had to pay for the parking costs and using public transport seemed to be too problematic.60
Link between practical barriers findings and changes proposed for the full-scale trial
One study reported changes to account for practical barriers but were not clearly linked to qualitative data.60 The study reported that patients should be reassured that participation in the trial should cause them the least amount of inconvenience. In one study, no changes to address practical barriers were reported.57
Facilitators for retention
There were no facilitators for retention reported in the included studies.
The CERQual Evidence profile is presented in online supplemental documents 7 and 8, which highlights each review finding along with its CERQual assessment.
Embedded qualitative investigations to illuminate barriers to recruitment and retention prior to a full-scale trial have increased in the last decade.20 63 This systematic QES was based on findings from 35 studies. The review provides important insights on how the findings of qualitative research methods at the pretrial stage were used to inform changes to the recruitment and retention plan of future full-scale trials.
The systematic synthesis identified an assortment of recruitment barriers (n=12) but only identified two barriers to retention. There were only three facilitators for recruitment, and there were no facilitators for retention. The findings of included studies tended to focus more on the challenges to recruitment and retention rather than the facilitators. Perhaps researchers are instinctively more interested in what is not working well (the barriers) and trying to make changes to remove those barriers. However, it is also important for researchers to take advantage of what facilitated recruitment and retention at the pretrial stage and to ensure ‘what worked well’ stays working well in the full-scale trial and that should be reflected in the reporting. Of the three recruitment facilitators identified, few studies46 59 explicitly reported how these facilitators would be used to improve the recruitment process in the subsequent full-scale trial. It is hard to believe that there are no facilitators for retention in the included studies; perhaps researchers were not looking for, or reporting, this.
The focus on recruitment may have meant that retention was overlooked, something that is in line with findings from a qualitative interview study with stakeholders from five trials.64 The study identified that extensive work on recruitment targets was deemed detrimental to retention activities and highlighted the need for efficient training and support for trial staff involved in retention practices and a wider recognition of the importance of retention from funding organisations. A recent evidence synthesis of qualitative studies identified only 11 studies that had explored any aspect of trial retention with participants who had not completed the trial until the end.65 While it may be hard to re-engage with former participants to understand why trials fail to retain them, the lack of knowledge about this issue is striking. To date, very few interventions have been shown to improve retention in RCTs, with only moderate certainty evidence available for the use of monetary incentives with a prompts or reminders to improve responses to postal questionnaires.10 Yet, none of the retention interventions to date has been informed by evidence on the perspectives of participants and/or former participants from a range of trials and what they experience as barriers and enablers to trial retention. A recent qualitative study with participants from several host trials provided participant reported evidence of behavioural reasons investigating two retention behaviours: questionnaire return and follow-up clinic attendance.66 Barriers frequently reported in relation to both target behaviours stemmed from participants’ knowledge, beliefs about their capabilities and the consequences of performing (or not performing) the behaviour. The findings can be used to develop participant-centred behavioural interventions where uncertainties remain about the most effective ways to increase retention. The study also highlighted that it is critical that researchers consider barriers and enablers of retention at the pretrial stage to prevent problems before they arise. Lawrie et al67 applied a behavioural framework to understand the barriers and enablers to questionnaire return within the C-Gall trial. The study outlined practical considerations other researchers may wish to consider to increase questionnaire return rate, such as managing participants’ expectations of trial-related activities (eg, how many questionnaires they will be expected to complete), highlighting the negative consequences of participant drop- out, tailoring the administration of questionnaires to suit individual preferences and circumstances and providing support where required.
The most common recruitment barriers reported in the included studies were lack of understanding the concept of randomisation, preference for a particular treatment option, and lack of clinical equipoise. The use of innovative qualitative data collection methods provided an in‐depth understanding of recruitment processes, how the trial was presented, and how patients were responding to the trial. Audio recording of recruitment consultations is a good example that provides specific recruiter feedback and opportunities to change practices.46 The approach was successfully implemented in six of the included studies.31 45 46 50 54 55 Exploring patient preferences, presenting information while being aware of framing effects, and avoiding the use of loaded terminology were identified as practical actions that recruiters could take to improve recruitment. The qualitative analysis of recruitment consultations highlighted communication practices that helped the multicentre pilot UK FASHIoN trial to achieve a 70% recruitment rate, although it had been assumed at the outset that it would be extremely difficult.46 On the other hand, retention was rarely discussed during clinical trial consultations. An embedded mixed-methods with a purposive sample of audio-recorded trial consultations obtained from four sites of a large multicentre UK-based surgical RCT revealed that there was no discussion of retention across 79% of consultations. If retention was discussed, it only made up 3% (at best) of the consultation content.68
The changes reported in the included studies to address recruitment barriers mainly aimed to clarify the concept of randomisation to study participants, maintain clinical equipoise, challenge patient treatment preferences and ensure clarity around the eligibility criteria. The changes reported to address retention barriers centred around identifying ways to ease the burden of follow-up questionnaires. However, in many cases, the link between the changes proposed for the full-scale trial and the pretrial qualitative findings was not explicit. This was the case in nearly 50% of the included studies, meaning that capitalising on the value of pretrial qualitative research when reporting these studies was not clear despite findings suggesting there was a problem that needed to be addressed. This might be because of limited article word count in papers reporting the results of the qualitative work alongside the pilot trial results, where very little space was allocated to the qualitative component and its impact was usually reported rather than demonstrated. It could also, of course, be because the proposed changes were not related to the pretrial qualitative findings. It is impossible to tell from many published reports.
The findings from our QES are in line with recently published studies on how qualitative work prior to an RCT can be invaluable in informing study design, especially for new interventions. A pretrial qualitative work with healthcare professionals conducted to refine the design and delivery of the Prepare for Kidney Care RCT identified challenges related to its design and recruitment and allowing changes to be made to the trial design in advance of the trial commencing.18 Likewise, clinicians’ views of patient-initiated follow-up in head and neck cancer were explored in a qualitative study to Inform the PETNECK2 trial.69 This study highlighted clinicians’ concerns that patients have unmet psychosocial needs during follow-up and that head and neck cancer community need to consider alternative follow-up protocols and justification for the PETNECK2 study.
Quality of the evidence and certainty of the findings
Since the main aim of this QES was to explore the practical utility of using qualitative research methods at the pretrial stage with the aim of maximising the chances of recruitment and retention success in a future full-scale trial, CERQual assessment of the overall confidence in the evidence was applied to assess whether qualitative findings were used to inform changes to the recruitment and retention plan. We considered a little less than half of the findings as of high certainty because the findings showed high levels of coherence and adequacy, while we assessed the remaining findings to be of moderate certainty because of concerns regarding both the coherence of the findings and the adequacy of data in the underlying studies. This means that for over half of the included studies, the contribution of pretrial qualitative research to the decision-making process and how it informed recruitment and retention processes for any subsequent full-scale trial was not explicit.
Limitations and strengths of the review
This qualitative synthesis brings together the evidence-base of barriers and facilitators to recruitment and retention identified in pretrial qualitative work together with an assessment of the practical utility of pretrial qualitative research in informing the recruitment and retention plan before the commencement of a full-scale trial. The comprehensive search strategy optimises the likelihood that we have identified all relevant studies published in the time period. Although we did not apply a quality assessment checklist to individual included studies to consider the relationship between quality and maximising the value of pretrial qualitative research, the systematic methodology and the use of GRADE-CERQual assessment of confidence in the findings is a strength of the review.70
There are, however, limitations. The review was based on what was written in published research and this may not reflect the breadth of qualitative research that is undertaken in practice. Every effort was made to contact corresponding authors to obtain a full account of qualitative data where information was lacking in the published report, or when researchers reported that a stand-alone article based on qualitative research will be published separately but was not yet available. However, not all authors provided these data, in which case it means the synthesis was limited to the findings and quotes published in the qualitative reports. Of the 35 included studies, 33 were UK based (the other two were conducted in Canada and Norway) and this resonates with the fact that both recruitment and retention are among the top three methodological research priorities in the UK.71 It does, however, mean it is uncertain whether and to what extent the findings apply to the trial environment outside the UK. The geographical spread of studies included in our QES is in line with the Cochrane review on factors that impact on recruitment to randomised trials.72 Of the 29 studies included in the review, 16 studies were conducted in the UK, 6 in other European countries (Austria n=1, Denmark n=1, Germany n=2, Sweden n=1, the Netherlands n=1); 3 in the USA; and 1 each in Australia, Canada, New Zealand and Tanzania.
Suggestions for good practice and maximising value
While pretrial qualitative research can be very illuminating in identifying barriers and facilitators to recruitment and retention, researchers need to clearly report how and if the findings from the qualitative research will be used to optimise their recruitment and retention approaches in the full-scale trial. This QES highlights the inefficient use of pretrial qualitative research; despite identifying an assortment of barriers to recruitment or retention, researchers failed, in most cases, to articulate how their qualitative findings would be put into a clear action plan to optimise the conduct of a future full-scale trial. The key issues identified by qualitative research need to be discussed with trial stakeholders and used in support of making practical changes to the trial design, presentation or amendments to the study protocol and that should be made explicit in the reporting. This could help make a stronger case when submitting funding applications for a planned full-scale trial and reassure funders that extensions will not be required. Examples of involving stakeholders at all phases of trial planning and conduct have proven effective in increasing both recruitment and retention.73 Crocker et al also investigated the impact of patient and public involvement (PPI) on rates of enrolment and retention in clinical trials.74 On average, PPI interventions modestly but significantly increased the odds of participant enrolment in the main analysis (OR 1.16, 95% CI and prediction interval 1.01 to 1.34). In exploratory subgroup analyses, the involvement of people with lived experience of the condition under study was significantly associated with improved enrolment (OR 3.14 vs 1.07; p=0.02). The findings for retention were inconclusive owing to the paucity of eligible studies.
This evidence synthesis provides some pointers for how researchers can improve their approach to pretrial qualitative work. Below we have suggested two summary recommendations that may help to maximise the value of undertaking this type of work:
Plan the qualitative research with the full-scale trial in mind
Researchers need to think about the recruitment and retention challenges their planned trial is likely to face and design the pretrial qualitative research to specifically address these, while of course allowing for a degree of openness and flexibility to address possible emerging issues as the trial progresses. Researchers need to prioritise the practical importance of qualitative research and its potential to optimise the conduct of the full-scale trial.
Be clear that changes were made to the recruitment or retention plan
In some cases, there was a clear link between qualitative findings and a particular change being made to the recruitment or retention plan for the full-scale trial. In others, there was no explicit link between findings and changes, or the lack of changes. For these the influence of pretrial qualitative work on the recruitment or retention plans for the full-scale trial remained unclear, either because of poor reporting or because there was no link. Researchers should provide a clear statement of their findings and the linked changes, if any, to the recruitment and retention plan for the full-scale trial.
A good example of how barriers to recruitment and the corresponding changes were reported in a study is that by Paramasivan et al 2017 ‘‘Enabling recruitment success in bariatric surgical trials: pilot phase of the By-Band-Sleeve study’’.31 This study was highlighted as a good example because qualitative findings were clearly reported, and the decision-making process was made explicit with regards to how the findings were transformed into actions to mitigate against recruitment problems before the commencement of a full-scale trial.
Many trial teams do pretrial qualitative work with the aim of improving, among other things, recruitment and retention in future full-scale trials. Just over half of all reports of such work do not clearly show how their findings will change the recruitment and retention strategy of the future trial. The scope of pretrial work needs to expand beyond looking for problems and also look for what might help and spend more time on retention.
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information.
Patient consent for publication
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Contributors AE, ST and KG conceptualised and designed the review. CF conducted the search. AE, ST and KG reviewed titles, abstracts and full-text papers for eligibility. AE extracted data from all the included studies along with either ST or KG or HB. Data synthesis was carried out by one researcher (AE) and verified by two researchers (KG and KH) for meaning and content. AE drafted the paper, and all authors reviewed drafts and approved the final version. AE is the guarantor of the paper.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.