Article Text
PDF
Abstract
Introduction In order for study results to be relevant for practice, the study participants should represent the source population. A common problem is recruitment of sufficient and representative subjects, threatening the external validity of the study and, ultimately, evidence-based practice. The aim was to highlight common challenges and to present possible solutions to recruitment.
Methods Using four recent randomised controlled trials as examples, common recruitment challenges were highlighted and solutions were proposed. The four studies represented some common and some specific challenges, but they investigated interventions for the prevention of the two major public health challenges of today: musculoskeletal pain and common mental disorders.
Results Identified challenges and suggested solutions were presented as a checklist to be used for future trials in order to aid recruitment and reporting thereof.
- primary care
- public health
- epidemiology
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
Suggestions for future recruitment are based on four good quality randomised controlled trials.
A checklist summarises these suggestions.
The checklist is not tested to evaluate usefulness.
Introduction
In the 21st century, evidence-based practice (EBP) should inform healthcare decisions; that is, scientific evidence, patient needs, values and preferences, the clinician experience and recourses, and context all influence the decision to implement EBP.1 2
Practice guidelines informing EBP rely on evidence synthesis from intervention studies. Usually, randomised controlled trials of high quality are the research design topping the evidence pyramid and the base on which evidence synthesis in systematic reviews is built.
Achieving high quality in an intervention study is, however, dependent not only on a high-quality research design but also on its execution. Even when there is sufficient knowledge about the design, the study quality may be seriously impaired by practical difficulties, of which recruitment is a common challenge.
Recruitment is meant to ensure that a sample representing the source population is included in the study; that is, the subjects included should resemble the individuals for which the intervention is intended, as much as possible. If sampling is biased due to poor recruiting, external validity (generalisability) is threatened. Moreover, poor recruitment might result in a study sample which contains a self-selected group of individuals more motivated and eager to change compared with the study sample of interest. Examining the effectiveness of an intervention in such a selected group will likely influence the external validity of the intervention.3
Recruitment is also important to achieve the necessary power, that is, the ability to draw statistically sound conclusions from the study. There are numerous examples of studies not reaching full power, resulting in a waste of resources without advancing knowledge or helping patients. In addition to the obvious ethical concern of gathering data that may be of limited value and exposing participants to risks without leading to scientific knowledge, participants may suffer the opportunity cost of being tied up in an unsuccessful trial without the possibility of joining another.
We could therefore say that one principal of EBP is the ability to recruit a relevant and sufficiently large sample in research studies. Unsuccessful recruitment may actually threaten patient care as treatment decisions are based on inadequate evidence. It is, however, difficult to get people to decide to participate in research studies.4 A recent systematic review found evidence that reminding subjects not responding to a postal invitation via telephone and telling people about the components of the intervention are improving recruitment, whereas producing an information leaflet with intended users did not.5
This paper aimed to highlight some of the recruitment difficulties we encountered while conducting intervention studies, to present some possible solutions to them, as well as to propose a checklist for future use. We will use four recent pragmatic studies as examples and discuss the available evidence in relation to our experience. These examples involve today’s major public health challenges: common mental disorders and musculoskeletal pain.
Methods
The points made in this paper are based on four existing Swedish randomised controlled studies. First, we will highlight some common as well as specific challenges encountered in recruiting participants. Second, we will present some solutions that may help other researchers using similar methodologies, summarised in a checklist.
We will look at recruitment challenges and solutions at different ‘levels’ (individual and organisational), different settings (school, primary care and private care) and use examples from primary, secondary and tertiary prevention.
The included studies are seen in table 1. The following are short descriptions:
Prevention of Low Back Pain- effect, cost-effectiveness and cost-utility of maintenance care (MC)6: a randomised controlled multicentre trial conducted between 2012 and 2016 to investigate the effect of secondary and tertiary preventive care on days with bothersome low back pain. Recruitment took place on two levels: private clinicians (chiropractors) and patients with low back pain.
Increasing return-to-work among people on sick leave due to common mental disorders,(PROSA)7: a cluster-randomised controlled multicentre trial conducted between 2018 and 2019 (follow-up not completed) to investigate the effect of a systematic secondary preventive approach on return-to-work after sick leave. Recruitment took place on three levels: the county council (primary care centres), primary care (rehabilitation coordinators) and among patients with common mental disorders.
Implementation of the Swedish Guideline for prevention of Mental Ill-health at the Workplace (iSKOL)8: a cluster-randomised controlled waiting list study conducted between 2017 and 2019 to investigate the effect of implementation strategies on the implementation of an evidence-based guideline for primary and secondary prevention of common mental disorders at the workplace (schools) to study adherence to a guideline on how employers can prevent work-related common mental disorders. Recruitment took place on three levels: municipality, managers (school principals) and school personnel.
Effect of spinal manipulative therapy on heart rate variability and pain in patients with chronic neck pain (SPARC)9: a randomised controlled multicentre trial conducted in 2019 and 2020 to investigate the effect of chiropractic care as a secondary and tertiary intervention on pain and heart rate variability. Recruitment took place on two levels: private clinicians (chiropractors) and patients with persistent and/or recurrent neck pain.
Characteristics of the included studies (PICO format)
Results
Challenges
The challenges in recruitment described further are represented in figure 1 along with possible solutions.
Challenges of recruitment to a research study at different levels as well as possible solutions. PBRN, practice-based research network.
The first challenge in any study is related to knowledge exchange between knowledge users (ie, stakeholders) and knowledge producers (ie, researchers).10 In other words, to get knowledge users to receive information about relevant studies to participate in and to get knowledge producers to receive information about users’ research needs.11 Very few administrators, union leaders, company owners, clinicians or patients scan websites such as Clinical Trials (www.clinical trials.gov) for relevant studies to participate in. In fact, most of these organisations and individuals will not approach the researchers, even though they may recognise that they deal with issues that need more knowledge. The researchers need strategies to get the information about relevant studies to the stakeholders and potential participants.
Even though the senior management (union leaders, company owners and managers) are on board and support a study, it does not mean that the recruiters (workers and clinicians) or potential participants (workers and patients) will be. If the decision to participate in a study is perceived as a top-down instruction, that is, a decision taken by management without consulting the employees, the relevant people may not recruit or participate successfully, likely resulting in recruitment difficulties.
Therefore, the second challenge is to make the research question relevant to participating recruiters and subjects. An organisation may be facing costs associated with sick leave or workers leaving due to ill health but may still not be motivated to participate in a study of the particular solution the researcher wants to investigate. A clinician may feel that the answer of the study will make little difference to the way they practise, and a worker or a patient may not get the benefit of the research results themselves. In order for any of these groups to participate, they must feel that the answer to the research question is vitally important to their organisation, profession, peers or, better still, themselves.
The third challenge concerns the burden of the study. Often, participating recruiters and subjects feel that the procedures of the study require some learning and getting used to, as they need to be standardised and adhered to in a high-quality research study. Generally, disrupting established routines feels uncomfortable. Participants may feel that the time commitment is more than they are willing to commit to or spend. Some are simply not willing to be randomised, as they know that one arm (control) will often imply that there is no added effect from existing procedures.
The fourth challenge concerns the stability of the study population between planning/eligibility screening and study start. Managers and workers may quit or be reassigned in an internal reorganisation (as experienced in iSKOL), or patients may finish a treatment regimen (as experienced in MC and SPARC). The advocates/supporters of the study may disappear; their roles may change or their contact details may disappear. Another challenge is patients who are simply too ill to answer questionnaires, participate in active interventions or be subjected to tests: this seem to be true for the diagnosis of mental illness (as experienced in PROSA).
Solutions
The solutions that worked for us are listed in table 2.
Use of different measures to maximise recruitment in four studies
In our studies, we have found personal contact to be a key component for recruitment. This is true at the level of the decision-maker (eg, the steering group of a county council (as experienced in PROSA), a hospital, a professional body (as experienced in MC and SPARC), a school (as experienced in iSKOL) or a company, and on an individual level (a recruiting clinician). Preferably, there should be a face-to-face meeting to present the research and to explain the relevance to motivate participation, systematically followed up by email, telephone and subsequent meetings.
In the interest of informing partners, we have also recently started using small film recordings (2 min) that are sent as follow-ups to participants with a motivational message from the research team (in iSKOL). Social media can be used (with permission) to send such filmed messages or small texts. If the recruiting clinic is sending newsletters to their patients regularly, this is a great vehicle for recruitment (as experienced in SPARC).
Contacts in the form of networks are invaluable. It is paramount to engage in partnership with stakeholders, be it companies,12 13 clinicians14 or patient organisations,15 before a study starts. In discussions, it will become apparent what issues need solving and what questions need answering. When a study is introduced in advance like this, the participants are already on board when the study starts; they know the research team; and they feel the research is relevant to them.
A specific type of network is a practice-based research network where clinicians, for example, with an interest in and understanding of the strict procedures of research engage in recruiting patients.16 17 Typically, the same researchers are in touch with the network time after time, and mutual respect and good collaboration ensue.
Procedures in the worksite, organisation or clinic should be disrupted as little as possible (unless it is the matter under investigation). Discussions with stakeholders will clarify where there is room for the study, and where it is impossible to disrupt routines. Some adaptability to individual variance is preferable in the study design. We have found that recruiters are quite inventive; they will often come up with solutions on how to find, screen and recruit eligible subjects within the framework of their normal procedures without compromising the strict processes of the study protocol.
The issue of time is really what may determine the recruitment success; if the recruitment or participation procedure steals time from daily routines, clinic or practice workflows, the recruiter will not participate. Again, stakeholders will let you know where it is possible to put an extra minute on a recruitment (as experienced in MC, PROSA and SPARC). We have had success in recruiting participants when they were able to participate during working hours (as experienced in iSKOL). Another suggestion is that the recruiter asks the potential subject for permission to send their contact details on, and then let the research team explain the study, screen, ask for consent and include the subject in the study.
However, it is our experience that individuals (recruiters and study subjects alike) who say they want to participate will not always do so. Therefore, we have found that a fair amount of reminding is needed to optimise recruitment as it is easy to forget to recruit participants. Equally, for trial participants, it is easy to postpone/forget to take the first step in participation, especially when the chores and stress of everyday life need attention. The reminding should be systematic and contain positive and motivational messages, as well as specific points about the study. Recruiters may need reminding of the inclusion or exclusion criteria; participants may need reminding why their contribution in the study is important. Email, telephone or social media may be used, as well as short films.
Reminding and communicating with recruiters and potential participants requires a substantial number of man-hours if recruiting is going to be optimal. The ideal solution is a research assistant/coordinator, if the project has funding for that. However, someone within the organisation or clinic can also remind potential subjects. This can, for example, be done by having recruitment on the agenda of regular personnel meetings.
In order to motivate participation, there is sometimes the opportunity to give something/reward the recruiters and participants: It can be increased competence (education/training) (as used in PROSA/ISKOL), subsidisation of treatment fees (as used in MC), cinema tickets and even feedback of health outcomes (as used in iSKOL).
Discussion/consequences
In this paper, we have highlighted some common issues that present challenges in recruiting study participants in intervention studies. In our research group, we discuss these issues frequently, and we have therefore also presented some of the solutions that have worked for us.
Recruiting a representative and sufficiently large sample is vital for the quality of the study, for the trust we have in the result, the quality of healthcare decisions that are made and, ultimately, the health of the public.
Our challenges and solutions may be context specific. We operate with the most common healthcare problems: musculoskeletal pain and common mental disorders; thus, we believe that many researchers will recognise the challenges we have encountered. However, the healthcare systems and partnership models may differ from those of other countries and settings, so the solutions may not be suitable to every context. In one recent evidence summary, similar findings regarding patient convenience, corresponding to ‘procedures’ and ‘time’ in our checklist, support for recruiters, corresponding to ‘personal contact’, monitoring and systems, corresponding to ‘reminding’ and the systematic use of this activity, incentives, corresponding to ‘relevance’ and resources and corresponding to ‘research assistant’ were identified.18 In another recent review, factors influencing the decision to participate were corresponding to ‘information’ and time19; thus, some of these factors are possibly generic.
A recent qualitative study explored the barriers to recruitment and found that organisational difficulties, corresponding to procedures and lack of time were important barriers, but also highlighted the conflict of roles that can arise when a clinician also has to take on the role of a researcher.20
Another study mentions partnership with end users to optimise recruitment.21 A specific form for partnership is to involve users, which has become an important consideration for planning and executing new studies. In a recent report, some advantages were highlighted, such as identification of urgent issues, and increased confidence in and improved dissemination of research results,22 matters indirectly linked to recruitment. In the four studies discussed in the present paper, we have included users in different ways. In the iSKOL project, for example, school principals were involved in the formation of the implementation strategies and gave input on optimal recruitment strategies. They were also actively involved in reminding subjects and sending out information.
We have mentioned social media in the context of informing and reminding, and this technology has been highlighted as a modern way of recruiting subjects to trials. Knowledge of special interest groups on these media may find subjects that are difficult to recruit through ordinary channels (worksite or clinic),21 23 but concerns have also been raised regarding representativeness of samples and protection of subjects.24 Also, a study explored Facebook to recruit males into health service research, but concluded that a multi-faceted approach was needed.25 In our four examples, recruitment has been optimised by targeting several aspects, and there is probably not a ‘one-fix’ solution to recruitment. Ongoing studies from America are exploring how we can map social media to identify which media channels specific target groups are using and even which hashtags are used.26
In order not to waste time and resources, we recommend that a feasibility or pilot study be performed before any full-scale study is launched. Such a study was recently published with a decision not to move forward with the full trial using the initially decided recruitment criteria and procedures.27 We suggest considering our proposed solutions, entered into the checklist as follows (table 3), to determine which are relevant in each specific case, and to work with as many aspects as possible.
Proposed checklist to ensure good recruitment in pragmatic intervention studies. The items listed may or may not be relevant in different contexts
In order to judge the quality of an intervention study, we encourage transparent reporting of the recruitment procedure, as this reflects directly on the generalisability of the results. We have found one example of specifying the recruitment strategy in the trial protocol,21 which also is a good idea for increased transparency. It is important to know if the included sample was indeed representative of the source population (a point in many quality checklists28 29). If a study did not reach full power, the trust in the results is limited, but if recruitment is described in detail, we may be able to judge the impact and not just dismiss the result.
References
Footnotes
Contributors The article was the result of discussions between all the authors. IA wrote the first draft and was responsible for the Maintenance Care study. EBB was responsible for the PROSA study, AGB for the SPARC study and LK for the iSKOL study. All authors edited and approved the final manuscript.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.