Article Text
PDF
Abstract
Introduction Recruiting participants into research trials is essential for the advancement of scientific knowledge that depends on clinical research studies. For the field of exercise and physical activity, there is an added difficulty in recruiting participants because participants must be willing to participate in an intervention that requires a significant commitment of both time and physical effort. Therefore, we have planned a systematic review to analyse how methodological factors, intervention characteristics and participant demographics impact recruitment rates in specific populations. This information will help researchers improve the design and recruitment approach in future studies.
Methods and analysis A mixed methods systematic review will be performed on studies that implement physical activity interventions and present data on participant recruitment. We plan on searching the Pubmed, Cumulative Index to Nursing and Allied Health Literature and Online Resource for Recruitment research in Clinical Trials databases for potentially eligible articles from database inception through 10 February 2017. A standardised approach will be used to identify studies through a review of titles, abstracts and reference lists. The process for each eligible study is to determine their eligibility, extract data from eligible studies and rate each eligible study’s methodological quality. Exploratory multivariate regression models will be used to determine the effects of methodological factors, intervention characteristics and participant demographics on the recruitment variables of interest.
Ethics and dissemination Because all of the data used in this systematic review has been published, this review does not require ethical approval. The results of this review will be disseminated through peer-reviewed publication as well as through conference presentations.
PROSPERO registration number CRD42017057284.
- quantitative review
- systematic review
- protocol
- participant recruitment
- physical activity and exercise
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
This systematic review protocol follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocol guidelines.
We will comprehensively search the Pubmed, Cumulative Index to Nursing and Allied Health Literature and Online Resource for Recruitment research in Clinical Trials databases to find all relevant studies pertaining to participant recruitment and exercise.
This systematic review will use the Cochrane Collaboration assessment tool to assess for risk of bias.
Some relevant exercise studies may not have produced a publication focusing on recruitment or included recruitment variables as a primary or secondary outcome, potentially limiting the number of eligible studies.
Introduction
In research studies, physical activity and exercise interventions have generally been found to be efficacious in a variety of populations. The efficacy of these interventions is noted in improved mortality rates,1 cardiovascular health,2–4 cancer risks,5 sleep quality6 and quality of life.7 Just as important, the benefits of these interventions also span a broad range of conditions, diseases and demographics including older adults,8 individuals with breast cancer,9 Parkinson’s Disease,10 smokers,11 osteroarthritis12–14 and diabetes15 16among others. As such, encouraging participation in exercise and physical activity intervention studies can have tremendous benefits for the participant as well as other individuals who could be positively impacted by the results of those studies.
A lack of, or inability to effectively recruit, research participants can lead to failed clinical trials.17 18 Only approximately half of studies reach their a priori recruitment goal,19 and the same proportion of multicentre studies encounter problems in recruiting enough participants.20
Recruitment has the potential to impact research studies in a positive or negative manner. In terms of positive effects, an appropriately recruited sample will be more representative of the target population.21 Furthermore, having a sufficient sample size becomes an ethical issue when researchers begin to decipher the significance of the analysis results.22 The underlying goal is that the results of a properly recruited study will be more generalisable because of the recruited sample and thus further scientific knowledge. However, inefficient recruiting can have negative impacts extending far beyond lower generalisability. Previous research has indicated that inefficient recruitment increases the length of the study due to the need for longer recruitment periods17 18 and increases the financial resources required to complete the study.19 Indeed, pharmaceutical studies are often forced to close prematurely primarily due to financial costs.23 Further, inefficient recruitment could delay the availability of new and effective treatments.
Traditionally, pilot and feasibility studies are carried out to demonstrate that a study is possible.24 These studies often examine whether the research design, intervention and even recruitment methods are reasonable to complete.24 These studies also require a financial budget, although they often require less time, money and resources.25 While these resources are generally less than what would be spent on a full-scale study, pilot and feasibility studies do require financial and human resources that could be more effectively focused on other aspects of study development if participant recruitment were optimised through the use of strategies supported by empirical evidence.
Although other reviews have examined participant recruitment,26–32 we are not aware of any reviews that have used a mixed methods approach to examine recruitment in exercise-related studies. Because no research has performed a mixed methods examination of exercise-related studies’ recruitment, researchers are unable to use an evidence-based approach to identify and maximise recruitment strategies that may be of particular benefit.
This mixed methods systematic review protocol will use the existing literature to empirically examine the impact of research methodology and participant characteristics on recruitment variables in exercise-related studies for specific populations. This manuscript describes our planned approach to analyse the quantitative effects of various methodological designs, intervention factors and participant characteristics and their impact on participant recruitment. These results will ideally be used to facilitate the design and construction of future studies that optimise participant recruitment and allow researchers to more accurately construct research study budgets and timelines.
Review question
Do methodological factors (eg, assignment method, independent and dependent variables), intervention characteristics (eg, length of the study, treatment setting), participant characteristics (eg, age, gender, race, ethnicity) and author/study characteristics (eg, year of publication, author’s primary discipline) affect participant recruitment in human exercise studies?
Methods and analysis
This mixed methods systematic review protocol follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocol guidelines (figure 1).33 34 This review protocol has been registered with the International Prospective Register of Systematic Reviews under the registration number CRD42017057284, and the protocol will be updated with amendments as needed.
Flow diagram for study selection based on the PRISMA guidelines. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.
Eligibility criteria
Our mixed methods systematic review will focus on clinical research studies that have recruited participants into exercise or physical activity interventions. The selected studies will include randomised trials (eg, randomised clinical trials, cluster randomised trials) as well as non-randomised studies (eg, quasirandom assignment studies, matched studies, non-matched studies, single group studies, pilot studies). Cross-sectional studies will be excluded, however, due to their lack of a physical activity intervention. Review articles (eg, literature reviews, systematic reviews, meta-analyses) will be excluded, but they will be used to identify additional eligible articles.
The abstracts of studies identified through the search strategy and through reference harvesting will be screened according to six eligibility criteria by any of the four review authors. These criteria are:
Is the study written in English?
Is the study using humans as the research subjects?
Is the study recruiting adults? Adult is defined as the mean sample age greater than or equal to 18 years of age.
Is the study abstract explicitly addressing recruitment? This is defined to mean that the study is focusing primarily on recruitment (eg, the written purpose of the manuscript is to evaluate recruitment) or the study is presenting recruitment outcomes as one of the primary outcomes (eg, articles listing recruitment variables as outcome variables to be analysed, articles presenting descriptive statistics on the number of participants contacted or screened).
Is the study implementing an exercise-related intervention? This is defined to mean any study requiring the participant to engage in some sort of physical activity intervention (eg, walking, riding a bicycle, resistance training).
Is the study peer reviewed?
If the reviewing author is unsure of how to classify any of these six criteria, the reviewer will mark the criterion as uncertain. The group of authors will collectively review any criterion that were marked as uncertain to determine its eligibility.
Data sources and search strategy
We plan on searching the Pubmed, Cumulative Index to Nursing and Allied Health Literature and Online Resource for Recruitment research in Clinical Trials databases for all existing articles from inception through 10 February 2017 to identify eligible studies. The database searches will be completed by one review author (JCH). Database searches will be restricted to English studies because of limited financial resources necessary for translating studies. Search strategies were formed according to the Cochrane handbook guidelines.35 The Pubmed search strategy is presented in table 1, and the full search strategy is presented in an online supplementary file. In addition to the database searches, we will attempt to locate additional eligible studies by reviewing the reference lists of all of the eligible studies and all relevant review articles (ie, review articles pertaining to recruitment and exercise in human adult research studies).
Supplementary file 1
Search strategy used in Pubmed—(inception to 2017). Search date: 10 February 2017.
Data management
The results from the database searches will be uploaded into EndNote citation manager software, which will allow for systematic storage of the search results as well as the ability to remove duplicate articles. To ensure a consistent eligibility screening process, all four reviewing authors (JCH, AMA, SA and MMA) will screen 100 randomly selected articles. Each reviewing author will mark each article as either eligible, ineligible or unclear. We will discuss any discrepancies in eligibility decisions in this random set of 100 articles until there is complete agreement between the reviewing authors. After agreeing on the initial 100 articles, the four reviewing authors will review the remainder of the identified articles. The reviewing author will determine whether the abstract appears to be eligible. If the reviewing author is uncertain of how to classify the abstract, he or she will indicate the uncertainty and the reviewers will collectively review that article and determine the article’s eligibility.
Data extraction
After all abstracts have been screened and potentially eligible studies have been identified, two review authors will independently extract data from eligible articles. The following data will be extracted from the studies:
Recruitment variables;
Number of patients/participants contacted;
Length of time spent recruiting;
Number of participants enrolled;
Number of people recruiting;
Background of recruiters;
Amount of training for recruiters;
Hours per week (total) spent recruiting;
Method of recruiting (eg, flyers, clinic visits, public advertisements);
Money spent on recruiting;
Methodological quality variables;
Intervention variables;
Participant demographics;
Descriptive characteristics of study and authors.
For missing data pertaining to the main recruitment variables, reviewing authors will use a preconstructed template for any necessary author queries. Review authors will only attempt to contact the author of the publication to gather the missing recruitment information during the first round of data extraction. This will serve to limit unnecessary contacts to authors. Information resulting from author queries will be made available to the reviewing author during the second round of data extraction so that the agreement between reviewers does not suffer as a result of one reviewer having access to more information than the second reviewer. In the event that the author of a publication does not respond to the author query, the requested information will be treated as missing data. For studies where some of the recruitment variables have missing data, these studies will still be included in the final analyses. If all of the recruitment variables for a study are categorised as missing data, that study will be deemed ineligible and subsequently excluded from the analyses since failing to have any data pertaining to recruitment does not meet the inclusion criterion for ‘addressing recruitment.’
When both of the independent reviewing authors have completed the data extraction, those two reviewing authors will compare the data that they extracted. If the two reviewing authors have discrepancies in the data that they extracted, they will discuss any discrepancies in the extracted data in an attempt to agree on how to appropriately classify the extracted data. In the event that the two review authors cannot agree on how to rate an aspect of the data, a third review author will be used to settle the disagreement.
Outcomes
These systematic reviews will focus on four aspects of recruitment as the primary outcomes. First, we will examine the efficiency rate of recruitment (ie, the number of participants recruited into the study divided by the number of participants contacted). Second, the rate that the participants were recruited into the study (ie, the number of participants recruited into the study divided by the number of months taken to recruit them). Third, the monetary cost of recruiting participants (ie, the total cost of recruiting divided by the number of recruited participants). Fourth, the percentage of enrolled participants who withdrew from the study.
Assessing methodological quality
Included articles will be independently reviewed by two reviewing authors using the Cochrane Collaboration assessment tool to assess risk of bias, which reviews selection bias, performance and detection bias, attrition bias, reporting bias and other sources of bias.36 The risk of bias will be rated using three categories: High risk, low risk and unclear risk. As with the data extraction process, any disagreements between the two authors will be discussed, and a third reviewing author will be used to resolve any outstanding disagreements. Because this systematic review is focusing on how studies’ methodological characteristics affect recruitment, the Cochrane Collaboration assessment tool for risk of bias36 will provide insight into the extent to which risk of bias is present in the included studies, which will provide helpful contextual details when discussing the results of the analyses.
Data analysis
A series of exploratory multivariate regression models will be used to determine the effects of methodological factors, intervention characteristics and participant demographics on the recruitment variables of interest. Per Osborne and Waters’ 2002 recommendations,37 preliminary analyses will first be conducted to ensure that the robust and non-robust assumptions of multivariate regression have not been violated. These multivariate regression analyses will provide additional information pertaining to how well the independent variables (IV) predict the dependent variables (DV), and the analyses will also demonstrate the relative importance of each individual IV in relation to the other IVs in terms of predicting the DVs (ie, recruitment variables). Finally, a series of multivariate regressions will allow for moderators and mediators to be added to the model in an attempt to improve overall model fit and predictive validity, which can serve to further improve recruitment.
Subgroup analyses
Provided that we have enough studies to provide information related to variables of interest, we hope to conduct subgroup analyses on logistical factors (eg, discipline of the lead author, where the study took place), participant demographic factors (eg, age, gender, race), types of intervention (eg, aerobic interventions, strength training interventions) and specific research designs (eg, randomised control trials, matched research designs). These subgroup analyses will be identical to the primary analyses. Our aim is to identify functional differences within specific participant populations in addition to identifying any differential functioning of methodological factors within certain interventions and research designs.
Discussion
We believe that our systematic reviews will address a gap in the literature by addressing how various factors (eg, methodological, intervention, participant) impact overall recruitment rates. By empirically demonstrating the effects of these factors, future researchers may be able to recruit participants into their research studies more efficiently and quickly to budget for recruitment costs more accurately and to allocate resources for staffing needs more appropriately. Furthermore, resources that are traditionally spent on pilot and feasibility studies24 25 to examine recruitment can now be spent on other aspects of the intervention and project. This protocol will guide these reviews in a standardised and systematic way. In sum, producing an empirical analysis of recruitment rates has the potential for a significant impact. Since future scientific discoveries in human research are dependent on participation from research participants, information that facilitates that participation can serve to improve future research studies and better the field of exercise research.
Limitations
Because of the requirement that eligible articles must directly focus on recruitment or clearly elucidate recruitment as one of the outcomes, the included articles may reflect a selection bias that is weighted more heavily towards feasibility and pilot studies. By selecting more pilot and feasibility studies, it is possible that the observed recruitment rates were negatively impacted by the smaller budgets typically associated with these smaller feasibility projects.25 Additionally, it is possible that relevant studies may have been omitted by not including grey matter publications as well as publications that are in the process of being prepared and/or published at this point in time. Finally, as a third limitation, many relevant exercise studies may not have produced a publication focusing on recruitment or included recruitment variables as a primary or secondary outcome, which would have rendered those studies ineligible.
Ethics and dissemination
Ethical approval is not required for this systematic review because all of the data included in the review has been published and is publically available. The purpose of this review is to disseminate the results so that other researchers can improve recruitment for their studies. A manuscript will be submitted to a peer-reviewed journal to present the results of this review, and the results of the review will also be presented at a national conference.
Review status
This is an ongoing review. The first manuscripts are estimated to be completed by March 2018.
References
Footnotes
Contributors Each author contributed to conceptualising this project, creating the search strategy, refining the inclusion and exclusion criteria and producing this manuscript. JCH, AMA, SA and MMA were involved in the data collection and extraction, the risk of bias assessment and data analyses. JR and PMK provided oversight to the project. All authors provided approval for the final version of this manuscript.
Funding This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient consent Not required.
Provenance and peer review Not commissioned; externally peer reviewed.