Article Text
Abstract
Objectives Our primary aim was to evaluate the impact of US National Institutes of Health (NIH)-funded bioethics training programmes (Fogarty bioethics training programmes, FBTPs) that trained individuals from Africa over the programme's first 10 years to examine changes between pretraining and post-training in individual achievement and to document any associations between individual, training programme and post-training accomplishments.
Design We surveyed trainees from the 10 bioethics programmes funded by NIH Fogarty International Center from 2000 to 2011 that included African trainees. McNemar's and Wilcoxon signed rank-sum tests were used to analyse pre–post levels of general and bioethics-related professional achievement. Likelihood of specific post-training achievement outcomes was measured using logistic regression including demographic, pretraining and intratraining variables.
Setting 10 different FBTPs that trained individuals from Africa from 2000 to 2011.
Participants Of 253 eligible respondents, 171 completed the survey (response rate 67.6%).
Primary outcome measures Pre–post comparisons of professional achievement indicators (eg, serving in leadership roles, teaching, publishing manuscripts); likelihood of specific post-training achievement outcomes.
Results Post-training, respondents were significantly more likely to report serving in a leadership role, being an investigator on a research grant, serving on international committees, serving as a mentor, and publishing manuscripts than at pretraining. Post-training, significantly greater numbers of respondents reported bioethics-related achievements including being a bioethics instructor, serving on an Institutional Review Board (IRB), being an investigator on a bioethics grant and publishing bioethics-related manuscripts than pretraining. Controlling for other factors, there were no significant differences by gender in the post-training success of these participants in terms of leadership roles, being instructors, investigators on grants and holding IRB roles.
Conclusions African trainees who participated in FBTPs reported significantly higher levels of professional achievement after training. There was no single factor—either demographic, related to a trainee's professional background, or in programme design—that consistently predicted greater levels of post-training achievement.
- evaluation
- bioethics
- research ethics
- training
- capacity development
- Africa
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
This study is one of the largest surveys evaluating multi-institutional capacity development investments in bioethics over a long period of time.
Our approach builds on previous frameworks for capacity development, integrates a chronological perspective for training (pretraining, intratraining and post-training), and captures multiple indicators relevant to professional success.
All data were self-reported and provided after training, potentially increasing the risk of recall bias and error and there was no comparison group not participating in a Fogarty bioethics training programme, so causal claims about programme impact cannot be made.
Introduction
More health research is conducted globally than ever before,1 with a growing portfolio in low-income and middle-income countries (LMICs).2–4 In the late 1990s, several HIV prevention trials, funded mostly by the US government and conducted in LMICs, sparked ethical controversy surrounding which interventions were appropriate when background standards of care differed between richer and poorer settings.5 Early debates around these issues were conducted primarily among scholars from wealthy countries; as such, the need for ethics training for individuals from the global South was recognised.6–8 Research ethics workshops were offered in LMICs,9 but for professionals from LMICs to participate meaningfully in debates about how trials within their borders should be conducted, longer term training was necessary.
The Fogarty International Center (FIC), part of the US National Institutes of Health (NIH), supports global health research through training and partnerships between US and international researchers and institutions. The FIC has provided training to LMIC professionals in research methodology for over 40 years. In 2000, the FIC first called for long-term training in bioethics for individuals from LMICs.10 Initially, five programmes were funded, and by 2013, FIC sponsored 19 bioethics training programmes throughout the globe.
In 2011, the FIC invited proposals to evaluate the impact of its investment from the first 10 years of their global bioethics programmes.11 Specifically, the FIC sought to assess trainees' professional accomplishments and to investigate whether trainee, programmatic or institutional-level factors were associated with post-training programme achievements in research ethics. We report here the findings of an empirical study evaluating the impact of the 10 FIC-funded bioethics programmes that trained individuals from Africa over the FIC bioethics programme's first 10 years.
This paper describes differences in pretraining versus post-training accomplishments, and documents associations between accomplishments and individual, training programme and institutional factors. Previous evaluations of training programmes have largely been single programme, qualitative or limited by time.12 ,13 This represents one of the largest surveys evaluating multi-institutional capacity development investments in bioethics over a long period of time.
Methods
Sampling
Between 2000 and 2011, FIC funded 10 bioethics training programmes from North America or Africa that either targeted African trainees exclusively or for which African trainees were eligible.10 These 10 programmes shared the same broad objectives of deepening conceptual understanding of ethics principles, increasing trainee capacity to serve as members and administrators of Institutional Review Boards (IRBs) and strengthening skills in teaching and publishing bioethics-related material and research. While programmes varied, all Fogarty bioethics training programmes (FBTPs) involved academic coursework, mentoring, research training, a practicum in the trainee's home country and IRB observation. Some FBTPs offered masters degrees; others offered a certificate of completion. Programmes have been described more extensively elsewhere.14–17
Individuals were eligible for this study if they were from a sub-Saharan African country and had enrolled at least 1 year before this survey was conducted in an FBTP funded between 2000 and 2011 (see online supplementary appendix table 1). Eligible participants were identified by querying CareerTrac, an NIH-maintained database of NIH-funded trainees and their professional activities; and by contacting the 10 FBTP directors for trainee contact information. In 2013, trainees received email messages inviting them to complete the survey, both from their own FBTP director and the Johns Hopkins Fogarty African Bioethics Training Program (JHU-FABTP) team charged with conducting this evaluation. JHU-FABTP team members followed up with trainees via email and telephone to encourage survey completion. Detailed methods for this evaluation are described elsewhere.18
Supplementary appendix table
Survey instrument
Participants completed a self-administered online survey measuring pretraining and post-training professional achievement. The survey was developed based on existing assessment tools, feedback from the 10 FBTP directors and our own work in bioethics training evaluation.13 ,18–20 The instrument was reviewed by FBTP directors and pretested with four graduate students in the USA with backgrounds in international health and research.
The instrument included five sections:18
Demographics, including birth year, gender, birth country, year respondent began training, country of residence when starting the FBTP, current country of residence;
Pretraining professional achievements, including questions about academic credentials, teaching activities, engagement in leadership roles and activities, service on IRBs or Research Ethics Committees (RECs), publications, and other activities;
FBTP content and structure, including questions about the content and structure of the respondent's training programme, eg whether any elements were provided online, engagement with an IRB/REC, practicum requirements and duration, and training duration;
Post-training professional achievements, including the same questions as section 2, focusing now on post-training achievements;
Attitudes and beliefs about the FBTP, including reflection and opinion questions about the value of different training components and the programme as a whole. This section included an open-ended question inviting any other comments.
Respondents who completed multiple training programmes were prompted to answer the questions in sections 2 through 4 for each training programme completed, as well as several additional reflection questions about the value of participating in multiple programmes.
Statistical analysis
Preliminary analysis focused on univariate descriptive statistics and frequency tabulations. ‘Don't know’ responses were coded as ‘missing’ and excluded from analysis. Bivariate analyses using McNemar's and Wilcoxon signed rank-sum tests were conducted to explore relationships between pretraining (demographic and professional achievement variables) and post-training indicators of professional achievement.
A logistic regression model was created to determine whether any pretraining indicators or programmatic features (intratraining) predicted post-training professional achievement. Pretraining and intratraining predictors were selected based on three considerations: whether variables had been significant in bivariate analyses; whether variables were the subject of debate and discussion in the literature as possible predictors (eg, whether a programme offered a master's degree); and to allow us to control for demographic factors like gender, length of time since training and age. We ran models for both general achievement outcomes (serving in any type of leadership role, instructor or investigator) and ethics-specific outcomes (serving as an ethics instructor, investigator on ethics grants or on an IRB).
We analysed data on subset of respondents missing no data across all indicators in models. Regression models compared respondents missing no data to those excluded and found no statistically significant systematic differences across demographic variables between groups.
Qualitative analysis
Responses to the open-ended question about experiences in and impact of the FBTP were reviewed and a coding scheme generated to capture emergent themes. All comments were coded by two independent team members and any discrepancies discussed and reconciled. Multiple codes could be applied to a single comment; themes were collated using tables.
Results
CareerTrac identified 222 long-term trainees; FBTP directors identified an additional 35 trainees. Three individuals were deceased, and one had no valid contact information, resulting in a target population of 253 trainees. After multiple contact attempts, 171 completed the survey, generating a 67.6% response rate. Almost all (164 respondents) had completed one training programme; seven had completed two programmes.
Almost half (41%) of respondents were women, the average age when beginning their first FBTP was 40.3 years, and average age at survey completion was 46.6 years (table 1). All but seven respondents were living in Africa at survey completion. Nearly all (92%) were living within an African country exhibiting high child mortality and high to very high adult mortality (AFR-E or D as defined by the WHO). The majority (93%) reported living in the same country where they had been living when admitted for their FBTP.
Pretraining and post-training activities and accomplishments were compared using McNemar's tests for variables with yes/no responses and Wilcoxon signed rank-sum tests for ordinal responses. Post-training, respondents were significantly more likely to report being an investigator on a research grant, serving in a leadership role, and serving as a mentor than at pretraining (p<0.0001 for all, figure 1A). Respondents were more likely to serve as course instructors after FBTP training than prior (not statistically significant; p=0.28). Trainees were also more connected to the internet and to each other post-training (figure 1B). Before training, 51% of respondents had ever published a manuscript, whereas 66% had published after training (p<0.007, figure 1C); more trainees had published more than two manuscripts post-training than pretraining (57% vs 42%; p<0.007, figure 1C).
Respondents exhibited much higher levels of bioethics-specific achievements post-training (figure 2A). Significantly more respondents reported post-training they had been an instructor for a bioethics course, served on international and national committees with a bioethics focus, received bioethics-related travel grants, attended bioethics conferences and published bioethics-related manuscripts (figure 2A). Significantly more trainees reported serving in IRB positions or administrative roles after training (figure 2B). After training, 17% had served as IRB chairs, 59% as IRB members and 20% as IRB staff. Additionally, the proportion of trainees reporting never having served on an IRB decreased from 51% to 23% after training. Prior to FBTP training, 22% of respondents reported spending at least half of their professional time on bioethics, as compared with 53% post-training (p<0.0001, figure 2C).
Using bivariate analyses, differences between pretraining and post-training accomplishments were mostly identical by gender. An exception was that prior to participation in an FBTP, female trainees were significantly less likely to identify themselves in leadership roles than male trainees (53.5% vs 68.7%, p=0.044), whereas after training there were no significant differences (79.1% vs 79.8%, p=0.913), and the proportion of female trainees reporting a leadership role increased. Both female and male respondents were more likely to report having published a manuscript after training than before, although the proportion publishing six or more total papers was higher among male than female trainees post-training (15.5% vs 36.6%, p=0.044). In logistic regression, gender was a significant predictor only for a model predicting whether or not the respondent was an investigator on an ethics grant or published manuscripts. A trainee's age when starting their programme did not significantly impact later professional outcomes—older trainees were slightly more likely to hold a leadership role and slightly less likely to hold an IRB role after training.
The influence of pretraining and intratraining factors on post-training achievement varied (table 2). Trainees were 1.3 times more likely (per annum) to have a leadership role, be an investigator and publish manuscripts following their training. An important predictor of all outcomes was having engaged in that same activity prior to training: those who had already served on IRBs were more likely to be on IRBs afterwards, and those who had been investigators prior to training were more likely to be investigators after training. Controlling for other factors, holding a PhD had no significant impact on any outcome variables.
We were interested in the impact of training programme components on outcomes. Trainees earning a master's degree through an FBTP were roughly 1.8 times more likely to become an instructor of an ethics course post-training and were one-sixth as likely to serve on an IRB post-training.
When asked to select the single greatest impact on individual success in bioethics, a majority of trainees answered ‘Fogarty bioethics training’ (66%), followed by ‘personal motivation’ (14%) and ‘supportive supervisors/leadership’ (7%; table 3). When asked which FBTP component had the single greatest impact on their individual understanding of research ethics, most reported ‘academic coursework’ (63%), followed by ‘practicum experience’ (12%), and ‘peer interaction’ (7%; table 3). Many trainees felt that FBTP participation increased the respect they received from colleagues (86%), and over half reported always identifying themselves as a former FBTP trainee in professional contexts (60%; table 4). Most trainees felt their goals were met (93%) and their cultural background was appropriately respected within their programme (86%). Almost one-quarter of trainees, however, wished training had focused more on Africa (24%).
Of the total sample, 135 (78.5%) respondents provided an optional, open-ended comment at the end of the survey. Of these respondents, 115 (85.1%) reported a least one positive comment about their FBTP experience. Specifically, about one-third provided a comment suggesting that participating in the FBTP positively contributed to their professional achievements and outputs (32.6%); 18.5% offered that the programme increased their understanding of ethics; 17.8% said that the programme was ‘life-changing’ or ‘broadened horizons’. The remaining comments (∼15%) suggested opportunities for programmatic improvement, described challenges faced during training or noted difficulties with integrating their acquired knowledge and skills within home institutions.
Discussion
This is the first project of which we are aware to evaluate long-term bioethics training for African trainees participating in any of multiple different training programmes. This builds on previous attempts to capture the returns on capacity development in LMICs.21 ,22 Respondents reported significantly higher levels of professional achievement after FBTP training across all indicators than at pretraining, including being more likely to do IRB work, teach ethics, publish and have leadership positions. While only one-fifth of trainees said they spent more than half of their professional time on ethics-related activities before training, more than half stated this afterwards. Further, 40% stated that they had spent none of their time on ethics before training, while after training 6% reported this. Almost all (93%) agreed their goals had been met through FBTP training, and many (86%) stated that, on returning home, they garnered more respect from colleagues. Findings are consistent with a systematic review of research capacity strengthening training programmes in sub-Saharan Africa, which documented increased trainee skills and knowledge and more research undertaken across all programmes reviewed and trainees publishing in journals across half of the programmes.22
No single factor—either demographic, in trainees' professional background or in programme components—consistently predicted greater levels of achievement. Involvement in a particular activity (eg, IRB service or teaching) predicted future involvement, but features such as whether trainees had doctoral degrees, their age or previous IRB service did not predict achievement post-training. Future programmes should note that no single trainee profile predicted higher productivity post-training. There were no significant differences by gender in most post-training outcomes, although female trainees were ethics investigators less than male trainees and published less, consistent with previous studies.14 ,19 Yet, while more men than women held leadership positions pretraining, this difference disappeared post-training. This is notable given reports on gender disparities in medicine and academia in Africa.23 It will be important to learn whether women's participation in other visible training programmes decreases the gender gap in professional success post-training.
At survey completion—6 years post-training, on average—nearly all trainees reported living in the African country where they had lived when admitted to FBTP. This is significant given concerns about brain drain from African countries.24 Further, at the time of survey administration, the majority of trainees reported living in African countries marked by high child and adult mortality rates. This signals that FIC bioethics training programmes are effectively training individuals in regions facing significant disease burdens and therefore most likely to benefit from increased health research and research ethics capacity. Our data suggest that trainees were connected with one another after training, important given other research suggesting that professional networks serve as a channel for affecting public health policies and programmes in African countries.12 A benefit of programme participation may include becoming part of a professional community and holding an identity with a named programme—indeed 60% of trainees reported introducing themselves as alumni of a Fogarty programme; a crucial outcome of organised training programmes may be their ability to foster lasting networks of professional researchers and academics within countries and across the African continent.
There were limitations to this work. All data were self-reported and provided after training, potentially increasing the risk of recall bias and error. Second, there was no comparison group not participating in an FBTP, so causal claims about programme impact cannot be made; undoubtedly many of these individuals, who were selected for being promising professionals, would have continued to have professional accomplishments and some likely would have become leaders in their fields regardless of additional training; we cannot know whether the degree of differences, particularly in areas focused specifically on bioethics, would have been as remarkable. While this attribution challenge pervades programme evaluation work, our chronological approach and separation of variables by pretraining, intratraining and post-training helps with plausibility of results. Third, we acknowledge this data set contains no indicators for income level, personal motivation or institutional support for ethics-related work, all of which may predict professional success. Fourth, qualitative interviews with trainees and programme directors, using an approach such as that used by Pratt et al,25 would likely have provided greater depth to our survey findings. As described above, respondents generally expressed positive reactions when provided with an open-ended question about their FBTP experience. Fifth, these programmes represent only one funding organisation's investment building research ethics capacity in Africa; evaluations of additional programmes would further strengthen this work. Sixth, while our response rate was good for this type of survey, we acknowledge the potential for non-response bias. Finally, in focusing on FBTP's sub-Saharan African trainees, we cannot speak to ethics capacity building in other parts of the globe.
This is the first evaluation of trainees' accomplishments after participating in any of 10 bioethics training programmes in Africa. Our approach builds on previous frameworks for evaluating capacity development, integrates a chronological perspective for training and captures multiple indicators relevant to professional success.18 ,26–29 We hope this work provides insight on the effects of investment in capacity development in bioethics in Africa and provides a new approach to help others evaluate other programmes.
Acknowledgments
The would like to thank all FBTP trainees for their participation, FBTP directors for their support and Tara White for input on survey design and administration.
References
Footnotes
Contributors NEK, JA and AAH created and administered the survey; participated in data analysis. KH prepared and analysed survey data. All authors were involved in manuscript drafting.
Funding Research reported in this publication was supported by the Fogarty International Center and National Institute of Allergy and Infectious Diseases of the National Institutes of Health under Award Number R25 TW 001604.
Disclaimer The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Competing interests All authors had full access to all the data in this study and had final responsibility for the decision to submit for publication. All authors have completed the Unified Competing Interest form at http://www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare that NEK, JA, AAH, KH have support from Fogarty International Center for the submitted work.
Ethics approval Johns Hopkins Bloomberg School of Public Health IRB.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Survey instrument available on request.