Article Text
PDF
Abstract
Objectives The overall purposes of this first US national pilot study were to (1) test the feasibility of online administration of the Bioethical Issues in Biostatistical Consulting (BIBC) Questionnaire to a random sample of American Statistical Association (ASA) members; (2) determine the prevalence and relative severity of a broad array of bioethical violations requests that are presented to biostatisticians by investigators seeking biostatistical consultations; and (3) establish the sample size needed for a full-size phase II study.
Design A descriptive survey as approved and endorsed by the ASA.
Participants Administered to a randomly drawn sample of 112 professional biostatisticians who were ASA members.
Primary and secondary outcome measures The 18 bioethical violations were first ranked by perceived severity scores, then categorised into three perceived severity subcategories in order to identify seven ‘top tier concern violations’ and seven ‘second tier concern violations’.
Results Methodologically, this phase I pilot study demonstrated that the BIBC Questionnaire, as administered online to a random sample of ASA members, served to identify bioethical violations that occurred during biostatistical consultations, and provided data needed to establish the sample size needed for a full-scale phase II study. The No. 1 top tier concern was ‘remove or alter some data records in order to better support the research hypothesis’. The No. 2 top tier concern was ‘interpret the statistical findings based on expectation, not based on actual results’. In total, 14 of the 18 BIBC Questionnaire items, as judged by a combination of ‘severity of violation’ and ‘frequency of occurrence over past 5 years’, were rated by biostatisticians as ‘top tier’ or ‘second tier’ bioethical concerns.
Conclusion This pilot study gives clear evidence that researchers make requests of their biostatistical consultants that are not only rated as severe violations, but further that these requests occur quite frequently.
- ethics (see medical ethics)
- medical ethics
- public health
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
First study to quantify bioethics violations in US biostatistical consulting.
Verified that the Bioethical Issues in Biostatistical Consulting Questionnaire detected differences in frequency and severity of bioethical violations.
Established sample size needed for full-sized study.
Established feasibility of recruitment and data collection methods.
Small sample size of pilot study.
Limited capability to conduct analysis of co-factors.
Introduction
This pilot study is the first US national survey to quantitatively identify a wide array of bioethical violations that arise between scientific investigators and their biostatistical consultants, a collaborative research consultation that underpins virtually all scientific studies. This descriptive survey quantifies, for the first time, the frequency of requests for ‘inappropriate data manipulation or practices’ by investigators via consultations with biostatisticians on a national level. While this phenomenon has been known to exist, the extent to which it exists has simply not been adequately studied, and this lack of research on bioethical research violations has been lamented by several authors.1–13
While six previous studies that attempted to quantify aspects of bioethical violations in research have suggested that violations levels were ’of concern’, each study has major limitations that preclude the drawing of firm and clear conclusions.1 2 6 9 11 12 One early study in 1993 only reported on the rate of exposure of doctoral students to perceived misconduct,1 while another study of that era that evaluated 23 possible ethical research violations reported that 10% of the membership of three surveyed professional research societies had observed data falsification or fabrication.2 Two later studies targeted research coordinators and asked only a very limited number of questions and achieved low response rates, one 31%, one 37%.11 12 A fifth study, a survey seeking the opinion of scientific meeting programme chairs from their annual international research meeting, which focused only on scientific abstracts submitted to their annual meeting, assessed 26 problematic research practices, achieved a response rate of 78% and reported that 30% had observed falsification of data and 54% had observed plagiarism one or more times.6 The sixth study, which sought to assess scientific fraud experienced by an international group of biostatisticians, reported that 51% were aware of at least one fraudulent study but only achieved a response rate of 37%.9
The overall purposes of this pilot study, conducted in collaboration with the American Statistical Association (ASA), were threefold: (1) to administratively pilot test the research methods proposed for use in a full-scale study using the newly developed Bioethical Issues in Biostatistical Consulting (BIBC) Questionnaire as administered to a random sample of US biostatisticians; (2) to establish, for the first time, the prevalence and relative severity of a broad array of bioethical violations requests that are presented to biostatisticians by investigators seeking biostatistical consultations; and (3) to gain estimates of the prevalence and relative severity of those bioethical violations to permit the planning and conducting of a full-scale, phase II study.
Methods
This phase I pilot national survey used a validated, pretested 18-item BIBC Questionnaire as previously developed within an NIH/NIDCR Oral Health Disparities Center (U54 DE14257) in collaboration with the National Center for Bioethics for Research and Health Care at Tuskegee University.13 In this phase I pilot study, the 18-item BIBC Questionnaire was administered to a randomly drawn sample of 112 professional biostatisticians who were members of the ASA, as drawn from their national membership list.
Each questionnaire item represents a different bioethical violation event. Specifically, the 18 items ask what bioethical violations the respondent has personally and directly been asked to do during their bioethical consultations over the past 5 years. Respondents were asked to make two assessments for each of the 18 items: (1) the total number of times they had been asked to do that specific bioethical violation over the past 5 years (using a five-point ordinal scale: 0, 1, 2–4, 5–9 and 10+) and (2) their own professional opinion on the ‘bioethical violation severity’ of that specific bioethical violation (using a five-point ordinal scale ranging from least to most severe: 0–5).
Of the approximately 18 000 total ASA members, approximately 5000 members who are categorised as ‘working statisticians’ (frequently performing data management and data analysis, consulting to other researchers in data analysis and statistics) comprised the available sample pool. They met the following eligibility criteria: (1) self-identified on their ASA annual registration forms as specialising in biomedical research consulting activities and (2) have at least 2 years of experience as biostatisticians. Our goal for this pilot study was draw a sample of 112 and to achieve a high response rate (>70%) via the use of an endorsement by the ASA and the use three specific incentives to participate: (1) a $99 Amazon gift certificate for completing the estimated 30 min BIBC survey; (2) a web tool online data collection system that avoided the use of any personal identifier for the respondent; and finally for this novel line of inquiry in reporting of violations, (3) the use of the concept of ’requests made to biostatisticians' as its dependent variable (as opposed to the alternative high-risk dependent variable of ’actually committed violations’) to ensure higher participation rates, as well as greater participant candour, in this first exploratory study. Data analysis for this initial pilot study consisted of descriptive analysis of the demographic variables, as well as for both the ‘perceived severity’ rankings and ‘frequency’ rankings of the 18 listed possible bioethical violations.
Results
First, from an initial working list of 800 emails as provided by the ASA, a random selection process was used progressing in subsets of n=50 to obtain an n=112 while avoiding an overenrolment which would exceed budgetary limits for incentives for enrolled subjects. The final response rate for randomly drawn ASA members was 67%. The demographic data on the respondents revealed that respondents self-reported working as biostatisticians between 2 and 55 years (median number of years=13), and 86.4% were employed full time, 7.3% were self-employed, 2.7% part-time employed with 3.6% not currently working. Of those currently working, 41.8% worked at a university (73.3% at a first tier research university and 11.1% at a second tier research university) while 58.2% were employed at non-university jobs.
Table 1, on its left side, shows the 18 bioethical violations items from the BIBC Questionnaire in ranked order by percentage of respondents rating the item as a ‘5’ (most severe) in ‘perceived severity’, and then subcategorised into severity group I (the top three most egregious violations), severity group II (the next eight most egregious violations) and severity group III (the seven least egregious violations). The bolded ‘q#’s’—within severity groups I and II—are marked by an asterick (eg, q#2*) and indicate ‘top tier violations’ (ie, have a ‘perceived severity’ score of 4–5 for at least 65% of the respondents AND a ’no. of times asked in last 5 years’ of 1–10+ times for>20% of the respondents). There were seven identified ‘top tier concern violations’.
Ranking of bioethical violations by ‘perceived severity’ and ‘number of times directly asked to do it over the past 5 years’: BIBC phase I (n=112) findings for q#1–18 which asked biostatisticians ‘to estimate the number of times—during the past 5 years—that you, personally, have been DIRECTLY asked to do this’
The unbolded ‘q#’s’ marked with a cross sign (†) (eg, q#7†)—all these are within severity group III—are labelled as ‘second tier concern violations’ (ie, have a ‘perceived severity’ score of 4–5 for at least 33%–64% of the respondents AND a ’no. of times asked in last 5 years’ of 1–10+ times for >20% of the respondents). There were also seven identified ‘second tier concern violations'.
Based on these pilot study findings that the observed effect size of most of the variables in relation to the demographic factors were moderate (ie, in the range of 0.3–04), our follow-up phase II study will seek a sample of 400 ASA members which will have a statistical power above 80% while being able to detect a minimum of 10% difference of the dependent variable between demographic and environmental variables.
Discussion and conclusions
Thus, 14 of the 18 BIBC Questionnaire items, as judged by a combination of ‘severity of violation’ and ‘frequency of occurrence over past 5 years’, were rated by biostatisticians as ‘top tier’ or ‘second tier’ bioethical concerns, that is, minimally having the characteristics of a ‘perceived severity’ score in the high range (ie, a score of 4 or 5) for at least 33% of the respondents AND having been ‘been asked during a biostatistical consultation’ over the past 5 years for at least 20% of the respondents. Inevitably, if unfortunately, the limited sample size of this pilot study prevents detailed subanalyses of the findings by demographic and work environmental factors. Finally, given that these findings are from a pilot study designed to answer methodologic issues, any detailed comparisons of our bioethical violations findings with prior studies would be inappropriate; those comparisons must await the findings from our funded—and now underway—full-sized, phase II study.
Nevertheless, there are clear public health implications from the findings of this phase I pilot study. First, the pilot US national survey quantitatively identified a wide array of bioethical violations that arise between scientific investigators and their biostatistical consultants, giving clear evidence that researchers make requests of their biostatistical consultants that are not only rated as severe violations, and that these requests occur quite frequently. Second, these phase I pilot findings provide strong evidence in support of future studies that will (1) provide replication of these findings in a large sample of subjects and (2) allow a more refined analysis of the findings by demographic variables.
Following our successful completion of this phase I pilot study, our research team submitted a phase II grant that was funded by the Office of Research Integrity at the US Department of Health and Human Services to conduct a follow-up phase II full-sized study which is currently underway, again in collaboration with the ASA. The findings from that phase II full-sized study will serve to more definitively describe both the frequency and severity of bioethical violations requested during biostatistical consultations, as well as guide the development of future educational bioethical training modules targeted at university-based clinical research training programme and their directors as well as to encourage and develop means for research universities and companies to improve their institutional environmental efforts regarding job and publication pressures to reduce the frequency of these bioethical violation requests.
References
Footnotes
Contributors MQW led the research team in the writing of phase I grant; designed the online data collection system; conducted the data analysis; reviewed and approved the final draft of this manuscript. AFY contributed to the writing of the phase I grant; administered and managed the online data collection system; contributed to the data analysis; reviewed and approved the final draft of this manuscript. RVK initially conceived of the project and research design method; contributed to the writing of this phase I grant; wrote the first draft of this manuscript.
Funding This work was supported by a grant from the Office of Research Integrity (ORI) of the U.S. Department of Health and Human Services (DHHS). Grant No. 1 ORIIR150017-01-00.
Competing interests None declared.
Ethics approval This pilot study was approved by the IRB at the University of Maryland School of Public Health and by the IRB at New York University as an Expedited Review category involving minimal risk for the subjects.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Technical appendix, statistical code and dataset are available from MQW (Department of Behavioral and Community Health, University of Maryland School of Public Health) via his email: mqw@umd.edu.