730 e-Letters

  • Prediction intervals - the beginning of an effectful future?

    There is considerable debate going on questioning the practical usefulness of a priori power calculations suggesting that “underpowered” studies are not unethical and that little scientific projection would be still better than no projection at all [1-4]. Some authors argue that “being underpowered is unethical” is a “widespread misconception which is only plausible when presented in vague, qualitative terms but does not hold when examined in detail” [1, 2]. Further review of the arguments reveals that the crucial assumptions implied in the reasoning do not reflect actual scientific practice. The main theoretical arguments assume a perfect “frequentist world” that may allow substitution of one big trial by a corresponding number of small trials that would, once being aggregated in a formal evidence synthesis i.e. meta-analysis, cumulate the same information as the big one [2, 4]. If the individual studies are non-representative samples of the target population, the practical value of estimating a pooled effect that is a weighted average of potentially disparate effects in different subpopulations is questionable.

    A widely considered answer to the threat of effect heterogeneity in meta-analyses are random-effect confidence intervals that are often assumed to better reflect variation in the effects across subpopulations than fixed-effects confidence intervals. However, while such intervals offer a valid solution to inference regarding the average effect across all c...

    Show More
  • Critical analysis of the Dumont et al. uterine balloon randomized controlled trial in Benin and Mali

    As postpartum hemorrhage (PPH) researchers, and leaders in education and care of maternal health emergencies, from the United States, UK, Canada, India, Peru, Honduras, Zambia, India, Kenya, Tanzania, Colombia and Nepal, we read the Dumont et al paper with great interest. We would like to share our review:

    The most fundamental flaw of this paper is that the authors confuse an intention-to-treat study of a clinical pathway of interventions and behaviors, with the efficacy of a device. These are two very different research questions. In order to test the latter via a randomized controlled trial (RCT) the two groups would need to be the similar and subjects that did not even receive the device (or received it in desperation two hours after the diagnosis of uncontrolled PPH) certainly could not be included in the intervention group. Thus, this study attempts to test intention-to-treat, not the efficacy of the uterine balloon tamponade (UBT) device.

    The second most obvious flaw is that degrees of illness are not accounted for. Clinically defined "uncontrolled PPH" is in no way a homogeneous group. For example, someone that has been referred in and is moribund from their advanced shock is an entirely different subject than someone who has mild uncontrolled PPH. Since this is not controlled for, these two groups are likely incomparable.

    Even taking into account the two issues described above, the two groups are different and heavily favor the non-...

    Show More
  • Breast cancer mortality is associated with factors linked to social and medical development in São Paulo State, Brazil

    Dear Editor:
    We have read the study conducted by Diniz et al. on the possible association between mammography and breast cancer-related mortality in the state of São Paulo, Brazil with the greatest of care. Despite the detailed statistical analysis, the ecological study design implies limitations to the hypothesis generated, as pointed out by the authors themselves (1). In our opinion, both the authors’ main conclusion and the assumed association of cause and effect are inappropriate.

    The factors associated with the incidence of breast cancer in Brazil and its resulting mortality have recently been evaluated in different studies (2-4). Mortality rates have been found to vary as a function of geospatial location (rural areas versus urban centers)(4). In addition, the reduction encountered in mortality was associated with the regions in which the human development index (HDI) was higher. On the other hand, the highest mortality rates have been found to occur in the states with the highest HDI (5). Diniz et al. and many other investigators have mentioned that a higher incidence of breast cancer occurs among more affluent women living in urban areas and in large cities (1,5). In this respect, we are certain that mortality is also related to the incidence of the disease; hence, the higher the incidence, the greater the resulting mortality will be. Conversely, women who do not have breast cancer will obviously not die from the disease.

    Therefore, we believe t...

    Show More
  • Cholesterol levels have negligible correlations with cardiovascular events

    Please see my article with the above name, published in the New Zealand Medical Journal, which used the Caerphilly data. This has not been mentioned elsewhere, presumably because of its obscure site. There was no relationship at all between cholesterol and heart attacks and only 4% of the variance was associated with strokes. NZ Med. J. 2012, 125, 1364.

  • Ask not only whether unpublished data were used but also how they were used

    This study asks the important question, what proportion of systematic reviews searched for and made use of unpublished data? However, an important follow-up question remains to be addressed: Among those cases in which unpublished data was used, how was it used? Unpublished data can of course address study publication bias, ie. data from unpublished studies can be simply added to data obtained from the published literature. However, unpublished data can also address outcome reporting bias,[1-3] ie. a trial publication conveys that the intervention is safe and/or effective while unpublished data on the same trial tell a different story. For example, in a study of 74 industry-sponsored antidepressants trials,[4] in addition to 23 (31%) unpublished trials, we found 11 (15%) trials with outcome reporting bias. If we had corrected for the former while ignoring the latter, we would have obtained an effect size estimate that was still inflated. Returning to the current study,[5] an informative follow-up would be to look within the cohort of systematic reviews that made use of unpublished data and determine how many used it to verify the published results.


    1 Kirkham JJ, Dwan KM, Altman DG, et al. The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ 2010;340:c365.

    2 Chan A-W, Altman DG. Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of author...

    Show More
  • Authors respond to "Comments on PROMISE data interpretation in Siemieniuk meta-analysis from PROMISE team"

    Dear Editor:

    We thank Dr. Fowler and colleagues for taking the time to consider and comment on our BMJ Rapid Recommendation (1). They speculate on reasons why tenofovir and emtricitabine increased the risk of neonatal mortality and early preterm delivery in their trial (2) and then say that the current evidence does not support a recommendation for alternative NRTIs over a tenofovir-based antiretroviral therapy (ART) regimen. We do agree that most, but not all, of the evidence comes from a single study, which may have overestimated harm. Our systematic review attempted to generate the current best evidence, and is not definitive: it is moderate-to-low quality for key outcomes (3). However, we disagree with the implication that based on this evidence, most women would choose a tenofovir-based ART regimen.

    The PROMISE authors suggest that results of the comparison between tenofovir-ART and AZT-ART are untrustworthy because the risk of neonatal death was lower in the AZT-ART arm in the earlier period 1 before the tenofovir-ART arm was introduced (2). However, the difference between the two time-periods in the AZT-ART arm could easily be explained by chance (neonatal mortality 1.4% in period 1 vs. 0.6% in period 2, p=0.39; very preterm delivery 3.4% in period 1 vs. 2.6% in period 2, p=0.60). Regardless, the only reliable comparison between tenofovir-ART and AZT-ART is during period 2 when randomisation to both AZT and tenofovir-based ART occurred. Despite these r...

    Show More
  • Authors’ response to Salemi and Zoorob

    We agree with Professors Salemi and Zoorob (1) that the 2016 UK guidelines’ message regarding the consumption of alcohol during pregnancy is clear: it is best to be avoided. However, this has not always been the case and previous guidelines implicitly suggested that drinking up to 4 units per week was likely to be safe. We also recognise the methodological challenges of studying long term effects of low maternal alcohol consumption in pregnancy. Like Salemi and Zoorob, we feel that the paucity of evidence is unfortunate and should be addressed. We also maintain that in light of the evidence that is available, basing guidelines on the precautionary principle is reasonable.

    Specifically, Salemi and Zoorob raise concerns about the inclusion of a study by Salihu et al. (2, reference 27 in the review) in the review (3), because maternal alcohol consumption was determined retrospectively, i.e. after delivery. Our predefined eligibility criterion was explicit - we required alcohol consumption to be ascertained prior to pregnancy. Salemi and Zoorob therefore question the inclusion of Salihu et al.’s study in our review. In the Methods section, Salihu et al. state that they used “the Missouri linked cohort data files…1989 through 2005.” No further details are provided The Discussion states that “Alcohol was based on maternal recall and therefore subject to bias. Women may underreport actual alcohol consumption because of societal stigmas, biasing study results toward the nu...

    Show More
  • Comment from Editor on reviewer Conflict of Interest

    During the peer review process of this manuscript, it came to our attention that one of the reviewers, Erling Solheim, may have an undeclared conflict of interest. We were told that he had worked in the same research group as the authors from June 2006 to September 2007 and had published research with the authors in 2008 and 2013.

    We attempted to contact Erling Solheim a number of times to verify these claims, but he has not responded to our emails. If true, we do not feel that this undeclared conflict of interest would have compromised the peer review process or altered our decision to publish the manuscript.

  • There are plenty of problems, but paucity of evidence is not one of them

    Dear Sir,
    In their recent paper,1 Mamluk et al describe the results of a systematic review and meta-analyses of the association between low alcohol intake in pregnancy and several adverse birth outcomes and long-term outcomes in children. The authors conclude that there is “limited evidence for a causal role of light drinking in pregnancy, compared with abstaining, on most of the outcomes examined” and that their “extensive review shows that this specific question is not being researched thoroughly enough, if at all.”

    I sympathize with the thorough work performed by the authors. Even so, a word of caution is needed with respect to the second part of their conclusion. The authors defined the intake of interest as a maximum of 32 g of alcohol per week corresponding to 4 standard UK drinks/week. While this makes sense from a British point of view, the definition of a standard drink being 8 g of alcohol, this cut-off makes little sense in most non-British countries, where a standard drink is typically defined as containing 10-12 g. Hence, a large number of non-British studies using three or four drinks as their cut-off for low intake have not been included in the present review, only because their definition did not correspond with the British.

    Several meta-analyses in recent years have used slightly more embracing definitions.2 3 These meta-analyses included far more studies, simply because a limit of 32 g fits poorly with international definitions. Using...

    Show More
  • Letter in response: Clinical Trial Transparency: A re-assessment of industry compliance with clinical trial registration and reporting requirements in the United States

    Dear Editors,

    Thank you for the opportunity to respond to Lassman et al.’s re-analysis of our study titled, “Clinical trial registration, reporting, publication and FDAAA Compliance: A cross-sectional analysis and ranking of new drugs approved by the FDA in 2012”.1 Our original study assessed the clinical trial transparency of novel drugs approved by the FDA in 2012 that were sponsored by large drug companies. We assessed the drugs by two sets of transparency standards: U.S. legal requirements and an ethical standard that all human subjects research should be publicly accessible to contribute to generalizable knowledge.

    Our original analysis included a review of 15 drugs, sponsored by 10 large companies, involving 342 trials. Lassman and colleagues’ reassessment examined 69 of these 342 trials and focused on only the U.S. legal requirements standard. Lassman et al did not elaborate on why they limited their assessment to this subset of trials and on compliance with legal requirements. As a reminder, US clinical trial disclosure requirements are defined by the Food and Drug Administration Amendments Act (FDAAA), passed in 2007.2

    We applaud efforts to replicate studies. We are glad that our policy of publicly sharing data through the Dryad Digital Repository enabled replication and re-analysis.3 Additionally, we generally agree with the Lassman and colleagues re-assessment of our study using today’s new and updated knowledge base and world-view....

    Show More