Do Certain Countries Produce Only Positive Results? A Systematic Review of Controlled Trials

https://doi.org/10.1016/S0197-2456(97)00150-5Get rights and content

Abstract

Objective: To determine whether clinical trials originating in certain countries always have positive results. Data sources: Abstracts of trials from Medline (January 1966–June 1995). Study selection: Two separate studies were conducted. The first included trials in which the clinical outcome of a group of subjects receiving acupuncture was compared to that of a group receiving placebo, no treatment, or a nonacupuncture intervention. In the second study, randomized or controlled trials of interventions other than acupuncture that were published in China, Japan, Russia/USSR, or Taiwan were compared to those published in England. Data extraction: Blinded reviewers determined inclusion and outcome and separately classified each trial by country of origin. Data synthesis: In the study of acupuncture trials, 252 of 1085 abstracts met the inclusion criteria. Research conducted in certain countries was uniformly favorable to acupuncture; all trials originating in China, Japan, Hong Kong, and Taiwan were positive, as were 10 out of 11 of those published in Russia/USSR. In studies that examined interventions other than acupuncture, 405 of 1100 abstracts met the inclusion criteria. Of trials published in England, 75% gave the test treatment as superior to control. The results for China, Japan, Russia/USSR, and Taiwan were 99%, 89%, 97%, and 95%, respectively. No trial published in China or Russia/USSR found a test treatment to be ineffective. Conclusions: Some countries publish unusually high proportions of positive results. Publication bias is a possible explanation. Researchers undertaking systematic reviews should consider carefully how to manage data from these countries.

Introduction

The Cochrane Collaboration, an international network of individuals committed to preparing, maintaining, and disseminating systematic reviews of clinical trials [1], has recently established a complementary medicine “field.” This group of researchers has various functions connected with supporting those who undertake Cochrane reviews, such as developing a register of randomized controlled trials, encouraging reviewers who are examining particular health problems (such as back pain) to undertake reviews of trials of complementary therapies, and offering methodologic advice where appropriate.

Because the field wants to promote systematic reviews of acupuncture, we are interested in obtaining data from China; however, researchers have suggested that, for reasons unknown, clinical trials from China always find acupuncture superior to the control intervention. Since this finding would have implications for systematic reviews, we wanted to learn whether it was true. We analyzed the results of trials referenced on Medline as a preliminary means of investigating whether clinical trials originating in any particular country had unusually high proportions of positive results. We are unaware of any similar published study.

Section snippets

Methods

The study comprised two separate searches and analyses. First, we retrieved and analyzed a set of trials on acupuncture that had been published worldwide. To test hypotheses resulting from this study, we then retrieved and analyzed clinical trials of interventions other than acupuncture that had been published in five specified countries.

The first search, made on Medline (January 1966–May 1995), used “exp acupuncture” as a focus term and “human” and “abstract” as limits. This search strategy

Study 1: Acupuncture

The initial Medline search located 1085 papers of which 252 were eligible for analysis. Overall, 171 trials (68%) were classed as “acupuncture superior to control” and 80 (32%) as “acupuncture equal or inferior to control.”

Acupuncture research was conducted in 27 different countries with the United States (47 trials), China (36 trials), Sweden (27 trials), and the United Kingdom (20 trials) contributing the greatest number of studies. Table 1 presents the raw outcome data for each country.

Discussion

Medline-indexed clinical trials conducted or published in certain countries favor acupuncture over control much more frequently that those conducted and published elsewhere in the world. All 51 trials conducted in China and East Asia and listed on Medline in the period studied favored acupuncture, suggesting that clinical trials of acupuncture originating in certain countries have an unusually high proportion of positive results. One possible explanation for this finding is that acupuncture is

References (12)

There are more references available in the full text version of this article.

Cited by (555)

  • Modernizing evidence synthesis for evidence-based medicine

    2023, Clinical Decision Support and beyond: Progress and Opportunities in Knowledge-Enhanced Health and Healthcare
View all citing articles on Scopus
View full text