What is new? To our knowledge, this study presents the first empirical comparison among Chinese, Indian, and Western randomized controlled trial publications. We found poor reporting of all quality measures among Chinese and Indian trials and strong evidence of bias in the reported results of Chinese trials.
The randomized controlled trial (RCT) provides the best evidence on the effects of health care interventions. Reports of RCTs provide evidence for health care providers and policy makers to make decision on whether to use new health technologies in their patients. To maximize the benefits from increased knowledge, a high-quality publication should always report the characteristics and results of an RCT. Previous work has indicated that the methodological quality of an RCT is reflected in the quality of such reports [1], [2]. Through the RCT publications, we can thus evaluate the trial's conduct and quality in developing research contexts.
China and India are two potentially important purveyors of RCTs given their substantial populations. They are also potentially important users of the results of high-quality trials. A search of ClinicalTrials.gov on October 7, 2009, identified that there were 29,919 ongoing randomized trials registered worldwide. China included 641 ongoing registered trials and India 423 ongoing trials, collectively almost 4% of those registered worldwide. This is likely to be an underestimate as many trials may not be registered on the ClinicalTrials.gov site or indeed at any registration site. In addition to investigator-led trials, the international pharmaceutical and device industries increasingly conduct their development programs for new products in lower income countries. Trials from low-income countries may also investigate treatments applicable to that setting that are not evaluated in higher income countries. Therefore, trial quality in low-income countries affects not only the substantial local population but also potentially influences health policy in other regions. In this article, we examine aspects of the quality of the reports of trials conducted in China and India and contrast these with a “gold standard” set of trials published in leading journals and conducted in Europe and North America. A better understanding of the quality of trials is important to both aid current interpretation and inform any future methodological work on the quality of RCTs in those countries. We deliberately chose to include trials published in top Western journals because we considered that these represented “best current practice,” a standard to which trialists can aspire. However, we are aware that this provides a tough comparison for trialists and journals from China and India.
The evaluation of journal articles reporting RCTs may include a number of qualitative indicators. Many of these are captured in the CONSORT statement. In our previous work, we identified that Chinese RCT reports frequently did not present adequate information [3]. We have been unable to identify a similar appraisal of the quality of reporting of RCT reports from India. In previous work, traditional Chinese treatments were invariably found superior to the control intervention in Chinese RCT papers [4]. However, no systematic studies have been done to confirm this assertion, which could be because of a selective consideration of Chinese trials or indeed through a high level of efficacy of the treatments concerned.
In our study, we examine whether reports of trials published in Chinese or Indian journals that were undertaken at least in part within those countries exhibit a high proportion of positive results compared with a “gold standard” selection from high-quality European and North American journals. We examine the extent to which any difference in the rate of positive results in trials may be because of trials examining traditional Chinese medicine treatments (traditional treatments considered mainstream in China but largely considered alternative medicine outside that country) or whether it may be because of markers of trial quality.
Relevant Chinese and Indian papers were identified through PubMed, “ISI Web of Sciences” databases, and the Journal Series of the Chinese Medical Association. PubMed includes MEDLINE and OLDMEDLINE (http://www.ncbi.nlm.nih.gov, 2006), but many non-English journals are not listed. The Journal Series of the Chinese Medical Association includes 71 journals, which comprise the main core medical journals in Mainland China and additionally the Chinese version of the British Medical Journal (BMJ). We
Among 372 identified Chinese papers, 29 papers were initially excluded as they were duplicate publications. Of the remaining 343 studies, 307 clearly described that their patients were allocated randomly to treatments and therefore were included. Thirty-six studies were excluded as they either were not or could not be confirmed to be RCTs.
Our search for Indian trials revealed 340 reports, identified by the use of the key term “India∗.” Of these, 124 papers were not Indian trials, 20 papers were
We compared Chinese and Indian RCTs with a gold standard set of Western trials from high-quality journals because we considered that there were some important similarities between these two countries. Both are developing countries with very large populations, and they are undergoing rapid socioeconomic transitions. They are also recognized as potentially the two largest markets for pharmaceutical trials in the world, which can contribute important data on medical research and development.
We
To our knowledge, this study presents the first empirical comparison among Chinese, Indian, and Western RCT publications. It provides important findings for the interpretation of trials from those countries.
We compared Indian and Chinese trials with trials published in a gold standard set from top Western journals. A limitation of our study may thus be that we have not compared “like with like,” although the Chinese and Indian journals included may also be considered the top journals for those countries. Our aim was explicitly to provide a point of reference for trialists and trial publishers in China and India and contrast current standards in those countries with Western journals, which aim to
Asian and Latin America regions have recently shown the largest annual increase in the number of registered clinical trials [5]. Trials done exclusively in less developed countries often have low methodological quality [6]. Only 56% of 670 surveyed researchers from developing countries reported that their research had been reviewed by a local institutional review board [7] and only 11% of published clinical trials conducted in China in 2004 report that their study protocol was reviewed by an ethical review committee [8].
This indicates that rigorous training on conducting trials for investigators in China is warranted. For MAs of TCM, the proportion was even as high as 96%, which demonstrates that most RCTs included in those MAs had poor design and implementation [26–29]. Wu et al. [30] found that more than 90% of RCTs published in core Chinese journals lacked an adequate description of randomization in 2009.
In particular, the potential for finding spurious international differences in trial results, which may have been found in other studies [3], was reduced by the fact that a random-effects model was used to allow for this latter type of heterogeneity. Also, the matching up of trials through the use of published systematic reviews avoided the type of pragmatic strategies for comparing trials between countries that have been used by other researchers [73,74]. The task of performing this kind of study in the future would clearly be made easier if researchers preparing the results of trials or systematic reviews for publication were willing to be more explicit about the countries in which trials have taken place.
