Article Text

Evidence for non-communicable diseases: analysis of Cochrane reviews and randomised trials by World Bank classification
  1. C Heneghan1,
  2. C Blacklock1,
  3. R Perera1,
  4. R Davis1,
  5. A Banerjee2,
  6. P Gill1,
  7. S Liew1,3,
  8. L Chamas4,
  9. J Hernandez1,
  10. K Mahtani1,
  11. G Hayward1,
  12. S Harrison1,
  13. D Lasserson1,
  14. S Mickan1,
  15. C Sellers1,
  16. D Carnes5,
  17. K Homer5,
  18. L Steed5,
  19. J Ross6,
  20. N Denny6,
  21. C Goyder1,
  22. M Thompson1,
  23. A Ward1
  1. 1Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
  2. 2Department of Cardiology, University of Birmingham, Birmingham, UK
  3. 3Department of Primary Care Medicine and Julius Center, University of Malaya, Malaya, Malaysia
  4. 4Oxford Centre for Diabetes, Endocrinology & Metabolism, University of Oxford, Oxford, UK
  5. 5Centre for Primary Care and Public Health, Barts and the London School of Medicine and Dentistry, London, UK
  6. 6Medical School, Oxford University, Oxford, UK
  1. Correspondence to Dr Carl Heneghan; Carl.heneghan{at}phc.ox.ac.uk

Abstract

Introduction Prevalence of non-communicable diseases (NCDs) is increasing globally, with the greatest projected increases in low-income and middle-income countries. We sought to quantify the proportion of Cochrane evidence relating to NCDs derived from such countries.

Methods We searched the Cochrane database of systematic reviews for reviews relating to NCDs highlighted in the WHO NCD action plan (cardiovascular, cancers, diabetes and chronic respiratory diseases). We excluded reviews at the protocol stage and those that were repeated or had been withdrawn. For each review, two independent researchers extracted data relating to the country of the corresponding author and the number of trials and participants from countries, using the World Bank classification of gross national income per capita.

Results 797 reviews were analysed, with a reported total number of 12 340 trials and 10 937 306 participants. Of the corresponding authors 90% were from high-income countries (41% from the UK). Of the 746 reviews in which at least one trial had met the inclusion criteria, only 55% provided a summary of the country of included trials. Analysis of the 633 reviews in which country of trials could be established revealed that almost 90% of trials and over 80% of participants were from high-income countries. 438 (5%) trials including 1 145 013 (11.7%) participants were undertaken in low-middle income countries. We found that only 13 (0.15%) trials with 982 (0.01%) participants were undertaken in low-income countries. Other than the five Cochrane NCD corresponding authors from South Africa, only one other corresponding author was from Africa (Gambia).

Discussion The overwhelming body of evidence for NCDs pertains to high-income countries, with only a small number of review authors based in low-income settings. As a consequence, there is an urgent need for research infrastructure and funding for the undertaking of high-quality trials in this area.

  • Epidemiology
  • Oncology
  • Neurology
  • Diabetes & Endocrinology
  • Cardiology

This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/3.0/ and http://creativecommons.org/licenses/by-nc/3.0/legalcode

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Article summary

Article focus

  • Non-communicable diseases (NCDs) such as cardiovascular disease, diabetes, chronic respiratory disease and cancer are increasing in prevalence.

  • NCDs are set to increase disproportionately in low-income and middle-income countries over the next 20 years.

  • We aimed to estimate the distribution of evidence relating to NCDs in all Cochrane systematic reviews by country and income distribution.

Key messages

  • The overwhelming body of evidence for NCDs relates to high-income countries.

  • Out of 8850 trials, we found that only 13 (0.15%) with 982 (0.01%) participants were undertaken in low-income countries.

  • Only a small number of review authors were based in low-income settings.

Strength and limitations of this study

  • In 15% of the reviews, we were unable to identify the country of origin for the trial.

  • Systematic reviews and trials can only serve as a proxy for high-quality evidence and information.

  • We did not review case-control, cohort studies and reviews published in the grey literature as they do not represent traditional streams of robust evidence.

Background

The global prevalence of non-communicable diseases (NCDs) such as cardiovascular disease, diabetes, chronic respiratory disease and cancer is increasing.1 ,2 Alarmingly, NCDs are set to increase disproportionately in low-income and middle-income countries over the next 20 years, placing additional burden on already overstretched health systems.1–3 WHO estimated that 36 million deaths in 2008 were due to NCDs, of which 9 million were in people younger than 60 years, and 80% of the 36 million deaths occurred in developing countries.4 In 2010, WHO published its ‘Package for essential NCD interventions for primary healthcare in low-resource settings,’ to provide a prioritised set of cost-effective interventions to improve equity and efficiency in NCD care.5 Calls to action from the international community have reported that NCDs present ‘a global crisis and require a global response’.6 A high-level United Nations meeting took place in September 2011 from which a detailed declaration was made devoted to the prevention and control of NCDs, with particular focus on the challenges faced by developing countries.4 In addition, attention has been drawn to significant issues arising around inequity, which will impact markedly on effective chronic disease care.2 ,7 ,8 For example, effective care requires access to vital medicines for NCDs, such as inhaled steroids in asthma,9 without which national management strategies become strained, if not impossible.

Systematic reviews provide high-quality evidence from which clinical guidelines and public health policy can be developed. However, recognition and evaluation of the potential differences between a trial population and a healthcare population and setting are important when applying such evidence to clinical practice and public health policy. Indeed, an intervention, whether non-drug or drug, may ideally have to be trialled in the population that it is intended for.10 ,11 However, in systematic reviews of interventions for NCDs, whether drug or non-drug, there is often no randomised trial evidence from low-income and middle-income countries.12–14 A number of trials have established effective methods for controlling the key risk factors for globally important NCDs, including reducing blood pressure, cholesterol, smoking. Yet, this evidence may be difficult to apply in low-income and middle-income countries due to the lack of generalisability of findings to LMIC settings.15

Therefore, to better understand the representation of populations by national income in global trial evidence for interventions for NCDs, and to scope potential gaps, we sought to answer the question: “in which countries are the randomized trials relating to NCDs performed, and how frequently is this reported?” We aimed to estimate the frequency of trial populations by country, included in all Cochrane systematic reviews relating to NCDs, and the distribution of evidence in these reviews by gross national income (GNI) of trial population.

Methods

We used the 2008–2013 WHO Action Plan for the Global Strategy for the Prevention and Control of Non-communicable Diseases to define NCDs for this study.16 This definition included cardiovascular diseases, cancers, diabetes and chronic respiratory diseases. We also chose to include tobacco use, to represent one common and accepted NCD risk factor.

We classified GNI of a country according to the World Bank Classification (see online supplementary table S1-correct as of May 2011 http://data.worldbank.org/about/country-classifications/country-and-lending-groups): low-income (GNI per capita $995 or less), lower-middle income (GNI per capita $996–$3945), upper-middle income (GNI per capita $3946–$12 195), high income (GNI per capita $12 196 or above). We included Taiwan as a high-income country in the analysis. We used the May 2011 classification, which is now revised (both classifications shown in online supplementary table S1).

We accessed the online Cochrane Database of Systematic Reviews (http://www.thecochranelibrary.com) in May 2011 to identify reviews that related to NCDs, for inclusion in the subsequent data extraction. Identification of reviews for inclusion was performed by a single researcher (CB), with all inclusion and exclusion decisions checked by a second researcher (CH). Reviews were included if the review related to one of the four NCDs: cardiovascular disease, cancer, diabetes, chronic respiratory disease or tobacco use. We first identified relevant Cochrane Groups from the overall database. The included and excluded groups are shown in online supplementary table S2. From the included Cochrane disease groups, we included all review categories within the group relating to the predefined NCDs, and excluded all other review categories. For each included review category, we excluded all reviews at the protocol stage only, and any obvious duplicate reviews.

Each included review and each Cochrane disease group was then coded by a single researcher (CB) to identify: (1) the NCD area (ie, cardiovascular disease, cancer, diabetes and chronic respiratory disease) and (2) the domain (ie, prevention, screening, management, diagnosis and other).

We divided the included Cochrane reviews into pairs of approximately 50 reviews (total 17 pairs), with electronic links to each review (Microsoft Excel 2007 version). Two researchers independently extracted data onto the standardised Excel spreadsheets. Repeated or withdrawn reviews were coded as such by the researchers and not included in the final analysis.

The following data were extracted from each full text Cochrane review: year of publication, year content assessed up to date, corresponding author details, whether a summary of countries of included trials was provided, year of earliest and latest included trials, total number of included trials and participants (but not age distribution), number of trials and participants from low-income, low-middle income, upper-middle income, mixed income countries and high-income countries. World Bank classification of country by GNI was included on all extraction sheets and was used to classify countries. Where the review did not provide a summary of countries of included trials, the researcher attempted to locate and review individual trial reports of included trials to establish the country in which each trial was conducted. Where only the countries of some trials were reported in the review, a pragmatic decision was made as to whether the review had provided a summary, based on the completeness with which the review reported the trial countries. Where data were unobtainable, they were recorded as missing. Each researcher was given a standardised instruction pack and spreadsheet for data collection. Queries in the data identified by individual researchers during the data extraction process were resolved by three coordinating researchers (CB, RD and CH), and responses were circulated to all researchers undertaking data extraction.

After the data had been extracted in duplicate for each review, a third independent researcher (CB or CH) compared the extracted data and constructed a single data sheet for analysis. Where there was disagreement between the first and second researchers, the third researcher checked the calculations and, if necessary, checked the data from the appropriate Cochrane review or the individual trials. If this was not possible, the data were considered unattainable and coded as missing. The third researchers (CB and CH) finally made a joint check of all completed data sheets.

The final data sheets were exported into SPSS software for analysis (IBM SPSS V.19.0). The dataset was first checked for any erroneous data entry by the third researcher (CB and CH) and amended where necessary. Descriptive analyses were used to present frequencies of the number of trials, and the trial populations by location, and StatPlanet (http://www.statsilk.com/) software was used to provide a visual world map of frequencies of the location of Cochrane review authors. We used κ to report agreement between the initial data extraction by the two independent researchers. In our analysis, the two independent researchers were recorded as being in agreement if the sum of their data entries for trials (or participants) from each category of GNI for a specific Cochrane review was exactly the same (ie, if the recorded number of trials or participants varied at all between the two researchers, this was counted as a disagreement).

Results

A total of 797 systematic reviews were included in the study, after exclusion of repeated and withdrawn reviews. Of these, we found 51 reviews in which no trials had been identified by the review authors as satisfying the inclusion criteria (figure 1). These were excluded from the final analysis of trial and participant data. Overall, the remaining 746 Cochrane reviews included a total of 12 340 trials involving 10 937 306 participants.

Figure 1

Flowchart of included reviews that reported country of trials, and in which country of trials could be established.

The countries listed by corresponding authors from the 797 identified reviews (including those 51 with no identified trials) are shown in figure 2 and online supplementary table S2. Over 90% of the corresponding authors of Cochrane NCD reviews were based in high-income countries by World Bank classification (720/797, 90.3%). By far, the most frequent country was the UK (n=327, 41%), followed by Australia (77, 9.7%), Canada (61, 7.7%) and the USA (54, 6.8%). China and India, low-middle income countries in May 2011, represented 2.6% (n=37) and 0.1% (n=1), respectively, of the corresponding authors (note: China has since been reclassified as an upper-middle income country by the World Bank). South Africa, Brazil and Russia, three upper-middle income countries, represented 0.6% (n=5), 1.5% (n=12) and 0.1% (n=1), respectively, of the corresponding authors. Other than the five Cochrane NCD corresponding authors from South Africa,17–21 only one other corresponding author was from Africa (Gambia).22

Figure 2

World Map of location of Cochrane non-communicable disease author N=797 (219 countries).

The mean number of included trials in the 746 Cochrane reviews which had identified at least one trial for inclusion varied considerably across review groups (table 1). Likewise, the mean number of participants also varied, from a mean of 575 per review (Childhood Cancer group) to 42 256 per review (Gynaecological Cancer group). The average year of review publication ranged from 2007 (Breast Cancer group) to 2011 (Childhood Cancer group).

Table 1

Cochrane NCD reviews included in analysis—table of characteristics

Of the 746 reviews which had included at least one trial, almost half (44.9%) did not provide a summary of trial countries within the review (figure 3). The agreement between the two initial independent researchers as to whether reviews had provided a summary of countries of trials was fair (κ 0.65). Reporting of trial countries in the review varied between the Cochrane groups, with one group reporting trial countries in 100% of reviews (Oral Health; however, only 6 reviews were from this group), and others reporting trial countries in less than a quarter of reviews (Haematological Malignancies 4/18, 22.2%; Pain, Palliative and Supportive Care 10/44, 22.7%). When present, reporting of trial countries was typically identified from the Characteristics of Included Studies table, whereas in others it was summarised in the text of the review.

Figure 3

Trials and number of participants from 626 Cochrane non-communicable disease reviews, by Gross National Income (GNI).

In two-thirds of the 335 reviews that did not summarise countries, we were able to subsequently establish the countries of included trials in two-thirds (221/335, 66%; figure 1). These reviews, in addition to the 411 reviews that had provided a summary of trials of countries in the review report, gave a total of 633 reviews for which we were able to determine the number of trials and participants by World Bank GNI category of trial country. Despite the provision of a summary, we were unable to confidently establish the countries of all included trials in five reviews—this was mainly due to ambiguous reporting of some trial countries, for example, ‘international’, that could not be resolved by our subsequent methods (these five reviews represented 168 (1.4%) trials).

Almost 90% of trials in the analysis (n=7869/8850; 88.9%) and over 80% of participants (8 053 378/9 806 291; 82.1%) were from high-income countries (figure 3). Low-middle income countries were the second most frequently represented category, comprising 4.95% of trials (n=438) and 11.68% of participants (n=1 145 013). Least represented were low-income countries, which contributed only 0.15% of trials (n=13) and 0.01% of participants (n=982) overall. All individual Cochrane disease groups, except Oral Health and Breast Cancer, included the majority of participants from high-income countries. When analysed by the Cochrane disease group, the Heart and Stroke groups had the largest proportion of included trials from lower-middle income countries (13.9% (n=92/662) and 11.8% (n=109/928), respectively). Also notable was the comparatively low proportion of trials from lower middle income countries in the Oral Health and Breast Cancer groups; however, despite this, these trials contributed a larger proportion to the overall number of participants (84.3% (n=194 439/230 584) and 45.2% (n=672 626/1 489 628) of participants, respectively).

Subgroup analysis of the distribution of reviews by type of review mirrored the overall findings above. Once again, high-income countries were the origin of a vast majority of prevention trials (873/942 (92.7%)) and participants (3 935 969/4 204 345 (93.6%)), screening trials (140/143 (97.9%)) and participants (1 692 826/1 885 858 (89.8%), management trials (6666/7570 (88.1%)) and participants (1 819 032/2 296 298 (79.2%)), and ‘other’ trials (133/135 (98.5%)) and participants (30 182/50 939 (59.3%)), apart from reviews relating to diagnosis. Of the four diagnostic Cochrane reviews, two included trials were from low-middle income countries and contributed 671 014 participants (49%), a single trial was from an upper-middle income country and contributed 122 468 participants (8.9%), and the remaining 57 included trials were from high-income countries and contributed 575 369 participants (42.1%).

Discussion

Overall, we found a blatant lack of evidence in low-income and middle-income countries. From 746 systematic reviews of 12 340 trials (10 937, 306 participants), only 13 trials (982 participants) were undertaken in low-income countries. The overwhelming body of evidence for NCDs pertains to high-income countries, with only a small number of review authors based in low-income settings.

In addition, low-income countries are poorly represented among reviews. For example, other than five review corresponding authors based in South Africa, only one further author was based in Africa (the Gambia). The real lack of any Cochrane review authors from many LMICs is surprising. It further reinforces the major developmental gap that is required in engagement and completing research in these settings.

A previous review, published in the New England Journal of Medicine between 1997 and 2004, reported less than 3% of research-addressed health issues in the developing world, and the majority of this addressed communicable diseases including HIV.23 A further review reported that >90% of published research by scientists comes from just 20 countries.24 The gap in scientific publications between low-income countries and the rest of the world has widened.24 Our work suggests that this is still the case. This issue has previously been raised by Richard Horton, editor of Lancet, “widespread systematic bias in the medical literature against disease that dominates the least developed regions of the world.”25

The question remains as to whether this lack of contextual evidence for LMICs matters. While there has been a dramatic increase in NCDs, particularly in highly populated transition countries, we have shown that there is a widespread lack of research into interventions directed at NCD prevention and treatments.26 The current evidence-base does not relate to the increasing burden of disease. As a result, some interventions (eg, cancer management) cannot be applied directly to LMICs, often because of the cost of the intervention. Cancer is not mainly confined to the high-resource countries and in the absence of good trial data; often, the most appropriate course of action is to modify interventions—according to cost and evidence. This was proposed as early as 1992, by WHO, which recognised that access to cancer services and drugs was limited and likely to worsen.26

In addition, many drugs differ in their effects due to ethnic and cultural diversity: β-blockers and ACE inhibitors in hypertension are commonly recognised examples, yet there are certainly many others. Multinational studies are often designed and powered to detect a single global treatment effect and not to detect subgroup differences that may occur. Yet systematic differences between treatment effects do occur, often due to variation in genetics, compliance, follow-up and concomitant medications.27

Limitations

A number of limitations in the present study are worth noting. First, in 114/747 of the reviews, we were unable to identify the country of origin for the trial. If the majority of trials and participants in these reviews were from LMICs, then our results may look different. In addition, there was some ambiguity in identifying whether or not the Cochrane reviews provided details about the country of origin of trials—agreement was judged as only fair. The task required many data extractors, as it was time-consuming and was not helped by how details or, in many cases, lack of details were reported.

Second, systematic reviews and trials can only serve as a proxy for high-quality evidence and information, but we did not evaluate the relevance and applicability of the completed trials in low-income countries. We also did not look at the year that each trial was published; therefore, we were unable to evaluate the trend in studies being published to determine if the quality of the literature is improving over time. We only included Cochrane reviews, and therefore the study does not reflect the entire literature base, but it is a good approximation given the recognised quality of such reviews.

Third, the data we analysed were from previously published Cochrane systematic reviews. There also exist sources of case–control, cohort studies and reviews published in the grey literature, the WHOLIS developing country database and ministries of health local papers specific to LMICs. These were not included as they do not represent traditional streams of robust evidence, although they have previously served in developing WHO strategies.

Implication for practice and research

The study findings raise significant concerns regarding the applicability of the current evidence base for NCDs to LMICs. Certain topics now have reasonable evidence to support them but most have a paucity of contextual trial data. There exists a major issue over the clarity of published papers and systematic reviews. This is an easy issue to fix. New Cochrane systematic review guidelines should require that a breakdown of population and trial countries be disclosed.

There is an obvious, urgent need for more research in low-income and low-middle income countries. The practicalities of funding and organising clinical trials in such varied circumstances are likely to be very difficult, but this should not act as a barrier. Many LMIC healthcare systems are severely underdeveloped with limited or no data collection and similarly limited experience of data collection/running clinical trials. There is therefore a need to develop infrastructure and capacity at a local level.

The high morbidity and mortality from infectious diseases have historically crippled LMICs. The epidemiological transition from infectious to chronic diseases is “more compressed in a shorter timeframe than high incomes,” which explains the lagging gap in evidence.28 Yet, of the research that is published in LMICs journals, around 40% focused on NCDs, suggesting that even given the focus on infectious disease research, capability exists to conduct such research.29 This research is often not translated into systematic reviews or bigger trials because of the economic constraints, language barriers and absence of some LMICs journals from MEDLINE.28

Part of the WHO Action Plan for the Global Strategy for the Prevention and Control of Non-Communicable Diseases is to respond to the epidemic by integrating disease prevention and control into local policies, and promoting research.16 But the medical information gap between rich and poor countries appears to be larger than the gap in funding for research.23 Commercial efforts are too often focused on where the money is—in providing end products to health professionals.30 As a consequence, many interventions will therefore require funding by non-commercial entities.

Conclusions

Scant attention has been paid to NCD research in LMICs. Even the research carried out is often within urban and relatively high-income settings within a given country.15 As a consequence, there is an urgent need for research infrastructure and the undertaking of high-quality trials.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Files in this Data Supplement:

Footnotes

  • CH and CB are joint first authors.

  • Contributors All authors contributed to the data extraction. CB and CH designed the protocol and the methods, and CB, CH and RP undertook the data analysis. All authors contributed to the drafting of the article and approved the final manuscript. CH is the guarantor.

  • Funding This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests CH is supported by the National Institute for Health Research School for Primary Care Research.

  • Provenance and peer review Not commissioned; internally peer reviewed.

  • Data sharing statement No additional data are available.