You are here

  • Home
  • Archive
  • Volume 5, Issue 2
  • HIV research productivity and structural factors associated with HIV research output in European Union countries: a bibliometric analysis

Article Text

Article menu
XML
HIV/AIDS
Research
HIV research productivity and structural factors associated with HIV research output in European Union countries: a bibliometric analysis
  1. A Uusküla1,
  2. K Toompere1,
  3. K T Laisaar1,
  4. M Rosenthal2,
  5. M L Pürjer1,
  6. A Knellwolf3,
  7. E Läärä4,
  8. D C Des Jarlais5
  1. 1Department of Public Health, University of Tartu, Tartu, Estonia
  2. 2Centre of Continuing Medical Education, University of Tartu, Tartu, Estonia
  3. 3Department of Drug Research and Evaluation, Italian National Institute of Health, Rome, Italy
  4. 4Department of Mathematical Sciences, University of Oulu, Oulu, Finland
  5. 5The Baron Edmond de Rothschild Chemical Dependency Institute, Beth Israel Medical Center, New York, New York, USA
  1. Correspondence to Dr A Uusküla; anneli.uuskula{at}ut.ee

Abstract

Objectives To assess HIV/AIDS research productivity in the 27 countries of the European Union (EU), and the structural level factors associated with levels of HIV/AIDS research productivity.

Methods A bibliometric analysis was conducted with systematic search methods used to locate HIV/AIDS research publications (period of 1 January 2002 to 31 December 2011; search databases: MEDLINE (Ovid, PubMed), EMBASE, ISI-Thomson Web of Science; no language restrictions).

The publication rate (number of HIV/AIDS research publications per million population in 10 years) and the rate of articles published in HIV/AIDS journals and selected journals with moderate to very high (IF ≥3) 5-year impact factors were used as markers for HIV research productivity. A negative binomial regression model was fitted to assess the impact of structural level factors (sociodemographic, health, HIV prevalence and research/development indicators) associated with the variation in HIV research productivity.

Results The total numbers of HIV/AIDS research publications in 2002–2011 by country ranged from 7 to 9128 (median 319). The median publication rate (per million population in 10 years) was 45 (range 5–150) for all publications. Across all countries, 16% of the HIV/AIDS research was published in HIV/AIDS journals and 7% in selected journals with IF ≥3. Indicators describing economic (gross domestic product), demographic (size of the population) and epidemiological (HIV prevalence) conditions as well as overall scientific activity (total research output) in a country were positively associated with HIV research productivity.

Conclusions HIV research productivity varies noticeably across EU countries, and this variation is associated with recognisable structural factors.

  • STATISTICS & RESEARCH METHODS
  • EPIDEMIOLOGY

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

https://doi.org/10.1136/bmjopen-2014-006591

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • To the best of our knowledge, this is the only study to date that used bibliometric methods to evaluate the output of peer-reviewed publications on HIV/AIDS from Europe for the past decade.

    • We present data on three different measures of research productivity. The measures used were highly correlated, indicating measurement of the same underlying phenomenon.

    • Defining research by publication in peer-reviewed journals and presentations (abstracts) at the leading international conferences of the field underestimates the total HIV research productivity.

    • The databases searched consist largely of English-language journals, thereby possibly contributing to selection bias due to language barriers.

    Introduction

    HIV infection is of major public health and research importance in Europe.1 In 2011, 28 038 HIV diagnoses were reported by countries in the European Union (EU) and European Economic Area.2 Across the region, there is a significant variation in the rates of new HIV diagnoses: the four countries with the highest rates of HIV diagnoses per 100 000 population in 2011 were Estonia (27.3), Latvia (13.4), Belgium (10.7) and the UK (10). The lowest rates were reported by the Czech Republic (1.5) and Slovakia (0.9).2

    An analysis by Ramos et al3 indicated that countries with the highest numbers of individuals living with HIV are not leading the scientific output on HIV/AIDS. An analysis based on articles published in three journals focusing on HIV/AIDS from 1986 to 2003 revealed that the USA and Western Europe together, and the five most developed world regions (of nine), accounted for the great majority of the world’s research production on HIV/AIDS (83% and 92%, respectively).4

    Results from bibliometric analyses can be a critically important source of objective information about the quantity and quality of scientific work.5 The primary assumption supporting the use of bibliometrics is that exchange and recognition of research results is desired and is one of the key driving forces in the advancement of science.6 Bibliometric studies of HIV research have described the distribution and variation in scientific output over time7–13 in addition to impact and linkage.5

    We present a study aimed at describing HIV/AIDS research productivity in EU countries, and at assessing the structural level factors associated with HIV/AIDS research output in these countries.

    Methods

    Evaluation of scientific output

    A bibliometric analysis was used to assess scientific output and to compare the relative scientific contributions of the 27 member states of the EU.14 Systematic search methods were used to locate HIV/AIDS research publications.

    Data sources and sample

    Criteria for considering studies for this review

    We included published research results (conference abstract and journal articles; hereafter referred to as publications), articles in the press as well as records that were ‘In Process’ (not yet fully indexed), irrespective of the study design (all publications, primary and secondary, commentaries, editorials, etc, were retained for the analysis). The search was limited to the period of 1 January 2002 to 31 December 2011 and to the 27 countries of the EU at that time.

    Search methods for identification of studies

    We searched the following bibliographic databases: MEDLINE (Ovid, PubMed; includes citations from approximately 5600 scholarly journals published), EMBASE (includes citations from approximately 8000 scholarly journals published), ISI-Thomson Web of Science (enables access to the published literature from conferences, symposia, seminars; selected conferences for the period 2002–2011: International AIDS conferences, Conferences on Retroviruses and Opportunistic Infections; The International Society for Sexually Transmitted Diseases Research conferences). To be comprehensive, we used a combination of controlled vocabulary (MeSH, Emtree, DeCS, including exploded terms) and free-text terms (considering spelling variants, synonyms, acronyms and truncation), and the filter used to search HIV-related publications was based on the search terms defined by Volmink et al15 (see online supplementary annex 1). This search filter was modified and adjusted to the search engine properties of each of the selected electronic databases. The filters for all databases and a full search strategy for PubMed can be found in online supplementary annex 1 (Electronic search strategies).

    Bibliographic information (title, journal, year of publication, authors’ names, the organisation to which the first author was affiliated and the country in which the first author's organisation was located) was derived from citations of publications and used for the analysis presented. The source (origin) country of the publication was defined on the basis of the first author’s affiliation.

    We used the online research management tool Endnote X5 for data processing (Thomson Reuters EndNote X5, http://www.endnote.com). Data abstracted from all databases searched were merged and duplicates deleted from the data set used for analysis. The data abstraction process was entirely conducted by computer. Given the very large number of publications identified through electronic searches, it would not have been feasible to have authors review the abstracts or titles to determine if the publications had been correctly classified as ‘HIV/AIDS’ publications or to verify that the organisational affiliations of the first authors were correct. For further analysis, the data were exported from Endnote into R (R V.2.15.1, http://www.r-project.org/).

    Measures

    We used three measures to assess HIV research productivity. First, we derived the number of HIV research studies (not restricted by study design or research area, ie, basic science, clinical, prevention or implementation research) indexed in selected data sources for a specific country. For comparison purposes, total scientific publication rates (per 1 000 000 population) were calculated.

    In addition, we hypothesised that if a researcher working in the HIV/AIDS area wants his or her work to be noticed by large numbers of researchers in the HIV/AIDS field, there are two main strategies to use. One is to publish in an HIV/AIDS journal, as other AIDS researchers will be paying attention to the contents of AIDS journals on a regular basis. The second strategy is to publish in a relatively high impact general journal as AIDS researchers will also pay more attention to what is published in high impact medical journals. Therefore, we assessed the numbers and rates of research studies published in all HIV/AIDS journals listed in the Web of Knowledge Journal Citation Reports (JCR)16 for the year 2012 (n=16) and in the selected journals with a 5-year impact factor (IF) ≥3 (n=19; table 1). Acknowledging that key research in the HIV/AIDS arena in the period of observation has been published in non-HIV journals, a set of infectious disease, general medicine and science journals with 5-year IF ≥3 was selected by the authors of the current analysis (journals such as NEJM, Lancet, PLoS Medicine; see table 1). For the last two measures (publications in all AIDS/HIV journals and in journals with IF ≥3), conference publications were excluded.

    View this table:
    Table 1

    List of HIV/AIDS journals and selected journals with moderate or higher 5-year impact listed in the Web of Knowledge Journal Citation Reports factor in the year 201216

    Information on journal IFs was obtained from the Web of Knowledge JCR.16 ,17

    Further, we evaluated the structural-level factors related to variation in HIV research productivity in EU countries. Data were obtained from reports published by the World Bank,18 WHO19 and Joint United Nations Program on HIV/AIDS.20 The following indicators were selected and data collected on explanatory factors for HIV research productivity on a structural level:

    1. Demographic indicator: population, total (mid-year);

    2. Economic indicator: gross domestic product (GDP) per capita (value, in US$);

    3. Education indicators:

      1. Public spending on education per capita,

      2. Public spending on education, total (% of GDP);

    4. Research and development (R&D) indicators:

      1. R&D expenditure per capita,

      2. R&D expenditure, per cent of GDP,

      3. Researchers in R&D (per million population),

      4. Overall research productivity (publications total per million population);

    5. Population health indicators:

      1. Health expenditure per capita (value in US$),

      2. Life expectancy at birth, total (years),

      3. Physician density (number per 1000 people);

    6. HIV epidemic indicators:

      1. Estimated number of HIV-infected persons, adults (15+),

      2. HIV prevalence, adult population (15–49 years).

    Three of the indicators were calculated using World Bank data—research productivity per million of population (using the number of scientific and technical journal articles and national population), public spending on education per capita (using public spending on education (% of GDP), GDP per capita) and R&D expenditure per capita (using R&D expenditure (% of GDP), GDP per capita). For the analysis, data were averaged over the years 2002–2011. If it was not possible to find data for all the years, the data were averaged over the years where it was available.

    Statistical analysis

    For each EU member country, the total publication rate (per million population in 10 years) as well as that of articles in selected journals with IF greater than 3 and all HIV/AIDS journals articles were supplied nominal 95% CIs, which were computed based on assuming Poisson distribution for the numbers of publications and applying the mid-P principle.21 Likewise, 95% CIs for the proportion of IF ≥3 articles out of all publications for all countries were computed based on binomial distribution and applying the mid-P principle here, too. Function pois.exact in package epitools and function midPci in package PropCIs, both contained in R environment for statistical computing,22 were used in these computations. To quantify the degree of dependence between the three publication productivity measures, Spearman's rank correlation coefficients were calculated.

    A negative binomial regression model was used to assess the impact of selected structural factors associated with variation in the overall publication rate describing HIV research productivity per million population in 10 years across the EU countries. This model was preferred over conventional Poisson regression because substantial over-dispersion on nominal Poisson variance was present, as indicated by the estimated value 9.8 and SE 3.4 of the dispersion parameter in the final negative binomial model. Owing to the strong correlations (r>0.9) observed between variables describing general economic conditions, spending on education and health indicators, only GDP per capita from those indicators was included in the final regression model. Research productivity as measured by the mean annual number of scientific publications per million population was selected to represent the overall level of research and development activity in the country. To describe the burden of the disease, the model included the estimated prevalence of HIV in adults in the year 2009. Finally, the size of the population was added to the model with the idea that in bigger countries there is more ‘critical mass’ in the scientific community as a whole than in smaller countries, and it is conceivable that this would be positively associated with the research output in any specific field, like HIV/AIDS research.

    Since each of the selected explanatory variables had a wide range and skew distribution, all of them were transformed to the natural logarithmic scale. The negative binomial regression model was fitted using the function glm.nb in the package MASS, also contained in the R environment. When presenting the results, the estimated regression coefficients (β) and their SEs were transformed to rate ratios (RR) and 95% CIs such that an estimated RR (RR=exp(β×log 2)) associated with a particular variable describes the multiplicative effect of doubling the value of this variable on the publication rate assuming ceteris paribus, that is, no change in the other variables.

    Results

    A total of 90 563 articles indexed by MEDLINE (PubMed, Ovid), EMBASE and Web of Science (WOS; selected conferences) from January 2002 to December 2011 for the selected 27 European countries were retrieved for this analysis. Six per cent of the entries where the origin country assignment was not possible were excluded from the analysis. The total numbers of HIV/AIDS publications in selected databases (with the numbers of publications from EU 27 retained for the analysis in parentheses) in the period January 2001 to December 2011 were as follows: PubMed 126 977 (28 231), EMBASE 119 278 (32 803), Ovid 89 160 (29 000), and WOS (ISI-Thomson Web of Science) 1646 (529).

    After deleting duplicate entries (duplicate entries of the same article by different databases), 39 645 publications from 27 EU countries were included in the analysis.

    The total numbers of HIV/AIDS research publications in 2002–2011 by country ranged from 7 to 9128 (median 319). The median publication rate (per million population in 10 years) was 45 (range 5–150) for all publications (see figures 1 and 2 and online supplementary annex 2).

    Figure 1
    Figure 1

    Publication rates (per million population in 10 years, with 95% CIs) of all research publications on HIV/AIDS in member countries of the European Union during 2002–2011.

    Figure 2
    Figure 2

    HIV/AIDS research publication rates (per million population in 10 years, with 95% CIs) in HIV/AIDS journals (JCR 2012), and of articles published in selected moderate or high impact factor journals (IF ≥3) in member countries of the European Union during 2002–2011 (absolute numbers by country are given in parentheses: publications in HIV/AIDS journals on the left, and publications in journals with IF ≥3 on the right).

    Across all countries and the period of observation, the median publication rate for articles in HIV/AIDS journals was 4.5 (0–36), and the proportion of total HIV/AIDS research published in these journals was 16% (from 0% in Latvia, Lithuania and Malta to 38% in Cyprus).

    For articles in journals with moderate or higher IFs (≥3), the median publication rate was 2.3 (0–13.2). In the graphical presentation of the ‘league table’ according to the total publication rate, the 27 European countries can be divided into three broad groups: (1) ‘top 10 countries’: UK to Ireland, in which the total rate (per million in 10 years) ranged from 75 to 150, and the rate of articles in journals with IF ≥3 varied from 4.4 to 13.2; (2) ‘intermediate six’: Portugal to Finland, the total rate varied between 35 and 50; and the rate of IF ≥3 journal articles from 1.8 to 3.7; and (3) ‘remaining 11 countries’, in which the total rate ranged from 5 to 25, and the rate of articles in IF ≥3 journals being at most one (per million population in 10 years). Across all countries and the period of observation, 7% of the publications were published in journals with a moderate or higher IF (≥3).

    The three measures of research productivity used—the total publication rate, publication rate in HIV/AIDS journals and publication rate in selected journals with moderate or higher 5-year IF—were highly correlated (for all pairwise correlations, the Spearman r>0.88).

    Table 2 summarises the impact of selected structural factors associated with variation in HIV research productivity in EU member countries, as estimated by a negative binomial regression model. GDP per capita had the strongest effect on publication rate, followed by HIV prevalence, size of the population and total scientific activity level in the country.

    View this table:
    Table 2

    Impact of structural-level factors associated with HIV research productivity (number of HIV publications per million population in 10 years) in EU countries as estimated by a negative binomial regression model

    Discussion

    Research is considered incomplete and cannot contribute to advancing knowledge unless it is published. To the best of our knowledge, this is the only study to date that used bibliometric methods to evaluate the output of peer-reviewed publications on HIV/AIDS from Europe for the past decade.

    Over the period of 2002–2011, a total of 40 000 publications on HIV/AIDS originated from 27 EU countries (in comparison to 53 500 publications from the USA (data not shown)). According to our results, HIV research productivity differs significantly across the European countries, being substantially higher in the majority of the old EU countries (pre 2004) in Western and Northern Europe than in the new members (since 2004), mostly from Central and Eastern Europe.

    In addition to significant differences in overall HIV/AIDS research productivity, there are observable differences also in the quantity and proportion of published research in journals with a higher IF between the countries of EU. It is noteworthy, however, that the IF (the median number of citations for a given period of the articles published in a journal17) of journals in which the work was published is a useful but highly controversial indicator of performance. While not a direct measure of quality, IF , as a citation measure, can be indicative of the influence journals have within the literature of a discipline (same subject area).23 ,24

    The structural level factors associated with HIV/AIDS research productivity in our analysis—economic well-being (GDP per capita) and disease epidemiology (adult HIV prevalence) in the countries—are intuitively understandable. Similar findings have been reported previously for other regions or areas of research.3 ,25 ,26 Interestingly, while GDP is a commonly used economic wealth indicator, scientific productivity is a much better predictor of economic wealth and human development according to the recent research by Jaffe et al.27

    The observation that HIV/AIDS research productivity is positively associated with the adult HIV prevalence in countries is in agreement with the ‘burden of disease’ approach to research priorities.28 Our analysis suggesting a ‘critical mass’ type effect (the size of total population contributing to the number of research publications per million population) may be indicative of the need to explore whether this holds for other areas of research, and if so, to then conduct research to understand the mechanisms through which it occurs (whether this relies on the critical mass of researchers, or patients, study subjects to be enrolled). There are other structural factors that could explain differences between countries such as wide variation in data protection laws restricting data linkage in public health and epidemiological research, especially for a sensitive diagnosis like HIV, but not for basic science.

    Factors in addition to the structural factors explored can be related to the differences in research output between high-income countries and medium-income/low-income countries in the EU. The results of randomised controlled trials of new treatments from pharmaceutical studies are more likely to be published in higher impact journals and conduced in more developed countries. Further, a subset of HIV research occurring within Europe is in the context of clinical cohorts such as COHERE, Cascade and EuroCoord. Although these are, by design, cross countries they often have a first author from the more developed countries as partner (ie, the UK).

    We would note that defining research by publication in peer-reviewed journals and presentations (abstracts) at the leading international conferences of the field underestimates the total HIV research productivity (by omitting works in non-indexed journals, and non-journal formats such as books, dissertations, reports, etc.). Further, it is important to note that the databases searched consist largely of English-language journals, thereby possibly contributing to selection bias due to language barriers. We tried to mitigate this effect by not setting language filters for the searches.

    We used a purposive list of journals to assess the volume and proportion of HIV research published in journals with IF ≥3. Notably, this might have omitted high impact research published in other journals. In addition, for journals’ IFs there is variation among subject areas; basic science journals tend to have higher IFs than journals in specialised or applied areas. Still, our analysis was limited to the specified field (HIV/AIDS) not restricted by the study design or research area (ie, basic science or clinical research, etc); basic science journals (Nature, Science), general medicine (Lancet, NEJM) and specialised journals (AIDS, JAIDS, BMC Infectious Diseases) were specified for IF categorisation. Of note, one could argue that given the relative availability of antiretroviral treatment in Europe, functioning health infrastructure and systems, the underlying factors for continued transmission are mostly behavioural and social factors and not biomedical. Thus, one would perhaps expect the emergence of more social and behavioural science studies of relevance to the European HIV context.

    By using the author addresses listed in the bylines of research articles, one can only identify countries (and organisations) where the authors were employed when the research was done or where the article was written, or both. This might lead to some misclassification of the ‘country of origin’ of the work. Also, we suspect that this misclassification is more likely to undercount research in low publication countries. In addition, since we assigned each publication to only one country, we de-emphasised studies with researchers and data from multiple countries. A special analysis of multinational research projects would be of interest for further research.

    For many countries, the numbers of publications in journals with a moderate or higher IF were very small; consequently, the associated error margins for both the pertinent publication rate and the proportion of these articles out of all publications are quite wide and should therefore be interpreted with caution. Finally, we studied European countries only, and thus cannot generalise our findings straightforwardly about the association between structural factors and research productivity outside of Europe, although the relationships would appear to be plausible for other countries, too.

    We presented data on three different measures of research productivity. However, the measures used were highly correlated, and indicating on measurement on the same underlying phenomenon, and that choice among the three sets of journals would not affect any conclusions to be drawn from the data.

    While these limitations are important, we feel it is extremely unlikely that they created the patterns that we observed in the data. Rather, we believe that these patterns were sufficiently strong enough to be observed in the data despite the limitations. Also, we would note that the analysis focuses on the research production from the first decade of the 2000s, and additional research to examine possible changes in research productivity would be useful.

    Our findings have several important implications. We believe that evaluating the publication output from HIV research in Europe sets the basis for analysing HIV research and setting strategies to increase research output from all EU countries and its visibility in the scientific community. Highly skewed distribution of productivity among individual scientists29 or universities30 has been explained by a process of accumulative advantage, and the accumulative advantage results in increasing inequality.29 The same model might also be applicable for countries: owing to feedback through recognition and resources and facilities, scientifically productive countries maintain or increase their productivity. Recognising the structural factors (economic well-being, size of the population) of research productivity stresses the significance of strategic decisions in research development. Johnston31 surveyed research productivity studies and found strong evidence from the existing literature that the scale and continuity of funding helped higher level research activity. The weight of the factors suggests recognising the need to further develop research capacity in countries that are currently not highly productive but face public health problems where research might contribute to successfully addressing the problems. This would probably involve both national and international (eg, European Union) resource commitments. The commitments would need to be sustained over time and not limited to a single public health problem. As the amount of scientific research proliferates and science becomes more costly to produce, scientific strategy and funding agencies are increasingly interested in objectively assessing the research output and factors associated with it.32 Despite the limitations, bibliometric analyses such as ours can be used to describe patterns of performance of scientific research across countries. Second, bibliometric analysis as a tool has the potential to assist decision-makers in understanding science and investing in science. This method can be used as an evaluation instrument in research together with other methods, such as expert opinion and panels.

    References

    1. ↵
      European Commission, research and innovation, health, HIV/AIDS. http://ec.europa.eu/research/health/infectious-diseases/poverty-diseases/aids_en.html
    2. ↵
      European Centre for Disease Prevention and Control/WHO Regional Office for Europe. HIV/AIDS surveillance in Europe 2011. Stockholm: European Centre for Disease Prevention and Control, 2012.
    3. ↵
      1. Ramos JM,
      2. Gutiérrez F,
      3. Padilla S, et al
      . Geography of medical publications. An overview of HIV/AIDS research in 2003. AIDS 2005;19:219–20. doi:10.1097/00002030-200501280-00020
      OpenUrlPubMedWeb of Science
    4. ↵
      1. Falagas ME,
      2. Bliziotis IA,
      3. Kondilis B, et al
      . Eighteen years of research on AIDS: contribution of and collaborations between different world regions. AIDS Res Hum Retroviruses 2006;22:1199–205. doi:10.1089/aid.2006.22.1199
      OpenUrlCrossRefPubMedWeb of Science
    5. ↵
      1. Rosas SR,
      2. Kagan JM,
      3. Schouten JT, et al
      . Evaluating research and impact: a bibliometric analysis of research by the NIH/NIAID HIV/AIDS clinical trials networks. PLoS ONE 2011;6:e17428. doi:10.1371/journal.pone.0017428
      OpenUrlCrossRefPubMed
    6. ↵
      1. Koskinen J,
      2. Isohanni M,
      3. Paajala H, et al
      . How to use bibliometric methods in evaluation of scientific research? An example from Finnish schizophrenia research. Nord J Psychiatry 2008;62:136–43. doi:10.1080/08039480801961667
      OpenUrlCrossRefPubMedWeb of Science
    7. ↵
      1. Sengupta IN,
      2. Kumari L
      . Bibliometric analysis of AIDS literature. Scientometrics 1991;20:297–315. doi:10.1007/BF02018160
      OpenUrlCrossRefWeb of Science
    8. ↵
      1. Small H
      . A sci-map study: building a map of AIDS research. Scientometrics 1994;30:229–41. doi:10.1007/BF02017225
      OpenUrlCrossRefWeb of Science
    9. ↵
      1. Onyancha OB,
      2. Ocholla DN
      . A comparative study of the literature on HIV/AIDS in Kenya and Uganda: a bibliometric. Libr Inf Sci Res 2004;24:443–7.
      OpenUrl
    10. ↵
      1. Macias-Chapula CA,
      2. Mijangos-Nolasco A
      . Bibliometric analysis of AIDS literature in Central Africa. Scientometrics 2002;54: 309–17. doi:10.1023/A:1016074230843
      OpenUrlWeb of Science
    11. ↵
      1. Patra SK,
      2. Chand P
      . HIV/AIDS research in India: a bibliometric study. Libr Inf Sci Res 2007;29:124–34. doi:10.1016/j.lisr.2006.08.010
      OpenUrl
    12. ↵
      1. Onyancha OB
      . Growth, productivity, and scientific impact of sources of HIV/AIDS research information, with a focus on eastern and southern Africa. Afr J AIDS Res 2008;7:55–70. doi:10.2989/AJAR.2008.7.1.7.435
      OpenUrl
    13. ↵
      1. Macias-Chapula CA,
      2. Rodea-Castro IP,
      3. Narvaez-Berthelemot N
      . Bibliometric analysis of AIDS literature in Latin America and the Caribbean. Scientometrics 1998;4:41–9. doi:10.1007/BF02457965
      OpenUrl
    14. ↵
      European Union. Member countries of the European Union. http://europa.eu/about-eu/countries/
    15. ↵
      1. Volmink J,
      2. Siegfried NL,
      3. van der Merwe L, et al
      . Antiretrovirals for reducing the risk of mother-to-child transmission of HIV infection. Cochrane Database Syst Rev 2007;(1):CD003510. Review. Update in: Cochrane Database Syst Rev 2011;(7):CD003510.
    16. ↵
    17. ↵
    18. ↵
      The World Bank: indicators. http://data.worldbank.org/indicator/
    19. ↵
      The World Health Organisation. Global Health Observatory Data Repository. http://apps.who.int/gho/data/node.main
    20. ↵
      Joint United Nations Programme on HIV/AIDS (UNAIDS). Global report: UNAIDS report on the global AIDS epidemic 2010. UNAIDS/10.11E | JC1958E. 2010. http://www.unaids.org/documents/20101123_GlobalReport_Annexes1_em.pdf
    21. ↵
      1. Berry G,
      2. Armitage P
      . Mid-P confidence interval: a brief review. Statistician 1995;44:417–23. doi:10.2307/2348891
      OpenUrlCrossRefWeb of Science
    22. ↵
      R Core Team. R: a language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing, 2013. http://www.R-project.org/
    23. ↵
      1. Amin M,
      2. Mabe M
      . Impact factors: use and abuse. Perspectives in Publishing 2000:1. http://cdn.elsevier.com/assets/pdf_file/0014/111425/Perspectives1.pdf
    24. ↵
      1. Van Raan AFJ
      . Measurement of central aspect of scientific research: performance, interdisciplinarity, structure. Measurement 2005;3:1–19.
      OpenUrl
    25. ↵
      1. Uthman OA
      . Pattern and determinants of HIV research productivity in sub-Saharan Africa: bibliometric analysis of 1981 to 2009 PubMed papers. BMC Infect Dis 2010;10:47. doi:10.1186/1471-2334-10-47
      OpenUrlPubMed
    26. ↵
      1. Rahman M,
      2. Fukui T
      . Biomedical research productivity: factors across the countries. Int J Technol Assess Health Care 2003;19:249–52. doi:10.1017/S0266462303000229
      OpenUrlCrossRefPubMedWeb of Science
    27. ↵
      1. Jaffe K,
      2. Caicedo M,
      3. Manzanares M, et al
      . Productivity in physical and chemical science predicts the future economic growth of developing countries better than other popular indices. PLoS ONE 2013;8:e66239. doi:10.1371/journal.pone.0066239
      OpenUrlCrossRefPubMed
    28. ↵
      1. Labonte R,
      2. Spiegel J
      . Setting global health research priorities. BMJ 2003;326:722–3. doi:10.1136/bmj.326.7392.722
      OpenUrlFREE Full Text
    29. ↵
      1. Allison PD,
      2. Stewart JA
      . Productivity differences among sceintists: evidence for accumulative advantage. Am Sociol Rev 1974;39:596–606. doi:10.2307/2094424
      OpenUrlCrossRefWeb of Science
    30. ↵
      1. Beerekens M
      . Competition and concentration in the academic research industry: an empirical analysis of the sector dynamics in Australia 1990–2008. Sci Public Policy 2013;40:157–70. doi:10.1093/scipol/scs076
      OpenUrlAbstract/FREE Full Text
    31. ↵
      1. Johnston R
      . Effects of resource concentration on research performance. Higher Educ 1994;28:25–37. doi:10.1007/BF01383570
      OpenUrlWeb of Science
    32. ↵
      1. Phelan T
      . Evaluation of Scientific Productivity. The Scientist 2 October 2000.

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

      Files in this Data Supplement:

    Footnotes

    • Contributors AU created the original idea for the article and outlined the design. EL, DCDJ and AK participated in the development of the methodology used. KTL, MLP and MR were responsible for data collection; KT and EL were responsible for data analysis. AU wrote the first draft of the manuscript to which all other authors contributed.

    • Funding This work was partially supported by the European Commission under FP7 Grant Agreement: EC-GA n°249697, HIVERA project (Harmonizing, Integrating and Vitalizing European Research on Aids/HIV), by the grant SF0180060s09 from the Estonian Ministry of Education and Research, and by the US National Institutes of Health grant P30 DA 011041.

    • Competing interests None.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement No additional data are available.