Article Text
PDF
Abstract
Objective To identify the top-cited articles in gastroenterology and hepatology, and analyse their characteristics.
Methods Two searches were conducted in the Science Citation Index Expanded database; a search of 69 journals under the category ‘Gastroenterology and Hepatology’ (list A) and a keyword search of all journals (list B). The search results were analysed and the inter-rater coefficient of agreement between evaluators was measured using Cohen κ.
Results The number of citations varied from 1049 to 2959 in list A and from 1929 to 5500 in list B. In both lists, the majority of articles were research papers. No significant correlations were found between the number of citations and the number of years since publication (R2=0.00992, p=0.473 and R2=0.00202, p=0.757, respectively). However, the mean number of citations of papers published before the year 2000 was lower than those published after 2000 (36.70±19.31 vs 106.03±39.22). No correlation was found between number of authors and the number of citations (R2=0.04352, p=0.130), but strong correlations were found between the number of institutes involved or number of countries and the number of citations (R2=0.275, p<0.001 and R2=0.16181, p=0.003, respectively). Females were under-represented in authorship (45 vs 254, p=0.004). Only 21 papers (of 54) in list A were supported by grants. No correlation was found between number of grants received and the number of citations (R2=0.02573, p=0.247). The inter-rater agreement between evaluators had a Cohen κ coefficient 0.76–0.84.
Conclusions Top-cited articles were not only published in highly ranked journals specialising in Gastroenterology and Hepatology but also in 14 journals not specialised in this field. The number of citations correlated with the number of institutes and the number of countries involved but not with the number of grants received or the number of authors. Females were under-represented in the authorship.
- MEDICAL EDUCATION & TRAINING
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
Two searches were conducted in the Science Citation Index Expanded database.
The search was based on journals with high impact factor and only those in the English language.
Analysis explored a range of parameters in the assessment.
Introduction
While the number of citations alone cannot reveal why a paper is considered important enough to attract citations by other researchers1 nor reflect fully the quality of a paper,2 the citations received by scientific publications have been used as a proxy measurement to assess the work of researchers and impact of research,3 and to rank researchers on the basis of differences in citation indices.3 ,4 Recently, Nicholson and Ioannidis5 explored whether there is a link between highly cited research and US National Institute of Health (NIH) funding. Their findings showed that too many US authors of the most innovative and influential papers in the life sciences do not receive NIH funding.5 While these findings raise a number of possibilities, there is ongoing debate on the importance of citations received by scientific publications.6 For example, using citation metrics to appraise scientists and their work has many pitfalls,7 yet the numbers of published research papers and their citations have been used as a measure to assess the quality of research on national scales and to set it in an international context.8 This may explain why top-cited publications are usually seen by researchers and universities as influential papers, and can be used in measuring the impact of the work of other researchers.9
The reputation of scientists and the influence of their work in a particular discipline can therefore be proportionally related to the number of citations received by their publications.3 This is particularly important when there is a pattern of consistency and progressive input into their discipline over time, as demonstrated from their publications’ record and the citation history of their publications. The more influential their papers, the more they are making an impact, not only in their institutes or at a national level, but at a global level as well.5
The identification of top-cited articles in gastroenterology and hepatology is useful for a number of reasons. First, the search identifies the articles that have contributed to the different topics related to the discipline. Furthermore, the top-cited articles enable readers to know authors who and institutions that have contributed to such work. Garfield and Welljams-Dorof10 showed that a simple, quantitative and objective algorithm based on citation data of high-impact research authors, can effectively corroborate and even help in predicting Nobel Prize award winners. Finally, the lists present useful information to authors and researchers regarding top-cited articles that can be used in teaching and learning of undergraduate and postgraduate students.11
Top-cited papers have been recently studied in several fields, including cardiovascular medicine,11 cardiac surgery,12 arthroscopic orthopaedic surgery,13 respiratory system,14 dermatology15 and medical education.16 An abstract on the top-cited articles in gastroenterology and hepatology was presented at Asia-Pacific Digestive Week in 2009.17 The objectives of this study are to identify the 50 top-cited articles in gastroenterology and hepatology, and to analyse their characteristics.
Methods
Study design
The Science Citation Index Expanded (SCI-Expanded) database of the Thomson Reuters Web of Science was used for citation tracking and the identification of most-cited articles. Although Scopus and Google scholar also provide citation tracking, it was decided to limit the search to the SCI-Expanded database. This is because the SCI-Expanded database is regularly updated and its 2014 Journal Citation Reports (JCRs) included 76 journals in the field of Gastroenterology and Hepatology. Google Scholar was not used because it is difficult to search and its citations include textbooks, monographs, conference proceedings, as well as non-peer-reviewed work.18 The Scopus database was not searched because it is not extensive in its coverage and its records only go back to 1966.19
To maximise the outcomes of this study, two search strategies were used. The first aimed at searching journals listed in the JCR 2014 under the category ‘Gastroenterology and Hepatology’. The second search aimed at identifying most frequently cited articles in all the database including journals not dedicated to gastroenterology and hepatology, such as general medicine, internal medicine and general surgery journals, as well as biology and related disciplines.
Searching the gastroenterology and hepatology journals
On 27 May 2015, the two authors (a professor of medical education as well as gastroenterology consultant, and a senior surgical registrar) along with a research assistant with a background in medicine, searched the SCI-Expanded database to retrieve top-cited articles. This search was conducted via the JCR 2014 under the category ‘Gastroenterology and Hepatology’. The category comprised 76 journals at the time of conducting the search. Seven journals were not searched because they were in languages other than English. Journals not in the English language were excluded because neither of the authors of this work are competent in the Spanish, Italian or German languages. Also, there are articles on the topic covering top-cited articles in languages other than English.20 Interestingly, after identifying the list of top-cited articles, and again checking these seven non-English journals, none had a paper with a citation higher than the paper ranked number 50 in the list.
A list identifying the 50 top-cited articles was reviewed again and checked regarding authorship, title of the article, number of citations and the institution of the first author (list A, see appendix 1). Articles that shared the same number of citations were given the same rank number.
Searching the Web of Science using keywords
The second search was conducted on 27 May 2015. The aim of the second search was to identify top-cited articles published in journals not dedicated to the field. The SCI-Expanded database was searched using the following keywords: ‘Bilirubin’, ‘Biliary disease’, ‘Esophageal disorder’, ‘Esophageal reflux disease’, ‘Esophageal cancer’, ‘Peptic ulcer disease’, ‘Helicobacter pylori’, ‘Gastric ulcer’, ‘Gastritis’, ‘Gastric cancer’, ‘Pancreatitis’, ‘Pancreatic cancer’, ‘Jaundice’, ‘Malabsorption’, ‘Celiac disease’, ‘Irritable bowel syndrome’, ‘Inflammatory bowel disease’, ‘Ulcerative colitis’, ‘Crohn's disease’, ‘Colitis’, ‘Diarrhea’, ‘Constipation’, ‘Esophageal varices’, ‘Chronic hepatitis’, ‘Viral hepatitis’, ‘Cirrhosis’, ‘Ascites’, ‘Chronic liver disease’, ‘Liver cell failure’, ‘End-stage liver disease’, ‘Gastrointestinal bleeding’, ‘Colon cancer’, ‘Diverticular disease’, ‘Liver function’, ‘Gallbladder disease’, ‘Gallstones’, ‘Cholecystitis’, ‘Medications and gastrointestinal diseases’, ‘Vomiting’, ‘Abdominal pain’, ‘Liver transplantation’; and ‘Gastrointestinal endoscopy’, ‘Gastrointestinal disease’, ‘gastrointestinal motility’, ‘Liver disease’.
These keywords were identified using the terminology used by major journals in gastroenterology and hepatology, and the major conference proceedings in the field such as the American Gastroenterological Association Annual Scientific Meeting, the American College of Gastroenterology Annual Scientific Meeting, the United European Gastroenterology Week, the Canadian Digestive Diseases Week, World Gastroenterology Congress and the American Association for the Study of Liver Diseases Annual Meeting.
Inclusion and exclusion criteria
Papers focusing on gastroenterology or hepatology as the main topic and in the English language were included. The exclusion criteria were: (1) articles in languages other than English, (2) articles focused on broad areas without giving the whole emphasis to gastroenterology or hepatology knowledge.
Assessing the articles
Following the methods of Lefaivre et al,21 each paper in the top 50 most cited articles list was reviewed. The full text of the articles included in lists A and B was obtained and a copy given to each evaluator. The following information was analysed: (1) the authors’ names and affiliations, (2) the city and country of publication, (3) the number of citations and (4) the year of publication and the calculation of the number of years since publication.
It was decided not to use the Web of Science classification because we noted that papers identified by the publishing journals as original research, article or practical guidelines were all grouped by the Web of Science and identified under the category ‘article’ or ‘review’. For consistency and for the purposes of this research, a definition of the category ‘review paper’, ‘article’, ‘educational guide’ and ‘research’ has been given in the glossary (box 1). Also, the type of research used in the top-cited papers has been placed under the following types: cross-sectional, case–control, cohort study, randomised controlled trial, experimental study and causal-comparative study. A definition of each type has been given in the glossary (box 1). The two authors independently ranked each paper identified with regard to paper category and research type for research papers. Articles that covered more than one topic were classified on the basis of the aim of the study and the main outcome. These articles were discussed among the researchers until a final topic was identified.
Glossary
Articles: Reports with conclusions that represent a substantial advance in the understanding of an important topic or problem. They provoke thoughts and ideas, and they aim at establishing new directions.
Case–control studies: In these studies, patients who developed a disease are compared with controls or referents groups (with no disease). The studies aim at estimating ORs or changes caused by the disease. The researchers have to identify potential confounding factors by making appropriate adjustment in the design of the study and in the analysis. This may be achieved either by matching cases and controls for exposure to confounders on an individual basis (by pairing each case with a control of same age and sex) or group comparison basis (controls have overall age and sex distribution similar to the patients).
Causal-comparative studies: These studies attempt to identify cause–effect relationships. The approach involves starting with an effect and seeking possible causes. The design involves comparison.
Cohort studies: In these studies, a group of individuals is followed over a period of time to assess the individuals’ health outcomes. In these studies, individuals who do not have the outcome of interest initially are identified and grouped in subsets that differ in their exposure to a particular factor, for example, hepatitis C infection, and non-exposure. The follow-up of the two groups over time enables the comparison of health outcomes. The cohort could be grouped according to whether they had or had not been exposed and the analysis of health outcomes could compare the frequency (the incidence) of a particular change (eg, liver cirrhosis) between the groups. Cohort studies could be prospective cohort studies or retrospective cohort studies.
Cross-sectional studies: These studies measure, in a population, at a point in time, the prevalence of health outcomes or determinants of health or both. They can also be used in planning healthcare. Cross-sectional studies are best suited to study aetiology of diseases that produce little disability in a population or the early phase of more serious diseases. However, the results of cross-sectional surveys (design) that explore aetiology have to be interpreted with great caution, as the findings identified may be associated changes rather than the causes of the change or the condition.
Experimental studies: In these studies, researchers are in control of the research design by determining the groups to be exposed and the groups not to be exposed. However, deliberate exposure of participants to potentially serious hazards does not follow the World Medical Association's Declaration of Helsinki, will not be approved by a formally constituted research ethics committee and may form a constraint on such research. In animal work, experimental study design may be in vivo or in vitro studies.
Practical guidelines: Resources usually written by a team of experts in the area/topic, and aimed at providing clinicians and researchers with a resource on principles, current evidence, applications and regulations.
Randomised controlled trials: Aim at evaluating therapeutic intervention by using experimental design and randomisation of participants. Participants are selected on the basis of inclusion criteria; those satisfying the entry criteria and representing the target population are asked to consent to participation. Participants are randomised to the intervention (treatment) under comparison using a valid randomisation method, usually conducted by a third party, such as web-based or phone-based randomisation. The use of randomisation means eventual distribution of any confounding factors and prognostic markers between the different treatment groups.
Reviews: Articles reviewing progress of knowledge in a particular topic, critically analysing the current status of the literature and presenting an understanding of the topic by discussing related literature, and identifying gaps in knowledge and highlighting future directions for further research.
Research papers: Original studies making systematic investigations into a problem, using valid and reliable methods in order to establish answers to the research questions made, and come with conclusions. Research methods used may be qualitative, quantitative or mixed methods.
The number of authors for each publication and the representation of females in the authorship of top-cited articles were identified. Collaboration type was determined by the addresses of the authors, and the term ‘country independent work’ was assigned if the researchers’ addresses were from the same country. Articles that were the result of collaborative work from more than one country were identified.22 Those from the same country were classified into those from one institute and those from more than one institute.23 Papers that received grant support were also identified.
Evaluating the journals
The journals in which the top 50 articles were published were evaluated with respect to the following: (1) the Impact Factor of each journal, determined as reported in the JCR 2014 and (2) the ranking of the article, at the time of the research, in comparison to other articles published in that journal on the basis of the number of citations obtained. For example, an article ranked number 1 in the journal it was published in means that the article received the highest number of citations in comparison to all other articles published in that journal. The aim of this evaluation was to assess the position of articles identified among the 50 top-cited articles in regard to their ranking among other articles published in their respective journals. Such assessment will give a better idea about the significance of the articles included in list B among other topics published in journals not dedicated to the field.
Statistical analysis
Using SPSS software (IBM SPSS Statistics Premium V.22.0 for Mac OS-SPSS Inc, Chicago, Illinois, USA), the data were analysed and reported as total and percentage. Pearson's correlation coefficient (r) was calculated to determine if the number of years since publication was correlated to the number of citations obtained. Also, the correlations between the number of authors, the number of institutes or the number of countries involved or the number of grants received against the number of citations were calculated. The degree of agreement between evaluators was calculated using Cohen κ index for inter-rater coefficient.24
Results
Top-cited papers identified by searching journals (list A)
Appendix 1 summarises the characteristics of top-cited articles published in the gastroenterology and hepatology journals (list A).25–78 Articles are listed in descending order from 1 to 50, with the highest absolute citation number ranked 1 and the article with the lowest citation ranked 50, as on the day of the search. Articles with the same number of citations were given the same rank number. Four articles49 ,56 ,64 ,78 had the same number of citations and were given the same ranks. Therefore, the total number of articles in list A is 54, and not 50. The denominator used in calculating the percentages is 54. All articles were published in the English language (list A).
Table 1 shows that these articles were published over a 66-year period (from 1945 to 2011). During the period from 1945 to 1987, only seven articles (12.9%) were included. However, the number increased progressively from 1988 to 2011, making a total of 47 (87%) articles.
Top-cited articles in Gastroenterology and Hepatology, identified by searching gastroenterology and hepatology journals (list A): by article type and year of publication*
Table 2 summarises the distribution of gastroenterology and hepatology topics in relation to the four categories. The majority of the top-cited articles were research papers (n=24, 44.4%), the remaining were practical guidelines (n=12, 22.2%) and reviews (n=12, 22.2%). Only six were articles (11.1%). The topics can be summarised as follows: chronic hepatitis and viral hepatitis (n=12, 22.2%), hepatocellular carcinoma (n=9, 16.7%), inflammatory bowel disease (n=7, 12.9%), colorectal cancer (n=7, 12.9%), fatty liver disease (n=6, 11.1%), gastro-oesophageal reflux disease (n=2, 3.7%), gastric ulcer and H. pylori (n=2, 3.7%), prostaglandins and gastric protection (n=2, 3.7%), and end-stage liver disease and liver failure (n=2, 3.7%). The remaining topics are shown in table 2.
Top-cited articles in Gastroenterology and Hepatology, identified by searching gastroenterology and hepatology journals (list A): by article topic and category*
The articles were published in the following journals: Gastroenterology (n=26, 48%), Hepatology (n=17, 31.4%), Journal of Hepatology (n=2, 3.7%), American Journal of Gastroenterology (n=2, 3.7%), Gut (n=2, 3.7%), Seminars in Liver Disease (n=1, 1.8%), Gastrointestinal Endoscopy (n=1, 1.8%), Diseases of Colon & Rectum (n=1, 1.8%), Digestive Diseases and Sciences (n=1, 1.8%) and Journal of Viral Hepatitis (n=1,1.8%) (see online supplementary appendix 1).
The most frequently cited article, by Bruix and Sherman (2005), was cited 2959 times over 10 years.25 Two articles were ranked number 50; an article by Baggio and Drucker77 was cited 1049 times over 8 years, and an article by Bedossa et al78 cited 1049 times over 12 years. No correlation was found between the number of citations of these papers and the number of years since publication (R2=0.00992, p=0.473) (figure 1A). However, the mean number of citations of papers published before 2000 was lower than those published after 2000 (36.70±19.31 vs 106.03±39.22) (figure 1B, C).
(A) Top-cited gastroenterology and hepatology articles identified by searching journals (list A): correlation between the number of citations and the number of years since publication, (B) Number of citations of papers published before the year 2000 (mean±SD), (C) Number of citations of papers published after the year 2000 (mean±SD), (D) Top-cited gastroenterology and hepatology articles identified by searching journals (list A): correlation between the number of citations and the number of institutions involved, (E) Top-cited gastroenterology and hepatology articles identified by searching journals (list A): correlation between the number of citations and the number of countries, (F) Top-cited gastroenterology and hepatology articles identified by searching journals (list A): correlation between the number of citations and the number of authors, and (G) Top-cited gastroenterology and hepatology articles identified by searching journals (list B): correlation between the number of citations and the number of years since publication.
Continued.
A strong correlation was found between the number of institutes involved (figure 1D) (R2=0.27531, p<0.001) or the number of countries, and the number of citations received (R2=0.16181, p=0.003) (figure 1E). Table 3 shows further analysis of top-cited articles in list A in regard to authorship, institutes involved, countries and grants received. No correlation was found between the number of authors of top-cited articles and the number of citations (R2=0.0452, p=0.130) (figure 1F).
Further analysis of the 50 top-cited papers on gastroenterology and hepatology*
Most articles originated from the USA (n=31, 57.4%), Spain (n=6, 11.1%), the UK (n=4, 7.4%), France (n=3, 5.5%), Canada (n=2, 3.7%), the Netherlands (n=2, 3.7%), Belgium, Italy, Japan, Germany, Switzerland and China (Hong Kong) (n=1, 1.8%) for each country.
Careful assessment of the authorship of top-cited articles shows that some authors contributed to more than one article in the list. Bruix was the first author of three articles,25 ,28 ,39 and was the second and third author of two other articles.52 ,63 In his pioneering work, the author focused on the clinical management of hepatocellular carcinoma25 ,28 ,39 ,52 and prognosis of hepatocellular carcinoma.63 Llovet co-authored three articles with Bruix.28 ,52 ,63 Also, Sherman co-authored three articles with Bruix.25 ,28 ,39 Lok was the first author of two articles44 ,66 and co-authored one article.58 These articles were on chronic hepatitis B,44 ,66 and assessment of a simple non-invasive index for predicting fibrosis and cirrhosis in patients with chronic hepatitis C.58 Ishak was the first author of one article27 and co-authored another article.26 These articles were on histological grading and staging of chronic hepatitis,27 and the use of a numerical scoring system for assessing histological activity in patients with asymptomatic chronic active hepatitis.26 Desmet was the first author of one article31 and co-authored another article with Ishak.27 Bedossa was the first author of two articles.36 ,78 These articles were on an algorithm for the grading of activity in chronic hepatitis C36 and sample variability of liver fibrosis in chronic hepatitis C.78 McCullough was a co-author of two articles.30 ,42 Both articles were on non-alcoholic fatty liver disease. Strader was the first author of one article59 and a co-author of another article,41 both articles were on management of hepatitis C. Winawer was the first author of two articles.47 ,50 These articles were on colorectal cancer. Fletcher co-authored with Winawer on two articles.47 ,50 No correlation was found between the number of grants received and the number of citations received (R2=0.0257, p=0.247).
Top-cited articles identified by keyword search (list B)
Online supplementary appendix 2 summarises the top-cited articles on gastroenterology or hepatology identified by searching the Web of Science across all journals (list B).25 ,33 ,79–119 Articles are listed in descending order by rank from 1 to 50 based on the absolute number of citations received as of the day of the search.
Table 4 shows that these articles were published over a 35-year period (from 1973 to 2008). During the period from 1973 to 1987, only six articles (12%) were published. However, the number increased progressively over the years from 1988 to 2008, making a total of 44 (88%) articles. Table 5 summarises the distribution of gastroenterology or hepatology topics in relation to the four categories. The majority of top-cited articles were research papers (n=38, 76%). The remaining were reviews (n=8, 16%) and articles (n=3, 6). Only one article (2%) was an educational guide. The topics can be summarised as follows: colorectal cancer (n=12, 24%), chronic hepatitis and viral hepatitis (n=9, 18%), hepatocellular carcinoma (n=7, 14%), inflammatory bowel disease (n=6, 12%), gastritis, gastric ulcer and H. pylori (n=4, 8%), and Escherichia coli and diarrhoeal diseases (n=3, 6%). The distribution of the remaining topics is shown in table 5.
Top-cited articles by searching keywords for all journals listed in the Science Citation Index Expanded database (list B): by article type and year of publication*
Top-cited articles by searching keywords for all journals listed in the Science Citation Index Expanded database (list B): by article topic and category*
The top-cited articles were published in 17 journals. Of these, three were specialised in the field (n=number of articles, %): Hepatology (n=4, 8%), Gastroenterology (n=3, 6%) and Journal of Hepatology (n=2, 4%). The majority were published in journals not dedicated to the specialty (n=number of articles, %): New England Journal of Medicine (n=16, 32%), Science (n=5, 10%), Nature (n=5, 10%), Lancet (n=3, 6%), Cell (n=2, 4%), Journal of Clinical Oncology (n=2, 4%), Nature Genetics (n=1, 2%), British Journal of Surgery (n=1, 2%), DNA Research (n=1, 2%), Clinical Microbiology Reviews (n=1, 2%), Cancer Research (n=1, 2%), Annals of Internal Medicine (n=1, 2%), American Journal of Surgical Pathology (n=1, 2%) and Proceedings of the National Academy of Sciences of the USA (n=1, 2%) (see online supplementary appendix 2).
The most frequently cited article was ‘Bevacizumab plus irinotecan, fluorouracil, and leucovorin for metastatic colorectal cancer’, by Hurwitz et al,79 published in New England Journal of Medicine and cited 5500 times over 11 years. The article ranked 50 was ‘Up-regulation of cyclooxygenase 2 gene expression in human colorectal adenomas and adenocarcinomas’, by Eberhart et al,33 published in Gastroenterology and cited 1929 times over 21 years. There was no correlation between the number of citations of these papers and the number of years since publication (R2=0.00202, p=0.757) (figure 1G). Most articles were from universities in the USA (n=32, 64%), Spain (n=4, 8%), the UK (n=3, 6), France (n=2, 4%), Japan (n=2, 4%), Germany, Italy, Finland, Greece, Canada, Belgium and Taiwan (n=1, 2%) for each country.
Some authors contributed to more than one article in list B. In addition to the five articles published by Bruix and included in list A,25 ,28 ,39 ,52 ,63 were two articles in list B.84 ,103 These articles were on hepatocellular carcinoma. Interestingly, 24 authors had authored or co-authored more than one article in list A and 29 authors had more than one article in list B. Those who contributed to both lists were: Bond JH, Bruix J, Llovet JM, Mann SM, JP, Miller LL, Rosen L and Winawer SJ (table 6).
Authors and co-authors of two or more articles of top-cited articles in gastroenterology and hepatology identified by searching gastroenterology and hepatology journals (list A) or by searching keywords for all journals listed in the Web of Science (list B)*
Based on the number of citations attracted by top-cited articles in list B, we looked at the ranking of these articles in the journals they were published in (the 14 journals that were not specialised in gastroenterology and hepatology). It was interesting to note that five of the top-cited articles were ranked number 1 in their respective journals (ranked number 1 means an article receiving the highest number of citations compared to all articles published in the journal). These articles were published in the following five journals—Journal Impact Factor (JIF)reference British Journal of Surgery 5.542,81 American Journal of Surgical Pathology 5.145,93 Journal of Clinical Oncology 18.428,94 DNA Research 5.47796 and Clinical Microbiology Reviews 17.406.101
Eleven top-cited articles were ranked between 2 and 50 in the journals they were published in. The articles were published in the following eight journals—JIFreference: Journal of Clinical Oncology 18.428,94 ,114 Annals of Internal Medicine 17.810,115 Lancet 45.217,82 ,103 Nature Genetics 29.352,97 New England Journal of Medicine 55.873,79 ,83 ,84 Cancer Research 9.329,102 Cell 32.24285 and Science 33.611.80
Comparing lists A and B
The total number of citations for all the 54 articles included in list A was 81324.0 and 138012.0 for the 50 articles in list B. The median number (IQR) of citations for top-cited articles in list A was 1340.0 (IQR=529.5) and 2585.5 (IQR=739.2) for list B.
Several universities or research centres contributed to more than one article. For example, Mayo Clinic and Mayo Foundation, Rochester, Minnesota;34 ,57 ,67 ,110 and the University of Michigan Medical Center, Michigan,44 ,58 ,66 ,91 each contributed four articles. Other research centres contributed two articles each. These were: The National Center Institute, Bethesda, Maryland;30 ,108 University Hospital of Cleveland, Cleveland;42 ,46 The Upjohn Company, Kalamazoo, Michigan;37 ,74 Duke University Center, Durham, North Carolina;43 ,79 Johns Hopkins Oncology Center, Baltimore, Maryland;85 ,100 and University of Wisconsin, Madison, Wisconsin.69 ,86 In Spain, the top-cited articles originated from the University of Barcelona25 ,39 ,52 and Barcelona-Clinic Liver Cancer Group, Institut d’Investigacions Biomèdiques August Pi i Sunyer, Liver Unit, Hospital Clínic, Barcelona, Catalonia.28 ,52 ,63
Figure 2 shows the citations (mean±SD) attracted by top-cited articles in lists A and B for each type of paper (Practical Guide, Research, Article and Review). Further analysis showed no significant differences between the number of citations attracted by each type in the two lists (p=0.803).
Citations (mean±SD) attracted by top-cited papers in lists A and B for each paper type.
Table 7 summarises the type of research in both lists. In list B, randomised controlled trials (n=15, 30%) and cohort studies (n=8, 16%) were dominant, while in list A, cross-sectional studies (n=3, 5.9%), a case–control study (n=1, 2%), experimental studies (n=9, 17.6%) and causal-comparative studies (n=5, 9.8%) were observed. Four research studies were identified in both lists.26 ,29 ,30 ,33 The inter-rater agreement between evaluators had a Cohen κ coefficient of 0.76–0.84.
Top-cited research articles in gastroenterology and hepatology identified by searching gastroenterology and hepatology journals (list A) or by searching keywords for all journals listed in the Web of Science (list B): by article topic and research type*
Discussion
The aim of this study was to identify the 50 top-cited articles in gastroenterology and hepatology, and to gain insight into the characteristics of the top-cited articles in the field. Citation analysis may offer the opportunity to gain insight into peer recognition of articles that added to the discipline. To ensure that our search had included articles published in journals other than those dedicated to the discipline, a second search was conducted using keywords (list B). The latter search covered all journals listed in the SCI-Expanded regardless of specialty. The number of citations attracted by articles included in list B was significantly higher than those in list A. This may be proportional to the JIF of the journals in which the articles were published. Interestingly, the articles in list A were published in 10 journals with the highest JIF in the category ‘Gastroenterology and Hepatology’ listed in the 2014 JCR. The finding that the majority of articles in list B were from journals not in the field, reflects the integrative nature of the specialty with the role of basic sciences and clinical studies in the development of Gastroenterology and Hepatology. It also shows the interest of editors and readers in journals such as New England Journal of Medicine, Annals of Internal Medicine and Lancet in gastroenterology and hepatology topics.
The study provided an insight into the trends in publications over the past 60–70 years. Top-cited articles from the 1950s to the late 1970s dealt primarily with animal models for the study of gastric ulcerations,45 cytoprotection of gastric mucosa by prostaglandins,37 ,74 animal models for intestinal ischaemia,65 the development of a numerical scoring system for assessing histological activity in patients with chronic active hepatitis26 and the development of Crohn's disease activity index.29 During the 1980s to early 1990s, the articles focused on animal models of inflammation and ulcerations in the colon,60 ,61 ,68 localisation of the multidrug-resistant gene product, p-glycoprotein,108 and the biology of bilirubin,112 non-A and non-B viral hepatitis,80 ,90 and colorectal cancer.49 ,51 ,105 ,106 ,109 ,116
From 1994 to 2005, top-cited articles focused on three hepatology topics: (1) steatohepatitis and fatty liver,30 ,35 ,38 ,42 ,48 ,62 ,110 (2) hepatocellular carcinoma,25 ,28 ,52 ,63 ,72 ,76 ,89 ,103 and (3) viral hepatitis C diagnosis and treatment,36 ,58 ,59 ,73 ,78 ,82 ,83 ,98 ,115 as well as three gastroenterology topics: (1) colorectal cancer,47 ,50 ,75 ,79 ,85 ,95 ,102 ,104 ,114 ,117 (2) inflammatory bowel disease and colitis,46 ,67 ,87 ,99 ,113 and E. coli and diarrhoeal diseases.86 ,96 ,101 The studies after 2006 continued to explore new aspects related to hepatocellular cancer,32 ,39 ,54 hepatitis B44 ,66 and hepatitis C.41
With regard to clinical relevance, the articles reflect the hidden burden of chronic hepatitis B and C infections, which the former US Assistant Secretary for Health, Dr Howard Koh, described as the ‘Silent Epidemic’,120 and highlight the need for strategies to prevent and manage liver cancer121 and colorectal cancer. In USA, over 1.3 million people suffer from inflammatory bowel disease and, despite extensive research in this area, we are still unable to identify the exact cause of the disease or to have clear preventive strategies or an effective cure.122 In the USA, colorectal cancer is the second leading cause of cancer-related deaths in males and in females. It is the third most common cancer in men and in women.123 Therefore, the scientific and research relevance of the top-cited articles identified in the two lists reflects the emerging needs of these areas and the new developments in our understanding of these disorders.
Although it may take 15 years or more for articles to reach a peak in the overall citation number,124 it was noted that a number of top-cited articles in both lists were only 8 or less years old.32 ,39 ,41 ,44 ,54 ,66 ,70 ,77 ,84 Only four articles were published after 2007. However, the article by Shay et al,45 on animal model for the study of gastric ulcers, was 70 years old (published in 1945). Careful scrutiny showed that the article is still attractive to researchers and was cited 10 times in 2015, 23 times in 2014 and 25 times in 2013. Top-cited articles are frequently cited but this tendency does not necessarily indicate that these papers are great. For example, the most cited work in history is the paper by Lowry et al, a 1951 paper describing an assay to determine the amount of protein in a solution. It has gathered, at the time of submitting this manuscript, 311 819 citations, although several new techniques for measuring protein in a solution and several modified techniques to this method have been developed and described in the literature over the past 65 years.125 However, such papers are exceptions and do not represent a general trend.
As shown from this study, there was no correlation between the number of citations and the number of years since the paper was published. This was demonstrated for articles included in both lists. The finding may be related to the tendency of researchers to cite particular papers. This may become standard practice to make clearer to other scientists the type of methods these individuals followed in their research.18 It has been shown from this study that the mean numbers of citations of articles published after the year 2000 are higher than those published before 2000, not necessarily because of their quality but due to the tendency of researchers to preferentially cite the most recent studies.
This study also showed no correlation between number of authors or number of grants received and number of citations, but demonstrated strong correlations between the number of institutes involved or number of countries and the number of citations. Females were under-represented in authorship (45 vs 254, p=0.004). In neither list were significant differences found between the number of citations attracted by each type of paper—Practical Guide, Research, Article and Review—which may highlight the equal significance and usefulness of each type to researchers and clinicians.
These findings are consistent with those of Danthi et al,126 who, in a large cohort of the National Heart, Lung, and Blood Institute (NHLBI) grant-funded research, reported that they were unable to find a monotonic association between better percentile ranking and higher scientific impact as assessed by citation metrics.126 Again, Fortin and Currie,127 in a study evaluating how scientific impact scales with funding, concluded that their findings are inconsistent with the hypothesis that larger grants lead to larger discoveries. The issue of female authorship in major academic gastroenterology journals has been recently studied. The authors found that the percentage of US female physician authors of original research in major gastroenterology journals is lower than expected, although it has increased over time.128
While writing this study, we came across a recently published article by Tang et al,129 who used three key search words to identify top-cited article on the digestive system. The method used was briefly described and there was no list of top-cited articles to compare with our list. However, it was noticed that, out of a list of 100 articles, Tang et al identified only eight articles from Gastroenterology, while this study identified 26 articles among top-cited articles in list A. They identified five institutions with two or more articles, while in this study, more than 10 centres were identified. They reported only two top-cited articles each from Mayo Clinic and Mayo Foundation, Rochester, Minnesota, and from the University of Michigan Medical Center, Michigan, while this study identified four articles from each of these institutions. Several centres that contributed to more than one article were not mentioned in Tang's study. Also, they only identified six authors who had authored or co-authored two or more top-cited articles, while, as per the results from this study, 24 authors in list A and 29 authors in list B had authored or co-authored more than one top-cited article (table 6). These differences may be related to differences in the methodology used and possibly the depth in analysing data collected.
However, this study is not without limitations. First, the search used was based on journals with high impact factor. This may have contributed to the increased number of articles from Western countries, especially the USA, the UK and Canada. Therefore, articles in languages other than English, which may have impact in the field, were not included. However, our findings show that Spain was the second country that contributed to most-cited articles in lists A and B. Recently, Iñigo and García-Samaniego, in an elegant article, conducted a bibliometric analysis of publications in gastroenterology and hepatology in Spain, from 2000 to 2009. The paper was written in the Spanish language.20 Second, authors’ self-citations were not excluded from the total number of citations, and absolute number of citations was used. Third, the search was conducted using the Science Citation Index Expanded database, and there is the possibility that the database filter is not sensitive enough to the search words. However, in this study, 69 journals specialised in the field were searched and more than 40 keywords were used in the search with the aim to maximise the yield. Fourth, the 50 top-cited articles in the two lists represent an arbitrary number and the findings represent the outcomes at the time of conducting the search. One of the strengths of this study is the search for the top 50 frequently cited papers, using two search methods.
Conclusions
The citation analysis in this study compiled two lists of the top 50 highly cited articles in gastroenterology and hepatology. The first list (list A) was constructed by searching 69 journals in the field. List B was constructed by searching the Science Citation Index Expanded database, using keywords. The citations varied from 1049 and 2959 in list A and from 1929 to 5500 in list B. The articles were published between 1945 and 2011, with the number of articles increasing progressively from 1988 to 2011. In both lists, research papers dominated top-cited articles. Randomised controlled trials and cohort studies dominated research in list B, while in lit A, cross-sectional studies, a case–control study and experimental studies were observed. The number of authors or co-authors with more than one article was 24 in list A and 29 in list B. Articles in list B were mainly published in New England Journal of Medicine, Science and Nature. The articles came from over 12 different countries, with the USA most frequently represented followed by Spain. While no correlations were found between the number of authors or the number of grants received, and the number of citations, strong correlations were found between the number of institutes or the number of countries involved, and the number of citations. In neither list was there significant correlation between the number of citations and the number of years since publishing. However, the mean number of citations tended to be higher in papers published after the year 2000, possibly indicating the significance of scientific content and the tendency of researchers to cite recently published work.
Acknowledgments
The authors would like to thank Diana Azer for her assistance and review of the manuscript. They also thank Dr Lily Scott for her assistance in this work and Ms Mae Eustaquio for secreterial assistance.
Appendix 1
Top-Cited Gastroenterology and Hepatology Articles Published in Gastroenterology and Hepatology Journals, identified by Category Search in the Web of Science™, Ranked from 1 to 50 (List A).
Appendix 2
Top-Cited Gastroenterology and Hepatology Articles Published Across All Journals Listed in the Web of Science™, Identified by Keyword Search, Ranked From 1 to 50 (List B).
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
- Data supplement 1 - Online supplement
Footnotes
Contributors SAA started the design of the study and contributed to its methodology. SAA and SA searched the databases, collected the data, analysed the findings and created the two lists. SAA and SA also interpreted the findings, ranked the articles, created the tables and figures and drafted the manuscript, as well as contributed to the revision of the manuscript and approved the final manuscript for submission.
Funding This work was supported by the College of Medicine Research Center, Deanship of Scientific Research, King Saud University, Riyadh, Saudi Arabia.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.