Article Text

PDF

Performance curves of medical researchers during their career: analysis of scientific production from a retrospective cohort
  1. Antoine Duclos1,2,
  2. Eléonore Herquelot1,
  3. Stéphanie Polazzi1,
  4. Muriel Malbezin3,
  5. Olivier Claris4
  1. 1Pôle Information Médicale Évaluation Recherche, Hospices Civils de Lyon, Lyon, France
  2. 2Health Services and Performance Research Lab (EA 7425 HESPER), Université Claude Bernard Lyon 1, Lyon, France
  3. 3Direction de la Recherche Clinique et de l'Innovation, Hospices Civils de Lyon, Lyon, France
  4. 4Department of Neonatal Intensive Care Unit, Hospices Civils de Lyon, Hôpital Femme Mère Enfant, Bron, France
  1. Correspondence to Dr Antoine Duclos; antoineduclos{at}yahoo.fr

Abstract

Objectives To establish the pattern of change in individual scientific production over the career of medical researchers.

Design Retrospective cohort based on prospectively collected data in a hospital information system.

Setting Multicentre university hospital in France.

Participants Two distinct populations of 1835 researchers (full professors vs non-academic physicians) having produced 44 723 publications between 1995 and 2014.

Main outcome measures Annual number of publications referenced in Medline/PubMed with a sensitivity analysis based on publications as first/last author and in high impact journals. The individual volume of publications was modelled by age using generalised estimating equations adjusted for birth cohort, biomedical discipline and academic position of researchers.

Results Averaged over the whole career, the annual number of publications was 5.28 (95% CI 4.90 to 5.69) among professors compared to 0.82 (95% CI 0.76 to 0.89) among non-academic physicians (p<0.0001). The performance curve of professors evolved in three successive phases, including an initiation phase with a sharp increase in scientific production between 25 and 35 years (adjusted incidence rate ratio 102.20, 95% CI 60.99 to 171.30), a maturation phase with a slower increase from 35 to 50 years (2.10, 95% CI 1.75 to 2.51) until a stabilisation phase with constant production followed by a potential decline at the end of career (0.90, 95% CI 0.77 to 1.06). The non-academic physicians experienced a slower pace of learning curve at the beginning of their careers (42.38, 95% CI 25.37 to 70.81) followed by a smaller increase in the annual number of publications (1.29, 95% CI 1.11 to 1.51).

Conclusions Compared to full professors, non-academic physicians had a poor capacity to publish, indicating a low productivity when medical doctors have limited time or little interest in undertaking research. This finding highlights the potential for rethinking the missions of medical doctors towards an enlargement of scientific prerogatives in favour of progress in global knowledge.

Statistics from Altmetric.com

Strengths and limitations of this study

  • This is the first study with a longitudinal design to evaluate the performance curves of individual researchers over their career, taking into account their biomedical field, academic position and birth cohort.

  • An accurate measurement of scientific production was available for all researchers.

  • The local context of the study may affect the generalisation of the results.

  • Bibliometric analysis based on referenced papers does not necessarily reflect the entire contribution of researchers to science.

Introduction

Productivity is a concern that initially arose in the manufacturing industry before spreading to all economic sectors, including research and innovation in healthcare. Industrial productivity tends to increase in all fields worldwide, but variations exist between firms, addressing the question of the determinants of productivity.1 ,2 In the context of research, pronounced differences exist between universities for scientific production.3 At the individual level, publication volume is now crucial for all researchers because it is often a prerequisite for the credibility of research projects and basically for obtaining funding or an academic position.4

The effect of age on scientific production of researchers has been explored in the past. Some studies stated the most novel theories were found before 40 years of age among scientists who have won the Nobel Prize. This supports the existence of an ‘obsolescence theory’ with major scientific breakthroughs emanating from young researchers.5 ,6 Other studies stated a high productivity for researchers after 50 years of age, in line with the ‘cumulative advantage theory’ or ‘Matthew effect’, suggesting that older researchers take advantage of their experience, position and network.5 ,7 However, the vast majority of investigations focusing on the individual determinants of scientific production were based on cross-sectional designs, comparing a heterogeneous population of researchers at a given time.4 ,5 ,8–10 A longitudinal follow-up of individual researchers during their entire career appears more appropriate to investigate this time-dependent phenomenon.7 ,11 Furthermore, exploring the change of scientific production with experience requires investigators to disassociate the effect of age from a possible secular trend and to consider several confounders related to the academic position and discipline of researchers.

This study aimed to establish the performance curve of two distinct populations of medical researchers, full professors versus non-academic physicians, based on the annual volume of publications over their career.

Methods

Study design and population

A retrospective cohort of medical researchers employed at the Lyon university hospital between 1995 and 2014 was constituted. This multicentre institution employs more than 23 000 healthcare workers divided between 14 sites and a large community of researchers from various biomedical disciplines generating more than 2000 citations per year in Medline/PubMed. The medical community comprises two groups of physicians: full professors and non-academic physicians. The full professors are affiliated both with the hospital and the university, undertaking care, teaching and research activities. Non-academic physicians are affiliated with the hospital, and their main activity is patient care with optional participation in research.

The cohort was selected among medical researchers with at least one publication during their period of employment and between 25 and 60 years of age. Researchers with uncertain positions or disciplines were excluded. In particular, young researchers with insufficient follow-up to determine their permanent position between full professor and non-academic physician were not considered in the analyses (eg, medical students, residents, fellows, assistant or associate professors).

The study was supported by the medical commission and research department of the host institution. Anonymous access and retrospective analysis of personal data was authorised by the national data protection commission (Commission Nationale de l'Informatique et des Libertés, CNIL; number 15-076), in accordance with the French legislation.

Data sources and main variables

We linked two databases that were prospectively collected in the institution data warehouse using an anonymous identifier readily available for every healthcare worker. On the one hand, the human resources database provided detailed information about career development of each medical researcher, including the change of his/her academic position and discipline during the period of employment. On the other hand, the annual number of publications by a given researcher and their characteristics (author ranking and journal impact factor) were available from the bibliometric system SIGAPS.12

The primary outcome was the annual number of publications referenced in the Medline/PubMed database for every researcher during the study period. As part of the sensitivity analyses, we used the annual number of publications in which the researcher was the first or last author to evaluate the work for which he/she was strongly involved. Jointly, to estimate the visibility of scientific production of each researcher, we monitored his/her annual number of publications in high impact journals, defined as the 25% of journals with the highest impact factors among all the journals in the same category of the Web of Science Journal Citation Report (JCR).12

The birth date was extracted from the human resources database allowing calculation of age at the time of publication and birth cohort for all researchers. In order to explore a potential secular trend, the birth cohort was categorised into three classes from the oldest to the youngest: 1935–1945, 1946–1965, and 1966–1985. Other determinants included the academic position and scientific discipline of the researcher. The academic position was the last known status of the researcher, either full professor or non-academic physician. The scientific discipline of the researcher was attributed according to the predominant biomedical field of interest during his/her career, as follows: medicine, surgery, emergency/intensive care, biology, medical imaging, or public health.

Statistical analysis

The main characteristics of the population were first described and compared by researcher position. Categorical variables were presented using absolute and relative frequencies, and they were compared between full professors and non-academic physicians using the χ2 test. Continuous variables were presented using the median and IQR.

The annual number of publications and the proportion of each type of publication were modelled using Generalised Estimating Equations (GEE) with a negative binomial distribution (or a binomial distribution for proportion) and a log link taking into account repeated publication measurement for each researcher according to his/her age.13 The working correlation matrix structure chosen was AR(1) and the results were presented on the empirical variance-covariance matrix. The mean number of publications per year was drawn on the entire follow-up according to age in class and academic position of researchers in univariate GEE models. The change with the age was modelled by quadratic spline with nodes a priori at 30, 35, and 50 years. The degree of splines was chosen by testing statistically the highest degree of spline until achieving a p value higher than 5%. The learning curves were successively drawn based on two intermediate multivariate models: the first adjusted for age and position, the second adjusted for age, position and birth cohort. The final multivariate model was adjusted for factors selected a priori: age, position, birth cohort and biomedical discipline. In all these models, the interactions of order two were explored one by one, particularly between age and other determinants, and were kept in the model presented when they reached the significance threshold of 5%.

In order to enhance the interpretability of model estimates, some incidence rate ratios (IRR) were combined, the effect of age was computed at several times points corresponding to each phase of the performance curve (25, 35, 50, and 60 years) and the trend between these time points was computed every 5 years. The results were presented as adjusted IRR with corresponding 95% CI. Similar analyses were repeated regarding the annual number of publications as first or last author and the annual number of publications in high impact journals.

Data manipulation and analyses were performed using SAS software (V.9.3; SAS Institute, Cary, North Carolina, USA).

Results

The study population included 1835 medical researchers who produced 44 723 publications from 1995 to 2014, corresponding to 12 518 years of research with at least one publication (see study flow chart in the online supplement, eFigure 1). As shown in table 1, these researchers were divided between 319 full professors (88.40% male) and 1516 non-academic physicians (46.44% male). Overall, 5.72% of researchers belonged to the oldest birth cohort (1935–1945), 48.23% to the intermediate cohort (1946–1965), and 46.05% to the newest cohort (1966–1985). The most frequent discipline of researchers was medicine (40.16%), followed by surgery (17.98%), emergency/intensive care (17.33%), biology (12.37%), imaging (6.70%), and public health (5.45%). The volume of publications during the two decades of follow-up ranged between 1 and 438 by researcher, with a median of 68 referenced papers among full professors and five among non-academic physicians.

Table 1

Characteristics of the study population

The annual number of publications increased with age, from a mean of 0.48 (95% CI 0.43 to 0.54) between 25 and 30 years to a mean of 2.24 (95% CI 2.01 to 2.49) between 50 and 55 years (figure 1). Averaged over the whole career, the annual number of publications was 5.28 (95% CI 4.90 to 5.69) among full professors compared to 0.82 (95% CI 0.76 to 0.89) among non-academic physicians (p<0.0001). Full professors published more papers as first/last author (42.84%, 95% CI 40.85% to 44.92% vs 25.90%, 95% CI 24.42% to 27.47%; p<0.0001) and in high impact journals (40.28%, 95% CI 38.27% to 42.40% vs 34.04%, 95% CI 32.62% to 35.52%; p<0.0001) compared to non-academic physicians.

Figure 1

Mean number of publications per year according to age (A) and academic position (B) of researcher. Interpretation: (A) Between 35 and 40 years, a medical researcher produced 1.38 (95% CI 1.27 to 1.51) publications annually, including 0.50 (95% CI 0.44 to 0.59) publications as first/last author and 0.52 (95% CI 0.46 to 0.58) publications in high impact journals. (B) Averaged over the whole career, a full professor produced annually 5.28 (95% CI 4.90 to 5.69) publications, including 2.24 (95% CI 2.05 to 2.46) publications as first/last author and 2.16 (95% CI 1.93 to 2.40) publications in high impact journals.

The performance curve of full professors was composed of three successive phases including a sharp increase in scientific production between 25 and 35 years of age (initiation phase), then a slower increase from 35 to 50 years of age (maturation phase), until a plateau was reached with constant production followed by a potential decline at the end of their career (stabilisation phase) (figure 2). Since starting their academic work, the annual number of publications among full professors was multiplied by adjusted IRR 102.20 (95% CI 60.99 to 171.30) at 35 years of age, 214.60 (95% CI 121.90 to 377.80) at 50 years of age, and 193.90 (95% CI 108.70 to 345.60) at 60 years (table 2). Accordingly, the annual number of publications was multiplied by 102.20 (95% CI 60.99 to 171.30) during the initiation phase, while it was multiplied by 2.10 (95% CI 1.75 to 2.51) during the maturation phase, and by 0.90 (95% CI 0.77 to 1.06) during the stabilisation phase. These slopes were more pronounced than those of non-academic physicians who experienced a slower pace at the beginning of their career followed by a smaller increase in the annual number of publications. This was evidenced through a 2.41 (95% CI 1.77 to 3.29; p<0.0001) fold higher slope during the initiation phase among full professors compared to non-academic physicians, then a 1.62 (95% CI 1.35 to 1.94; p<0.0001) fold higher slope during the maturation phase. Conversely, scientific production of professors declined compared to physicians after 50 years: IRR 0.79 (95% CI 0.65 to 0.96; p=0.0178) during the stabilisation phase. Similar results were observed regarding the annual number of publications as first/last author and in high impact journals.

Table 2

Multivariate analysis of scientific production over a career

Figure 2

Scientific production during career according to academic position of researcher (A. Annual number of publications; B. Annual number of publications as first/last author; C. Annual number of publications in high impact factor journals). Interpretation: (A) The mean number of annual publications at 35 years was 4.20 (95% CI 3.71 to 4.74) among full professors and 1.01 (95% CI 0.93 to 1.10) among non-academic physicians.

The birth cohort influenced the scientific production of medical researchers, irrespective of age, academic position and biomedical discipline (figure 3 and eTable 1 in the online supplement). Although the same shape of performance curves was observed across generations, the birth cohort was significantly associated with the annual number of publications in the final multivariate analysis: IRR 1.69 (95% CI 1.31 to 2.19) for the birth cohort 1966–1985, and 1.22 (95% CI 0.96 to 1.54) for the birth cohort 1946–1965, compared to the birth cohort 1935–1945 (p<0.0001). Hence, professors in the newest cohort published 9.23 (95% CI 7.80 to 10.93) papers annually at 50 years, compared to 6.59 (95% CI 5.94 to 7.32) papers in the intermediate cohort, and 4.56 (95% CI 3.50 to 5.94) in the oldest cohort. The birth cohort was also significantly associated with the number of publications in high impact journals (p<0.0001) but not with the number of publications as first/last author (p=0.1066).

Figure 3

Scientific production during career according to academic position and generation of researcher (A. Annual number of publications; B. Annual number of publications as first/last author; C. Annual number of publications in high impact factor journals). Interpretation: (A) Among full professors, the mean annual number of publications at 50 years was 4.56 (95% CI 3.50 to 5.94) for the birth cohort 1935–1945, 6.59 (95% CI 5.94 to 7.32) for the birth cohort 1946–1965, and 9.23 (95% CI 7.80 to 10.93) for the birth cohort 1966–1985. Among non-academic physicians, the mean annual number of publications at 50 years was 0.86 (95% CI 0.57 to 1.31) for the birth cohort 1935–1945, 0.99 (95% CI 0.87 to 1.12) for the birth cohort 1946–1965, and 1.33 (95% CI 1.15 to 1.54) for the birth cohort 1966–1985.

Discussion

This study established the pattern of researcher performance over an entire career in a medical context. There was a pronounced difference in scientific productivity between two distinct populations of researchers. Compared to full professors, non-academic physicians had a poor capacity to publish, indicating a low performance when medical doctors have limited time, poor incentives or no interest in research. The publication volume among full professors evolved in three successive phases: the initiation phase with a dramatic 100-fold increase in scientific production before 35 years of age, the maturation phase with a doubling in production between 35 and 50 years of age, and the stabilisation phase with constant production followed by a potential decline at career end. The performance curve for non-academic physicians showed the same change with a less pronounced dynamic and a gradual downturn in the slope of production improvement during their career. Furthermore, the scientific production of researchers was strongly influenced by their birth cohort, supporting the hypothesis of a secular trend. There was a significant increase in publication volume among the research community born more recently compared to older cohorts. This effect was observed among both full professors and non-academic physicians, suggesting an increasing production over time as the generations succeed one another.

The main strength of this work is its longitudinal design that provided a valid picture of performance curves for individual researchers by exploring the change of scientific production over a career according to their age, academic position and birth cohort. The chosen GEE model was appropriate to evaluate the mean performance trajectories according to various determinants, even though this approach did not allow comparison between models. Outcome measurement based on the SIGAPS bibliometric system was accurate because this required individual approbation by researchers with incentives to validate their publications in the system.12 Human resources data were also exhaustive and of high quality because this information was critical for payment of salaries.

The main study limitation is the local context that may affect the generalisation of results. In particular, the absolute number of publications by researchers may have been influenced by how the research teams and disciplines were locally organised. Although these findings would deserve to be replicated using a multinational community of researchers, we assume that the pattern of the performance curves highlighted and the relative differences between academic positions and birth cohorts would be identical in a more general context. Furthermore, the gender of the researchers was not considered in determining performance curves for robustness considerations, due to the small number of women in several stratums related to academic position, birth cohort and discipline. Resolving this issue would require a larger cohort of researchers to be investigated. Another limitation relates to the absence of consideration of the total number of co-authors in analyses to control for opportunistic authorship strategies.14 Collaborations within and across research teams or in some disciplines that systematically include an important number of authors with limited contributions can trigger a spurious inflation in the volume of publications and an overestimation of scientific production at the individual researcher level. This aspect could not be evaluated in the present study because the number of co-authors in each paper was not available, which limited the analysis to full counts of publications instead of fractional counts. However, our sensitivity analysis based on the number of publications as first/last author revealed unchanging results for most findings. Finally, volumetric analysis based on referenced papers does not necessarily reflect when a researcher has full capacity to make a scientific breakthrough during his/her career. Identifying qualitatively the ground-breaking nature and potential impact of research findings beyond the state of the art (ie, novel concepts across disciplines with high gain for scientific community and public health) requires another approach. This may reveal a different pattern of individual performance curves with an innovation peak occurring earlier during the scientific careers of young researchers. Jointly with bibliometric evaluation, researcher performance could also be assessed using other aspects of scientific production. Active collaboration with international research networks or the mentoring of future researchers would make sense for the most experienced researchers in the last part of their career.15

The definition of ‘scientific productivity’ in terms of volume is subject to much debate in the research community because this is a complex notion, the measurement of which can include a wide range of documents including not only publications in peer-reviewed journals but also books, reports, conference abstracts, oral communications or filed patents.16 To date, there is no consensus for a gold standard in measuring scientific production and a wide range of criteria exists in the literature.4–9 In this study, the basic criterion of publication volume was refined to reflect the substantial contribution of the researcher in scientific projects as first/last author and the visibility of his/her own works in high impact journals. We identified the same determinants of scientific productivity that have been reported in other investigations conducted worldwide, including age, discipline and academic position.5 ,8 ,10 While a similar pattern of performance curve by age was found in the literature and corroborates our findings, this result should be cautiously interpreted because it could reflect exogenous factors occurring at career milestones, such as shifts in job security or responsibilities. Previous work found a more important decline of scientific production at the end of career because publication volume was not adjusted for birth cohort and older researchers belonged to the oldest generation.7 One reason for this age-related decline in scientific achievement might be that a full professor cannot maintain a high level of scientific production passively by accumulating experience, which raises concerns about motivation throughout a career that extends over several decades. In many European countries, academics need publications under their names if they want to reach the rank of full professor, but this pressure to publish disappears once they have reached this goal. The average age for achieving the position of professor at our institution was closer to 45 than 55 years. A potential explanation for the absence of a decline in performance immediately after appointment is that personal status within the academic system also relies on the research funds one acquires, and because even senior professors have to continue publishing if they want to be respected by their colleagues and to continue mentoring top graduate students.17 This late peak and minimal decline in performance at the end of a researchers career is consistent with previous works.18

The shape of the observed performance curve for medical researchers seemed close to the ‘conceptual’ curve proposed by Ericsson in other fields such as chess or music.11 However, this ‘conceptual curve’ had no strong empirical basis and was directed at performance in well-defined task domains, as evidenced previously for systematic care delivery.19 ,20 Considerable evidence shows that creative productivity does not necessarily work the same way for researchers, yielding different expected longitudinal functions.21 There are fundamental shifts in the lifecycle of research productivity, and the frequency of great achievement at a young age would be more a function of time than field. Indeed, independent associations have been found between age dynamics within fields and both the prevalence of theoretical work and the measures of the stock of foundational knowledge.22 In the same way, the generational increase in productivity has been well established in various industrial sectors.1 ,2 ,23 Beyond the broadening in available space for publishing in biomedical journals, the effect of the birth cohort may reveal a growing productivity of researchers whose practices are impacted by more incentives to publish. Indeed, public institutions and research funding tend to prioritise career advancement based on metrics reflecting their publications in peer-reviewed journals.24 It is of note that this secular trend was found for overall publication volume and not the number of publications as first/last authors, which may indicate changing practices across generations towards more collaborations.25 Additionally, assuming that young researchers are most likely not independent and are collaborating with senior researchers already in their maturation or stabilisation phases, this may also represent a virtuous circle with increasing publication capacity over generations. Variations also exist within each generation and researchers who are highly productive in their 30s are also likely to be much more productive in their 60s than researchers who are not very productive at a young age.

Based on routinely collected data from a hospital information system over 20 years, this study established an accurate curve of individual performance among medical researchers during their career. Using this curve to evaluate researchers integrates the need to consider their personal characteristics for a fair interpretation of their scientific production. Indeed, it would be inappropriate to expect from a physician who has just started his/her training to perform similarly as a professor at the peak of his/her career. Each researcher can now follow his/her publication volume over time depending on what is expected in view of his/her experience, academic position and year of birth. Such an approach, both dynamic and researcher-centred, should enable realistic goals to be set by researchers to improve or maintain their performance throughout their career. A further implication regards the organisation of research at the macro level of university hospitals. To date, most of publications are produced by a limited number of professors, while there is a modest contribution of non-academic physicians to research effort in spite of representing most of medical workforce in university hospitals. Rethinking the missions of all medical doctors towards an enlargement of scientific prerogatives would represent a substantial investment at the level of each institution in favour of progress in global knowledge.

To this end, we need tangible elements about the optimal balance between research, teaching and care activities that can be performed by the same person. Although clinical activities may catalyse the emergence of original research ideas, overwhelming investment of medical doctors in patient care reduces even more their time dedicated to science. Spending adequate time in research activities is essential to allow principal investigators to lead creative and well-designed research projects. Better understanding of the effect on scientific production of time spent exclusively for research purposes compared to time spent in administrative tasks or patient care would be of interest to medical researchers and their host institutions. This poses the question of how to prioritise the time of medical researchers to increase their scientific production and the chance of major discoveries without compromising patient care.

References

View Abstract

Footnotes

  • ▸ Additional material is available. To view please visit the journal online (http://dx.doi.org/10.1136/bmjopen-2016-013572).

  • Contributors AD identified the research question, interpreted the results and wrote the manuscript. EH analysed the data and provided a first draft of the manuscript. SP produced the dataset to be analysed and provided a critical revision of the manuscript. MM and OC highlighted some new insights on the results and provided a critical revision of the article. All authors gave a final approval to the article. AD is the guarantor of this work.

  • Competing interests None declared.

  • Ethics approval The study was supported by the medical commission and research department of the host institution. Anonymous access and retrospective analysis of personal data was authorised by the national data protection commission (Commission Nationale de l'Informatique et des Libertés, CNIL; number 15-076), in accordance with the French legislation.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data access, responsibility, and analysis Authors had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. AD and EH are responsible for the data analysis.

  • Transparency declaration AD affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.

  • Data sharing statement Statistical codes are available from the corresponding author at antoine.duclos@chu-lyon.fr. No additional data are available.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.