Article Text

PDF

Surgical learning curves and operative efficiency: a cross-specialty observational study
  1. Mahiben Maruthappu1,
  2. Antoine Duclos1,2,
  3. Stuart R Lipsitz1,
  4. Dennis Orgill1,3,
  5. Matthew J Carty1,3
  1. 1Brigham and Women's Hospital Center for Surgery and Public Health, Boston, Massachusetts, USA
  2. 2Hospices Civils de Lyon, Pôle Information Médicale Evaluation Recherche—Université de Lyon, Lyon, France
  3. 3Division of Plastic Surgery, Brigham and Women's Hospital/Faulkner Hospital, Jamaica Plain, Massachusetts, USA
  1. Correspondence to Dr Mahiben Maruthappu; maruthappu{at}post.harvard.edu

Abstract

Objectives To evaluate the learning curves of three high-volume procedures, from distinct surgical specialties.

Setting Tertiary care academic hospital.

Participants A prospectively collected database comprising all medical records of patients undergoing isolated coronary artery bypass grafting (CABG), total knee replacement (TKR) and bilateral reduction mammoplasty (BRM) at the Brigham and Women's Hospital, USA, 1996–2010. Multivariate generalised estimating equation (GEE) regression models were used to adjust for patient risk and clustering of procedures by surgeon.

Primary outcome measure Operative efficiency.

Results A total of 1052 BRMs, 3254 CABGs and 3325 TKRs performed by 30 surgeons were analysed. Median number of procedures per surgeon was 61 (range 11–502), 290 (52–973) and 99 (10–1871) for BRM, CABG and TKR, respectively. Mean operative times were 134.4 (SD 34.5), 180.9 (62.3) and 101.9 (30.3) minutes, respectively. For each procedure, attending surgeon experience was associated with significant reductions in operative time (p<0.05). After 15 years of experience, BRM operative time decreased by 69.8 min (38.3%), CABG operative time decreased by 17.5 min (7.8%) and TKR operative time decreased by 94.4 min (48.4%).

Conclusions Common trends in surgical learning exist. Dependent on the procedure, experience can serve as a powerful driver of improvement or have clinically insignificant impacts on operative time.

Statistics from Altmetric.com

Strengths and limitations of this study

  • Use of high-granularity data on over 7000 procedures, performed by 30 surgeons.

  • Robust statistical demonstration of the effect of experience on operative efficiency.

  • Single centre, retrospective and focused on operative time, which although having clear relevance to operative efficiency and financial costs, is not a clear patient-centred outcome.

Introduction

Performance measurement and the application of quality, efficiency and safety metrics in surgery has dramatically expanded over the past two decades. This has met with variable degrees of compliance and success. The operating room (OR) represents a high-risk environment, requiring the coordination of technology, competence and resources under time pressure by the operating team.1 ,2 When combined with the technical and dexterous nature of surgery, it is apparent that several factors must be considered and accounted for to make accurate, equitable and comparable measurements of performance between surgeons. One of these factors is the learning curve inherent to operative procedures.3–8

Until recently, studies have lacked adequate data volume and resolution to gain an appropriate understanding of procedural dynamics, or to robustly identify suitable performance metrics. Work thus far has predominantly focused on surgery at the institutional or departmental level, rather than that of the individual surgeon. Furthermore, no studies to date have compared learning curves across different surgical specialties using statistically uniform methods.8

We performed a cross-specialty, comparative study evaluating the learning curves of three high-volume and well-defined procedures: bilateral reduction mammoplasty (BRM), isolated coronary artery bypass grafting (CABG) and total knee replacement (TKR). Our intent was to define commonalities and discrepancies in the learning curves of these procedures, and thereby identify overarching themes that may inform future surgical performance improvement efforts.

Methods

Design and population

Data for all CABG, TKR and BRM procedures performed at Brigham and Women's Hospital (BWH), 2001–2010, 1996–2009 and 1995–2007, respectively, were culled from a combination of electronic medical records, an electronic operative time tracking application and physician employee databases. The data sets were filtered to exclude CABGs that were accompanied by concomitant procedures (eg, valve surgery, MAZE, etc); erroneously coded procedures including partial knee replacements, combined procedures and bilateral TKR procedures; gynaecomastia, resections and partial/total mastectomies; and incomplete or incorrect data entries (details of the exclusion process are included in the online supplementary appendix). Data sets were subsequently filtered to only include surgeons who conducted more than 10 procedures annually, focusing on the first 20 years of the learning curve.

For CABG, the primary dependent variable was operative time defined as the sum of cardiopulmonary bypass time (CPB) and cross clamp time. For TKR and BRM, operative time was defined as the time elapsed from skin incision to skin closure. The operative experience of the attending surgeon was calculated as the difference between the date of the procedure and that of the surgeon's completion of training. This study was performed under Institutional Review Board approval (protocol 2006p000586).

Statistical analysis

The characteristics of patients and surgeons were described using absolute frequencies with percentages for categorical variables. The mean values with SDs and median values with minimum–maximum intervals were calculated for continuous variables.

The expected learning curve of surgeons over time was generated based on a multivariate generalised estimating equations (GEE) regression model. A number of possible shapes of learning curves were tested in order to obtain the best fitting.9 Outcomes were adjusted for clustering of patients by surgeon, as well as patient case-mix factors (for CABG: patient age, sex, preoperative cardiogenic shock, diabetes mellitus and preoperative congestive heart failure; for TKR: patient age, sex and comorbidities including coronary artery disease, chronic obstructive pulmonary disease, diabetes mellitus, hypertension, obesity and smoking; for BRM: volume of breast reduction).9 Selection of covariates for generation of case-mix models was achieved using the approach described by Collett.10 Model estimates were obtained using the GENMOD procedures in SAS V.9.2 (SAS Institute Inc, Cary, North Carolina, USA). The expected reductions in operative time associated with attending experience were plotted. All tests were two-tailed, and p values <0.05 were considered significant.

Results

Surgeon and cohort characteristics for each procedure are listed in tables 1 and 2. Figure 1 displays the learning curve of each of the procedures. The number of procedures per surgeon per year are displayed in the online supplementary appendix.

Table 1

Overview of study participants

Table 2

Surgeon and statistical model characteristics

Figure 1

Learning curves showing bilateral reduction mammoplasty (BRM), coronary artery bypass grafting (CABG) and total knee replacement (TKR) operative times versus surgeon experience.

A total of 1052 BRMs were completed by eight surgeons. Mean attending experience was 11.3±4.5 years, with a median case volume of 61 (range: 11–502). Mean operative time was 134.4±34.5 min. Multivariate regression showed breast reduction volume to be the only case-mix factor to have a significant effect on operative time. After adjusting for patient characteristics and clustering, we found a significant association between operative time and attending surgeon experience entered both as a linear (p=0.0002) and a quadratic (p=0.0427) term in the model. Five, 10 and 15 years of experience were associated with 33.74, 57.02 and 69.82 min reductions in operative time, respectively, equating to 18.51%, 31.28% and 38.30% reductions (table 3).

Table 3

Learning curve—reductions in operative time with surgical experience

A total of 3254 CABGs were completed by nine surgeons. Mean attending experience was 9.7±5.9 years, with a median case volume of 290 (range: 52–973). Mean operative time was 180.9±62.3 min. Multivariate regression showed the following case-mix factors to have a significant effect on operative time: patient age, sex, preoperative cardiogenic shock, diabetes mellitus and preoperative congestive heart failure. After adjusting for patient characteristics and clustering, we found a significant association between operative time and attending surgeon experience entered as a logarithmic term (p=0.0374) in the model. Five, 10 and 15 years of experience were associated with 5.85, 11.69 and 17.54 min reductions in operative time, respectively, equating to 2.61%, 5.22% and 7.83% reductions (table 3).

A total of 3325 TKRs were completed by 13 surgeons. Mean attending experience was 14.5±3.8 years, with a median case volume of 99 (range: 10–1871). Mean operative time was 101.9±30.3 min. Multivariate regression showed the following case-mix factors to have a significant effect on operative time: patient age, sex and comorbidities including coronary artery disease, chronic obstructive pulmonary disease, diabetes mellitus, hypertension, obesity and smoking. After adjusting for patient characteristics and clustering, we found a significant association between operative time and attending surgeon experience entered both as a linear (p<0.0001) and a quadratic (p=0.0002) term in the model. Five, 10 and 15 years of experience were associated with 38.20, 69.68 and 94.41 min reductions in operative time, respectively, equating to 19.60%, 35.75% and 48.44% reductions (table 3).

Discussion

We performed a cross-specialty evaluation of learning curves in surgery, investigating over 7000 procedures performed by 30 attending surgeons. By comparing these learning curves using statistically uniform data and techniques, our study permits identification of important commonalities and differentiations, yielding several important findings.

First, in contrast to the notion that efficiency is optimised within a fairly narrow temporal window following the start of clinical practice, our data suggest that operative learning curves, for some procedures, exhibit ongoing improvement in efficiency over the course of a surgeon's career, with time courses much longer than previously anticipated. This emphasises the necessity to draw equitable comparisons between surgeons at similar stages of the learning curve,11 ,12 and supports proposals for continual monitoring, training and behavioural interventions aiming to accelerate operative maturation.13

Second, our results demonstrate the different learning curve dynamics that exist between procedures. BRM is typified by an initial phase of variability, followed by a period of rapid improvement, followed by a relative plateau phase. TKR and CABG, on the other hand, demonstrate a more linear improvement over time. Such findings suggest that certain procedures may demonstrate characteristic learning curves, with some achieving maturation more rapidly than others. The factors contributing to these characteristics should be the subject of further investigation.

Third, the magnitude of improvements in efficiency over time varies from procedure to procedure. In some cases, efficiency is markedly augmented with increasing surgeon experience (BRM and TKR); in others, however, the improvement is substantially more marginal, and possibly clinically insignificant (CABG; <10% reduction after 15 years of experience). This observation may be because cardiothoracic fellows begin their training as proficient and highly competent general surgery graduates. As demonstrated by previous studies, this may result in attendings achieving a significant portion of their maturation as CABG surgeons during their cardiothoracic fellowship,14–16 diminishing the impact of subsequent increases in experience. Superimposed on this is the possibility that plastic and orthopaedic surgeons may learn a broader range of procedures during their training than cardiothoracic surgeons, precluding as much experience to be gained in a given operative technique—a difference of breadth and depth. This observation suggests that interventions aiming to accelerate surgical experience acquisition, such as simulation training, may be particularly effective for enhancing efficiency in TKR and BRM, but possibly less so in CABG, given the differential impact of individual surgeon experience across these procedures.

In addition to this, procedural learning curves were found to have differential sensitivities to case-mix factors. BRM was only affected by breast reduction volume; in contrast, the TKR learning curve was found to be influenced by a variety of patient-factors including age, sex and comorbidities. This differential is no doubt due, at least in part, to varying tolerances on the part of surgeons in each of the highlighted specialties to operate on patients with marginal health status; this must be taken into consideration when making cross-procedural comparisons.

Importantly, the development of experience-based learning curves, such as those elucidated in this study, has the potential to inform efforts to prospectively monitor individual surgical performance metrics on an ongoing basis. Through the application of iterative quality improvement systems, such as statistical process control (SPC), the depicted learning curves could be utilised as a baseline against which to chart the progress of surgical personnel—whether to meet administrative or educational goals.17 ,18 Such methodologies are designed to first identify and then address unwarranted variation in clinical processes, which generally leads to substantial improvements in efficiency, safety and quality. While initially conceived for the manufacturing industry, they have recently been recognised as having increasing relevance to the healthcare sphere.19 ,20 Compared to more traditional approaches of quality control, they offer a more accurate means of assessing performance and guiding improvement initiatives, particularly when they are, as in the case of the data presented here, tailored to a specific operative procedure.11 ,12

Implementation of SPC offers several potential benefits. Through improvement of surgical performance, these efforts have the potential to simultaneously improve surgical efficiency, safety, quality and cost at the individual as well as institutional levels. In the context of training, mapping of performance throughout a surgeon's career will permit the evolution of young trainees to be monitored, giving rise to appraisals based on performance rather than career chronology alone, potentially ensuring progression only on acquisition of sufficient expertise.3 ,11 ,12 ,21 Deconstruction of the performance curve and comparison of curves between high and low performers will permit the elements contributing to training to be dissected. By doing so, the mechanics of surgical learning will be better understood, providing methods by which the learning curve can be accelerated. SPC methodology also presents a means of accurately assessing the efficacy of new training tools or simulation programmes at the individual level, both rapidly and sensitively. Finally, this methodology may have the capacity to identify deteriorations occurring towards the end of a surgeon's career, serving as a safety monitor and supplementing the introduction of continuing education programmes.4 ,22

These implications, however, must be considered in the context of this study's limitations:

  1. Our focus on operative efficiency as an outcome for CABG, TKR and BRM did not explicitly incorporate considerations related to surgical safety such as complication rates, which may be considered more patient focused. However, in a variety of work both within surgery and outside, time of task completion has been used as a robust indicator of learning and outcome.3 ,23–27 Studies have indicated that faster completion of the TKR procedure is associated with better outcomes.28 Further, operative times in CABG are closely linked with patient outcome; specifically, elevated CPB times are associated with increased risks of stroke and renal failure.29

  2. We performed a retrospective investigation at a single academic medical centre, which may be criticised for limiting data quality. This approach, however, removed any Hawthorne effect with regard to efficiency assessment. We also did not account for changes in technology, operative technique or local hospital resources occurring during the retrospective capture of our analysis.

  3. Our investigation utilised years of training and/or practice as a proxy for surgical experience, rather than number of cases performed. The latter may have permitted volume-efficiency relationships to be better elucidated, and for the specific number of cases required to reach maturation to be determined. This limitation is due to our inability to capture procedures performed at other institutions by surgeons in our study cohort, either before or during the study phase; for the TKR and BRM data sets, several of the surgeons included in the data set performed procedures elsewhere; for the CABG data set, most of the surgeons operated exclusively at the host institution (with the exception of one). However, number of years of training and/or practice has been utilised as an acceptable substitute for surgical experience in prior published studies in the surgical literature.3 ,4 ,8 ,11 ,12 ,30

  4. Additional controls could have been incorporated into the model, such as team familiarity, team experience, fellowship-status and the presence of a resident. Although cross-specialty analyses of such work have yet to be conducted, evidence exists at the procedure-specific level.16 ,30

This study quantitatively characterised the learning curves of three procedures from distinct surgical specialties. We identified common trends in surgical learning and demonstrated that, dependent on the procedure, surgical experience can serve as a powerful driver of improvement or have clinically insignificant impacts on efficiency. Appreciation of these findings may guide implementation of performance-tracking and quality-improvement strategies in surgery.

References

View Abstract
  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Files in this Data Supplement:

Footnotes

  • Contributors AD, DPO, MJC, MM, SRL contributed by taking part in study concept and design. MJC, MM were involved in data collection. AD and SRL analysed the data. AD, MJC and MM participated in manuscript preparation. AD, DPO, MJC, MM and SRL revised the manuscript.

  • Funding This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests None.

  • Ethics approval Ethics approval was granted by the Institutional Review Board (protocol number 2006p000586).

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data sharing statement No additional data are available.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.