You are here

Article Text

Article menu
XML
Surgery
Research
Surgical learning curves and operative efficiency: a cross-specialty observational study
  1. Mahiben Maruthappu1,
  2. Antoine Duclos1,2,
  3. Stuart R Lipsitz1,
  4. Dennis Orgill1,3,
  5. Matthew J Carty1,3
  1. 1Brigham and Women's Hospital Center for Surgery and Public Health, Boston, Massachusetts, USA
  2. 2Hospices Civils de Lyon, Pôle Information Médicale Evaluation Recherche—Université de Lyon, Lyon, France
  3. 3Division of Plastic Surgery, Brigham and Women's Hospital/Faulkner Hospital, Jamaica Plain, Massachusetts, USA
  1. Correspondence to Dr Mahiben Maruthappu; maruthappu{at}post.harvard.edu

Abstract

Objectives To evaluate the learning curves of three high-volume procedures, from distinct surgical specialties.

Setting Tertiary care academic hospital.

Participants A prospectively collected database comprising all medical records of patients undergoing isolated coronary artery bypass grafting (CABG), total knee replacement (TKR) and bilateral reduction mammoplasty (BRM) at the Brigham and Women's Hospital, USA, 1996–2010. Multivariate generalised estimating equation (GEE) regression models were used to adjust for patient risk and clustering of procedures by surgeon.

Primary outcome measure Operative efficiency.

Results A total of 1052 BRMs, 3254 CABGs and 3325 TKRs performed by 30 surgeons were analysed. Median number of procedures per surgeon was 61 (range 11–502), 290 (52–973) and 99 (10–1871) for BRM, CABG and TKR, respectively. Mean operative times were 134.4 (SD 34.5), 180.9 (62.3) and 101.9 (30.3) minutes, respectively. For each procedure, attending surgeon experience was associated with significant reductions in operative time (p<0.05). After 15 years of experience, BRM operative time decreased by 69.8 min (38.3%), CABG operative time decreased by 17.5 min (7.8%) and TKR operative time decreased by 94.4 min (48.4%).

Conclusions Common trends in surgical learning exist. Dependent on the procedure, experience can serve as a powerful driver of improvement or have clinically insignificant impacts on operative time.

  • SURGERY
  • PUBLIC HEALTH
  • MEDICAL EDUCATION & TRAINING

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

https://doi.org/10.1136/bmjopen-2014-006679

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • Use of high-granularity data on over 7000 procedures, performed by 30 surgeons.

    • Robust statistical demonstration of the effect of experience on operative efficiency.

    • Single centre, retrospective and focused on operative time, which although having clear relevance to operative efficiency and financial costs, is not a clear patient-centred outcome.

    Introduction

    Performance measurement and the application of quality, efficiency and safety metrics in surgery has dramatically expanded over the past two decades. This has met with variable degrees of compliance and success. The operating room (OR) represents a high-risk environment, requiring the coordination of technology, competence and resources under time pressure by the operating team.1 ,2 When combined with the technical and dexterous nature of surgery, it is apparent that several factors must be considered and accounted for to make accurate, equitable and comparable measurements of performance between surgeons. One of these factors is the learning curve inherent to operative procedures.3–8

    Until recently, studies have lacked adequate data volume and resolution to gain an appropriate understanding of procedural dynamics, or to robustly identify suitable performance metrics. Work thus far has predominantly focused on surgery at the institutional or departmental level, rather than that of the individual surgeon. Furthermore, no studies to date have compared learning curves across different surgical specialties using statistically uniform methods.8

    We performed a cross-specialty, comparative study evaluating the learning curves of three high-volume and well-defined procedures: bilateral reduction mammoplasty (BRM), isolated coronary artery bypass grafting (CABG) and total knee replacement (TKR). Our intent was to define commonalities and discrepancies in the learning curves of these procedures, and thereby identify overarching themes that may inform future surgical performance improvement efforts.

    Methods

    Design and population

    Data for all CABG, TKR and BRM procedures performed at Brigham and Women's Hospital (BWH), 2001–2010, 1996–2009 and 1995–2007, respectively, were culled from a combination of electronic medical records, an electronic operative time tracking application and physician employee databases. The data sets were filtered to exclude CABGs that were accompanied by concomitant procedures (eg, valve surgery, MAZE, etc); erroneously coded procedures including partial knee replacements, combined procedures and bilateral TKR procedures; gynaecomastia, resections and partial/total mastectomies; and incomplete or incorrect data entries (details of the exclusion process are included in the online supplementary appendix). Data sets were subsequently filtered to only include surgeons who conducted more than 10 procedures annually, focusing on the first 20 years of the learning curve.

    For CABG, the primary dependent variable was operative time defined as the sum of cardiopulmonary bypass time (CPB) and cross clamp time. For TKR and BRM, operative time was defined as the time elapsed from skin incision to skin closure. The operative experience of the attending surgeon was calculated as the difference between the date of the procedure and that of the surgeon's completion of training. This study was performed under Institutional Review Board approval (protocol 2006p000586).

    Statistical analysis

    The characteristics of patients and surgeons were described using absolute frequencies with percentages for categorical variables. The mean values with SDs and median values with minimum–maximum intervals were calculated for continuous variables.

    The expected learning curve of surgeons over time was generated based on a multivariate generalised estimating equations (GEE) regression model. A number of possible shapes of learning curves were tested in order to obtain the best fitting.9 Outcomes were adjusted for clustering of patients by surgeon, as well as patient case-mix factors (for CABG: patient age, sex, preoperative cardiogenic shock, diabetes mellitus and preoperative congestive heart failure; for TKR: patient age, sex and comorbidities including coronary artery disease, chronic obstructive pulmonary disease, diabetes mellitus, hypertension, obesity and smoking; for BRM: volume of breast reduction).9 Selection of covariates for generation of case-mix models was achieved using the approach described by Collett.10 Model estimates were obtained using the GENMOD procedures in SAS V.9.2 (SAS Institute Inc, Cary, North Carolina, USA). The expected reductions in operative time associated with attending experience were plotted. All tests were two-tailed, and p values <0.05 were considered significant.

    Results

    Surgeon and cohort characteristics for each procedure are listed in tables 1 and 2. Figure 1 displays the learning curve of each of the procedures. The number of procedures per surgeon per year are displayed in the online supplementary appendix.

    View this table:
    Table 1

    Overview of study participants

    View this table:
    Table 2

    Surgeon and statistical model characteristics

    Figure 1
    Figure 1

    Learning curves showing bilateral reduction mammoplasty (BRM), coronary artery bypass grafting (CABG) and total knee replacement (TKR) operative times versus surgeon experience.

    A total of 1052 BRMs were completed by eight surgeons. Mean attending experience was 11.3±4.5 years, with a median case volume of 61 (range: 11–502). Mean operative time was 134.4±34.5 min. Multivariate regression showed breast reduction volume to be the only case-mix factor to have a significant effect on operative time. After adjusting for patient characteristics and clustering, we found a significant association between operative time and attending surgeon experience entered both as a linear (p=0.0002) and a quadratic (p=0.0427) term in the model. Five, 10 and 15 years of experience were associated with 33.74, 57.02 and 69.82 min reductions in operative time, respectively, equating to 18.51%, 31.28% and 38.30% reductions (table 3).

    View this table:
    Table 3

    Learning curve—reductions in operative time with surgical experience

    A total of 3254 CABGs were completed by nine surgeons. Mean attending experience was 9.7±5.9 years, with a median case volume of 290 (range: 52–973). Mean operative time was 180.9±62.3 min. Multivariate regression showed the following case-mix factors to have a significant effect on operative time: patient age, sex, preoperative cardiogenic shock, diabetes mellitus and preoperative congestive heart failure. After adjusting for patient characteristics and clustering, we found a significant association between operative time and attending surgeon experience entered as a logarithmic term (p=0.0374) in the model. Five, 10 and 15 years of experience were associated with 5.85, 11.69 and 17.54 min reductions in operative time, respectively, equating to 2.61%, 5.22% and 7.83% reductions (table 3).

    A total of 3325 TKRs were completed by 13 surgeons. Mean attending experience was 14.5±3.8 years, with a median case volume of 99 (range: 10–1871). Mean operative time was 101.9±30.3 min. Multivariate regression showed the following case-mix factors to have a significant effect on operative time: patient age, sex and comorbidities including coronary artery disease, chronic obstructive pulmonary disease, diabetes mellitus, hypertension, obesity and smoking. After adjusting for patient characteristics and clustering, we found a significant association between operative time and attending surgeon experience entered both as a linear (p<0.0001) and a quadratic (p=0.0002) term in the model. Five, 10 and 15 years of experience were associated with 38.20, 69.68 and 94.41 min reductions in operative time, respectively, equating to 19.60%, 35.75% and 48.44% reductions (table 3).

    Discussion

    We performed a cross-specialty evaluation of learning curves in surgery, investigating over 7000 procedures performed by 30 attending surgeons. By comparing these learning curves using statistically uniform data and techniques, our study permits identification of important commonalities and differentiations, yielding several important findings.

    First, in contrast to the notion that efficiency is optimised within a fairly narrow temporal window following the start of clinical practice, our data suggest that operative learning curves, for some procedures, exhibit ongoing improvement in efficiency over the course of a surgeon's career, with time courses much longer than previously anticipated. This emphasises the necessity to draw equitable comparisons between surgeons at similar stages of the learning curve,11 ,12 and supports proposals for continual monitoring, training and behavioural interventions aiming to accelerate operative maturation.13

    Second, our results demonstrate the different learning curve dynamics that exist between procedures. BRM is typified by an initial phase of variability, followed by a period of rapid improvement, followed by a relative plateau phase. TKR and CABG, on the other hand, demonstrate a more linear improvement over time. Such findings suggest that certain procedures may demonstrate characteristic learning curves, with some achieving maturation more rapidly than others. The factors contributing to these characteristics should be the subject of further investigation.

    Third, the magnitude of improvements in efficiency over time varies from procedure to procedure. In some cases, efficiency is markedly augmented with increasing surgeon experience (BRM and TKR); in others, however, the improvement is substantially more marginal, and possibly clinically insignificant (CABG; <10% reduction after 15 years of experience). This observation may be because cardiothoracic fellows begin their training as proficient and highly competent general surgery graduates. As demonstrated by previous studies, this may result in attendings achieving a significant portion of their maturation as CABG surgeons during their cardiothoracic fellowship,14–16 diminishing the impact of subsequent increases in experience. Superimposed on this is the possibility that plastic and orthopaedic surgeons may learn a broader range of procedures during their training than cardiothoracic surgeons, precluding as much experience to be gained in a given operative technique—a difference of breadth and depth. This observation suggests that interventions aiming to accelerate surgical experience acquisition, such as simulation training, may be particularly effective for enhancing efficiency in TKR and BRM, but possibly less so in CABG, given the differential impact of individual surgeon experience across these procedures.

    In addition to this, procedural learning curves were found to have differential sensitivities to case-mix factors. BRM was only affected by breast reduction volume; in contrast, the TKR learning curve was found to be influenced by a variety of patient-factors including age, sex and comorbidities. This differential is no doubt due, at least in part, to varying tolerances on the part of surgeons in each of the highlighted specialties to operate on patients with marginal health status; this must be taken into consideration when making cross-procedural comparisons.

    Importantly, the development of experience-based learning curves, such as those elucidated in this study, has the potential to inform efforts to prospectively monitor individual surgical performance metrics on an ongoing basis. Through the application of iterative quality improvement systems, such as statistical process control (SPC), the depicted learning curves could be utilised as a baseline against which to chart the progress of surgical personnel—whether to meet administrative or educational goals.17 ,18 Such methodologies are designed to first identify and then address unwarranted variation in clinical processes, which generally leads to substantial improvements in efficiency, safety and quality. While initially conceived for the manufacturing industry, they have recently been recognised as having increasing relevance to the healthcare sphere.19 ,20 Compared to more traditional approaches of quality control, they offer a more accurate means of assessing performance and guiding improvement initiatives, particularly when they are, as in the case of the data presented here, tailored to a specific operative procedure.11 ,12

    Implementation of SPC offers several potential benefits. Through improvement of surgical performance, these efforts have the potential to simultaneously improve surgical efficiency, safety, quality and cost at the individual as well as institutional levels. In the context of training, mapping of performance throughout a surgeon's career will permit the evolution of young trainees to be monitored, giving rise to appraisals based on performance rather than career chronology alone, potentially ensuring progression only on acquisition of sufficient expertise.3 ,11 ,12 ,21 Deconstruction of the performance curve and comparison of curves between high and low performers will permit the elements contributing to training to be dissected. By doing so, the mechanics of surgical learning will be better understood, providing methods by which the learning curve can be accelerated. SPC methodology also presents a means of accurately assessing the efficacy of new training tools or simulation programmes at the individual level, both rapidly and sensitively. Finally, this methodology may have the capacity to identify deteriorations occurring towards the end of a surgeon's career, serving as a safety monitor and supplementing the introduction of continuing education programmes.4 ,22

    These implications, however, must be considered in the context of this study's limitations:

    1. Our focus on operative efficiency as an outcome for CABG, TKR and BRM did not explicitly incorporate considerations related to surgical safety such as complication rates, which may be considered more patient focused. However, in a variety of work both within surgery and outside, time of task completion has been used as a robust indicator of learning and outcome.3 ,23–27 Studies have indicated that faster completion of the TKR procedure is associated with better outcomes.28 Further, operative times in CABG are closely linked with patient outcome; specifically, elevated CPB times are associated with increased risks of stroke and renal failure.29

    2. We performed a retrospective investigation at a single academic medical centre, which may be criticised for limiting data quality. This approach, however, removed any Hawthorne effect with regard to efficiency assessment. We also did not account for changes in technology, operative technique or local hospital resources occurring during the retrospective capture of our analysis.

    3. Our investigation utilised years of training and/or practice as a proxy for surgical experience, rather than number of cases performed. The latter may have permitted volume-efficiency relationships to be better elucidated, and for the specific number of cases required to reach maturation to be determined. This limitation is due to our inability to capture procedures performed at other institutions by surgeons in our study cohort, either before or during the study phase; for the TKR and BRM data sets, several of the surgeons included in the data set performed procedures elsewhere; for the CABG data set, most of the surgeons operated exclusively at the host institution (with the exception of one). However, number of years of training and/or practice has been utilised as an acceptable substitute for surgical experience in prior published studies in the surgical literature.3 ,4 ,8 ,11 ,12 ,30

    4. Additional controls could have been incorporated into the model, such as team familiarity, team experience, fellowship-status and the presence of a resident. Although cross-specialty analyses of such work have yet to be conducted, evidence exists at the procedure-specific level.16 ,30

    This study quantitatively characterised the learning curves of three procedures from distinct surgical specialties. We identified common trends in surgical learning and demonstrated that, dependent on the procedure, surgical experience can serve as a powerful driver of improvement or have clinically insignificant impacts on efficiency. Appreciation of these findings may guide implementation of performance-tracking and quality-improvement strategies in surgery.

    References

    1. ↵
      1. Catchpole K,
      2. Mishra A,
      3. Handa A, et al
      . Teamwork and error in the operating room: analysis of skills and roles. Ann Surg 2008;247:699–706. doi:10.1097/SLA.0b013e3181642ec8
      OpenUrlCrossRefPubMedWeb of Science
    2. ↵
      1. Leape LL
      . Error in medicine. JAMA 1994;272:1851–7. doi:10.1001/jama.1994.03520230061039
      OpenUrlCrossRefPubMedWeb of Science
    3. ↵
      1. Carty MJ,
      2. Chan R,
      3. Huckman R, et al
      . A detailed analysis of the reduction mammoplasty learning curve: a statistical process model for approaching surgical performance improvement. Plast Reconstr Surg 2009;124:706–14. doi:10.1097/PRS.0b013e3181b17a13
      OpenUrlCrossRefPubMed
    4. ↵
      1. Duclos A,
      2. Peix JL,
      3. Colin C, et al
      . Influence of experience on performance of individual surgeons in thyroid surgery: prospective cross sectional multicentre study. BMJ 2012;344:d8041. doi:10.1136/bmj.d8041
      OpenUrlAbstract/FREE Full Text
    5. ↵
      1. Harrysson IJ,
      2. Cook J,
      3. Sirimanna P, et al
      . Systematic review of learning curves for minimally invasive abdominal surgery: a review of the methodology of data collection, depiction of outcomes, and statistical analysis. Ann Surg 2014;260:37–45. doi:10.1097/SLA.0000000000000596
      OpenUrl
    6. ↵
      1. Tekkis PP,
      2. Senagore AJ,
      3. Delaney CP, et al
      . Evaluation of the learning curve in laparoscopic colorectal surgery: comparison of right-sided and left-sided resections. Ann Surg 2005;242:83–91. doi:10.1097/01.sla.0000167857.14690.68
      OpenUrlCrossRefPubMedWeb of Science
    7. ↵
      1. Sutton DN,
      2. Wayman J,
      3. Griffin SM
      . Learning curve for oesophageal cancer surgery. Br J Surg 1998;85:1399–402. doi:10.1046/j.1365-2168.1998.00962.x
      OpenUrlCrossRefPubMedWeb of Science
    8. ↵
      1. Maruthappu M,
      2. Gilbert B,
      3. El-Harasis MA, et al
      . The influence of experience on individual surgical performance: a systematic review. Ann Surg 2014. In press.
    9. ↵
      1. Ramsay CR,
      2. Grant AM,
      3. Wallace SA, et al
      . Statistical assessment of the learning curves of health technologies. Health Technol Assess 2001;5:1–79. doi:10.3310/hta5120
      OpenUrlPubMed
    10. ↵
      1. Collett D
      . Modelling survival data in medical research. Boca Raton, LA: CRC, Chapman Hall, 2003.
    11. ↵
      1. Maruthappu M,
      2. Duclos A,
      3. Orgill D, et al
      . A monitoring tool for performance improvement in plastic surgery at the individual level. Plast Reconstr Surg 2013;131:702e–10. doi:10.1097/PRS.0b013e3182865a0c
      OpenUrl
    12. ↵
      1. Duclos A,
      2. Carty MJ,
      3. Peix JL, et al
      . Development of a charting method to monitor the individual performance of surgeons at the beginning of their career. PLoS ONE 2012;7:e41944. doi:10.1371/journal.pone.0041944
      OpenUrlPubMed
    13. ↵
      1. Dawe SR,
      2. Windsor JA,
      3. Broeders JA, et al
      . A systematic review of surgical skills transfer after simulation-based training: laparoscopic cholecystectomy and endoscopy. Ann Surg 2014;259:236–48. doi:10.1097/SLA.0000000000000245
      OpenUrl
    14. ↵
      1. Murzi M,
      2. Cerillo AG,
      3. Bevilacqua S, et al
      . Traversing the learning curve in minimally invasive heart valve surgery: a cumulative analysis of an individual surgeon's experience with a right minithoracotomy approach for aortic valve replacement.
    15. ↵
      1. Song M-H,
      2. Tajima K,
      3. Watanabe T, et al
      . Learning curve of coronary surgery by a cardiac surgeon in Japan with the use of cumulative sum analysis. Jpn J Thorac Cardiovasc Surg 2005;53:551–6. doi:10.1007/s11748-005-0066-y
      OpenUrlPubMed
    16. ↵
      1. ElBardissi AW,
      2. Duclos A,
      3. Rawn JD, et al
      . Cumulative team experience matters more than individual surgeon experience in cardiac surgery. J Thorac Cardiovasc Surg 2013;145:328–33. doi:10.1016/j.jtcvs.2012.09.022
      OpenUrlCrossRefPubMedWeb of Science
    17. ↵
      1. Wheeler DJ,
      2. Chambers DS
      . Understanding statistical process control. Knoxville, Tenn: SPC Press, 1992.
    18. ↵
      1. Duclos A,
      2. Touzet S,
      3. Soardo P, et al
      . Quality monitoring in thyroid surgery using the Shewhart control chart. Br J Surg 2009;96:171–4. doi:10.1002/bjs.6418
      OpenUrlCrossRefPubMedWeb of Science
    19. ↵
      1. Tekkis PP,
      2. McCulloch P,
      3. Steger AC, et al
      . Mortality control charts for comparing performance of surgical units: validation study using hospital mortality data. BMJ 2003;326:786. doi:10.1136/bmj.326.7393.786
      OpenUrlAbstract/FREE Full Text
    20. ↵
      1. Duclos A,
      2. Polazzi S,
      3. Lipsitz SR, et al
      . Temporal variation in surgical mortality within French hospitals. Med Care 2013;51:1085–93. doi:10.1097/MLR.0b013e3182a97c54
      OpenUrl
    21. ↵
      1. Maruthappu M,
      2. Carty MJ,
      3. Lipsitz SR, et al
      . Patient- and surgeon-adjusted control charts for monitoring performance. BMJ Open 2014;4:e004046. doi:10.1136/bmjopen-2013-004046
      OpenUrlAbstract/FREE Full Text
    22. ↵
      1. Neumayer LA,
      2. Gawande AA,
      3. Wang J, Jr., et al
      . Proficiency of surgeons in inguinal hernia repair: effect of experience and age. Ann Surg 2005;242:344–52.
      OpenUrlPubMedWeb of Science
    23. ↵
      1. Pisano GP,
      2. Bohmer MJ,
      3. Edmondson AC
      . Organizational differences in rates of learning: evidence from the adoption of minimally invasive cardiac surgery. Manag Sci 2001;47:752–68. doi:10.1287/mnsc.47.6.752.9811
      OpenUrlCrossRef
    24. ↵
      1. Edmondson A,
      2. Winslow A,
      3. Bohmer R, et al
      . Learning how and learning what: effects of tacit and codified knowledge on performance improvement following technology adoption. Decis Sci 2003;34:197–223. doi:10.1111/1540-5915.02316
      OpenUrlCrossRefWeb of Science
    25. ↵
      1. Epple D,
      2. Argote L,
      3. Devadas R
      . Organizational learning curves: a method for investigating intra-plant transfer of knowledge acquired through learning by doing. Organ Sci 1991;2:58–70. doi:10.1287/orsc.2.1.58
      OpenUrlCrossRef
    26. ↵
      1. Argote L,
      2. Epple D
      . Learning curves in manufacturing. Science 1990;247:920–4. doi:10.1126/science.247.4945.920
      OpenUrlAbstract/FREE Full Text
    27. ↵
      1. Ballantyne GH,
      2. Ewing D,
      3. Capella RF, et al
      . The learning curve measured by operating times for laparoscopic and open gastric bypass: roles of surgeon's experience, institutional experience, body mass index and fellowship training. Obes Surg 2005;15:172–82. doi:10.1381/0960892053268507
      OpenUrlCrossRefPubMedWeb of Science
    28. ↵
      1. Peersman G,
      2. Laskin R,
      3. Davis J, et al
      . Prolonged operative time correlates with increased infection rate after total knee arthroplasty. HSS J 2006;2:70–2. doi:10.1007/s11420-005-0130-2
      OpenUrlCrossRefPubMed
    29. ↵
      1. Sniecinski R,
      2. Chandler W
      . Activation of the hemostatic system during cardiopulmonary bypass. Anesth Analg 2011;113:1319–33. doi:10.1213/ANE.0b013e3182354b7e
      OpenUrlCrossRefPubMedWeb of Science
    30. ↵
      1. Xu R,
      2. Carty MJ,
      3. Orgill DP, et al
      . The teaming curve: a longitudinal study of the influence of surgical team familiarity on operative time. Ann Surg 2013;258:953–7. doi:10.1097/SLA.0b013e3182864ffe
      OpenUrlCrossRefPubMed

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

      Files in this Data Supplement:

    Footnotes

    • Contributors AD, DPO, MJC, MM, SRL contributed by taking part in study concept and design. MJC, MM were involved in data collection. AD and SRL analysed the data. AD, MJC and MM participated in manuscript preparation. AD, DPO, MJC, MM and SRL revised the manuscript.

    • Funding This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

    • Competing interests None.

    • Ethics approval Ethics approval was granted by the Institutional Review Board (protocol number 2006p000586).

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement No additional data are available.