Article Text
PDF
Abstract
Purpose Standardised Training of Paediatric Resident (STPR) plays an essential role in training qualified paediatricians. Until now, China had no paediatric resident competency index system to effectively guide and evaluate the competence of paediatric residents. This study aimed to establish a competency index system for paediatric residents in China to provide a reference for improving the training system and quality of STPR.
Study design and setting This study conducted two rounds of Delphi expert consultation survey among paediatric medical experts (n=16), followed by screening, revising and supplementing indicators using the boundary value method. Next, the analytic hierarchy process was used to determine the weight of indicators and finally establish a competency index system for paediatric residents.
Results The results of the statistical analysis revealed a positive coefficient of 100% for both rounds of expert consultation. The expert authority coefficient values were 0.82 and 0.83, and the expert coordination coefficient test was p<0.01. After referring to experts’ opinions, a competency index system for paediatric residents with 5 primary indicators, 14 secondary indicators and 73 tertiary indicators was finally formed and the weight of each indicator was calculated. The five primary indicators were professional quality (0.3187), knowledge and skills (0.2734), communication and cooperation (0.1986), lifelong learning (0.1302), and teaching ability (0.0791).
Conclusions In this study, a competency index system for paediatric residents was constructed following the characteristics and quality requirements for paediatric residents in China and is expected to significantly improve the overall level of paediatricians’ medical service quality and supply.
- paediatrics
- medical education & training
- statistics & research methods
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This study fully incorporated the indicators of the existing Standardised Training of Paediatric Resident competency model and developed a competency index system suitable to paediatric residents in China.
A behavioural event interview was conducted to collect the opinions of 24 paediatric residents in order to determine actual clinical competencies in China.
A modified Delphi method was conducted with 16 experts to ensure authority and scientific foundation for the competency index system.
An interview to collect competence indicators was not conducted among paediatric patients because they were immature to explain themselves clearly.
The index system has not been implemented in a large sample of paediatric residents, and the reliability and validity of the competency indicators will be further verified by the research team in subsequent research.
Introduction
In China, the level of social and economic development has restricted the supply and demand for paediatric medical services.1 With the introduction of the ‘two-child policy’, shortage in paediatricians has become a prominent social problem. Currently, the number of children aged 0–14 years in China is approximately 235 million, accounting for 16.9% of the total population.2 By the end of 2018, the total number of paediatricians in China was around 230 000. The number of paediatric practising (assistant) physicians per 1000 children was 0.92, while the standard is 1.5 in developed countries.3 With the large paediatric population base in China, paediatric medical resources are insufficient and imbalanced, especially primary paediatric resources.4 Studies have suggested that the lack of stable paediatrician support is the root cause of this problem.5
Standardised training of residents (STR) is an international model designed to train clinicians with good professional ethics, solid medical theoretical knowledge and clinical skills so they can independently provide diagnosis and treatment of common diseases according to standards.6 In 2013, China published the STR policy,7 and ‘paediatrics’ was considered an ‘urgent-need major’ to further expand the Standardised Training of Paediatric Resident (STPR) enrolment scale, thus providing more comprehensive training to clinicians and providing better medical services.8 STPR is a crucial means to ensure enough paediatric residents and train them according to standards. The STPR policy in China needs to be reinforced in training competent paediatricians so they can better provide clinical diagnosis, differential diagnosis, treatment and prognosis.9
The third-generation medical education reform is focused on being patient-centred, system-based and competency-driven.10 Competency-driven STR is an effective measure to promote development among paediatricians. It is indispensable to build an STPR competency index system to better solve clinical problems.9 The Global Pediatric Education Consortium has proposed the Global Pediatric Curriculum and Guidelines for residents and has been recognised worldwide. It defines 10 core competencies for paediatricians.11 The American Board of Pediatrics also developed a paediatric residents’ milestone project based on a model of 6 core competencies for residents,12 including 7 primary indicators (adding personal and professional development capability) and 51 secondary indicators. Internationally, some developed countries have built a unified and preferable training assessment and admissions standard based on their competency index system to ensure standardisation of clinicians’ competence and medical treatment and rapidly develop medical services.13–15
In China, the current core competency definitions for STPR standards are not comprehensive and mainly focus on cultivating residents’ basic clinical knowledge and skills, ignoring humanistic qualities, communication and teamwork, which cannot guide standardised training and assessment. A preliminary study16 indicated that simply emphasising medical knowledge and clinical skills can no longer meet the needs of the complex health system. The deterioration of the doctor–patient relationship and the STPR restrictions in China limit the independent diagnosis and clinical thinking competence of residents.17 The STPR disease system in China does not contain ‘mental and behavioural health’, ‘child abuse and neglect’, ‘dermatology’ and ‘psychosocial issues’.18 Additionally, the medical environment in China is imbalanced, with high-quality resources concentrated in large cities while medical resources in marginal areas are insufficient.19 20 These further emphasised the importance of STPR competency in China.
Different environments will impact healthcare and disease incidence; thus, medical services should be indigenised.21 Cultivating homogeneity among paediatricians is a sustainable way to relieve the pressure of seeking paediatric medical treatments in China. To improve China’s medical specialty environment and training programme, this study aimed to establish a local STPR competency index system to improve the quality of paediatric healthcare services.
Methods
Patient and public involvement
Patients and the public were not involved.
Study design and participants
This study used a modified Delphi method, boundary value method and analytic hierarchy process (AHP) to determine the STPR competency index in China. The modified Delphi method is an effective group consensus consultation that is widely used in the medicine and public health fields.22 23 It uses literature review, ideas from stakeholders and judgement of experts in order to reach agreement.24 The results have high credibility as these are designed and collected through anonymous expert consultation questionnaires.25
The procedures employed in this study are shown in figure 1.
Study design. AHP, analytic hierarchy process.
Building the preliminary indicator pool
A literature review and behavioural event interviews were conducted before the Delphi survey to gather evaluation indicators and descriptions. A literature search was performed in the China National Knowledge Infrastructure (CNKI), WanFang and PubMed databases to gather initial tertiary indicators. The time span was from the establishment of the database to 25 December 2019. The following retrieval strategy was used: ((pediatrician [Title]) *(pediatric resident [Title]) *(resident [Title])) + ((competence [Title]) * (competency [Title])) and ((pediatrician [Title]) *(pediatric resident [Title]) * (resident [Title])) + ((competence [Title]) *(competency [Title])) + ((evaluation [Title/Abstract]) * (assessment [Title/Abstract])). According to the keywords and abstracts, excluding articles which are not relevant to the theme, 262 English references and 199 Chinese references were considered. The Chinese university hospital clinicians’ general competency model,26 the Chinese first residency core competency consensus,27 and the existing STR and STPR competency index systems11–15 were selected as references for this study. After intensive reading of team members, 147 indicators were identified, resulting in 84 indicators after screening according to exclusion criteria.
Meanwhile, behavioural event interviews were conducted among 24 residents who completed STPR at Beijing Children’s Hospital in 2017 and 2018 by purposive sampling, each of whom narrated three successes and three failures during STPR. According to grounded theory, two team members simultaneously encoded the interview text using NVivo V.11 software (QSR International, Melbourne, Australia) and summarised 6 initial primary indicators and 17 initial secondary indicators.
Further, the research team met to classify the 84 tertiary indicators collected through the literature review into primary and secondary indicators, obtained through the behavioural event interview method. Through this process, the three-level preliminary STPR competency index database was formed when consensus was reached without any discrepancies. The preliminary indicator pool was then formed with 6 primary indicators, 17 secondary indicators and 84 tertiary indicators. The primary indicators were (1) professional quality, (2) knowledge and skills, (3) childcare, (4) communication and cooperation, (5) teaching ability, and (6) lifelong learning.
Using the Delphi method to build an index system
The expert Delphi survey included three parts: experts’ basic information, assessment of their familiarity with STPR evaluation, and evaluation of the constructed index system, along with comments. The evaluation focused on the importance, feasibility and sensitivity of each indicator, with scores ranging from 1 to 10. Experts also scored their familiarity and judgement of indicators from 1 to 5 using Likert scale. Judgement scores represent the degree of influence from 1 to 3 (1=small, 2=medium, 3=high) based on theoretical analysis, work experience, understanding of domestic and foreign counterparts, and insight.
For authenticity and comprehensiveness, we selected experts from the STPR assessment specialist group of Health Human Resources Development Center, National Health Commission of the People’s Republic of China. The inclusion criteria were as follows: (1) owning a title of deputy senior or above; (2) more than 10 years of working experience in medical practice, teaching or STPR management; and (3) ability to provide comprehensive opinions and participate in the two rounds of consultations with enthusiasm.
Reasonable Delphi studies were conducted with 10–18 experts to ensure sufficient viewpoints from diverse disciplines.28 29 According to the inclusion criteria, 16 experts were selected from the following institutions: Beijing Children’s Hospital of Capital Medical University (6), Shanghai Children’s Medical Center-Shanghai Jiaotong University School of Medicine (2), Children’s Hospital of Zhejiang University (2), Children’s Hospital of Chongqing Medical University (2), Shengjing Hospital of China Medical University (1), Children’s Hospital of Fudan University (1), Guangzhou Women and Children’s Medical Center (1), and Xi’an Children’s Hospital (1).
The two rounds of the Delphi survey and data analysis were implemented with 16 experts via email. The purpose, significance, content, requirements and precautions of the study were explained in detail, and the experts were asked to evaluate each indicator and provide comments. SPSS V.22.0 was used to calculate the positive coefficient, degree of authority and coordination coefficient of participants in order to demonstrate the validity of the two rounds of Delphi expert consultation. Based on expert feedback from the first round, inappropriate indicators were modified or deleted. The AHP questionnaire was then added to form the revised survey from the second round of expert consultation. Based on expert feedback from the second round, we computed the validity of the Delphi process and used the boundary value method to modify or delete substandard indicators based on discussion with experts. After these steps, the weights of indicators were calculated according to the results of the AHP questionnaire to form the final STPR competency index system.
Using the boundary value method to screen the indicators
We used the boundary values to screen the indicators by full frequency, arithmetic mean and variable coefficient in terms of importance, feasibility and sensitivity. In calculating the full score frequency and arithmetic mean, the boundary value was set to ‘mean−standard deviation’, and the indicators with scores above the boundary value were kept. When the boundary value of the coefficient was set to ‘mean+standard deviation’, the indicators with scores lower than the boundary value were retained.30 31 The results of the second round are shown in table 1.
Indicator screening using boundary values
According to the indicator selection principle, an indicator has two aspects of importance, feasibility and sensitivity, and each aspect has two or more thresholds that do not meet the boundary value. We also discussed the deletion of indicators during the focus group interview with experts in order to arrive at a final decision.
Using AHP to assign weights
To ensure scientific foundation for the index system, after the second round of expert survey the Meshcade software (Nanjing Meshcade Software, Nanjing, China) was used to calculate the weight of the 5 primary indicators and 14 secondary indicators through the AHP method.32 The percentage weight method was then conducted to calculate the weights of tertiary indicators.
Data analysis
Expert positive coefficient
The expert positive coefficient is expressed by the questionnaire recovery rate and reflects the positive input of experts; a recovery rate greater than 70% was considered good.33
In this study, both rounds of Delphi expert surveys were distributed with 16 copies and all were recovered. The effective recovery rate was 100% (16), indicating the expert positive coefficient in both rounds was 100%.
Expert authority coefficient
The expert authority coefficient (Cr) is the arithmetic average of the judgement coefficient (Ca) and familiarity coefficient (Cs), namely: 
, where Cr ≥0.7 means acceptable reliability. The higher the degree of expert authority, the higher the prediction accuracy of the indicators.34 Ca represented the evidence when expert makes a judgement, while Cs represented the degree of the expert’s familiarity with the problem.34
Ca is calculated according to the expert’s judgement and the degree of influence of each indicator. In this study, the experts used the terms ‘practical experience (0.4)’, ‘theoretical analysis (0.3)’, ‘understanding of peers (0.2)’ and ‘insight (0.1)’ as judgement, and the corresponding degree of influence was large (1), medium (0.5) and small (0). The evaluation criteria35 are shown in table 2. Then to sum the evaluation criteria as the Ca of each indicator, when Ca=1, judgement has a great influence on the expert; when Ca=0.5, the influence on expert judgement is moderate; and when Ca=0, no influence is evident on expert judgement.36
Judgement basis and the degree of influence
Cs refers to the degree of the expert’s familiarity with the question. This study used the Likert scale method to assign the expert’s familiarity with the question from 0 to 1 (1=very familiar, 0.75=more familiar, 0.5=generally familiar, 0.25=less familiar, 0=unfamiliar) and to calculate the average degree of familiarity of the consulting expert statistically.
The Cr values from the two rounds of Delphi expert survey were 0.82 and 0.83 (>0.7), indicating the results of expert consultations were accurate and credible.
The degree of expert coordination37 is an important index to judge the consistency of indicators among experts, including Kendall’s W coordination coefficient and each index’s variation coefficient, and the variation coefficient is an important basis for index deletion. Using Kendall’s W coordination coefficient test to judge the degree of expert coordination, p<0.05 was considered statistically significant. The larger the Kendall’s W coefficient, the higher the degree of expert coordination and the higher the consistency of expert opinion.
This study performed Kendall’s W test on the experts’ coordination coefficient for each indicator. The p value of each round was less than 0.01 (see table 3), indicating that expert consultation was consistent.
Coordination factors of expert consultations
Results
Basic information of participants
In this study, all experts had high academic attainment in their respective fields, 14 (87.5%) with master’s degree or higher, 16 (100%) with senior deputy titles and above, and 15 (93.75%) engaged in relevant work for more than 15 years (see table 4).
Demographics of the Delphi survey experts
Index screening
Based on experts’ opinions during the first round of consultation, as the primary indicators the ‘patient care’ could be subsumed under the ‘knowledge and skill’ to form five primary indicators. Six tertiary indicators were deleted (good character, optimistic, arrange treatment plan for children, care of children during illness, prognosis follow-up and reasonable arrangement of priorities), 14 tertiary indicators were merged to 7 indicators, and 3 tertiary indicators were added, forming 14 secondary indicators and 73 tertiary indicators. In the second round of expert consultation, six tertiary indicators were deleted according to the boundary value method (see table 5), while after the focus group interview with experts to remove some indicators and from the perspectives of residents’ actual ability in Chinese medical environments, the experts strongly recommend retaining the indicators (see the Index deletion analysis section). The STPR competency index system was finally formed with 5 primary indicators, 14 secondary indicators and 73 tertiary indicators.
STPR competency index system and weights
Final index system with weights
The AHP method and the per cent weight method were used to determine each indicator’s weight in the STPR competency evaluation index system (see table 5). The primary indicators were as follows: (1) professional quality: possesses responsible medical professionalism, with professional ethics and humanities; (2) knowledge and skills: have sufficient knowledge and skills in basic medicine, clinical medicine and related disciplines, and apply them to clinical practice; (3) communication and cooperation: fully cooperates with children and their families, medical teams and other medical personnel to obtain effective information and preferably provide medical services; (4) teaching ability: actively provides medical education to other medical personnel, children and their families and the public; and (5) lifelong learning: constantly learns, self-reflects and improves in practice to meet the needs of medical development and self-development.
Discussion
The competency index system is significant in that it provides ideas and approaches to training of physicians and to further innovate the system. In this study, the Delphi method was used to construct a competency index system for paediatric residents following the characteristics and quality requirements for paediatric residents in China to better guide STPR in order to achieve the goal of real standardisation in China. Our study proposed a three-level indicator STPR competency index system, of which the classification and description of abilities are clearer and more specific, and the clinical skills are more suitable to China’s paediatric residents. This study also provides a theoretical reference to better the STPR training system and to improve the overall level of paediatricians’ medical service quality and supply in China.
Index weight analysis
The weight sequence of the primary indicators in this study is as follows: professional accomplishment (0.3187), knowledge skills (0.2734), communication and cooperation (0.1986), lifelong learning (0.1302), and teaching ability (0.0791). Of the primary indicators, ‘professional quality’ and ‘knowledge and skills’ have the two largest weights, indicating that STPR in China has strengthened doctors’ education on professionalism and that training is progressing towards the double improvement mode of clinical knowledge and skills and professional quality. With the development of medicine, medical education has gradually changed from ‘focusing on natural science knowledge and clinical skills’ to ‘providing medical services based on patient needs’.38 Many scholars believe that the inclusion of humanities in medical education can accelerate this development; therefore, it is crucial to cultivate residents’ professionalism.39 In the context of a ‘Healthy China’, medical professional quality has been recently emphasised, and experts have pointed out that students need not only strong clinical theoretical knowledge and excellent clinical skills, but also excellent medical humanities and professional ethics.40 41 Communication is an effective way to express medical humanities spirit to patients. Mature guidelines and evaluation scales that improve doctors’ communication skills exist internationally.42 43 In recent years, China had adopted many measures to improve patient safety, medical care quality, medical team cooperation and doctor–patient communication.44 Good communication and cooperation contribute to clinical decision-making and teamwork, which are crucial in managing medical affairs and improving medical work efficiency and patients’ medical experience.42 43 The cultivation of comprehensive and multilevel communication skills is a cornerstone for residents to comprehensively develop and adapt to the modern medical transformation model.
Research has shown that residents’ lack of autonomy, in terms of uncertain course schedules, heavy clinical work and insufficient financial support, affect the development of lifelong learning and teaching ability,45 46 especially with regard to paper writing and statistical analysis.47 In China, the increasing pressure on paediatric medical service has led to the inevitable problem of work–study contradiction. However, previous studies have shown that medical development depends on medical knowledge innovation and indepth medical research; thus, STPR should encourage residents to participate in clinical teaching and academic activities and cultivate scientific research thinking skills.48 49 The experts in this study suggest that paediatric residents should actively learn and accept new knowledge and technologies to better adapt to the rapidly changing social environment and to be well prepared for the rapid, modern, interdisciplinary medicine development under the global knowledge network platform.
Index deletion analysis
The six tertiary indicators were deleted in this study. With regard to ‘blood smear interpretation, bone marrow interpretation’, some experts claimed that only haematology/oncology doctors have more opportunities to practise this indicator and recommended that it should be included in the future specialist standardised training stage; however, according to expert discussion, residents should still have comprehensive paediatric capabilities. Because Chinese children’s medical resources are mainly provided by the paediatrics department in general hospitals and there are few specialty hospitals for children,19 20 the skills and ability to interpret blood smears and bone marrow should be included at this stage to ensure quality service from general paediatric residents. Regarding ‘independent practice (outpatient)’, the experts claimed that patient–doctor relationships and results-based evaluations had reduced residents’ autonomy; however, they cannot change the requirement that residents should have independent practice ability. Future training could apply simulation teaching for residents to be fully prepared.50 With regard to the indicators ‘provide information or training to other health practitioners (eg, other physicians, nurses, pharmacists, medical technicians and other support staff)’, ‘participate in various clinical experiments and trials’ and ‘participate in the exchange study of domestic and foreign institutions’, our experts considered that, although STPR should refer to clinical behaviour, residents should be familiar with the above three aspects according to their study situation. Thus, the above six indicators were retained according to experts’ opinions.
Strengths and limitations of this study
This study fully incorporated the indicators of the existing STR and STPR competency models and developed a competency index system suitable to China’s paediatric residents. The vast literature reviews and behavioural event interviews have ensured the comprehensiveness and representativeness of the indicators, which reflect the current status of STPR in China. The selection of experts from all over the country ensured the authority and validity of the Delphi survey data. However, an interview to collect competence indicators was not conducted among paediatric patients because they were immature to explain themselves clearly. So during the expert consultations, we emphasised that they should select indicators from the patients’ aspect. The index system has not been implemented in a large sample of paediatric residents, and the reliability and validity of the competency indicators will be further verified by the research team in subsequent research.
Conclusion
The construction of the STPR educational competency index system was not only related to the improvement of medical treatment, teaching and scientific research level, but also to the sustainable development of social medical services. Based on the Delphi method, this study finally formed the first paediatric resident competency evaluation index system in China and determined the weight of each indicator with scientific and application values. This index system highlights the combinations of medical theory and clinical practice, clinical ability and humanistic communication, and professional quality and medical ethics. Additionally, it specifically clarifies the connotation of each element to explicitly guide practical work, fills the research gap on the evaluation index system for paediatric residency in China, and provides a reference for better training of paediatricians in diagnosis and treatment.
Acknowledgments
The authors wish to thank all participants, especially the 16 experts who helped with the analysis and interpretation of this study.
References
Footnotes
Contributors YW is the first author and KM obtained funding. YW codesigned the study with KM, TW and SC, conducted the study with AW and LJ, and analysed the data with JYS and XF. YW and KM contributed to the interpretation of the results and critical revision of the manuscript for important intellectual content and approved the final version of the manuscript. All authors have read and approved the final manuscript. YW and KM are the study guarantors.
Funding This study was supported by grants from the Chinese Medical Association’s 2018 Medical Education Research Project (2018A-N17002) and the Capital Medical University Hospital Management Institute 2020 open project (2020YGS03).
Competing interests None declared.
Patient consent for publication Not required.
Ethics approval Ethics approval was granted by the Ethics Committee of Capital Medical University (No. Z2019SY055). The study design and information sheets were reviewed by the committee and considered appropriate for use. All 16 experts signed informed consent forms and voluntarily participated in the study. Participant information was confidential and participants could withdraw at any time during the study.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement No data are available.