Article Text
PDF
Abstract
Objectives With increased cross-border movement, ensuring safe and high-quality healthcare has gained primacy. The purpose of recertification is to ensure quality of care through periodically attesting doctors’ professional proficiency in their field. Professional migration and facilitated cross-border recognition of qualifications, however, make us question the fitness of national policies for safeguarding patient care and the international accountability of doctors.
Design and setting We performed document analyses and conducted 19 semistructured interviews to identify and describe key characteristics and effective components of 10 different European recertification systems, each representing one case (collective case study). We subsequently compared these systems to explore similarities and differences in terms of assessment criteria used to determine process quality.
Results Great variety existed between countries in terms and assessment formats used, targeting cognition, competence and performance (Miller’s assessment pyramid). Recertification procedures and requirements also varied significantly, ranging from voluntary participation in professional development modules to the mandatory collection of multiple performance data in a competency-based portfolio. Knowledge assessment was fundamental to recertification in most countries. Another difference concerned the stakeholders involved in the recertification process: while some systems exclusively relied on doctors’ self-assessment, others involved multiple stakeholders but rarely included patients in assessment of doctors’ professional competence. Differences between systems partly reflected different goals and primary purposes of recertification.
Conclusion Recertification systems differ substantially internationally with regard to the criteria they apply to assess doctors’ competence, their aims, requirements, assessment formats and patient involvement. In the light of professional mobility and associated demands for accountability, we recommend that competence assessment includes patients’ perspectives, and recertification practices be shared internationally to enhance transparency. This can help facilitate cross-border movement, while guaranteeing high-quality patient care.
- recertification
- continuing professional development
- performance assessment
- patient safety
- quality assurance
- professional mobility
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
- recertification
- continuing professional development
- performance assessment
- patient safety
- quality assurance
- professional mobility
Strengths and limitations of this study
Our research provides a comprehensive comparison of 10 European recertification systems and their assessment criteria used to ensure quality of care delivered. It highlights how physicians’ knowledge and competence are assessed, which stakeholders are involved and how the processes are regulated.
Our research focuses on European countries only as free cross-border movement of professionals is unique to the European context.
We cannot exclude that inter-regional variations were missed because recertification systems were decentralised in some countries, and we explored the national level only.
The diversity and ambiguity in terminology (recertification, revalidation, continuing professional development) underline the challenge of comparing various recertification systems.
Introduction
Increased mobility of health professionals can pose potential threats to the quality of care. Suppose, for instance, a high performing, Romanian doctor moves to the Netherlands. There, this person will face a new work environment in a distinct healthcare system with specific quality guidelines and different morbidity patterns, and patient demands. As this new work setting requires specific knowledge, skills and values that differ from the Romanian context and culture, you may wonder: Will this doctor still be competent to deliver high-quality care?
While the problem of safeguarding quality of care across borders is omnipresent, it is particularly pertinent in Europe where the free movement of professionals has long historical and legal roots. Although a European Commission directive has facilitated mobility by providing for international recognition of professional qualifications, it fails to guarantee that doctors actually meet the minimum and context-specific quality standards. To safeguard quality of patient care, regulatory bodies around the world have implemented different systems,1 2 such as recertification systems. Recertification entails lifelong learning and periodic assessment of doctors’ competence and performance through various methods.3 It describes the process designed to promote and demonstrate continuous professional competence.4 More specifically, it requires a formal procedure of assessing and attesting quality of service provided ‘in accordance with established requirements or standards.’5 By renewing initial certification, recertification aims to address any decline in performance as well as to ensure trained doctors’ adaptation to advances in knowledge and technology.6 7 This is particularly important in times of increased publicity over individual failures of medical performance, demands for doctors’ accountability and concerns about patient safety.8
Despite its well-intended aim, recertification harbours two inherent problems. First, current national recertification practices fail to ensure quality of care internationally, as they assess doctors’ competence and performance in accordance with national quality standards. Differences in standards across countries and the absence of international recertification systems may complicate international quality assurance and quality improvement.7 This begs the question of whether such discrete practices can respond to repeated calls for international accountability and transparency.4 Second, although research on assessment of professional competence provided a set of guidelines for assessment criteria to ensure high-quality assessment,9 the question on how to assess doctors’ competence has often turned into a political rather than an educational one,10 potentially impacting on effectiveness of recertification systems.
‘Competence’ is defined as the ability to integrate knowledge, skills and attitudes into a certain context to ensure safe patient care.11 12 This definition suggests to pay balanced attention to multiple competency domains relevant to a doctor, when assessing professional competence.13 Indeed, many scholars and institutions advocate the assessment of medical knowledge, skills, and competencies, such as communication, collaboration and clinical judgement, as well as cultural competence or critical consciousness.14–16 Assessment measures must also be robust and focus on the healthcare system’s needs and outcomes, implying involvement of key stakeholders, particularly patients when evaluating quality of care.17–20 It is furthermore acknowledged that, for each of the competencies, outcomes of different assessment methods must be combined to ensure robust decision-making about professional competence.21 22
To conclude, cross-border quality of care will be promoted if countries share their recertification practices and are willing to critically reflect on quality of assessment processes embedded in recertification procedures.8 23 In the present study, we attempt taking a first step in this direction by identifying different national recertification approaches. The question of the present study, therefore, was what are the key characteristics of recertification systems for doctors of different countries? More specifically, we aimed at exploring use of assessment criteria in design of recertification procedures. We used a collective case study design to describe and compare different national systems. We were particularly interested in the assessment criteria used, if any, and how they were applied. Although recertification is sometimes also coined ‘revalidation’, ‘reaccreditation’ and ‘maintenance of certification’ or used interchangeably with ‘continuing professional development’ in other contexts, this article keeps to the former term. The article builds on previous work on certification but primarily focuses on recertification.
Methods
Study design and case selection
We described and analysed the recertification systems of 10 individual European countries. Each country’s national recertification system represented a single case. We selected our cases using purposeful sampling to reach maximum heterogeneity in terms of geographical spread across Europe, demographics, health professionals’ migration profile and type of healthcare system (table 1).24
Sampling criteria
Based on these criteria, the final study sample included Denmark, Germany, Hungary, Ireland, Poland, Portugal, Spain, Switzerlandi, the Netherlands and the UK (table 2).
Overview of selected countries and their health insurance systems, their coverage and the existence of a gatekeeper system49–57
Data collection
We collected data on the respective recertification procedures by performing a document analysis for each case in addition to conducting semistructured interviews with one to three representatives from each country.
For the document analyses, we retrieved documents describing national recertification procedures for doctors from the websites of national certification organisations, and translated them into English if needed. The documents included national recertification schemes and regulations, rules and reports of medical education and training, user guidelines, laws and grey literature articles. We focused on documents that clarified rationale, form and procedure, as well as requirements and rewards of each recertification programme.
To validate and corroborate our interpretation of data from document analysis, we conducted one to three semistructured interviews with representatives of each national regulatory body responsible for postgraduate medical education and recertification or the recognition of professional qualifications (eg, international affairs offices) (n=19). These interviewees were directors of professional development and practice, heads of recertification departments, experts on continuing professional development and official secretaries or legal advisers to national medical education offices, medical or scientific societies, accreditation bodies, medical royal colleges, councils, or chambers (table 3).
Number and profile of respondents per country
The first author (CS) conducted all interviews via video or phone, based on an interview protocol adapted from a study on continuing professional development and lifelong learning for health professionals.25 The interview protocol was piloted in the Netherlands. Questions addressed competency frameworks as well as rules and regulations of recertification, asking about regulatory authorities involved, main objective(s), structure, requirements and consequences of compliance or non-compliance. Before the interview, we explained the research purposes to participants and asked them to give informed consent. Interviews were audio taped and lasted 50–90 min, during which notes were taken. Notes were subsequently presented to interviewees to approve or to add information.
Data collection took place from April to September 2016.
Patient involvement
No patients were involved in this research, given our specific aim.
Data analysis
Data analysis spanned a two-step process. First, we analysed the data from the document analyses and interviews to identify and describe key characteristics of each case. We asked at least one interviewee per country to comment on the accuracy and completeness of the described recertification system. We subsequently reanalysed the data, specifically focusing on the application of criteria for high-quality assessment: validity, reliability, educational and catalytic effect.9 14 26 For that purpose, we identified specific strategies used to ensure assessment quality in terms of validity, reliability and educational consequences, for each of the recertification system (table 4).
Strategies embedded in recertification, affecting assessment quality
These strategies included programme of assessment, assessment goals and methods (ie, authentic and suitable methods which aim at measuring day-to-day performance and professional competence), as well as frequency of assessment (ie, consistent outcomes across measurements and decisions). We also addressed the involvement of different stakeholders including patients, and consequences for learning and development. Self-assessment as tool for lifelong learning and assessment of practice performance were the two major components of recertification considered.27 Finally, we compared recertification systems across cases to identify similarities and differences with respect to use of the aforementioned assessment criteria.
Results
In the following paragraphs, we highlight differences and/or similarities across countries in terms of the purpose, focus, frequency and methods of recertification, and the stakeholders involved in the process. Exact details are provided in table 5, while table 6 outlines the bodies (Medical specialities, Ministries of Health or Medical Authorities) responsible for recertification. The final paragraph provides a synopsis of the most striking results.
Competence assessment in recertification systems of investigated cases
Regulation of recertification process in the countries under scrutiny
All systems uncovered applied to all registered practising doctors, irrelevant of whether they were trained nationally or internationally, as they are automatically enrolled in the national scheme on registration.
Purpose of recertification
As shown in table 5, the purpose of recertification constituted a major source of variance. While several countries aimed to improve quality of care and patient safety, a minority (n=2), essentially those countries where recertification was not mandatory, upheld personal development and career advancement as their primary objective (table 5).
Participation in a recertification programme was voluntary in three countries only, Denmark, Spain and Portugal, though all countries imposed a professional or legal obligation to engage in lifelong learning. Consequences of non-compliance were non-existent in voluntary systems; in the mandatory systems (n=7), however, they varied from financial sanctions (Switzerland and Germany) or work under supervision to suspension of the licence to practice (Germany, the UK, Hungary and partly the Netherlands), with two countries allowing for licence recovery. Finally, one country conferred a lifelong registration on doctors, obviating the need to impose any sanctions in practice (table 6).
Information obtained from interviews confirmed information from documents with the exception of handling of non-compliance: compared with the rules laid down in official documents, interviewees reported a more lenient handling of non-compliance in practice.
Focus of the assessment
As regards focus, almost all recertification systems emphasised the lifelong learning of doctors. Likewise, most systems relied on the collection of a minimum number of credits per year, mostly 50 (n=5), where one credit typically represented 1 hour of learning activity. Although the three voluntary systems did not require credits to be earned for recertification, one did recommend it (Denmark). Such practice was often embedded in a continuing professional development framework as part of a voluntary recertification process. In Hungary, doctors must take a specific course followed by an examination. Generally, they received more credits for courses if these were concluded with an examination. Of the countries that assessed practice performance, only five did so through audits and appraisals or multisource feedback. Four countries evaluated doctors’ individual and team functioning focusing on communication and collaboration skills.
Frequency of recertification
The frequency of recertification and time frame within which requirements must be fulfilled varied widely: some countries had annual appraisals (n=2), others 3-year procedures (n=2), but most of the countries undertook quinquennial assessments (n=5).
Assessment methods
To demonstrate their knowledge and engagement in lifelong learning, doctors in most countries must earn credits, for instance by participating in workshops and national or international conferences, doing individual reading, teaching, writing scientific articles, spending time as visiting doctor and/or e-learning. Denmark assessed performance on the basis of a dialogue between employer and employee who jointly discussed learning needs. The UK counted reflection on significant events, that is, unintended critical events which potentially harmed the patient, to measure patient outcomes. Yet other countries (n=4) used clinical audits, number of complaints, reviews or appraisals and peer reviews to measure processes of healthcare delivery. Finally, some countries deployed portfolios (n=6), clinical audits (n=4) and multisource feedback (n=4) to reflect on individual and team functioning.
Stakeholder involvement
In most cases (n=9), doctors decided which learning activities to take based on their self-assessed learning needs. Several countries, however, also based the assessment of performance outcomes and the process of care on feedback from peers (n=5) or patients (n=2), yet only one country (the UK) demanded involving patients in the assessment regularly.
Synopsis
All things considered, what stood out was that most recertification systems relied heavily on doctors’ self-assessments, attached little weight to patient outcomes, patient involvement and the assessment of practice performance, as well as lacked an overarching competency framework. Only four countries seem to match the content of assessment programmes with evaluation of professional practice. These findings clearly contrast with the aim to ensure quality of care and patient safety most systems pursued. Evaluation of practice performance seems to be a sine qua non, an indispensable condition, for assessment of competence, that is, what doctors actually do in day-to-day practice. Two countries (the Netherlands and the UK), however, did use a more comprehensive system, covering both self-assessment and practice performance through multisource feedback, including patients’ feedback.
Three other countries deserve mention for their apparent distinctness from the rest. Denmark, though not formally requiring continuing professional development, assessed practice performance based on an annual dialogue between doctor and employer. This left little room for individual doctors to self-assess their performance and independently decide on activities to be taken, which was the case in all other countries where the individual doctor was responsible for high-quality patient care. The systems in Spain and Portugal stood out as being career focused: they did not require doctors to engage in lifelong learning and professional development for purposes of patient safety and quality patient care, but rather encouraged the use of a portfolio to enhance chances of promotion.
Discussion
The purpose of this study was to investigate how recertification is organised across different countries. We found substantial differences in recertification requirements and procedures. Moreover, these requirements in many respects seemed to conflict with aims to ensure quality of care and patient safety.
First, we observed that only a few systems included feedback from patients in the assessment. Involving patients in assessing quality of healthcare and doctors’ performance seems inevitable for accountability and transparency purposes.28 Although many patients are needed to obtain reliable evaluations, their involvement in recertification procedures can help respond to public calls for doctors’ accountability.9 Wright et al recommend including data from 34 patient questionnaires and 15 colleague questionnaires to obtain reliable performance evaluation for appraisal purposes.29 Despite the fact that the literature reports peers to give accurate, credible and valid assessments of performance, peer feedback was absent in most systems investigated but is for example employed in some Canadian provinces.9 14 30 31 Use of multisource feedback to assess practice performance, requires high quality and credible feedback to induce reflection on practice.32 Multisource feedback, including patients’ feedback, can be especially effective when the feedback received contrasts with individual perceptions and is facilitated by a mentor or coach.33 A mentor can help to deal with the emotional aspects of the multisource feedback and to structure individual reflection and follow-up.34 Use of multisource feedback and mentoring systems could thus help countries transitioning from a system based on self-assessments to ‘directed’ self-assessments as suggested by Sargeant et al.35
Second, most systems relied on self-assessments and lifelong learning activities doctors selected themselves without attending to external assessment of practice performance. More specifically, by relying on credit accumulation systems that allowed doctors to choose their learning activities,36 it was entirely at the doctors’ discretion to judge their performance and learning needs. There’s strong evidence, however, that several individual and social factors obscure the validity of self-assessments such as age and experience.36 37 Additionally, self-assessments tend to mirror self-confidence and self-efficacy which are not necessarily good measures of doctors’ competence.36 This evidence provides ample ground to question both the effectiveness of recertification systems that rely on doctors’ self-assessments and the autonomy granted to clinicians.17 38 Hence, assessments of competence will become more meaningful when they involve multiple assessors, including patients.
Another deviation from the purpose of recertification constituted the assessment methods used. Whereas activities such as reading written materials, and attending conferences or presentations have been shown to deepen specific knowledge, there is no evidence that such didactic and passive learning interventions alone improve performance and patient outcomes.39–41 A causal link between educational activities and improved patient health status yet remains to be established.42 This casts doubt on the impact of the recertification systems in our study on doctors’ performance. Consequently, our findings reinforce concerns about the validity of recertification procedures and emphasise the need to combine various assessment methods, likely resulting in greater accountability as previously been proven.43 As stated by Forsetlund et al, a combination of multiple media, multiple instructional techniques and multiple exposures can help to induce change in performance towards improved patient outcomes.44
Other non-European countries have experienced similar challenges in implementing adequate assessment methods for recertification purposes.4 31 45 46 Also Australia, the USA and Canada investigate new methods to evaluate competence and practice performance, cautiously moving away from self-assessment.4 31 46
Since medical specialists invest substantial time and money in their professional development, the feasibility, applicability and acceptability of recertification are topics worth exploring in the context of quality assurance. We, therefore, invite future studies into stakeholders’ perceptions of recertification and their effectiveness and impact,47 and also to bring into focus the content and formal aspects of learning activities which, by facilitating its design and implementation, may improve recertification. To shed light on the full picture, we would furthermore welcome studies investigating the feasibility and acceptability of involving patients in evaluating physicians’ competency.
Limitations
Since recertification systems were decentralised in some countries and we explored the national level only, we cannot exclude that inter-regional variations were missed. Moreover, although the interviewees ideally represented at least two different national organisations, interviews were mostly limited to two or three respondents per country. A final and possibly the most complex and intervening limitation constituted the diversity in terminology and language. This may have affected the translation of national concepts into English during the interviews and of written descriptions, potentially causing loss of detail during the analyses. These language differences and ambiguity in terms underline the challenge of comparing various recertification systems.
Practical implications for professional mobility
Defining universal criteria for assessing professional competence will be no easy feat, especially not when considering the differences between national recertification approaches and rising cross-border mobility. Since each system is customised to a specific context, culture and healthcare system, a universal recertification system may neither be desirable nor achievable, as doctors are required to consciously reflect on the local culture, and adapt to the unique features of their work setting and healthcare system.16 To our knowledge, currently there is no requirement or overarching effort in striving towards harmonising recertification processes across countries within the European Union (EU). Its member states have agreed that each individual country will remain responsible for national healthcare affairs, without European regulations interfering. Moving towards a standardised system would however require an EU-wide regulation, which is currently interrupted by those strong nationally regulatory powers. For transparency purposes, however, national bodies and medical societies could share their competency assessment procedures and quality standards, turning a political matter into an educational (and quality assurance) matter.8 Moreover, national bodies can incorporate performance evaluation, involve multiple stakeholders including patients and use other assessments besides clinicians’ self-assessments in their recertification procedures to enhance liability.48 Considering the increasing internationalisation of healthcare, doctors’ cultural competency should also be incorporated into recertification programmes.
Achieving an overarching quality assurance system being an unrealistic goal, we need to have a shared understanding of what are minimum standards for a doctor38 thereby creating a base for international comparison while allowing for local adaptations. This, however, asks for an increased collaboration between countries and understanding of differences inherent to each system and culture. Such standards of training content and certification directives could meet the challenges posed by the free, cross-border movement of professionals, improving patient safety, and enhancing accountability and transparency.
Conclusion
Recertification can help assess and improve knowledge, skills, professional performance and ultimately, patient outcomes. Yet, systems vary widely across countries in terms of being compulsory or not, requirements, patient involvement and consequences of compliance or non-compliance. A shift towards a broader programme of assessment focused on competence assessment and lifelong learning might create a more valid, credible and reliable basis for recertification, meeting growing demands for accountability and transparency.
References
Footnotes
↵i Although Switzerland is not a member of the European Union, it is part of the European Economic Area and characterised by a high migration rate, and high reliance on foreign-trained doctors, which made it relevant for our study.
Contributors CS wrote the research plan, collected, cleaned and analysed the data, and drafted and revised the paper. She is the guarantor. All authors were involved in conceptualising the research. MJG analysed the data, and drafted and revised the paper. SM revised the paper. GGUR drafted and revised the paper. FWJMS drafted and revised the paper. EWD analysed the data, drafted and revised the draft paper.
Funding CS is a PhD student at Maastricht University, on a project funded by the European Respiratory Society. The European Respiratory Society was however not involved in the design or in conducting this research.
Competing interests None declared.
Patient consent Not required.
Ethics approval Ethical approval was obtained from the Netherlands Association for Medical Education (NVMO; file number 669).
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement The dataset(s) supporting the conclusions of this article is available from the author on request by emailing the corresponding author.