The need for needs assessment in continuing medical education

Introduction

Maintenance of professional competence is a critical component of professionalism. However, traditional methods, which rely on individual self assessment, are inadequate. Conversely, legislated recertification programmes are difficult to individualise and can be perceived as draconian. What is required are better methods of standardised individual needs assessment. We suggest some possible strategies.

Background

Like all professions medicine is granted professional autonomy by society under the assumption that its practitioners will be deemed competent on entry into practice and will maintain competence for as long as they practise. Traditionally it is the responsibility of the individual practitioner to do whatever is necessary to remain competent.

In the past maintaining one's competence was not problematic because relevant knowledge accreted slowly. Today, however, without a programme of active learning no doctor can hope to remain competent for more than a few years after graduation. One response to this challenge has been for education programmes, particularly problem based ones such as our own, to focus on the development of self assessment skills and self directed learning skills in order to equip graduates to maintain competence. The evidence, however, while not abundant, shows that this was a quixotic quest. The evidence that graduates from problem based learning are better at “keeping up” is weak.1 2 Moreover, many studies have shown that self assessment is far more difficult than we thought.3 Finally, self assessment does not emerge on graduation as a consequence of the demands of changing practice. Sibley et al observed that practitioners tend to pursue education around topics they are already good at while avoiding areas in which they are deficient and where there may be room for improvement.4 The evidence shows therefore that self monitoring programmes such as the maintenance of competence (MOCOMP) programme,5 which leave practitioners to their own devices, may be hopelessly optimistic. According to Norcini all attempts at voluntary recertification strategies initiated by members of the American Board of Medical Specialties failed and have been replaced by mandatory procedures.6

As a consequence, much effort has been invested in more formal approaches to maintenance of competence. Several models exist—for example, formal peer review and, where necessary, remediation7 or formal written recertification examinations.6 All are expensive. Peer review processes, which are necessarily individualised, are difficult to implement on the scale needed to ensure adequate monitoring in large jurisdictions. Further, as Norcini points out,6 standardised audits can be applied only to relatively common conditions, yet much of specialist practice is devoted to diagnosis and management of rare, but clinically important, conditions. Examinations have an economy of scale, but their relevance to actual competence or individual practice needs is often challenged, and they remain costly to create and maintain. Finally, both of these strategies tend to operate under the long arm of the licensing law and are therefore unlikely to be adopted voluntarily.

What is required is something between the anarchy of self assessment and the “Big Brother” approaches; strategies that would be seen as supportive and individual yet objective. The recent focus in continuing medical education (CME) on learning needs assessments shows promise.

Why it's important for needs in continuing medical education to be assessed

The ultimate goal of continuing medical education is to improve outcomes for patients by changing doctors' practice behaviours. Evidence from systematic reviews of the literature shows that programmes in continuing medical education that are predicated on well conducted needs assessments are effective in changing doctors' behaviours.8 In line with these findings a shift has occurred recently in thinking about continuing medical education, its purpose, and the keys to its effectiveness.9 Rather than be passive recipients of offerings of continuing medical education, often provided by enterprises with non-educational goals, doctors are encouraged to actively choose what to learn, how to learn it, and to reflect on the implications of what has been learned. To meet the needs of this self directed learning approach, formats of continuing medical education have broadened to include workshops, small, practice based study groups, individualised programmes delivered on CD, interactive computer programs, websites, workbooks, journals, practice guidelines, peer consulting, and “academic detailing.” However, this proliferation of course formats still represents a diversity of methods to deliver knowledge. Without a grounding and justification of the content through a specific needs assessment, offerings are unlikely to be effective, regardless of the presentation format.

Learning needs differ from educational needs

A fundamental gap remains between the learning needs of the individual practitioner and the priority educational needs identified by bodies for continuing medical education for course offerings. The two are not synonymous. Learning needs are personal, specific, and identified by the individual learner through practice experience, reflection, questioning, practice audits, self assessment tests, peer review, and other sources.10 Although, in theory, doctors should use these methods to create self directed learning plans, there is no evidence for most doctors that this actually happens. In contrast, educational needs can be defined as the interests or perceived needs of a whole target audience and can be identified through surveys, focus groups, analysis of regional practice patterns, and evaluations of CME programmes. They are necessarily more general than learning needs and may be directed as much by participants' curiosity and academic interests as by well defined individual learning needs. An example of an educational needs assessment is a discrepancy or “gap” analysis, in which current practice behaviour is compared with an ideal or accepted standard of practice.11 In contrast, an exploration of the issues that created the gap, in individual cases, would identify the learning needs.

What is the way forward?

If doctors cannot assess their own learning needs reliably and large scale surveys can assess only education needs, what is the way forward? We need to increase the objectivity of learning needs assessments while making the process simple enough for doctors to do regularly. Little research directed at improving the technology of needs assessment has been reported.