Article Text
PDF
Abstract
Purpose Studies document that primary care improves health outcomes and controls costs. In regions of the world where primary care is underdeveloped, building capacity is essential. Most capacity building programmes are expensive and take physicians away from their clinical settings. We describe a programme created, delivered and evaluated from 2013 to 2014 in Jordan.
Design Cohort study.
Setting Physicians providing primary care in the United Nations Relief and Works Agency for Palestine Refugees clinics in Jordan.
Participants Eighty-four general practitioners (GPs) were invited to participate and completed the training and evaluation. GPs are physicians who have a license to practice medicine after completing medical school and a 1 year hospital-based rotating internship. Although GPs provide care in the ambulatory setting, their hospital-based education provides little preparation for delivering ambulatory primary care.
Intervention/Programme This three-stage programme included needs assessment, didactics and on-the-job coaching. First, the learning needs and baseline knowledge of the trainees were assessed and the findings guided curriculum development. During the second stage, 48 hours of didactics covered topics such as communications skills and disease management. The third stage was delivered one on one in the trainee’s clinical setting for a 4 to 6-hour block. The first, middle and final patient interactions were evaluated.
Primary and secondary outcome measures Preknowledge and postknowledge assessments were compared. The clinical checklist, developed for the programme, assessed eight domains of clinical skills such as communication and history taking on a five-point Likert scale during the patient interaction.
Results Preknowledge and postknowledge assessments demonstrated significantly improved scores, 46% to 81% (p<0.0001). Trainee’s clinical checklist scores improved over the assessment intervals. Satisfaction with the training was high.
Conclusion This programme is a potential model for building primary care capacity at low cost and with little impact on patient care that addresses both knowledge and clinical skills on the job.
- education
- family medicine
- general practice
- middle east
- postgraduate
- training
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
The programme described and evaluated is one of the first low-cost training programmes with both didactics and on-the-job components, designed to enhance the knowledge and clinical skills of general practitioners delivering primary care in the region.
The programme was designed to have little negative impact on patient volumes and neutral costs for the organisation.
The Clinical Checklist, designed to address all aspects of the clinical encounter, functioned as a tool for training and evaluation, but reliability and validity were not formally conducted.
One board-certified family physician faculty developed and delivered all the stages of the module which offered consistency in the delivery and the evaluation of the participants in the on-the-job component.
Long-term retention of the programme’s curriculum was not evaluated.
Introduction
Studies conducted over several decades have documented that primary care improves health outcomes and controls costs.1–3 Developing the primary care specialties and creating educational opportunities for physicians practising primary care in regions where the specialty is not fully realised are imperative. Jordan and other Middle Eastern countries face these challenges as do other areas of the world.4 5
The biomedical and hospital-centred model has shaped most medical educational programme in the Middle East. The model of 6 years of medical education that follow secondary school, or the equivalent of high school in the USA, has not kept pace with the rapid changes in healthcare delivery, the increase in chronic disease or the focus on quality of care and outcomes. In addition, Jordanian physicians are licensed to practice after medical school and a 1-year hospital-based rotating internship (paediatrics, surgery, internal medicine and gynaecology) without further specialty training. These physicians are known as general practitioners (GPs). They have little outpatient experience and limited education in the psychological, social, ecological and economic dimensions of health.6 Due to United Nations Relief and Works Agency (UNRWA) for Palestine Refugees’s budgetary constraints, the GPs employed are not permitted to participate in specialty training.7
Until the late 1990s when the specialty of family medicine was created in Jordan, GPs saw the bulk of primary care patients. The need for enhanced primary care skills is magnified because 3000 Jordanian medical students graduate annually, but only 10% can do a specialty in Jordan due to limited positions.8 At the time of this programme, family medicine offered 12 residency spots each year for the entire country. Approximately 2700 students graduate without specialty training beyond a 1-year hospital internship and very limited, if any, ambulatory skills. Therefore, an efficient and cost-effective way to enhance primary care skills is imperative to provide quality and cost-effective care. With limited primary care knowledge and skills, GPs make numerous referrals to specialists, overburdening the specialty clinics and escalating costs. For example, it is common practice to refer any patient with chest pain to cardiology.7
The goal of this programme was to enhance the primary care knowledge and skills of GPs providing primary healthcare in the outpatient clinics serving Palestinian refugees in Jordan with a low-cost model that had minimal impact on patient care loads. The literature suggests that engaging GPs in quality improvement activities is essential to improve the health of the population, enhance patient experiences and outcomes and reduce the per capita cost of care.9 A needs assessment guided the development of course material that was delivered during classroom sessions and in one on-the-job training visit. Pre-evaluation and postevaluation were completed for both the knowledge and on-the-job skills portions.
Methods and analysis
Program setting and participants
UNRWA for Palestine Refugees in the Near East is a United Nations agency established by the General Assembly in 1949 and mandated to provide assistance and protection to registered refugees from Palestine. Its mission is “to help Palestinian refugees in Jordan, Lebanon, Syria, the West Bank, and the Gaza Strip to achieve their full human development potential, pending a just and lasting solution to their plight.”10 UNRWA services encompass education, healthcare, relief and social services, camp infrastructure and improvement, protection and microfinance. They are available for about five million Palestinian refugees, of which about two million live in Jordan.11
As part of ongoing efforts to improve access and to strengthen the quality of primary healthcare services in the outpatient setting, UNRWA introduced the Family Health Team approach as part of its reform agenda in 2013.12 This is a comprehensive person-centred and family-centred model which moves from the historical vertical approach where patients went to different clinics to address different needs such as immunisations, school health, non-communicable or chronic diseases (NCD) and reproductive issues. With the new model, a family receives the majority of their outpatient care from one healthcare team that includes a GP, nurse and pharmacist. This approach requires training of the team. For example, new skills are necessary for the same doctor to do well-child care, as well as care for the father with diabetes and hypertension, and the mother who is needing contraception or is pregnant. The Family Health Team approach is integrated into a wider range of efforts across UNRWA to improve quality and achieve universal access.12
Between 2013 and 2014, the author (AAS), an experienced and specialty trained, board-certified family physician and teacher, set up a primary care training programme for all 24 UNRWA health centres in Jordan. These health centres are distributed throughout the country, in urban and rural settings, in formal settlements and refugee camps. The 84 GPs working in these centres were invited to participate in the training which occurred over 2 years, during their work day and had no additional out of pocket cost to them. The programme was approved by the UNRWA review committee, which is equivalent to an ethics review, and all trainees gave written consent.
Structure of the training programme
The training programme was organised in three stages (figure 1) and conducted by one family physician faculty (AAS). In the first stage, a needs assessment gathering qualitative and quantitative data was distributed to the trainees to assess baseline knowledge and identify the main training needs. The questionnaire was developed by a scientific committee from the UNRWA, The Hashemite University School of Medicine and the Jordan University of Science and Technology Department of Family Medicine. It asked the following open-ended questions: If you have a clinical question how do you find the answer? What resources do you use to answer medical questions? How do you keep up to date with medical information? What kind of conferences, if any, have you attended in the last year? What are the 10 most common illnesses you treat in your UNRWA clinic? Knowledge of most common topics in primary healthcare were tested with multiple choice questions selected from Graber’s board review for the American family medicine examination.13 Questions with relevance to primary care and the patients seen in the UNRWA setting were chosen. AAS also gathered information and data during his year-long consultation work for UNRWA to help implement new budgetary initiatives and the electronic health record.12 During this time, he shadowed a third of the physicians in clinic and saw first hand the challenge of implementing the Family Health Team approach without additional didactics. For example, a GP who worked in the NCD clinic for 20 years was expected to deliver reproductive care after shadowing a physician in the reproductive health clinic for 1 day. AAS lobbied UNRWA–Jordan leaders that additional training was needed that he would do for no additional cost. Referral patterns and medication prescriptions data also informed the development of the training curriculum. While patients were not directly involved in this pilot, UNRWA patient satisfaction reports and the high number of referrals to specialists and the lengthy wait times were made available to AAS.
Building Family Medicine Capacity Pilot: Training Model and Evaluation Delivered to 84 general practitioners in 24 United Nations Relief and Works Agency for Palestine Refugees in the Near East (UNRWA) clinics delivered by one faculty trainer (Author:AAS). Evaluation included preknowledge and postknowledge tests and serial evaluations with the Clinical Checklist during Stage 3.
The needs assessment and UNRWA data guided the selection of the training materials and presentation methods for the second stage. The didactic training included PowerPoint presentations with lecture, role play, interactive learning and audio-visual materials. The 48 hours of face-to-face course work was repeated over 6 weeks so that 10 to 12 physicians participated in each day-long session, allowing colleagues to cover for each other so that a clinic’s capacity to care for patients was not affected. The course work included: The Family Health Approach which is having one healthcare team address the health needs of a family. Communication skills employed the Calgary Cambridge Modules which cover 71 key communication points.14 Evidence-based medicine stressed that textbooks are inadequate for point-of-care up-to-date information and reviewed free Internet resources. Rational drug use discussed avoiding polypharmacy, using guidelines for the treatment of chronic illnesses such as hypertension and diabetes and the misuse of antibiotics for viral infections. Basic clinical skills reviewed focused interviewing and physical examinations for patient problems and identifying red flag symptoms to better determine what needs specialist referral or what can be sent home and monitored. Disease surveillance reviewed appropriate Center of Disease Control guidelines and which diseases needed to be reported to UNRWA and the Jordanian Ministry of Health. The disease-oriented topics addressed approaches to commonly seen illnesses such as headache, upper respiratory tract infection, dizziness, low back pain, dyspepsia, diabetes mellitus, hypertension, gestational diabetes and hypertension, micronutrients, anaemia in pregnancy and preconception care. Special attention was spent on the diagnosis and management of psychosomatic disorder. For example, a common UNRWA patient scenario is a 20 or 30-year old refugee with multiple somatic complaints who has received numerous prescription medications. The Diagnostic and Statistical Manual of Mental Disorders 5th Edition description of psychosomatic illness was reviewed as well as how to differentiate ‘red flag’ symptoms from those that might benefit from reassurance and monitoring over time. An introduction to behavioural therapy skills was also presented.
The third stage of the training was conducted ‘on the job’. AAS spent 4 to 6 hours with each physician trainee during his/her clinic following a format that included: negotiating the agenda and the trainee’s learning needs, joint examination of a patient with teaching, using equipment such as the fetoscope, case discussions of difficult or challenging patients and then observation and coaching throughout the session. The physician trainees were not informed about the timing of these visits in advance. The evaluation was reviewed with the trainee at the end of the session.
Evaluation of the training
The impact of the training was evaluated with two instruments developed with input from the scientific committee mentioned above. Knowledge before and after the didactic sessions was assessed with 20 multiple-choice questions from Graber’s board review for the American family medicine examination.13 Second, a clinical checklist was designed to assess performance in the clinic. AAS piloted the 20-item checklist on 15 physicians, made adjustments and then used the instrument to evaluate trainees’ performance during the on-the-job sessions. The checklist assesses eight domains of clinical skills: communication, history taking, examination, diagnoses, treatment planning, follow-up plan, closure and documentation, using a five-point Likert scale, with five representing well done or present. The faculty trainer (AAS) completed the checklist during the first patient of the on-the-job training session (pretest), for another patient about 2 hours into the session and on the final patient of the session (post-test) (table 1).
Clinical checklist used for evaluation during the on-the-job training sessions
Satisfaction with the training was evaluated with three questions using a four-point Likert scale response: clarity of the training objectives, relevance of the training to daily practice and the positivity of the on-the-job learning environment. These were collected anonymously.
Statistical analyses
The scores of the preknowledge and postknowledge assessments were tallied and compared with paired student t-tests. SPSS V.23 (SPSS,) was used for statistical analyses. All tests were two sided, and the significance level was set at p<0.05. The scores for the first, second and final checklists were tallied for all the trainees and compared. In addition, the scores of the trainee’s three checklists were averaged and shared with the trainee at the conclusion of the session. Finally, trainee satisfaction responses were compiled.
Results
All 84 physicians invited to the training attended the entire 48-hour didactic course, completed the preassessment and postassessment and participated in the clinical on-the-job observation and pre/mid/postevaluations. The mean knowledge test score prior to the didactic training was 46%. After programme completion, the mean score increased to 81%, a statistically significant improvement (p<0.0001). Scores are presented in table 2.
Prestage and post-stage two knowledge scores of physician trainees
The clinical checklist demonstrated high performance on all the main indicators assessed, and scores improved between the on-the-job initial and final assessments (see figure 2). Finally, satisfaction with the training was high. Ninety-seven per cent thought that training objectives were clear or mostly clear. One hundred per cent found the information to be relevant and useful or mostly relevant and useful for their daily practice. Ninety per cent felt that the on-the-job component created a positive learning environment.
Clinical skills evaluation performed at three intervals during the on-the-job session: first encounter serves as the pretest, the second assessment is 2 hours into session, and the third assessment is the final encounter or post-test.
Discussion
A training programme designed to enhance the knowledge and clinical skills of GPs delivering primary care in all the UNRWA clinics in Jordan was created, delivered and evaluated. Because there are not enough physicians trained with up-to-date primary care skills to care for refugee patients in Middle Eastern countries, approaches like this are important for building capacity. Efforts to increase or enhance the primary care skills of GPs are being implemented all over the world. In the Middle East region, courses are expensive and take physicians away from their clinical setting for weeks or months at a time.15 UNRWA cannot afford the tuition or the loss of clinical time. In addition, some courses deliver content online and cannot include an on-the-job component.16–18
Because of the brevity of this didactic curriculum along with the on-the-job component delivered during the participating physician’s clinical day, the programme did not impact the daily census of patients served. This is a critical element for UNRWA given the limited budget and high daily volume of patients. This model may be a useful in other settings where keeping costs to a minimum and maintaining patient volumes are important. In addition, the knowledge and skills were practical and focused on what GPs were seeing and what they needed to learn and apply in their clinical settings.
This programme introduced the clinical checklist which functioned both as a tool for training and evaluation. Development of this new tool was necessary because existing tools were not adequate. The Arizona Clinical Interview Rating Scale developed in the USA assesses interview skills but not clinical examination skills.19 The MAAS History-taking and Advice Checklist-GP developed in the Netherlands20 and the recent UK Leicester Assessment Package21 have not been adequately tested in settings with real patients. While reliability or validity testing of the clinical checklist was not formally conducted, it did evaluate all aspects of the clinical encounter. Further testing is needed.
This programme had additional limitations. The performance of the trainees was only evaluated immediately after the training, and we cannot demonstrate that the training generated any improvement on their ongoing practical performance. Additional visits and sessions would need to be conducted to obtain this information. However, reports by UNRWA–Jordan show that the Family Health approach, for which this programme trained the GPs, decreased the average number of daily medical consultations per doctor, increased consultation time and decreased antibiotic prescription rates.22 Second, physician trainees might have been eager to perform well during the on-the-job component to impress the faculty trainer which may have led to an overestimation of their routine performance. To minimise this effect, faculty arrived unannounced preventing any preplanning for the session. The faculty trainer was paid part time by UNRWA for consultation work described above. This gave AAS the ability and time to understand UNRWA GPs’ knowledge deficits and practice reality. He had a unique perspective that another faculty may not have and allowed him to create and deliver the programme without additional costs, making this cost neutral for UNRWA–Jordan. While helpful to UNRWA, it is likely impossible in other settings. In addition, AAS developed and delivered all the stages of the module which offered consistency of delivery and evaluation. While his curriculum and approach are reproducible, his skill and enthusiasm as a teacher may be harder to replicate. Finally, serving as trainer and evaluator may introduce leniency bias,23 since AAS both created and evaluated the programme. To minimise this, the second author was added to review and interpret the data.
The main strength of the programme was the inclusion of a large number of physicians working in primary healthcare. Their learning needs and knowledge were assessed before and after a structured training that included both classroom and on-the-job components. The delivery of the programme had little negative impact on patient volumes and costs were neutral for the organisation. The findings from this programme are being used to guide the development of a primary care capacity building training programme for the Jordanian Ministry of Health. Classroom components could be translated into online modules or asynchronous learning sessions with faculty; however, the on-the-job component requires real time. Trainees valued this component, and studies suggest that it is a critical element for affecting behaviour change.24
Multiple strategies to build primary care capacity are being pursued around the world. Some of these include empowering advanced-practice providers, expanding public health programme and the roles of public health nurses, various telehealth models and training community health workers to serve in the communities where they live. A thorough discussion of these is beyond the scope of this paper.15 In conclusion, this programme underlines the importance of training programme designed to address the needs of the learners and impact both knowledge and skills. In regions where there are large numbers of physicians who are not adequately trained to provide primary care, this programme may offer a model for building capacity without excessive expense or impact on patient care volumes.
Acknowledgments
We acknowledge all the doctors from the United Nation Relief and Works Agency for Palestine Refugees who participated in the training and the programme. We are grateful to Sandra Eldridge, PhD, Professor at Queen Mary University, London, England for her assitance with statistics.
References
Footnotes
Presented at AAFP Global Health Summit, Jacksonville, Florida, 13 September 2018.
Contributors AAS created and delivered the training programme, developed the pre-evaluation and postevaluation with input from the scientific committe described, analysed the data and drafted the manuscript. TZ assisted with data analysis, further drafting and editing of the manuscript and tables.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient consent for publication Not required.
Ethics approval The program was approved by the United Nation Relief and Works Agency for Palestine Refugees’ review committee, and all participants gave written consent to participate.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data are available upon reasonable request.