Objective To assess health informatics (HI) training in UK postgraduate medical education, across all specialties, against international standards in the context of UK digital health initiatives (eg, Health Data Research UK, National Health Service Digital Academy and Global Digital Exemplars).
Design A mixed methods study of UK postgraduate clinician training curricula (71 specialties) against international HI standards: scoping review, curricular content analysis and expert consultation.
Setting and participants A scoping literature review (PubMed until March 2017) informed development of a contemporary framework of HI competency domains for doctors. National training curricula for 71 postgraduate medical specialties were obtained from the UK General Medical Council and were analysed. Seven UK HI experts were consulted regarding findings.
Outcomes The International Medical Informatics Association (IMIA) Recommendations for Biomedical and Health Informatics Education were used to develop a framework of competency domains. The number (maximum 50) of HI competency domains included in each of the 71 UK postgraduate medical specialties was investigated. After expert review, a universal HI competency framework was proposed.
Results A framework of 50 HI competency domains was developed using 21 curricula from a scoping review, curricular content analysis and expert consultation. All 71 UK postgraduate medical curricula documents were mapped across 29 of 50 framework domains; that is, 21 domains were unrepresented. Curricula mapped between 0 (child and adolescent psychiatry and core surgical training) and 16 (chemical pathology and paediatric and perinatal pathology) of the 50 domains (median=7). Expert consultation found that HI competencies should be universal and integrated with existing competencies for UK clinicians and were under-represented in current curricula. Additional universal HI competencies were identified, including information governance and security and secondary use of data.
Conclusions Postgraduate medical education in the UK neglects HI competencies set out by international standards. Key HI competencies need to be urgently integrated into training curricula to prepare doctors for work in increasingly digitised healthcare environments.
- health informatics
- information technology
- information management
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Strengths and limitations of this study
In the first comprehensive study across all 71 specialties in UK postgraduate medical training, we showed that health informatics (HI) is grossly under-represented in postgraduate clinical training curricula.
A mixed methods design (scoping literature review, curricular content analysis and expert consultation) allowed iterative development of a contemporary, generalisable HI competency framework for all doctors.
A possible limitation was the subjective nature of determining keywords and mapping competencies between an established framework (International Medical Informatics Association [IMIA]) and curricular documents.
Not all competencies listed in the IMIA framework are relevant to every specialty.
HI competencies for training doctors were studied, but those for other clinicians, such as associate specialists, staff grade clinicians and consultants, have no national curricula, making it impossible to assess if HI skills are being promoted as part of their continuing professional development (CPD).
The scale and the complexity of modern healthcare data have resulted in successive policy recommendations for investment and innovation in healthcare information technology.1–4 Following the 2017 Wachter review, the UK is embracing ‘big data’, ‘digital medicine’ and ‘data-driven healthcare’ through the Global Digital Exemplar Programme in hospitals, and Health Data Research UK (HDR-UK), the new national health informatics (HI) institute. HI in this context may be defined as ‘the intelligent use of information and technology to provide better care for patients’.5
Both in the UK and internationally, there is growing appetite to quantify and improve the digital maturity of health systems.6–9 The universal relevance of HI is exemplified by the WHO’s recent adoption of a digital health resolution, led by India,10 and the urgency is amplified by the ambition of recent investments in precision medicine11 12 and artificial intelligence.13 Despite acknowledgment that ‘upskilling’ and ‘increased capacity’ are necessary for ‘digital readiness’ across all health professions,4 7 14–16 there has been far less research and policy focus on training of doctors and other health professionals in HI, the variably defined discipline that has evolved into the application of information science and computer science to healthcare.17 Even in the USA, where a separate subspecialty accreditation programme in HI has been developed, there has been limited success with digital transformation.18
In the UK, there are approximately 113 500 doctors,19 of whom up to 65 700 are estimated to be in postgraduate training and less than 4000 are clinical academics.20 The Wachter review recognised that previous large-scale HI programmes in the National Health Service (NHS) have suffered from ‘a shortage of individuals with relevant skills’.4 The UK trains 7500 medical students per annum.21 Despite repeated General Medical Council (GMC) guidance,22 deficiencies in UK undergraduate HI education persist.23 Therefore, postgraduate HI training will be crucial in bridging the knowledge and skills gap in HI among doctors. The NHS Digital Academy will train chief clinical information officers (CCIOs),24 25 and the Faculty of Clinical Informatics (FCI) seeks to ‘professionalise’ HI accreditation, but these recently formed organisations will emphasise leadership. The Digital Academy will train only 300 digital leaders in the next 3 years, which cannot alone effect change in postgraduate training of the clinical or clinical academic workforce.
Although leadership will be important for digital transformation projects, all doctors will need to be trained in HI. However, despite successive attempts in the UK3 26 27 and across Europe,28 existing postgraduate curricula for HI have not been evaluated systematically and have not led to agreed HI standards for all doctors. There is evidence of substantial heterogeneity in HI training at undergraduate and postgraduate training across countries in Europe.29 Without a coordinated effort to map the use of these standards across all postgraduate medical specialties in a particular country, it will be difficult to develop robust, universal HI competencies for all doctors.30
We therefore sought to: (1) develop up-to-date, evidence-based, international HI standards for postgraduate medical training and (2) to evaluate HI competencies currently required across all 71 medical specialties in the UK, compared with these standards.
We conducted a mixed methods study with three components (scoping review, curricula content analysis and expert interviews) to iteratively develop universal HI standards for postgraduate medical training.
A scoping review31 was undertaken to identify existing HI standards for doctors using PubMed. The search was limited from 1 January 1998 to 10 March 2017 to ensure up-to-date literature following the National Program for Information Technology,32 using search terms:
(((((((health informatics) OR clinical informatics) OR medical informatics)) AND ((((curricula) OR curriculum) OR syllabi) OR syllabus)) AND (((clinician) OR doctor) OR physician))) AND (((‘1998’[Date - Publication]: ‘3000’[Date - Publication])) AND English[Language]).
A ‘snowball’ technique was used to identify additional relevant references. Where two or more versions of a curriculum were identified, the most recent was included and duplicates were excluded.
A three-stage document screening process—title, abstract then full review—was conducted by two researchers (LJ and MA) independently, and where there was disagreement, a third reviewer was involved (AB). Documents were eligible for inclusion if they described postgraduate HI competencies that specified doctors within their target population (ie, curricula relating to nursing, pharmacy and allied health professionals, as well as undergraduate medical training were excluded).
Data extraction included: author details, year of publication, country of study, type of curricula (eg, academic or clinical), study population (competency target population), place of intended dissemination (workplace, university and so on), aim(s) and methodology.
Huang33 used the 2000 International Medical Informatics Association (IMIA) Recommendations for Biomedical and HI Education34 as a framework for mapping HI competencies, to which additional unidentified competencies were iteratively added. We applied the same method, using revised 2010 IMIA recommendations35 (table 1). These two studies34 35 were therefore excluded from the scoping review. The IMIA framework was selected as it is the most comprehensive and most recent framework, distinguishing between ‘IT Users’ and ‘Biomedical Health Informatics (BMHI) specialists’, and suggesting skill levels for each competency: introductory (+), intermediate (++) and advanced (+++). ‘IT User’ domains are limited to introductory and intermediate levels, with blank domains signifying a BMHI specialist-targeted domain. Blank ‘IT User’ domains were classified as advanced (+++) to ensure the domain was included in the analysis, regardless of the BMHI level allocated by IMIA. Core clinical knowledge taught at undergraduate level was excluded; for example, competencies 2.1 (‘Fundamentals of human functioning and biosciences: anatomy, physiology, microbiology, genomics, and clinical disciplines such as internal medicine, surgery, etc.’) and 2.2 (‘Fundamentals of what constitutes physiological, sociological, psychological, nutritional, emotional, environmental, cultural, spiritual perspectives and its assessment’). Competencies 3.2 (‘Ability to use personal computers, text processing and spread sheet software, easy-to-use database management systems’) and 3.3 (‘Ability to communicate electronically, including electronic data exchange, with other health care professionals, internet/intranet use’) were combined to form a ‘Basic IT skills’ competency, leaving 45 competency domains in total.
Mapping involved identification of the ‘nearest match’ for each ‘framework competency domain’ in each of the curricula documents identified by the scoping review. If there was no identical match, keywords were identified and searched in the framework (using Microsoft Word 2016 ‘Find’ function), locating all instances of the keyword (or alternative keywords, for example, ‘statistical software use’ searched by ‘stat[istics]’ and ‘math[ematics]’), allowing selection of the most relevant domain. ‘Umbrella’ competencies, for example, ‘core informatics skills’, were not mapped as they provided no specific details. In some cases, multiple framework domains were mapped to one document competency. In other cases, the framework domain was more granular than the review document, for example, the document may have specified only one parameter ‘nomenclatures’, which would be mapped to domain 17, ‘Nomenclatures, vocabularies, terminologies…’.
Curricula content analysis
Content analysis of published postgraduate medical curricula identified from, or via the GMC website36 was conducted to identify HI in postgraduate training (online supplementary appendix 1 for the full list of included curricula and clinician training structure). Curricula (parent and subspeciality where applicable) were viewed in Adobe Acrobat Reader using the search function to identify keywords signifying HI content (informatic*, technolog*, ‘IT system’, ‘electronic [health/medical/clinical/patient] record’ and ‘e[-]Health’). Any additional identified competencies were added to the framework.
Two experts were interviewed by telephone (each 40–50 min), and five experts had face-to-face interviews (40–80 min). Four experts were selected via convenience sampling, and a further three were selected using a snowball technique, based on prior involvement or knowledge of HI curricula development or NHS digitisation. Interviews were continued until saturation. Expert backgrounds included physicians, curriculum developers, educationalists and a data scientist to provide a range of attitudes and knowledge bases and avoid bias. Two pilot interviews led to streamlining of questions. Experts were consulted regarding outputs of the scoping review and the curricular content analysis and invited to consider universal HI competencies for all doctors.
Experts gave informed consent prior to interview for recording, storage and use of identifiable quotes. One expert requested anonymity. Subjects were free to withdraw at any point without explanation and were under no obligation to answer questions.
Patient and public involvement
Patients and the public were not involved directly in this study.
From 2734 references, 21 curricula documents were eligible for inclusion (figure 1), ranging from 1998 to 2017. There were 12 papers from the USA30 37–47 with the remainder from Canada (n=3),48–50 Germany (n=2),51 52 the Netherlands (n=2),53 54 Australia (n=1)55 and a collaboration between the Netherlands, USA and Germany.56 Documents described academic (n=10), clinical (n=6), a combination of academic and clinical (n=2) and ‘comprehensive’ curricula (n=3) (developed by professional HI bodies not affiliated to one academic establishment or workplace, eg, Canada’s Health Informatics Association). Curricula were aimed at all HI professionals (n=10) doctors (n=6) and all healthcare professionals (n=4). One study45 did not specify the target audience. The intended learning setting was university (n=11), workplace (n=9) or online (n=1). Most documents did not specify methodology for formulating competencies (table 1).
Curricula documents were mainly for higher education qualifications, requiring active uptake by the student and too detailed for the frontline clinician.38 43 52 The competencies described varied widely even between MSc degrees with the same target audience, for example, focused on technical knowledge and computer science51 versus more comprehensive coverage of topics including ethics and communication.39 One curriculum document identified 10 core HI competencies for healthcare professionals by asking delegates attending the American Medical Association (AMA) 1999 Spring Conference to discuss their views on HI education in small group workshops.30 However, it does not specifically target doctors, and although suggested competencies are relevant, the broad themes overlook more recent topics, such as e-prescribing.
Mapping of documents to ‘IMIA+’ domains identified three missing competencies, leading to the iterative addition of domain 46: international developments,53 domain 47: medical physics51 and domain 48: teaching50 (table 2). Domains 5 (information literacy), 9 (information systems in supporting patients and the public) and 12 (structure design and analysis of the health record including data quality) were present in all documents, highlighting their importance. Health information science, chemoinformatics and nanoinformatics (domains 41–43) were not present in any documents, which is compatible with the ‘optional’ status given to these domains for BMHI specialists within the IMIA recommendations.35 Any competencies relating to health information science were usually mapped in an HI context, for example, domain 10: regional networking and shared care, present in 81% of documents. Document-derived domains 46 and 47 (medical physics and international developments, respectively) were both present in 10% (n=2) of documents. Domain 48 (teaching) was present in 33% (n=7) of documents. The three iteratively added domains demonstrate that the IMIA recommendations represent the majority, but not all, of the HI curriculum. Full mapping is in online supplementary appendix 2.
Wide variation in definitions of ‘core’ and ‘introductory’ competencies across the reviewed documents underscored the need for expert consultation. The majority of competencies across the documents were attributed as core: 41.2% (n=247); the remainder were classified as core standard: 6.3% (n=38), standard: 42.5% (n=255), standard specialist: 4.7% (n=28) and specialist: 4.7% (n=28). Mixed knowledge combining core–standard–specialist skills accounted for 0.7% (n=4): an uncommon situation where document competencies with multiple skill levels mapped to a single framework domain. Forty-three domains were identified as core skills ranging from 71% of mapped documents advocating domain 48 (teaching) as a core competency to 13% for domain 15 (biomedical modelling and simulation).
Curricula content analysis
A total of 71 curricula were analysed (online supplementary appendix 1). Child & Adolescent Psychiatry and Core Surgical Training curricula did not contain any of the keywords searched. No curricula contained the keyword e[-]Health. Of the remaining 69 curricula, 96% (n=66) contained Technolog*, 46% (n=32) IT System, 29% (n=20), Electronic [Health/Clinical/Medical/Patient] Record and 14% (n=10) Informatic*. Mapping of the curricula led to the addition of domains 49 (employ new technologies appropriately, including information technology) and 50 (proactive approach to new technology).
Out of the 50 domains, 29 were present across 71 curricula documents. Domains 3 (efficient and responsible use of information processing tools to support healthcare professionals’ practice and their decision making), 4 (use of personal application software for documentation, personal communication including internet access, for publication and basic statistics), 11 (appropriate documentation and health data management principles), 26 (basic IT skills) and 49 (employ new technologies appropriately, including information technology) were mapped in over 50% of curricula (table 2; online supplementary appendix 3). Chemical pathology and paediatric and perinatal pathology mapped the highest percentage of domains (32%; n=16), followed by histopathology, general practice (GP) and anaesthetic curricula all mapping 30% of domains (n=15). Surgical specialties mapped particularly poorly with 9/10 specialties mapping only 2% (n=1) of the domains (figure 2). Thirty per cent (n=21) of all curricula (n=71) mapped between 0 and 2 of the total 50 domains. Full curricular mapping is in online supplementary appendix 3.
HI competencies were unanimously felt to be essential for front-line clinicians. Experts found the IMIA framework useful as a guide, but not comprehensive, making iterations necessary. Introductory domains did not alone fulfil needs of a clinician; most of the introductory and intermediate competencies were identified as necessary skills. Many domains were felt to comprise a mixed skillset, for example, introductory domain 12 (structure, design and analysis of the health record). Analysis of the health record is a universally required skill, but system design is an advanced skill more appropriate to CCIO level for system procurement. Similarly, some advanced IMIA domains, such as domain 18 (use of informatics tools to support education), were considered universally relevant. Regarding skill levels, ‘universal’ competencies for all clinicians were felt to be more important than the IMIA framework’s IT user/BMHI specialist distinction, validating findings of the scoping review, where curricular documents identified core competencies across all IMIA skill levels.
Experts felt that the iteratively added domains had mixed utility. Domain 48 is a vital skill throughout clinical training and therefore naturally extends to HI in both teaching other clinicians and empowering patients to engage with HI resources. Domain 46 (international developments) was acknowledged as important but not considered a universal requirement. Domain 47 (medical physics) was not considered to be in the remit of HI. Experts agreed that domains 49 (employ new technologies appropriately, including information technology) and 50 (proactive approach to new technology) were too vague to have impact on HI skill dissemination throughout the workforce and were symbolic of a ‘tick-box attitude’ in curricula development.
All experts agreed that HI was grossly under-represented in postgraduate medical training. While the framework and iteratively added competencies covered the majority of HI education (at all levels, not limited to working clinicians), some areas were felt to assume more detail than was explicitly stated, including secondary use of data, digital communication, system and staff safety and patient empowerment. These areas were all identified in the recommended universal HI competencies, in addition to information governance/security, information and knowledge management and emerging technologies (figure 3). Experts involved in curricula development during their career felt that HI should be integrated into pre-existing competencies, since a dedicated HI subsection would neglect relevance of HI across healthcare.
To our knowledge, this is the first review of HI curricula available for postgraduate medical training and the first comprehensive mapping across all specialties at country level, with four key findings of global relevance. First, existing curricula for HI are mostly developed in North America (15/21; 71%). Second, curricula have been developed without standardised methodology, often omitting key HI competencies. Third, across 71 UK postgraduate medical specialty curricula, the 50 competencies identified as important for HI training were poorly represented and absent in two specialties (Child & Adolescent Psychiatry and Core Surgical Training). Fourth, given poor adoption of exhaustive lists of HI competencies (eg, IMIA), we proposed a pragmatic list of six universal competency domains for all postgraduate doctors.
The emphasis of heads of government and the WHO on digital health, precision medicine and artificial intelligence highlights the need for critical awareness and skills in HI for every doctor.10 11 57 However, our scoping review shows that existing postgraduate curricula for HI are developed in a very small number of high-income countries. It is likely that existing digital competencies are neither tailored nor relevant to all doctors in all contexts. Middle-income or low-income countries are unrepresented in the literature. The ‘digital divide’ has been extensively described58–60 but is understudied with respect to education and training of doctors in HI, which may especially exacerbate inequities within and across countries. The USA has led in the field of HI, in terms of research, training and practice. The vast majority of EHR providers and other HI-related companies are based in North America, predominantly in the USA. Moreover, in the UK, recent reviews of NHS healthcare information technology have been led by US experts, including the ongoing Topol review of the role of artificial intelligence, genomics and digital medicine61 and the Wachter review.4 Although IMIA competencies are comprehensive and the most up-to-date international standards for HI, local adaptation is necessary for training, research and practice in all countries, including the UK, which requires local training and expertise.
Current HI curricula are stand-alone, require independent uptake by the clinician or focus on specialist HI skills. The current model of motivated clinicians seeking out short-term training or qualifications at masters or doctoral level in HI, often at considerable expense, is neither sustainable nor scalable. For example, of 7660 doctors registered to undertake NHS e-Learning for Healthcare modules between 2012 and 2016, only 250 doctors completed all five modules and 151 doctors completed one to four modules.62 ‘Medical education’ is often interchangeably used to signify both postgraduate and undergraduate education, meaning that several curricular documents had an unclear target audience.37 44 45 51 53 54 56 Institutions may be approaching HI education in the same way for undergraduate students and postgraduate doctors,53 despite differing educational63 and knowledge needs64 of the two groups. In a rapidly moving discipline, standardisation is essential, but curricula have not been produced with reproducible methodology, making the evidence base for existing standards questionable at best. Our method, adapted from a prior study, uses literature review, curricular content analysis and expert review to iteratively develop competency domains and could be used routinely in areas other than HI.
HI competencies in current UK training curricula for clinicians are limited, fragmented and often neglected. It is perhaps unsurprising that specialties such as pathology, general practice (GP) and anaesthetics mapped to the highest percentage of domains. These specialties have embraced digitisation,65 66 especially GP, which has near universal EHR use in the UK.67 Conversely, it is of concern that while the government is aiming for a paperless NHS by 2020,68 Child & Adolescent Psychiatry and Core Surgical Training curricula do not contain any of the HI keywords. Fifty-nine per cent of curricula contained the stock phrase ‘Employ new technologies appropriately, including information technology’, but 14% of curricula contained no other HI-related competencies. Such ‘token’ competencies lack specificity and are difficult to implement for both trainees and trainers. Creating ‘future-proof’ training at postgraduate level requires regular review and updating of competencies for doctors, but the curriculum is already overcrowded. Therefore, our list of six universal HI competency domains offers a solution, to which areas of increasing importance can be added iteratively, for example, the role of research, artificial intelligence and multiomics (which are largely ignored, even in IMIA competencies). HI skills must also be balanced with the non-analytical, humanistic aspects of medicine to produce an ‘even more effective physician for the future’.69
Although evidence that HI training improves the likelihood of success of digital implementation programmes is currently lacking, there is strong evidence that current IT programmes and current efforts to train clinicians in HI are suboptimal.14 70 Therefore, universal postgraduate training in HI represents the best way to create change at scale, alongside training of leaders and individuals seeking more specialist training. Generic, stand-alone competencies are likely to be ineffective because training curricula, trainee needs and working environments vary greatly. Lack of competency integration could explain the suboptimal implementation of NHS-endorsed HI skills to date. Competencies in evidence-based medicine71 and medical ethics72 have been successfully integrated into training of all doctors. Given current deficiencies in HI curricula, both in terms of literature review and mapping of UK postgraduate specialties, we provide a simpler framework of universal HI competencies for all postgraduate doctors. Improving ‘HI literacy’ among doctors is an international agenda that requires three actions. First, specialty organisations (eg, Royal College of Physicians) must identify opportunities to integrate HI competencies across multiple curricula, for example, record keeping, audit and research. Second, the validation and adaptation of our six universal competency domains must be overseen by postgraduate leads in medical education in collaboration with professional HI standards bodies (eg, FCI). Third, the optimal implementation plan for universal HI competencies for all postgraduate doctors will require consultation with a wide range of stakeholders, including regional/national leads in HI (eg, CCIOs), the HI research community (eg, HDR-UK), government (eg, NHS Digital and Department of Health) and professional regulators (eg, GMC).
The major strengths of this study are its comprehensive consideration of HI training across all postgraduate specialties; the use of previously used methodology and frameworks to iteratively develop HI standards for curricula; and independent expert feedback. A possible limitation was the subjective nature of determining keywords and mapping competencies between the IMIA+framework and curricular documents. Additional keywords could have been included in the curricula keyword search, uncovering further HI competencies. Bias was reduced by addressing inter-rater reliability with a second researcher when identifying papers for inclusion, involving experts in academic and clinical HI to determine keywords and consulting a range of experts. Another limitation is that not all competencies listed in the IMIA+framework are relevant to each specialty; for example, domain 29 (technical informatics/computer science) is only mapped in the Aviation & Space Medicine Curricula. HI competencies for training doctors were studied, but those for other clinicians, such as associate specialists, staff grade clinicians and consultants, have no national curricula, making it impossible to assess if HI skills are being promoted as part of their continuing professional development (CPD).
This study has identified that HI education for postgraduate doctors is not fit-for-purpose, partly due to inconsistencies in HI terminologies and scope within existing HI curricula. It is unsurprising that without agreement on what to teach, postgraduate training curricula often represent HI with a ‘token competency’ approach. Our method, adapted from a prior study, uses literature review, curricular content analysis and expert review to iteratively develop competency domains and could be used routinely in areas other than HI. We propose 20 universal HI competencies for integration into pre-existing curricula for doctors (see figure 3). A focus on ‘digital maturity index’ of hospitals and clinical institutions alone, without a ‘digitally-ready workforce’, will restrict the enormous potential benefits of digital healthcare. ‘Technology is as much an integrated tool of the practitioner as is the stethoscope’30 and should be recognised as such within clinician curricula for the digital healthcare era.
Contributors The study was conceived by AB. Together, AB and LJ designed data collection tools, monitored data collection, wrote the statistical analysis plan, analysed data and produced the initial draft of the manuscript. LJ, MA and AB were responsible for collecting data. AB was guarantor. MA collected data for the scoping review. PB contributed to design of data collection tools and monitoring of data collection. All authors had full access to all data and can take responsibility for the integrity and accuracy of the data analysis. All authors contributed to the writing and revision of the manuscript. All authors have accepted the final version.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Ethics approval Ethical approval was granted by University College London.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement The full dataset is available from the corresponding author. All data were publicly available (no patient-level data were used), and therefore consent was not required.
Patient consent for publication Not required.