Article Text
PDF
Abstract
Objectives This study aimed to determine the impact of the United Kingdom's National Health Service Digital Academy (NHSDA) Digital Health Leadership course on high-level recommendations in digital preparedness and the development of a proficient leadership to oversee digital transformation, which has been a longstanding priority within the NHS.
Design A mixed methods study incorporating an online questionnaire, in-depth interviews and focus groups that were then analysed through a thematic analysis, underpinned by a constructivist approach.
Settings An online mixed methods study of a defined cohort of participants who had completed the NHSDA course.
Participants 26 participants were recruited to the study, of whom 50% were clinicians, 26.9% in management and 19.2% in data science. All had completed the 2-year NHSDA programme in Digital Health Leadership more than 6 months prior.
Results Interviews and focus groups elicited two key areas of impact of the course: loco-regional digitisation and the development of a network of change agents. The dissertation project had direct effects on local digital transformation efforts. Most of these projects focused on clinician (11.7%) or service user (10.3%) engagement, as oppose to de novo digital processes (9.4%). The development of a network of digital leaders has facilitated communication between organisations and improved the efficiency of the national digital infrastructure.
Conclusions A bespoke course incorporating a dissertation of practice model for digital health leaders can have broader impact for the attainment of digital priorities. This includes helping trusts to successfully adopt digital solutions, as well as fostering shared organisational learning. These influences, however, are mediated by resource and cultural barriers, which continue to hinder transformation efforts.
- Medical education & training
- Health informatics
- Education & training (see Medical Education & Training)
Data availability statement
Data are available on reasonable request. The data that support the findings of this study are available from the authors but restrictions apply to the availability of these data, which were used under licence for the current study, and so are not publicly available.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- Medical education & training
- Health informatics
- Education & training (see Medical Education & Training)
Strengths and limitations of this study
This study is the first to critically examine the impact of the NHS Digital Academy on high-level digital priorities.
Using a robust mixed methodology, we enrolled previous candidates, who hold prominent national roles in digital health transformation, to determine spheres of impact.
Our findings provide a blueprint for the field of digital health leadership including similar digital courses globally
By highlighting areas that require further examination, we also set future directions in the education of digital health leaders.
This study is limited to those who have completed the NHSDA course and cannot provide comparisons with the effects of similar courses.
Background
Delivery of a digitally transformed healthcare service has been at the forefront of NHS plans for over a decade. Estimates indicate that this digital transformation strategy will cost £8.1 billion, with all NHS trusts mandated to reach a core level of digitisation by 2024.1 Progress has been variable. 17% of trusts still rate their digital maturity as medium or low,2 despite the increasing importance placed upon achieving national digital priorities.
The COVID-19 pandemic galvanised the move towards digital solutions, with traditional models of care no longer appropriate. Policies to avoid transmission of COVID-19 have pushed digital solutions for facilitating patient–clinician interaction, transitions of care and the accessibility of healthcare services.3 There is therefore greater need for NHS trusts to attain greater digital capabilities to carry out even routine functions.
Crucial to achieving these aims and enabling local digital transformation is the development of a digitally prepared workforce and a digitally proficient leadership team. The Wachter Review highlighted the importance of establishing a digital workforce and advocated for growth of local digital leaders and the chief clinical information officer (CCIO) role.4 The remit of these digital leaders is to oversee the rapid uptake of digital solutions and to guide sustainable organisational transformation in a manner that avoids the mistakes of previous failed attempts. Moreover, leadership must now examine future opportunities such as integrating electronic health records across providers and the safe application of artificial intelligence.5 Ensuring these leaders are qualified and supported with a network of resources is central to the realisation of this vision.
As a response, the NHS Digital Academy (NHSDA) developed its flagship course to provide the necessary training, mentorship and support to these leaders to facilitate meaningful change at a local, regional and national level. Originally commissioned for delivery by Imperial College London, in conjunction with the University of Edinburgh and Harvard University, the programme is divided into two fully accredited components.6 The first, culminating in a postgraduate diploma (PGDip) in Digital Health Leadership, uses a blended learning approach to provide a theoretical foundation in key transformational topics including user-centred design, decision support and actionable data analytics. Following the year-long PGDip, candidates are given the opportunity to proceed to a 1 year master’s (MSc).7 The MSc uses a Dissertation of Practice model to enable project based learning and practicable applications of theoretical constructs to achieve digital transformation. While curriculum evaluations and feedback have focused on individual aspects of the course, little is known about the programme’s impact on the delivery of national and regional targets for digital transformation.8 The aim of this mixed-methods study is to critically evaluate the impact of the NHSDA Digital Health Leadership course on high-level recommendations in digital maturity and preparedness. This study enables critical understanding of the extent to which the needed levels of digital health leadership have been established within NHS trusts and provides a blueprint for further courses development both nationally and internationally.
Methods
Recruitment
Participants in cohort 1 of the NHSDA’s flagship Digital Leadership were recruited into the study. To assess the impact of the course on attainment of organisational and national digital priorities, the study targeted participants from the first cohort, who completed the MSc. This mitigated recency effects, ensuring that participants had time to apply acquired knowledge. The study diagram is demonstrated in figure 1.
Diagram of the participant flow throughout the study.
Scoping questionnaire
An online scoping questionnaire was developed (online supplemental material 1), which was mapped to programme learning objectives. The questionnaire focused on assessing impact of the programme on individual skill acquisition, organisational aims and NHS priorities. The survey also encapsulated some aspect of the existing curriculum feedback that participants undergo during the course. This focus supported exploration of the influence of the dissertation project on high-level priorities. Two authors (RCB and AA) developed the questions of the survey, with a third (AS) independent author was involved to derive consensus and discuss disagreements. All study participants were recruited via an email containing an anonymous link, such that central tendency biases were reduced.
Supplemental material
Interviews
Following completion of the survey, anonymous responses were quantitatively (multiple choice questions) analysed. Two authors (RCB and AS) used the results from these responses to develop topic guides for both semistructured interviews and focus groups (online supplemental material 1). Enrolment to an interview was not dependent on completion of the survey. Interviews were conducted online via Microsoft Teams (Microsoft Corporation, USA) due to the restrictions imposed by the COVID-19 pandemic during the conduct of the study. Interviews were undertaken by an author not affiliated with the NHS Digital Academy or its faculty (AA) to reduce response bias.
Focus groups
Two online focus groups were subsequently conducted by an author not affiliated with the NHSDA (AA). Each focus group involved up to five participants. Participation in the focus groups were not dependent on previous completion of either the survey or interview. Focus groups used open-ended prompts with discussion between participants facilitated to understand shared experiences and to provide a deeper understanding of issues raised in the previous stages of the study.
Analysis
Responses to the survey conducted online using the experience management software, Qualtrics (USA), were aggregated. Frequencies were calculated for each response across each option. Given the relatively small sample size for this study, no further analysis was conducted on quantitative responses. The response frequency was discussed between authors and used to guide questions in semistructured interviews. Specifically, areas authors felt were contentious, or where there was a strong consensus among respondents were the focus of further examination.
To facilitate analysis of online interviews and focus groups, audio was recorded with the explicit consent of participants. Transcription was conducted using an online tool Descript (USA), with the transcript fidelity verified by the researcher who conducted the interviews (AA) directly following the interview/focus group. Anonymised transcripts were analysed using an established thematic analysis approach, until thematic saturation was reached.9 Initially, transcripts were reviewed to provide familiarisation of the data. Transcripts were then coded by two independent authors (AA and RCB) and uploaded to a qualitative analysis tool, MAXQDA (Germany) to facilitate data exploration, for example, by determining the frequency of each code. Codes were aligned with recommendations from the Wachter Review to highlight key areas of impact,4 which was the driving force for the establishment of the NHSDA. Subsequently, three authors (AA, RCB and AS) reviewed codes and developed subthemes, and then high-level impact themes drawing together codes into broader areas of impact. These themes were reviewed against the anonymised transcripts and amended as appropriate until there was thematic consensus. Disagreements regarding allocations of codes, themes and subthemes were discussed among authors until consensus was determined. Subthemes that were consistently mentioned by participants, those which aligned with findings from the survey and those that were highlighted as strong determinants were considered more prominent influences. A constructivist approach was used to underpin the conduct of this study paralleling the active learning undertaken throughout the MSc. This paradigm focusses on the importance of learning as an active process of engagement and reflection, influenced by context and motivation.
Patient and public involvement (PPI)
As this study represented an educational evaluation of a novel digital health leadership course, no PPI was involved.
Results
A total of 26 out of 28 eligible participants completed the online questionnaire. A percentage of 34.6 of responses were female. Fifty per cent of respondents to the questionnaire were primarily clinicians, while 26.9% were management focused, and 19.2% were involved in data science or information technology. Interviews were conducted among 10 participants. The demographics of this subgroup were similar to those participating in the original survey, with 40% female and 40% based in non-clinical roles. Two focus groups were subsequently undertaken, in which 44.4% of the participants had not been interviewed previously.
From the analysis, two broad area of impact of the NHSDA were elicited: (1) loco-regional digitisation of NHS Trusts and (2) the development of a network of digital change agents. Table 1 demonstrates how the course impacted these areas through quotations from respondents.
Demonstrating the breadth of impact in across areas, with participant interview quotes highlighting the main themes and subthemes elicited
Theme 1: loco-regional digitisation
The NHSDA was shown to impact on regional digital development, primarily because of participants’ MSc dissertation projects. A percentage of 67.6 of respondents reported their choice of project was influenced by high-level national priorities outlined in documents including the Wachter or Topol reviews. These priorities included the attainment of digital maturity of NHS Trusts by 2023, with 19.4% respondents reporting direct influence of these aims on their project. A description of the dissertation projects for the 28-person cohort mapped to the four main themes from the Wachter Review are listed in table 2. The impact of the course on loco-regional digitisation efforts was further subdivided into direct and indirect influences.
Cohort dissertation topics mapped to Wachter review main themes
The majority of projects were focused on increasing engagement of healthcare professionals (11.7%). This was greater than those concerning service user engagement (10.3%) or organisational barriers to digital transformation (9.7%). Fewer projects aimed to develop de novo digital processes with 9.4% of respondents reporting that integrating care through digital technology was a project priority. Key to the projects aiming to improve engagement was the effect of the MSc on candidates’ perceptions of digital transformation in their organisations. One CCIO of a tertiary service had ‘previously trialled everything’ to transform outpatient services. Within the MSc project, however, the participant found ‘it [the NHSDA] made me realise it’s not about persuading them [consultants] with data…but ask for explicit reasons’.
Similar themes were appreciated by participants trying to engage healthcare professionals in non-clinical scenarios. Following the development of a shift suggestion engine to address workforce shortages, one participant recalled that ‘the knowledge from the MSc made me understand the motivating factors that made people pick up additional work’. As such, this impact on candidates’ perspectives had the potential to improve the outcomes of digital transformation targets. As one of the informaticians examining ‘bring your own devices’ noted, rather than resisting contrary opinions, understanding ‘there is a real reason why people think what they think…and valuing that, drove a better digital product’.
The NHSDA course also had indirect influences on digital transformation. Several respondents reported how the lessons from their MSc projects were used as templates in other areas. One CCIO noting they were ‘able to take my research and go to NHSI and say this is what I’ve done with NHSDA… can we deliver a wider digital solution’. As such the impact of the course was not limited to directing digital transformation in isolated organisations or fields. A CCIO involved with the implementation of an advice and guidance platform for GPs, mentioned that with organisational support the project was to be ‘scaled up throughout the organization…in all different specialties’. It was also mentioned that ‘other trusts who want to implement it off the back of our experience’, highlighting indirect impact.
The extent to which these projects could be integrated on a wider scale, and even within organisations, however, was context dependent. As one participant, who was head of information at their Trust noted, ‘ you can’t broad scale culture change’, finding that with a more forward-looking project, ‘you need to find a way to bring everybody together to discuss first’. Logistical issues also curtailed the impact of the projects, with a divisional nurse informatics lead noting, despite ‘executive coach buy-in…and the corporate team saying it’s a great idea… we didn’t have the resources’. It was noted that was ‘before COVID…now I think it will be easier for us [the organisation] to reach our goal’. Which is contrary to another CCIO who found ‘when COVID hit…I was suddenly overwhelmed and couldn’t do it [implementing the project] justice’.
Theme 2: a network of digital change agents
In addition to the impact on digitisation efforts, the NHSDA also led to the development of a network of digital leaders. When asked of the main lessons from the course, only 4.4% of survey respondents mentioned of the alumni network. However, throughout focus groups and interviews, the importance of the network was apparent, suggesting an unintended effect of the course. This, according to one CCIO, was fostered by ‘the residentials in which networking was at the fore’.
This network directly impacts how programme alumni undertake their roles. One respondent noted that having ‘a network of people, that is the number one thing that is useful on a weekly, if not daily basis’. By leveraging shared experience, respondents improved task efficiency for those in leadership roles. One CCIO said, ‘when I reach out to another organisation, I immediately ask who does X. If I don’t know their CIO it takes me a second to find out, although 9/10 times its someone I know. We then get things together, share resources, connect, and make things work’. As a result of this network, individuals have access to a breadth of experience from not only CCIOs, but informatics and data leads, ‘with a shared understanding of digital to allow transformation at pace’. The value of this was also being recognised by organisations. One CCIO said, ‘by demonstrating the value of the network I’ve [they’ve] created’, they were being ‘pulled into bigger and bigger projects’ by their Trust.
Participants did acknowledge the ongoing development of a formal professional community of informaticians and CCIOs independent to the NHSDA. They found the course provided accessibility not previously available, with one clinical manager noting ‘digital health was a very lonely place before the PG Dip from a provider point of view’. Moreover, they suggested since the NHSDA, this group of digital leaders is now setting the agenda for change. As one CCIO noted, the community previously lacked ‘direction or focus, as they hadn’t had people who understand transformation in the way that the MSc has put it across’. As such, the network provided not only a forum to facilitate digital transformation locally, but a way to set and support wider priorities. These wider projects and priorities can be more effectively undertaken as a result of the MSc, as it has helped to harmonise perceptions of digital transformation and given an identity to digital leadership. One informatician noted ‘we’ve got a real understanding of who the leaders [in digital] are now, and the PG dip really helps with identifying those [leaders]’. This suggests the NHSDA course has positively impacted the NHS’s wider digital agenda.
Discussion
This research is one of the first mixed methods studies to demonstrate the impact of a focused programme on digital health leaderip on the attainment of national digital priorities. Findings show this impact can be broadly categorised into two themes: loco-regional digitisation and the wider digital health leadership. By integrating a dedicated period of research within an MSc year, the NHSDA provides a unique opportunity for candidates to directly lead digital projects within their organisation. The focus of these projects varies greatly, including healthcare professional engagement (11.7%), service user utilisation (10.3%), organisational barriers to transformation (9.7%) and digital care integration (9.4%). The successful adoption of these projects has been suggested to act as a template to transformation efforts in other fields and regions. However, this impact was variable among participants and was largely context dependent. We also found unexpected outcomes that may facilitate longer term digital transformation beyond the completion of the course. Most notable of these was the development of a network of digital leaders and change agents, who leverage their shared experiences to improve the efficiency of transformation. This first cohort of MSc alumni are now being recognised as digital leaders and appear to be setting the directions of the regional and national digital agenda.
The drive towards achieving digitally enabled healthcare has been a long-standing priority for the NHS.10 The COVID-19 pandemic has greatly accelerated this push, not only through necessity, but by changing the perceptions of what can be achieved through digital care.11 As recovery from the pandemic continues, digital leaders must facilitate the adoption of technology in a way that is sustainable and benefits relevant stakeholders. According to the Five Year Forward Review, the NHS Digital Academy was developed to upskill digital leaders, in order to increase the chances of successful adoption of new technologies.12 This evaluation shows that this 2-year flagship course has directly achieved this aim by providing an opportunity for candidates to undertake a local transformation project. These projects directly align with national benchmarks set for Trusts, such as improving accessibility of healthcare records and using data to improve population health outcomes, thus assisting regional digital transformation.13 By enabling participants to focus on the needs of their organisation, and undertake the project while faculty support is available, these projects are able to drive organisational change at pace. This organisational and system transformation is augmented by the theoretical foundation established during the first-year diploma in digital health. The combination of didactic and practical learning appears to improve the likelihood of project success. For example, the project on outpatient services succeeded where previous attempts had failed by acknowledging alternative perspectives of users.
This appreciation of user-centred design, a module during the PGDip, was an important lesson for many participants. Almost a quarter of projects from the cohort centred on user (clinician or patient) engagement. Moreover, several interviewees attributed the successes of local adoption of projects to their newfound ability to integrate user perspectives. This is one of the most important, and potentially, long-lasting influences of the course. Prominent failures in digital adoption, such as the National of Program for IT, have often been ascribed to a lack of engagement.14 This is particularly important as the freedom afforded to digital initiatives by users during the unprecedented COVID-19 response subsides, and the need for longer term adoption strategies grows.15 By not only teaching but having future digital leaders use this learning in their practice, the course may help avert the costly mistakes of the past.
In addition to the impact of the dissertation projects, one of the most notable long-term effects of the NHSDA, though not by intention, has been a network of digital leaders. By incorporating writing residentials and collaborative work groups, the course has fostered the formation of a highly supportive and engaged digital leader collaborative. As stated in the NHSDA’s scoping report, one of the limitations of a didactic course is its ability to affect leaders from vastly different organisations.8 By developing a network of change agents, however, digital leaders can share ideas, translate digital successes and appreciate the setbacks of other organisations. This network, therefore, helps to breakdown silos and help regional digital assimilation, in way that integrated care systems will aim to build on.16 Unlike the Global Digital Exemplar programme, however, this shared learning is purely born out of collaboration and support, as opposed to competition.17
As shown within our findings, such a network also helps delineate the identity of digital health leaders in the UK. This is also facilitated by the prominence of the work of many of the candidates, such as the NHSX adoption of ‘bring your own device’ or through academic publications on SNOMEDCT.18 19 The output however is highly variable, and as with similar digital leadership courses, the impact is mediated by external factors. A number of candidates echoed sentiments from other publications, suggesting that factors such as resource limitation and governance inevitably have a greater effect on the adoption of digital technologies than leadership.20 21 While the network can impart learning, without the appropriate infrastructure and recognition, digital leaders will not be able to effectively enact change. Organisational support and adequate credentialling of digital leaders, therefore, remain important barriers to impact and transformation.22 23 While courses such as the MSc in Digital Health Leadership provide an accredited qualification, this does not yet necessarily translate into professional certification for those in NHS management positions.24 This is beyond the scope of this manuscript, however, which aims to examine wider impact only. Future work will examine the influence on personal progression and professional identity that such courses have on current and prospective digital health leaders.
Despite a robust methodology, this study must be considered in relation to its limitations. First, this evaluation represents the experiences of only a proportion of individuals that have undertaken the course, which is now entering its fourth year. While this affects the generalisability of the results, the use of a mixed methods approach, including an anonymised survey distributed to a larger group, does mitigate this limitation. The response rate to this survey was high, with 26 from 28 eligible candidates answering (92.9%). This would ensure that the breadth of experiences from candidates was included within the current evaluation. This high response rate would also reduce the effect of response biases, seen in similar qualitative assessments. While there is still potential for such selection biases to impact on the study’s findings, as the demographics of the interviewees were similar to the wider cohort, this reduces the influence of these effects. The study also reduced recency biases by ensuring that only individuals who had completed the course at least 6 months previously were recruited.
Conclusions
Despite these limitations, this study has shown the relative successes and shortcomings of the NHSDA 2-year course on the attainment of wider digital priorities. Through a blended approach incorporating didactic and practical aspects, it is possible to directly and indirectly impact on the digital transformation efforts of the NHS at a loco-regional level. This is principally achieved through a dissertation project that is directly relevant to candidates’ organisations. Resource and cultural barriers, however, continue to hinder digital transformation. Future work should examine how such courses can help overcome these challenges, as well as how they affect individual learning and the professional development of digital leaders. In this way, a holistic view of such courses can be understood, which will in turn facilitate the creation of a digitally prepared workforce.
Data availability statement
Data are available on reasonable request. The data that support the findings of this study are available from the authors but restrictions apply to the availability of these data, which were used under licence for the current study, and so are not publicly available.
Ethics statements
Patient consent for publication
Ethics approval
Ethical approval was sought from the Institutional Review Board at Imperial College London (EERP2021-026a). All participants provided informed consent prior to taking part in any stage of the study and provided written consent. Participants gave informed consent to participate in the study before taking part.
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
AA and RCB are joint first authors.
AA and RCB contributed equally.
Contributors AA, AS and RCB were all involved in the study design, conduct and data analysis. AA and RCB drafted the manuscript with AS involved in editing and review of the submission. RCB is the guarantor of the study. AD provided infrastructural support that enabled the study to occur and oversaw study conduct.
Funding Imperial Biomedical Research Centre (BRC) provided infrastructure to design and conduct the study. The funder had no role in the conduct, analysis or dissemination of this work. Researchers were independent from funders, and all authors had full access to all the data in the study and can take responsibility for the integrity of the data.
Competing interests AD is the codirector of the NHS Digital Academy. RCB is the programme head of the MSc Digital Health Leadership. Both authors acted independently during the conduct of this study.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.