Article Text
PDF
Abstract
Objectives To explore and explain success and limiting factors in UK health service innovation.
Design Mixed methods evaluation of a series of health service innovations involving a survey and interviews, with theory-generating analysis.
Setting The research explored innovations supported by one of the UK’s Academic Health Science Networks which provides small grants, awards and structural support to health service innovators including clinical academics, health and social care professionals and third-sector organisations.
Participants All recipients of funding or support 2014–2018 were invited to participate. We analysed survey responses relating to 56 innovation projects.
Results Responses were used to conceptualise success along two axes: value creation for the intended beneficiaries and expansion beyond its original pilot. An analysis of variance between categories of success indicated that participation, motivation and evaluation were critical to value generation; organisational, educational and administrative support were critical to expansion; and leadership and collaborative expertise were critical to both value creation and expansion. Additional limiting factors derived from qualitative responses included difficulties navigating the boundaries and intersections between organisations, professions, sectors and cultures; a lack of support for innovation beyond the start-up phase; a lack of protected time; and staff burn-out and turnover.
Conclusions A nested hierarchy of innovation needs has been derived via an analysis of these factors, providing targeted suggestions to enhance the success of future innovations.
- health services administration & management
- health policy
- change management
- quality in health care
- organisational development
Data availability statement
Data are available upon reasonable request. Due to the highly individual nature of healthcare innovations and the limited geographic area of this study, we are unable to provide our raw data. We undertake to provide a redacted data set upon reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- health services administration & management
- health policy
- change management
- quality in health care
- organisational development
Strengths and limitations of this study
A strength of this research is that it compares many innovations in a consistent way, and that it provides insights across a range of categories of success.
A limitation of this research is that it is situated in a single geographical context; however, repeating these methods in different contexts should produce locally relevant results.
Few mid-level theories relating to innovation are grounded in data that include projects that have not achieved their intended outcomes; therefore, we may have identified novel insights.
Many of the success factors we have identified are not unique to this study; however, they have been subjected to further statistical analysis and found to differentiate significantly across categories of success in this context.
More research is needed to examine whether addressing these factors prospectively enhances the success of future innovations.
Background
Healthcare systems worldwide are faced with increasing demand linked to the rising burden of disease within a resource-constrained environment.1 This has led to a pressing need to find and disseminate innovative ways of meeting the healthcare needs of patients and communities in ways that are more sustainable.2 The WHO characterises health service innovation as ‘a novel set of behaviours, routines, and ways of working that are discontinuous with previous practice, are directed at improving health outcomes, administrative efficiency, cost-effectiveness, or users’ experience and that are implemented by planned and coordinated actions’3 (p 7).
Academic efforts in the health sciences continue to sharpen the focus on impact, rather than the creation of ‘new knowledge’ as the primary goal of research activity. At the vanguard are implementation scientists who work to translate research and innovation into clinical practice, navigating institutional, organisational, structural and cultural complexities to improve services.4 New support structures have emerged, such as the 15 Academic Health Science Networks set up in 2013 by the National Health Service (NHS) England, with funding streams that aim to support and encourage innovation at various levels.5 After more than half a decade of programme grants, the impact of these innovation programmes is a legitimate subject of enquiry: how and why have certain innovations become normalised, sustained or spread, and why have others struggled or stopped?
The knowledge created through an individual innovation is likely to be complex and context dependent, providing insights that may not necessarily be generalisable.6 Meta-analyses are faced with the complexity of synthesising multiple project evaluations that may be reported in different ways. The published literature on health service innovation contains few analyses of unsuccessful innovations, despite attempts to encourage negative reporting.7 By evaluating a large corpus of projects across one of these academic health sciences networks in a systematic way, we have an opportunity to directly compare innovations including those that may have struggled or stopped and not reached the literature.
This study thus sets out to explore a large number of innovations, both as individual projects in their unique local contexts, and as part of a larger integrative study. By isolating the factors that differentiate between categories of success, our aim is to produce an empirically derived explanatory model, and thereby to inform and enhance the success of future innovations.
Methods
Research aim
To explore and explain success and limiting factors in health service innovation.
Methodological orientation and theory
This study is situated at the intersection of policy, social sciences and organisational research. Our philosophical assumptions are that there are real differences in the success of innovations, but also that success is fundamentally a subjective construct. Any research will only produce an approximation of the truth, and findings must be interpreted with an appreciation for context. We therefore position this research at the boundary of critical realism and constructivism.8
Adopting Varpio et al’s terminology on the philosophy of research, we are taking an inductive approach that works towards a theoretical framework, rather than applying a pre-existing theoretical framework to this study.9
We have adopted what Creswell et al refer to as a sequential mixed methods design.10 According to Creswell, insight can emerge from exploring first through qualitative methods (in our case a published qualitative review and interviews) the types of factors that might be important, and then designing questionnaires to explore their salience to a population (called an ‘exploratory sequential design’). Insight can also emerge from collecting survey data initially and then following up with interviews to help explain the survey results in more detail: an ‘explanatory sequential design’. Where both qualitative and quantitative data are collected simultaneously, one set of data can be used to triangulate the other (eg, where the meaning of one is unclear), or they can be used in complementary ways to illuminate each other (eg, one determining which factors are important, the other illuminating why that might be). Our research process involves both exploratory and explanatory aspects as well as triangulation and illumination. It is summarised in figure 1.
Research overview. A mixed methods sequential research process to explore and explain success factors and limiting factors in health service innovation.
Context
The Health Innovation Network (HIN) is one of the nationally funded Academic Health Science Networks set up by NHS England in 2013. It provides small grants, awards and structural support to academics, health and social care professionals and third-sector organisations, supporting service-level innovations to improve outcomes and value, including the sustainable use of resources. In addition, in the years 2014–2017, Health Education England (South London) provided investment in training and education innovation projects across healthcare settings in South London, through its strategic investment programme.
Participants and sampling
All recipients of HIN funding and Health Education England (South London) strategic investment programme funding and support during the years 2014–2018 were invited to participate. As our sample size was moderate, we aimed to analyse all responses and retrospectively determine whether the sample size was sufficient for thematic saturation and statistical inference. We achieved a priori thematic saturation for success factors (exemplar comments for each significant factor that we found) and inductive thematic saturation for limiting factors (content coded until no new themes arose).11
Research team and ethics
The research was commissioned by HIN in collaboration with Health Education England and conducted by an independent research team at King’s College London. The research team comprised a postdoctoral educational psychologist/learning scientist (GBR), a postdoctoral occupational psychologist/health services researcher (AK) and a medical education research fellow (KLG). None were in a position of power or influence over participants, and the research was carefully designed to be conducted at arm’s length from the funding agency. Survey responses were collected anonymously and decontextualised by the research team to encourage innovators to comment critically and safely about their projects. Innovation funding was not conditional on taking part in this research. Ethical approval was granted on 26 March 2019 by the Research Ethics Committee of King’s College London (LRS-18/19–10432). Written informed consent was obtained from interviewees. Consent was implied from participation in the survey.
Patient and public involvement
No patient was involved. The primary stakeholders in this research were health service innovators who were involved in the survey design and in checking back and refining our interpretation.
Data generation methods
Survey design
The survey design began with the extraction of potential success factors for health service innovation from a recently published qualitative systematic review.12 This review aimed to identify all the factors and theories associated with sustainability and scale-up (capacity building) of innovations in health services research. KLG validated and expanded these factors through scoping interviews with five experienced health service innovators. The interviews started with an open exploration of what the innovator felt had impacted on the success of their project, followed by discussion on the factors identified through the literature. Personal factors were mentioned by all stakeholders in addition to the factors from the review, suggesting these may be under-reported. An additional theme (personal factors) with related subfactors was therefore included, based on these interviews. Themes and factors are listed in figure 2. These were used to create a mixed methods nested design survey13 using Qualtrics software (full text in online supplemental data).
Supplemental material
Survey scope. The survey scope was based on a qualitative review of theories and findings relating to the sustainability and scale-up of health service innovations12 supplemented by five scoping interviews. The questions are listed in table 1, and the full-text survey is in the online supplemental material.
Analysis of variance of potential success factors across categories of success
The survey asked respondents to:
Categorise and describe their project’s current status (no longer running/likely to finish soon/stable at the level of the original pilot/scaled up beyond the original pilot/too early to say/other).
Score statements (listed in table 1) relating to the impacts of each factor on their project’s outcomes, grouped into nine themes on a 5-point disagree/agree Likert scale.
Describe the status of their project and provide qualitative insights into each of the nine themes.
Our five stakeholders helped to improve the clarity, acceptability and usability of the survey questions and instructions.
Survey distribution
A neutral administrator from Health Education England distributed the survey by email in August and September 2019 to all 176 named recipients of HIN and Health Education England (South London) funding awards, grants and bursaries. A reminder was distributed 4 weeks later to participants who had not responded. Projects that had received more than one award were sent a single survey, and participants who had run more than one project were sent a separate survey for each project.
Stakeholder follow-up interviews
KLG checked back our results and interpretation with five stakeholders identified by HIN as experienced innovators, one of whom was also involved in the original scoping interviews. Interviews lasted 30–45 min and transcription was facilitated by automated software (otter.ai). These stakeholders helped to refine the model and confirmed its applicability and utility in their context. No new themes arose; however, quotes were used to enrich our survey data.
Data analysis methods
Development of categories of success
KLG and AK categorised projects into grades of success based on how the respondent self-categorised their project, triangulated against their qualitative survey responses. The categories of success were derived through an iterative process, involving both researchers agreeing a descriptive summary of the status of each project (eg, scaled down despite achieving better than expected patient outcomes; scaled down because the intervention did not achieve its aims). We grouped projects with similar project outcomes together, and through a process of constant comparison14 constructed a categorisation framework that accounted for all the cases in the set.
Determination of salience of success factors
We adopted an exploratory approach to data analysis, which aims to generate rather than test theory.15 KLG conducted an analysis of variance (ANOVA) for each of the scored factors (Kruskal-Wallis non-parametric ANOVA on rank using IBM SPSS V.25) to see whether there were significant differences between categories of success. The Kruskal-Wallis test does not assume a normal distribution in the data and can be used when the data are ordinal, for example, Likert scores. For asymmetric group sizes, the non-parametric Kruskal-Wallis test performs better than the parametric equivalent ANOVA method.16
For each factor that was identified as being significantly different between categories of success, we conducted a secondary analysis (box plot for each category) to confirm the direction and consistency of the association. This is generated automatically by SPSS after a Kruskal-Wallis test. A graded ‘exposure-response’ relationship across all grades of success would be expected if a factor genuinely drives success.17 Where a graded relationship was not present, this is discussed in table 1.
Illumination of success factors
KLG and AK extracted quotes from the survey and interviews relating to each significant success factor to generate a rich description within each theme.
Inductive analysis of limiting factors
KLG coded the content of all qualitative data relating to challenges within projects that had not achieved their intended outcomes or that had scaled down or stopped (n=21) facilitated by NVivo V.12 software. GBR and KLG refined the codes and both authors worked together to inductively arrange the content into themes.18
Development of final model
We mapped significant factors onto a 2×2 grid using a natural logarithmic scatter plot so that factors that were significant to one dimension of success were mapped to the right half of the grid, factors that were significant to a second dimension of success were mapped to the top half and factors that were significant to both were mapped to the top right quadrant. We grouped success factors into themes through a process of collaborative discussion, and we explored which themes predominated in each quadrant to generate our model which was checked back with stakeholders.
Results
Descriptive summary
We received 63 responses, but seven were incomplete or duplicate so a total of 56 responses (31.8% of 176) were included in the analysis. Each response related to a different innovation project. Survey respondents self-identified within one or more of the following groups: the project leadership team (n=54); service delivery team (n=9); training team (n=9); administrative team (n=6); service lead (n=2); and patient/service user (n=1). Several respondents identified within multiple groups.
Projects were situated in secondary care (n=19); community care (n=14); academic sector (n=5); mental health sector (n=4); online (n=4); primary care (n=3); and the hospice sector (n=2), with the remainder working at the interfaces between services, or across sectors. Their scope ranged from national programmes at hundreds of sites, local programmes supporting tens of thousands of patients, to small intensive innovations working in new ways with a few dozen complex patients, and their duration ranged from 1 to 5 years. The innovation areas related to new ways of working in end-of-life care; disability enablement; support for complex or vulnerable patients; discharge support; pain management; patient safety innovations; recovery and rehabilitation; personalised care; chronic conditions; new models of integrated health and social care; health promotion; and novel simulation and workforce development strategies. Typical projects can be explored at the HIN website19; however, for reasons of confidentiality, we cannot specify which were included in this study.
Categories of success
Our emergent framework categorised each project’s success across two dimensions: the first relating to whether the innovation was reported as generating more or less than its anticipated value for patients/carers (‘value creation axis’), the second according to whether the project became sustained or scaled up beyond the initial pilot, or whether it was scaled down or stopped (‘expansion axis’). Innovations that were within the scope and intentions of the original pilot were positioned centrally. We initially scored projects into five categorisations across the expansion axis, as some projects expanded locally and some nationally; however, there were not enough projects in each group and statistics became unreliable, so we made a pragmatic decision to adopt relative rather than absolute categories.
The resulting categorical framework is illustrated in figure 3, with the number of innovations in each category shown in brackets.
Framework for categorisation of success within health service innovation Health service innovations were categorised across two dimensions of success through an inductive classification process. The horizontal axis relates to how successful innovations were in realising their intended value: ‘did it create more or less than its intended value for beneficiaries?’ The vertical axis relates to expansion: ‘was the innovation scaled up or scaled down from the original pilot?’ Numbers in brackets indicate how many innovations were found in each category.
Success factors
Our analysis compared the variance of success factors across innovations that had demonstrated lower than expected value (n=11), value as expected (n=25) and higher than expected value (n=20). Next, we compared variance across innovations that had diminished in scope or stopped (n=17), innovations that were running as expected (n=16) and innovations that had scaled up (n=23). Finally, we excluded low-value projects and analysed again across the expansion axis (n=10, n=12 and n=23, respectively), seeking to explore why innovations with proven value had not been scaled up.
Our analysis is presented in table 1 with significant results (p<0.05) shaded in green. At this level of significance, there is a 1 in 20 probability that a result is in fact random. We have used lighter shading to indicate factors that might potentially be significant, or which could be found not to be significant if the power of the study was increased. The final column gives our interpretation of the more significant findings (p<0.05) that takes into consideration our secondary analysis.
Many factors were similarly scored across all categories of success, for example, information technology (IT) infrastructure. This does not mean that these factors are not important, only that they were experienced similarly across all categories of success and are therefore unlikely to be the underlying cause of the relative success or failure of a project.
We have collated the significant factors together in table 2 with illuminative quotes, and we discuss both positive and negative findings within each of the nine survey themes below.
Factors that are significantly associated with innovation success with illustrative quotes
Project-related factors
Interestingly, the aims of the project did not appear to be critical to success. Both successful and unsuccessful innovations were similarly reported as being designed to address an important healthcare need that was concerning to the public. All funded projects were required to articulate a credible evidence base arguing that stated benefits could be achieved through the project plan.
Resourcing and expertise
All significant resource-related success factors were associated with the workforce. Non-critical factors included infrastructure (such as buildings, materials and supplies), which were reported as sufficient; IT, which was moderately good across all categories of success; and funding issues, which were also similar across all categories of success. However, having the right numbers of staff with the right skills appeared to be highly significant, both in terms of the project being able to realise its intended value, as well as for it to become scaled up beyond the original pilot. Staff with time and energy appeared critical to whether successful innovations became scaled up, as were administrative and educational support, including the availability of ongoing educational support (eg, orientation and training for new staff, or to build capacity). Expertise appeared to be critical across all categories of success, both in terms of the innovator feeling they had the right skills, experience or training; the project having access to staff with the right skills; and having external expert input where needed.
External factors
Alignment to societal needs appeared to correlate with whether a project was able to realise its intended value, but less so with its expansion. However, as the effect size did not grow consistently across categories of value creation, we cannot necessarily infer a causal relationship.17 Qualitative comments indicated that projects that were able to align themselves to current political or societal agendas, such as mental health, were more successful. Conversely, those attempting to work in relatively less topical areas of practice described difficulty securing strategic funding, so we have tentatively included this factor in our model.
Organisational factors
Our analysis of organisational factors indicated that the ability of an innovation to integrate into existing organisational structures, programmes or policies may be critical to whether it scales up, and possibly also to its ability to create value (p=0.059). Successful projects described adapting where necessary to achieve a good fit within organisational priorities. For the most part, host organisations were described as having a positive learning culture and were ready and able to undertake innovative initiatives; however, even innovations with proven value were unable to survive if there was opposition within the host organisation.
Personal factors
Few respondents reported being released from other duties so that they could implement their initiative. However, most respondents said they benefited from a supportive peer culture. Respondents who were able to realise value were significantly more like to say they were internally motivated and found working on the project rewarding.
Project management
Most projects measured or assessed the outcomes and impacts of the project, though this appeared to be more common in successful projects (p=0.060). Projects with high value were able to demonstrate and share this success. Leadership appeared to be a highly significant success factor across all categories of success, with struggling or unsuccessful projects citing leadership failures.
The tasks of the project
Similar to theme 2, which explored the aims of the project, the tasks of the project did not appear to be significantly different across categories of success.
Collaborative and participatory practices
Valuing team members’ opinions was highly significant across all categories of success and was present in all projects that were scaled up (hence variance not calculable). Participatory approaches were significantly associated with the ability of an innovation to generate value. These participatory processes related to the staff delivering the innovation, the intended beneficiaries and the communities in which the innovations were situated.
Finally, one of the most significant differentiating factors across all categories of success was engagement with a collaborative network that helped to support and sustain the initiative.
Limiting factor analysis
In addition to the above success factors, which were quantitatively identified, the following limiting factors were identified through our qualitative analysis of failed or struggling projects. As our limiting factor analysis is qualitative and interpretive, we present our data in line with our analysis.
Boundaries between commercial, voluntary and public sectors
While UK healthcare is primarily publicly funded and provided by the NHS, social care is often commercially provided,20 creating the potential for friction at the interfaces between these sectors.
As the care homes are private businesses, there was some lack of political will to embrace the training, as there was a view that although there was the potential to improve health outcomes for the residents, the manager did not feel there were sufficient resources to implement the required training. (R4)
Commercial organisations were reported as unwilling to release staff for training unless the value of that training was felt within the organisation. Valuable initiatives by the voluntary sector to train social care staff, but which provided benefits in the healthcare sector, fell between sectors and were potentially unviable without direct funding.
The voluntary sector is happy to participate but there is no spare capacity within it unless there is a financial package that can go with it. (R35)
There was concern that privately funded organisations were not subject to the same standards and mandates as the publicly funded bodies, and were failing to invest in training.
Because it is not mandated, organisations do not have to engage with or release staff for education. (R2)
Restructuring within the NHS has created a set of semiautonomous institutions and organisations with different and sometimes competing priorities.21 Some participants described difficulty aligning project aims to multiple organisational goals.
There were tensions between the two boroughs in relation to approach & resourcing. There was also a tension between commissioner expectations and practice/federation expectations which have impacted on the programmes sustainability. (R16)
So, this intervention has a good return on investment, for every £1 you spend you get a return of £5.20. And they’ll say, I’m the one making the investment, but he’s the one making the return here. I’ve got a budget; he’s got another budget. We might both be in the health system, but I’m not going to spend my money if he’s the one getting the return. (FI2)
Workplace cultures and priorities
Some projects reported finding non-healthcare workers receptive to health-related training; however, some failed or struggling projects found this a challenge.
Medicines delivery teams unwilling to take on additional role. (R61)
There were concerns raised by care home managers that the initiative would cause undue responsibility on individuals to make clinical decisions. (R16)
Participants described differences between academic and workplace learning cultures, and variable receptiveness of front-line clinical staff to change. Some described resistance to outsiders telling healthcare workers how to improve. This may reflect the inverse of high-value projects, which were found to engage in participatory practices, engaging patients, front-line staff and communities in codesigning their innovation.
It has been difficult to embed these products due to structural issues within the staff teams. (Nursing) It was clearly not seen as a priority. (R20)
I think the main insight I would have is that when working with mental health nursing teams the researcher and research team needs to be fully integrated into team(s) and seen as part of the culture. Being an outsider does not seem to work as day-to-day practice seems to regulate research. (R20)
Participants also described tensions between management priorities and the priorities of those working directly with patients.
No interest on part of management. I don’t think they have even read it. (R57)
The initiative was welcomed at service level, however there was little interest at senior management level. (R52)
There is such a dislocation between commissioning and what is happening on the ground. (FI2)
Lack of support beyond the start-up phase
Participants noted ongoing privileging of new innovation over sustaining or scaling up innovations that have already demonstrated their value. For example, clinical academics do not gain publications for ongoing maintenance of innovative practice:
‘Research remit probably wouldn’t cover [further dissemination] unless there was a good likelihood of REFability’ [‘REF’ refers to the Research Excellence Framework, a scoring system used to fund the university sector]. (R1)
Participants described innovation funding streams, but articulated difficulty securing funding beyond the start-up phase.
The project was resourced sufficiently for the pilot. However, once the pilot finished so did the project. (R5)
The education faculty and funding is driven towards innovation and not sustainability—this de-incentives individuals from continuing with existing projects. (R8)
Burn-out, turnover and lack of protected time
Participants described projects that were limited by staff burn-out, turnover and a lack of protected time.
My commitment to the project was there however the resources I had to continue with project were limited due to competing pressures on my time. (R12)
The programme required more administrative support than anticipated & this ended up being an ask over & above someone’s day job for a prolonged period of time. (R16)
The most important person was our pharmacist who moved from the pharmacy a few months after we started! (R61)
Risk as integral to innovation
Finally, it is worth noting that participants felt that risk was a necessary ingredient of healthcare innovation. Innovations that fail to demonstrate value should be supported in folding without hesitation, and lessons shared.
The project demonstrated that this initiative was not a model that would work in the hospital environment hence could not be embedded. (R30)
Final framework
We created two 2×2 matrices containing all the significant success factors across each dimension of success, shown in figure 4. The matrix on the right excluded low-value projects in the calculation of factors significant to expansion and served to support the inclusion of some marginal factors in the final model as they became more significant despite lower power.
Critical success factors plotted according to their salience to success. Critical success factors plotted according to their significance to success on a natural logarithmic scale so that factors above and to the right are probably significant (p≤0.05). The expansion axis indicates significance to whether a project is scaled up or down beyond the original pilot. The value creation axis indicates significance as to whether it creates more or less than its intended value for beneficiaries.
Figure 4 shows clearly congruent clusters of factors in each quadrant, indicating that some types of factors may be more important to expansion, while others are more important to value creation. These clusters relate to skills and expertise, leadership and motivation, organisational fit and structural support, societal alignment and participation, and evaluation.
As outlined in table 1, there are questions as to whether evaluation and motivation are dependent rather than independent variables: does finding working on a project personally rewarding drive success or vice versa, and does a positive evaluation drive success or vice versa? Triangulation with qualitative comments (in table 2) suggests that evaluation and motivation may drive success, so they have been tentatively included in our final model.
Themes that were predominantly related to value creation (participation, motivation and evaluation) were labelled value creation factors. Themes that were predominantly significant to expansion (organisational fit and structural support) were labelled expansion factors. Themes that were significant to both axes (expertise, leadership and a supportive network) were labelled core success factors. We arranged success factors into a nested hierarchy, as innovations that do not generate value are unlikely to be scaled up. Our final model in figure 5 also lists potential limiting factors identified through our inductive qualitative analysis.
Nested hierarchy of success factors and limiting factors for health service innovation. Factors that may be significant to both value generation for the intended beneficiaries and to whether the innovation is scaled up beyond the original pilot are labelled as core needs. Factors that may be significant specifically to value generation are the next priority (middle layer) as innovations that do not generate value will not become embedded or spread. Finally, factors that may primarily be significant to expansion are presented in the outermost layer. Additional limiting factors were identified through an inductive analysis of qualitative responses from projects that had scaled down or failed to produce their intended value.
Discussion
This analysis of 56 health service innovation projects has enabled us to propose a model for understanding success in health service innovation that has two discrete axes: one relating to whether or not the innovation created value for its intended beneficiaries; the other relating to whether or not it was scaled up beyond the original pilot. Comparing projects across these dimensions of success has enabled us to hypothesise that:
The core drivers of success are leadership and collaborative expertise (leadership skills and commitment, expert input, sufficient staff with the right skills and expertise, and a supportive collaborative network).
The drivers of value creation for the intended beneficiaries are participation, motivation and evaluation (involvement of patients, public, practitioners and communities, alignment to societal needs, internal motivation, finding the project work rewarding, ability to demonstrate benefits and having opportunities to share impacts).
The drivers of sustainability and scale-up are organisation fit and structural support (organisational fit and alignment, administrative and educational support, staff with time and energy).
Additional limiting factors included difficulties at the boundaries and intersections between organisations, professions, sectors and cultures; a lack of structural support beyond the start-up phase; and staff burn-out and turnover.
Within healthcare services, the issue of diffusion and sustainability of innovation has received widespread academic attention pioneered by Greenhalgh et al22 who drew on Rogers’ seminal text on diffusion of innovations.23 There have been many subsequent notable academic contributions.24–28 Nilsen proposed an overarching framework of healthcare implementation theories according to the aim of the theory.29 Theories such as those about innovation sustainability, which include the diffusion of innovation theory, were categorised as ‘determinant frameworks’, as they posit general types of factors that can influence the success of an innovation. We believe that our findings contribute through empirical evidence to theoretical development at this level, and thus may have wider implications than programme-level data would normally allow. According to Nilsen, such theories have typically been analysed and formulated across individual studies, at the level of meta-analysis or review29 and may therefore be one or more steps removed from the underlying data. This study is different in that we have developed mid-range theory that is empirically grounded in programme-level data, and there is a clear line between our data and the generated theory.
Conceptions of innovation success tend to focus on sustainability30 and scale-up.24 We suggest that both are contingent on the ability of an innovation to provide value to its intended beneficiaries in the first place. There are few theories grounded in empirical data that explain this dimension of success. Our findings highlight the importance of patient, public and practitioner involvement, alongside the core success factors of leadership and collaborative expertise. We suggest that these are fundamental preceding factors to either sustainability or scale-up.
This ‘value for intended beneficiaries’ dimension of success also allows us to conceptualise a valuable innovation that is not growing or expanding. This, we argue, is important: healthcare innovations may have parameters within which growth and expansion are constrained, perhaps because their aims have been achieved, or because the context changes. An innovation that has met its aims but has not expanded beyond its natural boundary should be properly positioned as such.
Our ability to research a set of potentially valuable projects that were scaled down or stopped, many of which never reach the literature, may have afforded novel insights. Fixsen et al suggest that sustainability can only be asserted when the funding to support implementation is withdrawn.31 Wiltsey Stirman et al’s systematic review suggests sustainability can be asserted after a period of 2 years.32 Our findings suggest that continued structural support, particularly organisational, administrative and educational support, may be critical to a project’s sustainability and scalability, and that their withdrawal may destroy potentially valuable innovations.
Finally, our findings further validate the work of Dopson et al,33 whose qualitative exploration of a similar set of health service innovations highlighted the importance of context and process over content: it is not so much what you are trying to achieve, it is how you do it and the organisational and interpersonal contexts that you work within that matter.
A limitation of this research is its highly contextual nature. Our results may not be generalisable to all contexts; however, repeating these methods may produce locally relevant results. The ANOVA depends on the universe of potential factors having been correctly identified and a large enough number of innovations to produce statistical significance. The research could be improved by more extensive validation of factors, patient and public involvement, further testing the directionality of tentative factors, a greater geographical spread and a greater number of projects to allow for finer grading across the expansion axis.
Conclusions
Our findings suggest that organisations and policy makers wishing to support service-level innovation in similar healthcare contexts address the factors identified through this research as critical to success.
Such strategies might include:
Supporting innovators with the right skills and expertise, including leadership skills, implementation support and evaluation expertise.
Innovation networks to provide opportunities to showcase success and provide a peer community of expertise and support.
Emphasising participatory practices and collaborative approaches, so that innovations are more likely to align to societal and organisation goals and generate value for patients, communities and practitioners.
Providing administrative and educational support during the scale-up phase, and ensuring that this support is maintained or handed over rather than withdrawn to schedule.
Recognising and enhancing the internal motivation and drive of innovators as well as more goal-oriented motivations such as career needs.
At a structural level, the boundaries between organisations, professions and the health and social care sectors may need to be addressed as potential barriers to successful innovation.
More research is needed to confirm whether addressing these factors prospectively enhances the success of future innovations.
Data availability statement
Data are available upon reasonable request. Due to the highly individual nature of healthcare innovations and the limited geographic area of this study, we are unable to provide our raw data. We undertake to provide a redacted data set upon reasonable request.
Ethics statements
Ethics approval
Ethical approval was granted on 26 March 2019 by the Research Ethics Committee of King’s College London (LRS-18/19-10432).
Acknowledgments
We would like to thank Josh Brewster from the Health Innovation Network and Sian Kitchen from Health Education England for their commitment to this project, and Professor Sue Smith from the Medical Education Research Unit of Imperial College for ongoing support and advice.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @doctorkayleigh
Contributors KL-G and GBR contributed to the conception and design of the work and to the acquisition of data. KL-G, GBR and AK collaborated on the data analysis and interpretation. KL-G and AK drafted the work, all authors revised it critically for important intellectual content. All authors have approved the final version and agree to be accountable for all aspects of the work and to resolve questions relating to accuracy or integrity.
Funding This work was funded by Health Education England (grant number XXMLIVESEY).
Disclaimer Gabriel Reedy is affiliated to the National Institute for Health Research Health Protection Research Unit (NIHR HPRU) in Emergency Preparedness and Response at King’s College London in partnership with Public Health England (PHE), in collaboration with the University of East Anglia and Newcastle University. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR, the Department of Health or Public Health England. These funders had no input in the writing of and the decision to submit this article.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.