Article Text
PDF
Abstract
Background The National Institute for Health and Care Excellence (NICE) was established in 1999 and provides national guidance and advice to improve health and social care. Several steps in the research cycle have been identified that can support the reduction of waste that occurs in biomedical research. The first step in the process is ensuring appropriate research priority setting occurs so only the questions that are needed to fill existing gaps in the evidence are funded. This paper summarises the research priority setting processes at NICE.
Methods NICE uses its guidance production processes to identify and prioritise research questions through systematic reviews, economic analyses and stakeholder consultations and then highlights those priorities by engagement with the research community. NICE also highlights its methodological areas for research to ensure the appropriate development and growth of the evidence landscape.
Results NICE has prioritised research questions through its guidance production and methodological work and has successfully had several research products funded through the National Institute for Health Research and Medical Research Council. This paper summarises those activities and results.
Conclusions This activity of NICE therefore reduces research waste by ensuring that the research it recommends has been systematically prioritised through evidence reviews and stakeholder input.
- research waste
- priority setting
- research priorities
- resource allocation
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
Highlights the importance of reducing research waste and the prioritisation of research as a key step in that process.
Describes the systematic prioritisation of research done by National Institute for Health and Care Excellence (NICE).
Illustrates the methodological research undertaken from prioritised topics by NICE in recent years.
This paper is limited to the prioritisation of research undertaken at NICE and it does not reflect all the work the nstitute does to reduce research waste.
Introduction
There is no doubt that the substantial public and private investment in biomedical research (estimated to be around US$240 billion in 2010)1 has led to significant improvements to the quality of life of people. However, not all research yields such benefits, and though some waste is unavoidable due to the nature of science, many improvements to the way we conduct our work can be made to ensure that our investments are sound and the value of our research is increased.1 2 Getting the first step of the research cycle, that is, appropriate priority setting of research or identifying the correct research question to fund, is key to this being successful.1
Previous research has shown that there is a huge mismatch between the research that is funded and the research needed by the end users. A study found that only 9 out of 334 research articles highlighted priorities relevant to patients or clinicians.2 A second study found that while the majority of trials funded were related to drugs, the end users were much more interested (80%) in research about ‘education and training, service delivery, psychological interventions, physical interventions, exercise, complementary interventions, diet etc’, compared with research in drugs (20%).1
Another problem noted is that often the new research ignores what is already known about the problem and does not take into account previous research and therefore can duplicate information and give little additional value.1 2 Moreover, such research may miss the true knowledge gaps that need answering, and this can often be compounded by the failure to publish negative research (publication bias) that can go undetected unless methodology such as funnel plots are used. Even if an appropriate research gap is identified, inappropriate design or statistical analyses or the problem of ‘effect to bias ratio’ (where the magnitude of the effect size and biases like selection or confounding, are similar) may mean that the validity of any results may be questionable.3 4
The National Institute for Health and Care Excellence (NICE) is an independent organisation responsible for providing evidence-informed guidance on health and social care for England.5 6 NICE guidelines (clinical, public health and social care), quality standards, Health Technology Assessments (HTA) and other products help to deliver the best possible care within the limited resources available.5 6 The Science Policy and Research (SP&R) programme7 works together with various academic, charity and research funding organisations to enable the appropriate research to be done. It is now also working with life sciences companies, using European Commission funding through the Innovative Medicines Initiative (IMI) to take this work forward. In this paper, we summarise these processes and the work that has been undertaken through these mechanisms.
Research priorities identified through systematic reviews (guideline or HTA production)
NICE guidance is based on the synthesis of evidence primarily through the process of systematic reviewing and, if appropriate, health economic modelling and cost-effectiveness decision analysis that are presented using Grading of Recommendations Assessment, Development and Evaluation (GRADE) profiles that allow for a transparent representation of the confidence in the evidence available for decision making.8 The results of this work are discussed by independent advisory committees (consisting of members external to NICE), which can include a diversity of National Health Service staff, healthcare professionals, social care practitioners, commissioners and providers of care, patients, service users and carers, industry and academics. The committees, through a process of guided deliberation of the evidence, reach conclusions by formally making draft recommendations. During a period of public consultation, external stakeholders can comment on draft recommendations before they are finalised and eventually published.9 The evidence base supporting guidance recommendations are reviewed every few years to ensure that any results potentially demonstrating a change in practice or care are formally considered for updating recommendations.
This process explicitly describes the evidence base and identifies where there are gaps, uncertainties or conflicts, in the existing evidence. Many uncertainties identified may be interesting to answer but could have little impact on people’s care or NICE’s ability to make appropriate evidence-based and evidence-informed recommendations. Uncertainties can arise because there is no evidence available or due to the absence of relevant research or if research has been done but not yet published. Or there may be evidence available but there is still insufficient information on which to base a recommendation (eg, due to inadequate reporting or poor quality), or the research is out of date (eg, a systematic review that needs updating with recent trials or if clinical practice has changed). However, if these uncertainties could have an impact, and support future NICE guidance recommendations, it is important for the institute to make recommendations for research (see table 1).
Example format for research recommendation rationale to support prioritisation
These recommendations are deliberated, discussed, agreed and formulated by the independent advisory committees and the external community (both specialist and public) that are then invited to comment and contribute.9 The final recommendations are identifiable in the guidance and are also collated into the publicly available NICE research recommendations database.10 NICE then liaises with the research community to ensure they are addressed as detailed in the paragraphs that follow. By making research recommendations, NICE is looking to steer the research community for them to generate new evidence to inform the future guidance update and review cycle. If gaps remain in the evidence base after several years, the committee may consider keeping, removing or archiving the research recommendations based on any new findings. In 2010 (updated in 2015), NICE established a formal guide to support this process and to provide a route to ensuring they are picked up as key topics for research funding.11 The guide describes a step-by-step approach to identifying uncertainties, formulating research recommendations and research questions, prioritising them and communicating them to researchers and research funders (see figure 1).
The NICE research recommendations process. NETSCC, NIHR Evaluation, Trials and Studies Coordinating Centre; NICE, National Institute for Health and Care Excellence; NIHR, National Institute for Health Research.
NICE works closely with National Institute for Health Research (NIHR; a government body that supports health and care research of national research priorities)12 to prioritise, promote and commission its research recommendations. The identified topics are considered by a number of their different research programmes, particularly the HTA, Public Health Research and the Health Services & Delivery Research programmes. If they are found to be suitable and fulfil their criteria, they enter the commissioning process. This process has evolved over the last 10 years and since 2005, over £59 million worth of these research projects have been funded (see table 2). In 2015, NICE and NIHR Evaluation, Trials and Studies Coordinating Centre (NETSCC) agreed a fast-tracked route for the most important research recommendations, to be flagged under the ‘NICE Key Priority’ designation. These research recommendations have the potential to have the highest impact on future guidance recommendations and people’s care.
Summary of the cost of National Institute of Health Research (NIHR)-funded projects (as of 31 March 2016)
Methodological research priorities: ‘research on research’
The Edinburgh Reducing Waste conference in 2015 highlighted the importance of being efficient in the ‘what’ but also the ‘how’ research is done. While the basic methodology of systematic reviews and cost-effectiveness analyses may be well established, there is still controversy about ‘best practice’ and challenges frequently arise in the context of specific technologies, particularly when decisions are negative. It is therefore vital that methodological research is undertaken. Inadequacies in the evidence base have required the exploration of newer methods such as indirect comparisons and the use of network meta-analyses that allow for comparisons across different drugs and interventions where those trials have not been undertaken.13 Different perspectives exist about what data constitute as evidence and about what should be used for decision making. Due to the paucity of the evidence base for some interventions, particularly in the public health and social care domains, NICE has always considered the ‘best available’ evidence to inform its decisions, which move beyond randomised controlled trials, using GRADE to determine the validity of the collective evidence.8 9 13 In order to ensure that the evidence NICE receives is fit for decision making, each of NICE’s guidance programmes have a methods guide that are regularly updated to incorporate new developments. NICE Scientific Advice works directly with commercial evidence producers on a fee-for-service basis to ensure the appropriate evidence is available for decision making.
In 2008, NICE had its first comprehensive review of the methodological research needed for future development of the work of the institute (methodological priorities for research), funded by the Medical Research Council (MRC) and NIHR joint Methodology Research Programme (MRP).14 The methodological review identified and prioritised methods research topics through a focused literature review, interviews, an email survey, a workshop and web-based feedback exercise. Participants were members of the NICE secretariat and its advisory bodies, representatives from academia, industry and other organisations working closely with NICE. The first prioritised project the MRP funded in 2008 was on the appropriate use of ‘only in research’ decision15 and a further nine projects were funded through another MRP call via their needs-led route in 2009 (over £2.3 million, see table 3).
Methodological research projects funded through Methodology Research Programme (MRP) in 2009 (approximately £2.3 million)
To ensure that NICE had robust processes for timely identification, prioritisation and communication of methodological research needs, an Internal Research Advisory Group (IRAG; formerly the internal methods advisory group) was established in 2010. This group was tasked to develop the institute’s methodological development needs in both the short term and long term by continuing the systematic identification and prioritisation of key research uncertainties on an ongoing basis and working with MRP to commission research.12 The group consisted of representatives of the different guidance and non-guidance producing centres and programmes across NICE, who were supported as and when needed by external research advisors from academia who were methodological experts in the fields relevant to the work of NICE. Each programme provided their methodological research priorities, and 45 uncertainties were collated from across the Institute. These were clarified and reprioritised into themes that included modelling service delivery, using qualitative evidence to capture patients experience, extrapolating data for comorbid and paediatric populations and using observational data from large datasets for decision making. These were then highlighted by MRC through a NICE-specific funding call (see figure 2).
Process of identifying methodological research priorities at NICE and relationship with Medical Research Council (MRC): example 2010–2012. NICE, National Institute for Health and Care Excellence.
Some of the prioritised topics did not receive any bids and other internal support mechanisms were used to fund them as small scoping projects. This led to more clarified research questions, of which one was undertaken internally by the SP&R programme at NICE on the use of colloquial evidence13 and the other to support the use of observational data in healthcare decision making was put forward as a highlight notice by the MRP in 2017.16 They also funded two more NICE priorities on developing a reference protocol for expert elicitation in healthcare decision making in 201617 and going beyond health-related quality of life towards a broader quality-adjusted life-years measure for use across sectors in 2017.18
The SP&R programme has also participated in a project considering the role of social values in priority setting as part of an international collaboration to try and unpick how cultural contexts of well-being interacted with healthcare decision making.19 20 Additionally, the programme supported the establishment of a Research Support Unit in 2013 to undertake short projects identified through internal prioritisation by IRAG on methodological areas that included reviewing the literature on social value judgements relevant for NICE and identifying and appraising promising sources of UK clinical, public health and social care real-world data.21 Other arrangements for NICE to access research commissioning include the Decision Support Unit, the Technical Support Unit and the External Assessment Centres that facilitate methodological research for HTAs, clinical guidelines and medical technologies evaluation work, respectively. NICE has also recently partnered with Myeloma UK to explore how patient preferences could be captured and included in HTAs more readily.
NICE has also been looking at internationally important and relevant research priorities that will potentially have a global impact on health and care decision making. It has been successful in gaining international cooperation and European Commission funding through the IMI for some of the key priorities currently facing the field. These include the use of real-world data for early decision making (GetReal),22 the medicines adaptive pathways to patients activities to foster access to beneficial treatments for the right patient groups at the earliest appropriate time in the product life-span in a sustainable fashion (ADAPT-SMART)23 and using big data for obtaining better outcomes for patients (BD4BO).24 The Institute has also been actively involved with other European Commission funded projects such as EUnetHTA (an effective and sustainable network for HTA across Europe)25 and DECIDE (patient and public focused strategies for communicating evidence-based recommendations).26
In 2017 NICE published nine areas that it had, through the SP&R programme and the IRAG, identified as priorities for methodological research. These are: (1) real-world evidence; (2) data science; (3) adaptive pathways; (4) patient preferences; (5) improvements in cross-sector comparisons; (6) expert elicitation; (7) complex data visualisation; (8) precision medicine; and (9) implementation of NICE guidance. Methodological research in these areas will help NICE assess the need for improvement in the methods and processes it uses to produce guidance and anticipate and adapt to policy developments and changes in health and social care delivery in the next few years. By only highlighting methodological research areas to funders that were systematically prioritised, NICE ensures that only questions whose answers will impact its future work are studied, therefore doing its part to reduce research waste.1
Discussion
Priority setting is an explicit method that considers what to fund by weighing the trade-offs between the various options in the process.27 It is imperative to avoid research waste such that only key gaps in knowledge are fulfilled by undertaking new research that help build a more complete evidence landscape for future policy development and better clinical practice. Systematic approaches to research priority setting improve the transparency of research management and are often based on a ‘research cycle’ approach that includes the identification of research questions, ranking their priority, identifying existing research, and setting priorities for primary research, and involving relevant stakeholders at key points throughout the cycle.28 It is therefore essential that a formal process of prioritisation of research be established within guidance-producing organisations, with the involvement of all legitimate stakeholders, to increase the ownership of the ensuing research and the likelihood of the results influencing practice and policy.29
Creating research recommendations is part of the guidance production cycle, and they should:
identify any uncertainties that may affect people’s care
be developed using an appropriate technique to frame research question development, for example population, intervention, comparator and outcome (PICO) or evidence, population, intervention, comparator, outcome and time (EPICOT).
go through a process of stakeholder consultation and be reviewed as part of the guidance review and update cycle.
This is the path that NICE has taken and therefore has a strong emphasis on reducing research waste. The NICE research recommendations are derived from the uncertainties identified through the NICE guidance development process and are an integral part of the guidance development and review cycles. The institute therefore reduces research waste by supporting the redirection of resources to more valuable activities.30 By making its research priorities and funded research publicly available, it minimises the duplication of research. It also drafts research questions with appropriate detail (eg, type of study design and sample size required), such that the answers obtained from its commissioned research are meaningful. Publicly funded guidance development bodies have a responsibility to ensure that they are guiding appropriate future research based on identified need and gaps in the evidence base. This is also true for research funders, who also have the same responsibility. NIHR has shown that only 5 (11%) of the 47 trials funded over the 2-year period (2006–2008) were not based on identified gaps from systematic reviews, and they had valid reasons for doing so.31
Methodological uncertainties derived from NICE also go through a process of systematic identification and prioritisation and are promoted to funders to encourage research being funded into the various aspects of how to undertake the development of future guidance. Though this paper is limited to describing the prioritisation of research undertaken at NICE, we feel it is an important part of the role of any HTA or guideline-producing agency globally to undertake in order to reduce research waste.
In recent years, NICE has entered into a few international research collaborations. These are becoming increasingly important as sources of research funding, particularly within the current economic climate, are dwindling. Such partnerships support the identification and prioritisation of cross-cutting research needs and also potential joint funding routes. There should therefore be a shared responsibility between research partners to ensure that key research is undertaken and cooperation to allow sharing of information to avoid duplication and improve efficiency. NICE has partnered with The National Center for Biotechnology Information so that there will be a PubMed bookshelf dedicated to methods research, and the methodological research reports undertaken by organisations like NICE would also be indexed and accessible in the future.
A number of methods to identify priority areas for research have emerged. These include measuring the burden of disease or the expected return from research as well as estimates of the welfare losses resulting from variations in clinical practice.32 The value of information analysis is a more novel approach to prioritising research uncertainties that quantifies the expected net benefit from the results of the additional research to society, against the cost of conducting that piece of research and its implementation. Through this framework, the value of acquiring additional information to inform the decision problem helps alleviate some of the uncertainty had less definitive evidence been used instead.33 34 This method has a firm foundation in statistical decision theory and has been successfully used in other areas of research.32 It has also been employed for developing research recommendations from a number of NICE guidance but not yet been undertaken routinely.
Other successful initiatives to identify, prioritise and promote research uncertainties include the James Lind Alliance (now part of NIHR and carried out via NETSCC) Priority Setting Partnerships (PSPs). These PSPs bring together patients, carers and clinicians using transparent methods to prioritise ‘known unknowns’ that have been elicited primarily from evidence-based knowledge.35 Until January 2016, these research priorities were collated and published in the UK Database of Uncertainties about the Effects of Treatments. This was launched in 2006 to collate uncertainties from reports of systematic reviews and clinical guidelines, protocols for systematic reviews (such as those published in the Cochrane Database of Systematic Reviews) or from registered information about ongoing clinical trials. Some of the treatment uncertainties came directly from patients or carers or from clinicians and cover a wide variety of health problems, for example, cancer, mental health and skin disorders.35 Another good example can be seen from the field of international development where the use of evidence gap maps have been developed by the International Initiative for Impact Evaluation. Evidence gap maps are evidence collections that map out existing and ongoing systematic reviews or primary studies in a sector or subsector, such as maternal health, HIV/AIDS and agriculture. They present a visual overview of existing evidence and therefore highlight the areas of gaps within the evidence landscape.36
Conclusions
It is important that HTA and guideline organisations use their systematic processes to identify research gaps and then subsequently link with national research funders to ensure they are addressed. NICE therefore supports the reducing research waste campaign by ensuring that the research it recommends has a beneficial impact on the health and care of the people, as it has been systematically identified as a genuine gap in the health evidence or a method that needs further clarification or development.
Acknowledgments
The authors would like to thank Dr Nick Crabb for some early discussions on this paper.
References
Footnotes
Contributors All authors had complete access to the data in the study. TS and MC contributed to the concept and design, acquired the data for the study and developed the first draft of the manuscript; all authors contributed to the analysis and/or interpretation of data and critically revised the manuscript and approved the final version.
Funding This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.
Disclaimer The authors’ views expressed in this paper are their own.
Competing interests TS was an employee of NICE, and the remaining authors are currently employees of NICE. The authors would also like to note that TS was previously a senior analyst at NICE in the United Kingdom.
Patient consent Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement The data can be requested from NICE, UK.