Objectives There is a mismatch between research questions considered important by patients, carers and healthcare professionals and the research performed in many fields of medicine. The non-alcohol-related liver and gallbladder disorders priority setting partnership was established to identify the top research priorities in the prevention, diagnostic and treatment of gallbladder disorders and liver disorders not covered by the James-Lind Alliance (JLA) alcohol-related liver disease priority setting partnership.
Design The methods broadly followed the principles of the JLA guidebook. The one major deviation from the JLA methodology was the final step of identifying priorities: instead of prioritisation by group discussions at a consensus workshop involving stakeholders, the prioritisation was achieved by a modified Delphi consensus process.
Results A total of 428 unique valid diagnostic or treatment research questions were identified. A literature review established that none of these questions were considered ‘answered’ that is, high-quality systematic reviews suggest that further research is not required on the topic. The Delphi panel achieved consensus (at least 80% Delphi panel members agreed) that a research question was a top research priority for six questions. Four additional research questions with highest proportion of Delphi panel members ranking the question as highly important were added to constitute the top 10 research priorities.
Conclusions A priority setting process involving patients, carers and healthcare professionals has been used to identify the top 10priority areas for research related to liver and gallbladder disorders. Basic, translational, clinical and public health research are required to address these uncertainties.
- chronic liver disease
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
A research prioritisation process involving clinicians, patients and carer, and public representatives was performed in the field of liver and gallbladder disorders. This will help to address the mismatch between research questions that are considered important jointly by patients, carers and healthcare professionals and the research performed in the field of liver and gallbladder disorders.
A Delphi consensus method was performed. This prevented dominance of ‘loud voices’, a problematic issue with small and large group discussions.
Because of the predominance of people with chronic liver disease on the Delphi panel, many of the top research priorities related to chronic liver diseases.
Failure to address treatment uncertainties by research can lead to significant suffering and deaths.1 It is important that research in any field of medicine takes into account the shared interests of patients, carers and clinicians.2 However, there is a mismatch between research questions that are considered important jointly by patients, carers and healthcare professionals and the research performed in many fields of medicine.3 4 The James-Lind Alliance (JLA) exists to help ensure a patient-centred process and enables the limited research resources to be used in addressing the research questions that are considered important jointly by patients, carers and healthcare professionals2 (‘top research priorities’). This is achieved by forming ‘priority setting partnerships’ (PSPs) between patients, carers and healthcare professionals.2 Formal prioritisation of research topics jointly by patients and healthcare professionals can lead to increased research on the topic.5 6
There has only been one formal research prioritisation process involving patients, carers and healthcare professionals in the field of liver and gallbladder disorders.7 However, the scope of that PSP was limited to alcohol-related liver disorders.7 The Non-Alcohol-Related Liver and gallbladder disorders Priority setting partnership (NARLIP) was established to address the prevention, diagnostic and treatment uncertainties related to the majority of liver disorders which were not covered by the JLA PSP on alcohol-related liver diseases7 and to include gallbladder disease.
The aims and objectives of the NARLIP were to work with patients, their carers and healthcare professionals treating them (‘stakeholders’) to identify uncertainties about the diagnostic tests and effects of prevention and treatments for non-alcohol-related liver and gallbladder disorders, to agree by consensus a prioritised list of those uncertainties for research, to publicise the results and process, and to take the results to research commissioning bodies to be considered for funding and researchers to encourage them to submit grant applications addressing these uncertainties.
Formation of the partnership: organisations and individuals representing people affected by non-alcohol-related liver or gallbladder disorders, their carers and healthcare professionals treating people with non-alcohol-related liver and gallbladder disorders. A partnership was formed between KG representing University College London and the British Liver Trust initially, but following reorganisation in the British Liver Trust, PSC Support9 became the leading patient organisation partner of this process. A steering committee was formed. The members of the steering committee who participated in the complete process were KG, MW, BRD, CF, BF, AM, RM, SM, IS and ET.
Establishment of the scope: the steering committee members discussed and decided that the scope should include adult and paediatric liver and gallbladder disorders which required medical and surgical treatments. The protocol was registered with JLA PSP.
Identifying potential research questions: research questions were collected through online surveys and searching UK Database of Uncertainties about the Effects of Treatments (UK DUETs), research recommendations in high-quality systematic reviews and clinical guidelines, and registers of ongoing research.
Refining research questions: the research questions identified in the above step were reviewed and where necessary combined to result in a set of unique research questions. Research questions were considered ‘answered’ when recent high-quality systematic reviews (based on low risk of bias studies) concluded that further research was not required. Removal of such ‘answered’ research questions was planned. The remaining questions were ‘uncertainties’.
Interim prioritisation: to shortlist the set of questions to manageable levels for the final prioritisation process, the members of the steering committee ranked the uncertainties after stratifying the questions as medical and surgical questions. The members of the steering committee agreed that the interim prioritisation list should consist of 75% medical questions and 25% surgical questions. This decision was an arbitrary decision made by the steering committee based on the rationale that majority of individuals with liver and gallbladder disorders are treated medically but a minority require surgery which have a major impact on patients’ lives.
Final prioritisation by consensus: a modified Delphi consensus method was followed to identify the top priorities using methods described by Jones and Hunter.10 The Delphi was performed electronically using Excel for managing the process. The steps in the modified Delphi consensus method were as follows.
A Delphi panel consisting of patients, their carers and healthcare professionals treating them was formed. The Delphi panel was formed by using ‘snowballing’ sampling methods and by contacting people through emails, online liver patient forums (British Liver Trust Health Unlocked forum) and newsletters. A total of 42 people expressed interest in joining the Delphi panel and 33 panel members completed all three rounds. Details of the Delphi panel composition and drop-outs are reported in the results section.
A total of three rounds were conducted.
Delphi panel members scored the shortlisted questions in the interim prioritisation process on a scale of 1–9 with 1 being considered least important and 9 being considered most important. Scores of 1–3 were categorised as ‘less important’, 4–6 as ‘moderately important’ and 7–9 as ‘highly important’. Panel members were requested to score the questions according to the importance of the question to them/the persons that they represent or treat and could leave questions that they were unable to score empty. Each Delphi panel member could add a maximum of two questions in the first round to ensure that the questions most important to the Delphi panel members were included in the prioritisation process even if they were not identified in the earlier steps. In the subsequent rounds, the panel members were shown the summary scores and their previous score for each question. They were able to retain or change their score in each of the rounds after the first round. For calculation of the summary scores and the proportion considering a question ‘highly important’, non-responses were excluded.
Consensus about a specific research question being a top research priority was reached when 80% or more Delphi panel members considered the research question as highly important (allocated scores between 7 and 9).
When fewer than 10 research priorities were obtained by consensus, the remaining priorities were completed by uncertainties based on the highest proportions of panel members agreeing that the research question was highly important (scores between 7 and 9).
There was no restriction on the Delphi panel to consult others while scoring the questions. However, only one final response on the set of questions was accepted from each Delphi panel member.
When there were no recent high-quality systematic reviews on the research question, we have recommended high-quality systematic reviews. When recent high-quality systematic reviews recommended high-quality research, we have recommended randomised controlled trials (RCTs) for prevention and treatment, as such studies carry the lowest risk of bias if conducted well; we would have recommended well conducted diagnostic test accuracy studies for diagnostic uncertainties. All online surveys were completed using Google Forms designed by KG. The Delphi process was completed using Microsoft Excel and email.
In addition, we had full support of patient organisations with involvement of patient representatives throughout the whole process rather than patients visiting the hospitals.
Patient and public involvement
Patients and public were involved in all aspects of this project: they were part of the steering committee and were involved in the definition of the scope, methodology used for the prioritisation process, identification of further patients and public representatives, participation in the Delphi panel, interpretation and critical revisions of the draft report. They will be involved in the dissemination of the findings through patient websites, patient forums and to research funders.
Identifying and refining of research uncertainties
A total of 126 patients, carers, and those at risk of developing non-alcohol-related liver and gallbladder disorders, and 13 healthcare professionals participated in the first survey which was conducted between July and December 2015. This survey resulted in a total of 209 unique research questions. In addition, 219 unique questions were identified from searching the UK DUETs, Pubmed and ClinicalTrials.gov on 2 January 2016. A total of 428 unique valid (ie, falling within the remit of this priority setting partnership) research questions (247 medical-related and 181 surgery-related) were identified from these sources. None of the research questions had been answered by recent high-quality systematic reviews based on low risk of bias studies which concluded that further research was not required. Therefore, all the 428 research questions were considered research ‘uncertainties’. The complete list of 428 unique valid uncertainties in no particular order is available in online supplementary appendix 1. This has been converted to the population, intervention, control and outcomes format whenever possible.
To identify a shortlist of questions (from the list of 429 questions) that were to be considered for the next step, a total of 48 research priorities (36 medical questions and 12 surgical questions) were identified on the basis of being selected by at least one patient or carer and healthcare professional of the steering committee (24 questions) and obtaining the highest ranks among the members of the steering committee (additional 24 questions). The list of 48 questions identified as interim priorities in no particular order is available in online supplementary appendix 2.
A total of 42 people expressed interest in joining the Delphi panel and 33 panel members completed all three rounds. Five people dropped out before they returned the scores of the first round (all patients, carers and general public), three between first and second rounds (all healthcare professionals) and one between the second and third rounds (healthcare professional). Of the 33 panel members who completed all the three rounds, 17 were healthcare professionals and 16 were patients, carers and general public. Of the 17 healthcare professionals, six were hepatologists, four were surgeons, two were hepatology nurses and the remaining were general practitioner, hepato-pancreato biliary surgery nurse, organ preservation biologist, dietician and pharmacist (one each). Of the 16 patients, carers and general public, there was representation from general public and various liver diseases including autoimmune diseases such as primary sclerosing cholangitis (PSC), primary biliary cholangitis, autoimmune hepatitis, viral hepatitis, metabolic diseases such as non-alcohol-related fatty liver disease (NAFLD), and other diseases such as hepatocellular carcinoma and polycystic liver disease. There was also representation of liver transplanted patients in the Delphi panel. In total, 23 panel members were from England, seven were from Scotland and three were from Wales. There were no panel members from Northern Ireland despite attempts to include panel members from Northern Ireland.
A total of 22 additional questions were added by the Delphi panel members in the first round of the Delphi process. The Delphi panel achieved consensus that an uncertainty was a top research priority for six research questions. Four additional research questions with the highest proportion of Delphi panel members scoring the question as highly important (scores between 7 and 9) were added to constitute the top 10 research priorities. The list of the top 10 research priorities (in the order of proportion who agreed that the uncertainty is a very important research priority) is available is table 1. All the top 10 research priorities were prevention and treatment uncertainties, and none were diagnostic test uncertainties. None of the panel members thought the first two questions as least important (scores 1–3). For the remaining eight questions, 3%–6.5% of people considered the questions to be least important (scores 1–3).
A summary of the availability of systematic review of RCTs on the topic of the individual questions, RCTs on the topic not included in the systematic review (if one exists), and the outcomes evaluated in these RCTs are listed in table 2. Table 2 also contains a suggestion for the next research steps. The list of the existing trials was compiled by searching ClinicalTrials.gov on 7 April 2018. The references to the trials not included in the systematic reviews is available in online supplementary appendix 3. As seen in table 2, a well-designed RCT is the next step for eight of these top 10 research questions. This is because it appears that the outcomes in those trials will not address the outcomes listed in the research questions.
The complete list of questions in the Delphi process, the proportion of respondents who considered a research question as very important and the summary scores in each Delphi round is available in online supplementary appendix 4. This appendix also has the breakdown of the proportion of patients, carers and general public who considered a research question as very important and their summary scores in each Delphi round along with similar summary measures for healthcare professionals.
This is the first priority setting partnership on non-alcohol-related liver and gallbladder disorders. This included a wide range of disease processes and a total of 428 unique research questions that met the scope of this priority setting partnership were identified. All the research questions were considered unanswered as there had been no high-quality systematic reviews which indicated that no further research is required, that is, all the research questions were uncertainties. Consensus that an uncertainty was a very important research priority was reached for six research questions. Four additional research questions with the highest proportion of Delphi panel members ranking the question as highly important were added to constitute the top 10 research priorities.
As evident from the online supplementary appendix 1, longevity of life and health-related quality of life are two major outcomes that appear important to patients, their carers and healthcare professionals. However, even when there are ongoing trials, it appears that the outcomes in those trials will not address the outcomes listed in eight of the top 10 research questions (table 2). Therefore, the next step in addressing these uncertainties is the design and conduct of RCTs. Such RCTs may involve qualitative studies to determine the design and should compare the treatments that improve the longevity of life and/or health-related quality of life.
It should be noted that uncertainties ‘what are the best treatments that cure or delay the progression (worsening) of NAFLD?’ and ‘what are the best treatments that cure or delay the progression (worsening) of non-alcohol-related steatohepatitis (NASH)?’ are related to each other. Although NAFLD includes NASH, most of the panel members felt that the research questions related to NAFLD and NASH should be kept separate uncertainties. While the same systematic review can cover both the uncertainties, the primary research study designed to address these two questions differ in terms of the setting, the outcomes used and the period of follow-up. Any primary research that tries to answer these two questions in a single RCT will be inefficient.
Similarly, for the uncertainties ‘what are the best treatments that cure or delay the progression (worsening) of PSC’ and ‘are there any treatments that reverse the liver damage in PSC?’, a single RCT will be inefficient and the preference of most of the panel members was to keep these uncertainties as separate uncertainties.
There are several limitations to our priority setting process. The first one is deviation from the original protocol. To select the final top priorities, the initial plan was to arrive at consensus by open small group and large group discussions of patients, carers, and healthcare professionals as suggested by the standard JLA process,8 which provides an opportunity for a knowledge exchange of viewpoints and experience. However, part of the steering committee with experience in a similar priority setting partnership felt that open discussions resulted in ‘loud voices’ being given more importance resulting in an unrepresentative list of top priorities. While this can be mitigated by facilitated group discussions by neutral JLA facilitators to ensure that all voices were heard in the discussions, this was considered by the steering committee as an important source of bias based on their prior experience in participating in open discussions. The steering committee therefore decided to follow the Delphi consensus method which is one of the major consensus methods.10 The advantages of Delphi consensus method over open discussions include anonymity of the response and the equal weight given to the opinions of all members.10 In addition, they are less costly to conduct without any limitation by geographical location compared with other methods of consensus10 because of the lack of necessity to travel and take time off regular work. However, there is considerable variability in the previous performance of Delphi processes with regard to the number of rounds and the criteria for achieving consensus.11 Arriving at consensus depends on people revising their scores based on the other’s scores. Our initial plan was to extend the Delphi to four rounds if consensus on 10 top research priorities was not reached in three rounds. However, there was minimal change in scores between the rounds for most questions (online supplementary appendix 3) and the Delphi process was completed in three rounds. Consensus on a top research priority was achieved for six questions only. However, the proportion of Delphi panel members ranking a question as highly important was greater than 70% for the remaining four questions added to the list of top research priorities. Previous Delphi consensus processes have used various cut-off points for defining consensus: greater than 70% agreement among panel members is well within the definition of consensus used in previous Delphi consensus processes.11
The other major limitation of our priority setting process is the representativeness of the people who completed the survey and took part in the Delphi process. The online survey was shared among clinicians and members of general and disease-specific patient organisations. Most questions resulting from the online survey relate to chronic liver disease (in particular, autoimmune liver diseases), perhaps reflecting the high motivation to support research from those groups. The Delphi panel also had a high representation of people related to chronic liver disease (in particular, autoimmune liver diseases) as patients, carers or healthcare professionals. While people affected by different liver and gallbladder disorders were actively sought through both general and disease-specific patient support groups and organisations, only a few responded and completed all three rounds of the Delphi process. The potential bias towards prioritising chronic liver diseases is evident as nine of the top 10 research priorities relate to chronic liver diseases (four relate to autoimmune liver diseases, three related to NAFLD and two related to liver transplantation). It was surprising that the uncertainties related to the treatment of chronic viral diseases such as chronic hepatitis B and chronic hepatitis C were not identified within the top 10 research priorities. This may be because of the perception by some of the panel members that the research questions related to the treatment of chronic hepatitis C were answered with the advent of directly acting antivirals (personal communication). The reason for non-prioritisation of chronic hepatitis B is not entirely clear. This may be because chronic hepatitis B may not have been considered as important as other chronic liver diseases or under-representation of chronic hepatitis B in the panel.
Cancer-related questions, childhood-related liver diseases and other benign disorders did not end up in the top research priorities (except for the treatment of very early hepatocellular carcinoma, which is managed by hepatologists and surgeons) probably for the reasons described above. We recommend that separate prioritisation processes are carried out for people with gallstones, a condition that affects approximately 5%–25% of the population,12 for people with primary and secondary liver cancer, and childhood liver disorders where significant uncertainties remain on the effectiveness of different treatments in decreasing mortality and improving health-related quality of life.
As well as the above limitation, we are aware of the inherent limitations of using solely technology to carry out the Delphi exercise. These are limitations that can potentially lead to bias in any consensus-building method including that of face-to-face consensus methods normally used in a JLA PSP.
One solution which might address the limitations of this priority setting process and the standard JLA process may be to collect information routinely from patients visiting hospitals using paper forms and conduct online meetings (video conferencing and presentation) before the final round of the Delphi (or the standard face-to-face priority setting workshop used by the JLA. Some JLA PSPs do use methods such as face-to-face interviews and group discussions rather than solely online surveys). By collecting information on paper forms and conducting the meetings in hospitals, it is possible to engage with people who do not have access to or are not familiar with computers. It is also possible to engage with people who have concerns regarding data confidentiality with the use of computers or social media by collecting information using paper forms. Ethical and confidentiality issues will need to be considered prior to engaging patients attending hospital in the research prioritisation process.
Another limitation of our priority setting process is the drop-outs during the Delphi process. While some of the drop-outs may be related to the ability to complete online surveys and use Microsoft Excel, some patient representatives or clinicians may have dropped out because they did not find any research question to be of direct relevance to them. Other reasons include lack of understanding of the conditions, feeling that the process was too complicated, feeling that the process would not work, and the time commitment for the process. This is because of the broad scope of this research prioritisation process and may be overcome by choosing a narrower focus while defining the scope of the prioritisation process, and by better explanation of the disease processes through presentations.
It should also be recognised that the Delphi panel was constituted of representatives from England, Scotland and Wales. Therefore, the findings are applicable in only these countries. However, the findings are likely to be applicable throughout the NHS and in other European and Western countries with a similar spectrum of chronic liver diseases and similar treatment options available.
In summary, there are significant uncertainties in the management of liver and gallbladder disorders. Further high-quality research is necessary to address these uncertainties which may require programmes of basic, translational, clinical and public health research. For issues with diverse and unproven treatment options, RCTs may be the only mechanism for identifying the most effective treatment and the treatments that represent good value for money for the NHS. Such RCTs should assess the effect of different treatments in improving longevity of life and/or health-related quality of life.
We thank Mr Andrew Langford, previous chief executive of British Liver Trust, participants in the survey, Ms Caroline Whiting and Ms Beccy Maeso, JLA PSP co-ordinators for their advice on the process and comments on the manuscript, and the Delphi panel members who preferred to remain anonymous in contributing to this research prioritisation process.
Contributors KG: conceptualisation, healthcare professional and methodological lead of steering committee, Delphi panel member, analysis, author of the manuscript. MW: patients and carers lead of steering committee, Delphi panel member. BRD, CF, BF, AM, SM, IS, ET: steering committee, Delphi panel member, suggested revisions to the manuscript. RM: steering committee, James-Lind Alliance advisor. IA, MC, JFD, GE, AME, LF, AG, NK, EJM, SR, AS, LS, AY: Delphi panel member, suggested revisions to the manuscript.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests The decisions made by healthcare professionals involved in the research prioritisation process might have been influenced by their professional interests, in addition to their own, or family member’s experience of health conditions. Decisions made by patients and carers in the research prioritisation process might have been influenced by their particular experiences, health needs and interests.
Ethics approval Ethical approval was not deemed necessary because no personal identifiable information was being collected, and the questions were being asked of healthcare professionals, patients and their carers were not considered sensitive questions.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement All data are available in the manuscript or in the online supplementary file.
Patient consent for publication Not required.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.