Article Text
PDF
Abstract
Objective To summarise study descriptions of the James Lind Alliance (JLA) approach to the priority setting partnership (PSP) process and how this process is used to identify uncertainties and to develop lists of top 10 priorities.
Design Scoping review.
Data sources The Embase, Medline (Ovid), PubMed, CINAHL and the Cochrane Library as of October 2018.
Study selection All studies reporting the use of JLA process steps and the development of a list of top 10 priorities, with adult participants aged 18 years.
Data extraction A data extraction sheet was created to collect demographic details, study aims, sample and patient group details, PSP details (eg, stakeholders), lists of top 10 priorities, descriptions of JLA facilitator roles and the PSP stages followed. Individual and comparative appraisals were discussed among the scoping review authors until agreement was reached.
Results Database searches yielded 431 potentially relevant studies published in 2010–2018, of which 37 met the inclusion criteria. JLA process participants were patients, carers and clinicians, aged 18 years, who had experience with the study-relevant diagnoses. All studies reported having a steering group, although partners and stakeholders were described differently across studies. The number of JLA PSP process steps varied from four to eight. Uncertainties were typically collected via an online survey hosted on, or linked to, the PSP website. The number of submitted uncertainties varied across studies, from 323 submitted by 58 participants to 8227 submitted by 2587 participants.
Conclusions JLA-based PSP makes a useful contribution to identifying research questions. Through this process, patients, carers and clinicians work together to identify and prioritise unanswered uncertainties. However, representation of those with different health conditions depends on their having the capacity and resources to participate. No studies reported difficulties in developing their top 10 priorities.
- James Lind Alliance
- priority setting partnership
- patient and public involvement
- patient involvement in research
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- James Lind Alliance
- priority setting partnership
- patient and public involvement
- patient involvement in research
Strengths and limitations of this study
This is the first scoping review of published studies using the James Lind Alliance (JLA) approach available with involvement of patients, carers and the public in the setting the research agenda.
The weakest voices often lack representation, which could limit the generalisability of these priorities to these populations.
Because a scoping review approach was used, the quality of the articles was not assessed prior to inclusion.
We were not in contact with the JLA Coordinating Centre and search in all relevant literature, such as grey literature and studies, which do not described all steps of the JLA process, might have limited our results.
A limitation of this scoping review was our inclusion of only English-language articles.
Introduction
Over the past decade, patient and public involvement (PPI) has been highlighted worldwide in both health research agendas and the development of next-step research projects.1 PPI has been defined as ‘experimenting with’ as opposed to ‘experimenting on’ patients or the public.2 PPI allows patients to actively contribute, through discussion, to decision-making regarding research design, acceptability, relevance, conduct and governance from study conception to dissemination.3 However, PPI may also involve active data collection, analysis and dissemination.4
Researchers have noted that involving healthcare service users, the public and patients improves research quality, relevance, implementation and cost-effectiveness; it also improves researchers’ understanding of and insight into the medical and social conditions they are studying,1 5 although such evidence is still relatively limited.4
The James Lind Alliance (JLA) is a UK-based non-profit initiative that was established in 2004. The JLA process is focused on bringing patients, carers and clinicians together, on an equal basis, in a priority setting partnership (PSP) to define and prioritise uncertainties relating to a specific condition.6 Hall et al 7 note that the JLA aims to raise awareness among research funding groups about what matters most to both patients and clinicians, in order to ensure that clinical research is both relevant and beneficial to end users. According to the JLA Guidebook,6 uncertainties and how to prioritise these are key features of the JLA process. The process begins by defining unanswered questions (ie, ‘uncertainties’) about the effects of treatment and healthcare—questions that cannot be adequately answered based on existing research evidence, such as reliable, up-to-date systematic reviews—and then prioritises the uncertainties based on their importance. The most recent version of the JLA Guidebook explains that many PSPs interpret the definition of treatment uncertainties broadly. They may interpret ‘treatments’ to include interventions such as care, support and diagnosis. This approach has been an important development and one that helps the JLA adapt to the changing health and care landscapes, as well as to the changing needs of its users.6
The JLA provides facilitation and guidance in the identification and prioritisation processes. This process forms part of a widening approach to PPI in research. The characteristics of the PSP process are (1) setting up a steering group to supervise all aspects of the study; (2) establishing a PSP; (3) assembling potential research questions; (4) processing, categorising, and summarising those research questions; and (5) determining the top 10 research priorities through an interim process and a final priority setting workshop using respondent ranking and consensus discussion. To ensure that all voices in the workshop are heard, the JLA supports an adapted nominal group technique (NGT) for PSPs when choosing their priorities. NGT is a well-established and well-documented approach to decision-making.6
To our knowledge, there is a gap in existing research given that no review has yet been published describing how the JLA approach is used to establish steering groups, set up PSPs, gather uncertainties, summarise uncertainties and determine the lists of top 10 priorities. Thus, the objective of this scoping review was to summarise study descriptions of the JLA approach to the PSP process, and how this process is used to identify uncertainties and develop lists of top 10 priorities.
How do the studies describe the characteristics of the PSPs and, elaborating on aspects, how have they operationalised the JLA methods?
How do the studies describe involvement of different user groups?
What processes are used to gather and verify uncertainties?
Methods
Identifying relevant studies
A systematic search was conducted up to October 2018 using five databases: Embase, Medline (Ovid), PubMed, CINAHL and the Cochrane Library. The search strategy in each database was «james lind*» OR «priorit* setting partnership*». We also searched in JLA website. This search identified 746 records and 431 potentially relevant citations. After removing duplicates and screening titles and abstracts based on our inclusion and exclusion criteria, the full text of 171 studies was examined in greater detail. A total of 37 studies met all criteria for review and were subsequently investigated. These numbers were verified by a university librarian (see flowchart, figure 1).
Flow diagram. JLA, James Lind Alliance; PSP, priority setting partnership; RCT, randomised controlled trial.
Selecting relevant studies
A prescreening process included reviewing the search results and excluding all articles that were not research studies, that were unavailable in full text or that clearly did not involve the JLA PSP approach. At least two authors screened the remaining articles using the inclusion and exclusion criteria presented in table 1.
Criteria for inclusion and exclusion
Charting data
A data extraction sheet was created to collect studies’ demographic details, aims, samples and patient groups. The sheet was used to collect methodological details about the studies’ PSPs, including descriptions of stakeholders, lists of top 10 priorities, descriptions of the roles of JLA facilitators and PSP stages.
Procedure
In addition to the first author, one of the other authors evaluated each article, and individual and comparative appraisals were discussed among the authors until agreement was reached. At least two authors were involved in each of the study selection procedures. A predefined procedure was developed for consulting a third author, or the whole research team, in cases of discrepancies; however, this was never necessary (ie, decisions to accept or reject unclear articles were based on a dyad consensus). The first author and one other author extracted the characteristics and findings of each study.
Quality appraisal
The most recent JLA Guidebook 6 served as the context for investigating the descriptions of the studies’ methods. A quality assessment was not included in the remit of this scoping review.8
Patient and Public Involvement
No patient was involved.
Collating, summarising and reporting results
Findings related to the scoping review’s research questions, based on the JLA approach, were extracted and documented. The information shown in table 2 includes the studies’ aims, suggested uncertainties and—depending on the version of the JLA guidelines used—how these uncertainties were determined. We also collected information on the stakeholders (including members of the PSP), whether a JLA advisor/facilitator was used, and the JLA process stages: (1) setting up a PSP, (2) gathering uncertainties, (3) data processing and verifying uncertainties, (4) interim priority setting and (5) final priority setting. The results are presented based on the JLA Guidebook steps, which have remained consistent across versions.6 9–11
Characteristics of included studies
Results
In total, 37 studies met the inclusion criteria; their characteristics are summarised in table 2.
The publication years ranged from 2010 to 2018. The number of studies using this process has increased annually, with 12 published in 2017. In our sample, 27 of the studies were from the UK,1 5 7 12–35 8 were from Canada,36–43 and 1 each was from India44 and Spain.45
The JLA process participants were patients, carers and clinicians aged ≥18 years. The studies collectively represented patient groups with heterogeneous ages and health conditions/diseases, with later studies generally more focused on symptoms and function than on diseases (table 2). Totally, 15 of the studies gave information about ethnicity.13 14 16 19 21 23 25–27 32 33 35 36 40 42 One of the studies also gave information about socioeconomic status.26 Another study gave only information about socioeconomic status.44
Three of studies described that patient and carers submitted more questions on psychosocial issues, psychosocial stress, depression and anxiety compered with clinicians.13 23 40 No studies described disagreement in the prioritisation stages. However, 24 other studies also mentioned psychosocial issues without noting who had done so.1 7 14–19 25–27 29 31–39 41–43 Ten studies did not mention psychosocial issues.5 12 20–22 24 26 28 44 45 The types of health conditions that were addressed included gastrointestinal,26–28 neurological,1 5 7 16 21 38 dermatological,13 15 17 29 34 45 endocrinal14 32 42 and cancer22 25 41 43 conditions.
Setting up a PSP
The JLA steering group is made up of key organisations and individuals who can collectively represent all or the majority of issues related to the PSP, either individually or through their networks.6
All included studies had a steering group, although they were described differently. Nineteen studies1 5 12 14–17 19 20 22 23 25 31 36 37 39–41 45 included patients, carers and clinicians in their steering groups; 16 studies7 13 18 24 26–29 32–35 38 42–44 did not include carers in their steering group (ie, only patients and clinicians). In one study,30 the titles of the members on the steering group were not reported; in another,21 the steering group did not specifically include patients, carers or clinicians, but rather stated that representation from all stakeholders was ensured.
The number of JLA steps in the PSP process varied across studies from four steps1 32 33 37 39 42 to eight steps.20 22 44 Five steps, corresponding to JLA Guidebook V.4, V.5 and V.6, were most common,12 13 15–19 23 24 26–29 31 34 36 38 40 41 43 45 with step 1, initiation; step 2, collection of uncertainties; step 3, collation of uncertainties; step 4, interim priority setting; and step 5, final priority workshop.
Gathering uncertainties
PSPs aimed to gather uncertainties from as wide a range of potential contributors as possible, ensuring that patients were equally confident and empowered compared with clinicians in submitting their perspectives on uncertainties.6
With regard to recruitment, various partner organisations, local advertisements, social media, patients, carers and clinicians were PSP information targets. In addition to an online and paper survey, two studies also used face-to-face methods to reach and facilitate involvement by their identified groups.5 42
The questions were usually deliberately open-ended to encourage full responses regarding the experiences of patients, carers and clinicians. One of the 37 studies44 used an online survey to collect uncertainties; patients and clinicians were invited via email to endorse their priorities based on a table that had been developed from abstracts collected in a literature search. Among the other 36 studies, 12 used open-ended questions1 15 18 23 25 32 35 40–43 45 such as ‘What questions about the management of hypertension or high blood pressure would you like to see answered by research?’ In seven studies, participants (patients, carers and clinicians) were asked to submit three to five research ideas.16 17 20 21 27 28 33 In eight studies, no limit was placed on the types of questions that could be submitted.5 13 24 30 31 36 37 39 One study asked about eight open-ended questions requesting a narrative answer.38 Close-ended questions were used in three studies,22 29 34 such as ‘Do you have questions about the prevention, diagnosis or treatment of hair loss that need to be answered by research?’ Five studies did not report their question format.7 12 14 19 26
The number of submitted uncertainties ranged from 8227, submitted by 2587 participants,32 to 323, submitted by 58 participants.45 All studies except two7 44 reported involving patients, carers and clinicians in the initial survey. Two of the studies addressed verifying uncertainties for example by content experts or librarians.40 43 The steering group or researchers were involved in addressing verifying uncertainties in 22 of the studies and5 7 14–16 18 20 21 23–27 30 31 33 35 37 39 41 44 45 in 13 of the studies not describing verifying the uncertainties.1 12 13 17 19 22 28 29 32 34 36 38 42
Data processing and verifying uncertainties
Unlike most surveys that are designed to collect answers, JLA PSP surveys are designed to collect questions. The survey responses must then be reviewed, sorted and turned into a list of ‘indicative’ questions, all of which are unanswered uncertainties.6
According to Lechelt et al,43 uncertainties are organised through coding, with natural clusters emerging. During this step, duplicates such as similar and related uncertainties are identified. Clinician–patient dyads consolidate and rephrase each cluster of related questions into a single indicative uncertainty, written in lay language using a standard format. Lomer et al 28 specified that similar uncertainties are combined to create indicative uncertainties. Among our included studies, 20 described refining questions into indicative uncertainties,5 13–15 17 19 20 23 24 27–29 31–34 38 39 42 43 while 17 did not describe a concept of indicative uncertainties.1 7 12 16 18 21 22 25 26 30 35–37 40 41 44 45
In total, 16 of the studies described directly ranking and assessing survey-generated uncertainties from a long list ranging from 43 to 226 uncertainties.1 5 13 14 19–21 23 24 26 27 30 38 41 43 44
The wording of the long list of uncertainties was reviewed by the steering group, and, in some cases, wording was altered to make the uncertainties more understandable and to explain complex words not generally well known to the public.1
Interim priority setting
Interim prioritisation is the stage at which the long list of uncertainties (indicative questions) is reduced to a short list for the final priority setting workshop.6
All studies described an interim stage, using the terms: interim priority setting14 32; interim prioritisation1 5 42; and ranking exercise.13 44
Their short lists varied from 2226 to 30 uncertainties.12 17–19 22 25 30 36 39 Sixteen of the studies used an interim prioritisation of their top 25 uncertainties that were taken to a final prioritisation workshop, where the participants agreed on their top 10 priorities.1 7 13 20 21 23 24 28 29 31 33–35 37 38 40 Three of the studies did not describe the number of shortlisted treatment uncertainties.15 27 44
To reduce the number of uncertainties, an interim prioritisation exercise was conducted by email or by post.5 18 32 Patients, carers and health professionals were initially invited to examine the long list18; 14 of the studies used a second online survey,1 19 20 23 25 28 29 31–35 38 40 and in one study, the steering group members facilitated an interim ranking exercise.39
Final priority setting
The JLA’s final stage is a rank ordering of the uncertainties, with a particular emphasis on the lists of top 10 priorities. For JLA PSPs, a final face-to-face priority setting workshop is conducted with both small group and whole group discussions. The NGT can be used by groups, with voting to ensure that all opinions are considered6; 21 of the studies reported using the NGT in the final priority setting workshop.5 12 16–19 21 23 25 26 29 32–34 36 37 39 41–43 45
All of the studies implemented a final priority setting workshop to agree on their top 10 priorities. In most of the studies, these final workshops included patients, carers and clinicians; nine studies mentioned including only patients and clinicians.7 23 28 29 33 34 42–44
Discussion
To our knowledge, this is the first scoping review of published studies using the JLA approach, although the number of steps used by PSPs differed and not all papers describe in detail every aspect of the JLA approach. However, overall they incorporated the same procedural content, which indicates no implications or small implications for our findings. Thus, this scoping review provides unique insight into a broad and varied range of perspectives on PPI using the JLA approach. Interestingly, there were some differences between the questions submitted by patients and carers compared with those submitted by clinicians. The patients focused more on symptoms and function than on disease, while clinicians focused on general treatment. Compared with clinicians, patients submitted more questions about psychosocial issues, psychosocial stress, depression and anxiety.13 23 40 There were no studies that described disagreement in the prioritisation steps. The health conditions addressed in these studies were primarily somatic diseases, although one study was about life after stroke and included mental health.5 Thus, the JLA approach is an appropriate and important method for defining research from the perspectives of end users that is, patients and carers.46
A key value that informs such partnerships is often described as equality. Equitable partnerships might be defined as a gradation of shared responsibility negotiated in a collaborative and cooperative decision-making environment. Whether such values always align within the JLA process is an open question. Thus, reflecting on and clarifying values about involvement before starting collaborative work might enhance the positive impacts while avoiding the negative impacts of public involvement.47
The number of priority setting exercises in health research is increasing,48 and our review indicates that the use of the JLA approach is also growing . This approach facilitates broad stakeholder involvement, and it is transparent and easy to replicate. This is consistent with findings by Yoshida,48 who argues that there is a clear need for transparent, replicable, systematic and structured approaches to research priority setting to assist policymakers and research funding agencies in making investments. Increased public involvement can lead to a wider range of identified and prioritised research topics that are more relevant to service users.49 A key strength of involving the public and patients, rather than only academics, throughout the partnership process is described in these studies, including having a project led by representatives of a wider range of consumer and clinician organisations.1 The number of resulting uncertainties reflects this breadth. The studies examined tended to conclude that the JLA principles were welcomed, but consistently emphasised the need for an even broader understanding, better conceptualisation and improved processes to incorporate the results into research. However, few studies focused on how to reach the weakest voices for survey participation. After critically reading these studies, one might ask whether they included the lowest socioeconomic groups and most vulnerable patients. Many respondents, particularly those associated with charity organisations, are likely to be white and middle class and to have high education attainment levels. Yet it is the individuals who are more difficult to reach, such as those in low socioeconomic groups and those who are vulnerable patients, who may have the greatest unmet needs and stand to gain the most from improved treatment.25 26 35 42 Given that the JLA is designed to identify shared research priorities, such individuals and their needs may not be reflected in what is typically reported studies. In one case, to better facilitate patient and carer involvement, and to reach those who may not receive and/or respond to email or postal information, a steering group member visited existing support groups and arranged the distribution of information leaflets at local meetings.5 Although great efforts were reportedly made25 to include participants from black and minority ethnic groups and care home populations, they were not particularly successful. Lough et al 33 reported that the use of an online survey may introduce a bias in favour of patients who use the internet and social media. It is also likely that those with literacy issues will not participate.16 Three of the studies5 18 42 attempted to facilitate participation among those with language barriers and literacy issues, which implies that efforts need to be made to enable minority groups and learning disabilities to participate in the PSP process. Stephens et al 22 note another major challenge to involving users in research and patients in the steering group who have incapacitating symptoms and short expected survival durations. Another important issue is that all but two studies44 45 were from English-speaking countries and thus represent a relatively limited global population.
According to the JLA Guidebook,6 PSPs usually report their process and methods, the participants involved, results, reflections on successes, lessons learnt or limitations, and the next steps. It is important that these reports be written in a language understandable to everyone with an interest in the topic, not just to clinicians. Lough et al 33 explained that all of the unanswered questions generated by their PSPs would be available on the JLA website and widely disseminated to research commissioners, public health and research funders. However, such reports can be difficult to obtain by those without ready online access or by those with literacy issues. Eleftheriadou et al 13 included implementation of a feasibility study as one of their top 10 priorities; the authors hoped that, following its publication, along with their list of the most important uncertainties, relevant studies would be developed.
Running a PSP and involving the relevant stakeholders in deciding which research should be funded seem to be an effective and sustainable model.24 Without doubt, the essential advantage is integration of this involvement in both research and healthcare. Identifying research priorities is perhaps where the PSP’s greatest effect can be achieved.26 Nevertheless, one might ask whether PSPs emphasise basic research less than applied research. Abma et al 50 have argued that the international literature describes corresponding challenges in research agenda setting and follow-up; patient involvement is limited to actual agenda setting, and there is limited understanding of what happens next and how to shape patient involvement activities in follow-up phases. This scoping review process gathered a large number of research priorities from a diverse set of respondents.32 34 There has been a clear paradigm shift from a reactive to a more proactive approach described as ‘predictive, personalised, preventative and participatory’.25 It is expected that the JLA process will have a clinical impact by driving relevant research studies based on PPI. Crowe et al 51 reported that a critical mismatch between the treatments that patients and clinicians want to have evaluated and the treatments actually being evaluated by researchers. This apparent mismatch should be taken into account in future research.
Strengths and limitations
A major strength of this paper is the application of a rigorous and robust scoping review method, including independent screening and data extraction. The search strategy was carefully performed in conjunction with a research librarian. To strengthen the review’s validity, several databases were used, and we have reported them with complete transparency. The studies selected for inclusion were manually searched. Although we searched multiple databases for the period since their inception, we may not have identified all relevant studies. We did not search the grey literature, assuming that empirical research using the JLA approach would be found in indexed databases. As a scoping review, the findings describe the nature of research using JLA’s approach and provide direction for future research; hence, this review cannot suggest how to operationalise the JLA process or how to use it in a given context. Another strength is that several of the researchers contributing to this project also work in the clinical areas represented in the studies. In addition, while a quality analysis was beyond the scope of this paper, we have noted varying descriptions within the selected studies (ie, sample sizes, health status and age of groups). Finally, the included studies do not provide information about the impact of involvement, regarding development of consensus, the discussions among all those who took part, the distribution of power and the politics. In future work, it may be important to evaluate how much influence patient/public partners had during the process, besides the impact of the number of participants in the respective groups. Another limitation might involve our inclusion criteria with respect to requirement for peer-reviewed publications, which by definition will use more academic language and may not be readily accessible to the layperson. Lastly, the cost and time involved in a PSP are described in one publication only.24 According to the JLA Guidebook, the PSP process will last approximately 12–18 months.6
Conclusions
JLA-based PSP makes a useful contribution to identifying research questions. A range from 327 to 8227 uncertainties were published, with 27 studies from UK. The number of reported steps varied from four to eight. In total, 33 studies mentioned the involvement of a JLA facilitator. Twenty-four included studies that addressed methods for verifying uncertainties, and the use of NGT was reported in 21 studies. Finally, it is important that the results of these studies, including the top 10 priorities, reach those who answered the survey, including the vulnerable groups. Online publishing might contribute to this. Future studies should focus on factors influencing patient and carer involvement in priority setting projects.
Acknowledgments
We thank research librarian Malene Wøhlk Gundersen for her helpful and knowledgeable assistance.
References
Footnotes
Contributors AN, LH, SL, EKG and AB designed the study. AN coordinated the project and is the guarantor. AN, LH, SL, EKG and AB screened articles and performed data extraction. AN conducted the literature search. AN, LH, SL, EKG and AB interpreted the data. AN drafted the manuscript and all authors critically reviewed it. All authors read and approved the manuscript.
Funding This work was supported by the Research Council of Norway (grant number OFFPHD prnr 271870), Lørenskog Municipality and Oslo Metropolitan University. The funders had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement All data relevant to the study are included in the article.