Article Text

Original research
Which outcomes should always be measured in intervention studies for improving work participation for people with a health problem? An international multistakeholder Delphi study to develop a core outcome set for Work participation (COS for Work)
  1. Margarita Ravinskaya1,
  2. Jos H Verbeek1,
  3. Miranda Langendam2,
  4. Ira Madan3,4,
  5. Suzanne M.M. Verstappen5,6,7,
  6. Regina Kunz8,
  7. Carel T.J. Hulshof9,
  8. Jan L. Hoving1
  9. Delphi participants
    1. 1Department of Public and Occupational Health, Coronel Institute of Occupational Health, Amsterdam Public Health Research Institute, Cochrane Work, Amsterdam UMC Location AMC, University of Amsterdam, Amsterdam, The Netherlands
    2. 2Department Epidemiology and Data Science, Amsterdam Public Health Research Institute, Amsterdam UMC Location AMC, University of Amsterdam, Amsterdam, The Netherlands
    3. 3Guy's and St Thomas' NHS Foundation Trust, London, UK
    4. 4King's College London Faculty of Life Sciences and Medicine, London, UK
    5. 5Centre for Epidemiology Versus Arthritis, Faculty of Biology, Medicine and Health, University of Manchester, Manchester, UK
    6. 6Manchester University NHS Foundation Trust, NIHR Manchester Biomedical Research Centre, Manchester, UK
    7. 7MRC Versus Arthritis Centre for Musculoskeletal Health and Work, University of Southampton, Southampton, UK
    8. 8Research Unit EbIM, Evidence Based Insurance Medicine, Division of Clinical Epidemiology, University Hospital Basel, Basel, Switzerland
    9. 9Department of Public and Occupational Health, Coronel Institute of Occupational Health, Amsterdam UMC Location AMC, University of Amsterdam, Amsterdam, The Netherlands
    1. Correspondence to Margarita Ravinskaya; m.ravinskaya{at}amsterdamumc.nl

    Abstract

    Objective Synthesising evidence of the effects of interventions to improve work participation among people with health problems is currently difficult due to heterogeneity in outcome measurements. A core outcome set for work participation is needed.

    Study design and setting Following the Core Outcome Measures in Effectiveness Trials methodology, we used a five-step approach to reach international multistakeholder consensus on a core outcome set for work participation. Five subgroups of stakeholders took part in two rounds of discussions and completed two Delphi voting rounds on 26 outcomes. A consensus of ≥80% determined core outcomes and 50%–80% consensus was required for candidate outcomes.

    Results Fifty-eight stakeholders took part in the Delphi rounds. Core outcomes were: ‘any type of employment including self-employment’, ‘proportion of workers that return to work after being absent because of illness’ and ‘time to return to work’. Ten candidate outcomes were proposed, among others: ‘sustainable employment’, ‘work productivity’ and ‘workers’ perception of return to work’.

    Conclusion As a minimum, all studies evaluating the impact of interventions on work participation should include one employment outcome and two return to work outcomes if workers are on sick leave prior to the intervention.

    • occupational & industrial medicine
    • public health
    • rehabilitation medicine
    • statistics & research methods

    Data availability statement

    All data relevant to the study are included in the article or uploaded as online supplemental information.

    https://creativecommons.org/licenses/by/4.0/

    This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.

    Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    STRENGTHS AND LIMITATIONS OF THIS STUDY

    • The core outcome set was developed in accordance with the Core Outcome Measures in Effectiveness Trials guidelines.

    • A five-step approach was used to reach international multistakeholder consensus on a core outcome set for work participation.

    • Five key stakeholder groups were involved in the consensus process: researchers, occupational health professionals, policy makers, employee/patient representatives and an employer.

    • Provided with comprehensive background information the stakeholders participated in two group discussions to discuss perspectives following a transparent Delphi procedure.

    • The majority of the stakeholders were researchers from Europe and North America.

    Introduction

    Authors of systematic reviews (SRs) state that inconsistent reporting of work participation outcomes in clinical trials hampers evidence synthesis in the field of occupational health (OH).1–3 Coordinating editors of Cochrane review groups have indicated that the reliability and quality of SRs could be improved by using core outcome sets (COS).4 In 2019, the Coronel Institute of OH at the Amsterdam University Medical Centre established an international research collaboration to develop a COS for Work Participation (COS for Work), based on the Core Outcome Measures in Effectiveness Trials (COMET) methodology.5 Our aim was to develop a generic COS for Work to be used in intervention studies which expect to have an effect on work participation. As our COS is generic, and relevant to different diseases, it can be used for any type of intervention. For example, pharmaceutical studies for people with rheumatoid arthritis may not only improve clinical outcomes but indirectly also help participants participating in work. Such studies mostly measure work outcomes as a secondary outcome. Other types of interventions aim directly in helping people with work participation, such as return to work interventions for cancer survivors or workers with long COVID.

    The first phase of our project involved an SR on the spectrum of work outcomes used to measure the effect of interventions in published randomised controlled trials (RCTs).6 The SR showed extensive heterogeneity in work-participation outcome measurements and confirmed the need for a COS for Work. In addition, we saw that it was unclear why authors chose to measure outcomes in a certain way and that to create COS for Work we would need to have a framework for outcomes which would ensure meaningful, pragmatic and mutually exclusive categories in which outcomes could be placed. In the second phase, we created a framework to aid the selection of work participation outcomes.7 The framework outlines four main stages of work participation: outcomes assessing whether a person is successful in acquiring or initiating employment (stage 1), outcomes indicating whether a person is in employment, can retain employment or lost work within the duration of the study, that is, having employment (stage 2), outcomes that measure increasing or maintaining productivity at work (stage 3) when persons experience limitations or restrictions with working or have less output, and outcomes that assess return to employment and sick leave from work (stage 4) when people (temporarily) stop attending work and are on sick leave. Phase three involved a survey among reviewers and trial authors showing different reasons and preferences for choosing and using work outcomes. In particular, researchers choose outcomes because of their use in similar studies or their relevance for the health problem or type of intervention.8

    In the fourth phase, we sought international consensus on COS for Work via a Delphi study. In the fifth and final phase, we will make recommendations for clinimetrically sound methods to measure the core outcomes selected during phase 4.

    The overall aim of the Delphi study is to define a comprehensive and minimal set of outcomes that is relevant and feasible for measuring the effectiveness of interventions that result in a change in work participation, based on consensus of an international, multistakeholder group.

    Methods

    On-line multistakeholder discussions and Delphi voting

    To develop methods, we followed the COMET guidelines,5 that is, the COS-STAndards for Development (The COS-STAD).9 For reporting we used the COS-STAndards for reporting (COS-STAR).10 The protocol for this study was published on the COMET website prior to the commencement of the study (https://www.comet-initiative.org/studies/details/1195, online supplemental file 1), and includes an extensive explanation of the methodology. The five steps are summarised briefly below.

    Step 1: preparation

    We invited five groups of stakeholder representatives:

    1. OH researchers—experienced researchers in OH field from varying disciplines, such as: health economy, occupational mental health, insurance medicine, workplace interventions, RtW interventions.

    2. OH professionals—physiotherapists, orthopaedic surgeon, rehabilitation specialist.

    3. Workers/employees and patients—stakeholders representing patients and workers as members of patients federations, workers alliances and labour unions.

    4. Policy makers—stakeholders working for organisations creating OH policies such as the National Institute for Insurance against Accidents at Work, European Agency for Safety and Health at Work.

    5. Employer—OH case manager.

    These five stakeholder groups were chosen as key stakeholders. Researchers will use the COS for Work and have experience with work participation outcomes measurement and the remaining stakeholder groups will use the results of what is researched in their professional or daily life.

    Stakeholders were recruited from our phase two international survey8 and our professional network. We anticipated that an effective group discussion with up to 30 participants would be feasible to allow diversity of opinion in the various subgroups. Our aim was to organise two group discussions of 30 participants, in the morning and late afternoon to account for the varying time zones. We sent out 90 invitations.

    Stakeholders who agreed to participate received an information package that included: (1) instructions for Delphi participation (online supplementary file 2), (2) a document containing the outcomes that would be included in the Delphi with worksheets to prepare for the first discussion (table 1, online supplementary file 3) and (3) a list of all participants in each discussion group. Item generation for the 24 outcomes in the preliminary set (table 1) was based on the SR6 and categorised under the 4 work participation stages as described in the framework (table 1).

    Table 1

    The preliminary set of outcomes that was presented to the stakeholders prior to the first discussion

    Participants were invited to familiarise themselves with the material and add any outcomes, which they deemed missing from the preliminary list prior to the first discussion (online supplementary file 4). The research team evaluated whether these new outcomes fitted the COS for Work criteria (table 2) and presented the outcomes during the first discussion round.

    Table 2

    Criteria for COS outcomes which the stakeholders were asked to consider when deciding for or against inclusion of outcomes in COS for work

    Step 2: first online consensus stakeholder meeting

    In the first online meeting, we presented the steps and research findings of the COS development including the four stages of work participation (see table 1) and criteria for COS outcomes (table 2). Further, we elaborated on the various stakeholder perspectives and its effect on outcome prioritisation, and discussed the proposed additional outcomes. In breakout rooms, groups of stakeholders created and discussed ‘absolutely in’ and ‘absolutely out’ outcomes for each stage. The results were discussed in the plenary session.

    Step 3: first round of Delphi voting

    COMET online DelphiManager software participants voted on a total of 25 outcomes (outcomes from table 1 plus one outcome suggested by a stakeholder) accompanied by clarifying text on how it could be measured. With a personal login, the DelphiManager software allowed each outcome to be anonymously ranked on a Likert Scale from 1 to 9: from not important (0 points) to critical (nine points), including an ‘unable to score’ option. Participants were asked to provide rationale for their ratings and additional outcomes could be suggested. A single voting reminder was sent to participants 2 days before the closing of the first round.

    Step 4: second online stakeholder meeting

    The results of the first online Delphi voting round and a collated list of arguments on why to include or exclude an outcomes were first distributed via email and then discussed in the second online stakeholder meeting. We also presented our analysis of suggested additional outcomes (online supplementary file 5). Based on the a priori defined consensus definition (see ‘data analysis’) and input from the stakeholder meeting, the authors then decided which outcomes would be dropped, included, reformulated or added to the second Delphi round.

    Step 5: second Delphi voting round

    The same online Delphi procedure allowed each participant to individually rate the outcomes that remained from the first Delphi round. As before, reminders were sent to non-completers.

    Patient and public involvement

    Consultation of patients’ is seen as a crucial aspect in the development of COS.5 We have consulted patient representatives, who were also representing the ‘employee’ perspective, in the discussion rounds and the selection of core outcomes for COS for Work. The patient representatives were members of, among others: The Netherlands Patients Federation, Canadian Injured Workers Alliance and labour unions. In addition, we also involved policy makers active in the field of OH. The results were disseminated to all participants via e-mail.

    Data analysis

    After the first and second stakeholder meetings we held a meeting with the steering group to discuss how we could use and present the stakeholder input for the Delphi voting to inform all Delphi participants. The results from the Delphi rounds were extracted from the DelphiManager software into a comma-seperated file format and analysed in SPSS (version number 28) using descriptive statistics.

    For each round, we calculated the distribution of scores for all outcomes as well as the percentage of stakeholder scoring between 7 and 9. As per protocol, outcomes scored between 7 and 9 by at least 80% of all stakeholders were deemed as core—showing as ‘definite core outcome IN’ on the spreadsheet distributed among stakeholders. Outcomes receiving a mean score between 7 and 9, from 50% to 80% of the participants in the first Delphi round were ‘candidate outcomes’ to be discussed with the stakeholders and rerated in the second Delphi round—labelled as ‘possible core outcome IN’. Outcomes which received a score between 7 and 9 from less than 50% of the participants were dropped after the first round. For outcomes that were not considered definite core outcome but for which there was strong disagreement between stakeholder groups in the first round, the steering group still considered them for being discussed and rated in the second round. It was possible to upgrade or downgrade the ranking of an outcome for inclusion or exclusion in the second Delphi round based on argumentation of stakeholders. The final COS is based on the rankings of the second round.

    Results

    Respondents

    Initially, we approached 90 potential stakeholders and 59 stakeholders agreed to participate (not everyone from this group registered for the Delphi). It was difficult to recruit employer and employee representatives; we had no replies from employee representatives and one employer. After the first discussion round, participants helped us to recruit 11 employee representatives through their professional network. In total 70 stakeholders agreed to participate. Twelve participants took part in either one or two of discussions but did not register for the DelphiManager software, some of whom emailed that they did not manage to due to time constraints (see table 3 for participant characteristics).

    Table 3

    Characteristics of survey respondents for two Delphi rounds (n; %)

    First online consensus stakeholder discussion

    We presented all Delphi voting information (ratings and arguments) to the stakeholders as outlined in the methods section. The stakeholders discussed their preferences for and against the outcomes in four breakout rooms. They then presented their arguments in a plenary session that included discussion on topics such as:

    • When work can be considered as sustainable and healthy and the challenges of measuring such concepts as part of this COS.

    • Which types of outcomes would be applicable and feasible to measure in an international context.

    • Why COS outcomes should have a positive focus (such as work-ability) rather than negative (such as sick leave).

    We received fifteen outcomes (online supplementary file 4) which stakeholders found missing from the preliminary set (table 1). We did not include the outcomes which did not match with the COS criteria (table 2), outcomes which were already included but phrased slightly different, and outcomes which were a specification on the level of measurement. Two outcomes were added to the list for the first round of Delphi voting: (1) ‘(self)-perceived stigma’ to the perceptions affecting productivity outcome concept and (2) ‘time to employment’ to the initiating employment stage.

    First round of Delphi voting and second online stakeholder discussion

    The ratings for the first round of Delphi voting are presented in online supplementary file 6. None of the outcomes within the four stages received our cut-point for ‘definite core outcome IN’ (>80% score of 7–9 points).

    Stage 1: initiating employment

    The highest-ranking outcome was ‘time to employment’ with a mean score of 5.6 and 44% of stakeholders giving it a score between 7 and 9. To not entirely exclude this stage from the second round, the steering group included the newly proposed outcome ‘any type of employment including self-employment’ for the second Delphi round.

    Stage 2: having employment

    ‘Any type of employment’ was the highest-ranking outcome with a mean score of 7.4 and 75% of the stakeholders scoring it between 7 and 9. Stakeholders indicated that this outcome was feasible to measure internationally irrespective of social insurance schemes. In addition, this outcome was also seen as an indicator of the degree of work participation when sickness absence is not measured, and people lose work because of a health problem—relevant for countries where sickness benefits are limited. An additional outcome was proposed by a stakeholder, discussed and agreed for inclusion in the second round: ‘sustainable employment’ (online supplementary file 5). The employee representatives advocated to define ‘sustainable’ in terms of quality of the working life, such as having decent work and healthy work.

    Stage 3: increasing or maintaining productivity at work

    ‘Work productivity loss’, ‘Work activity impairment’ and ‘Work ability’ received ‘possible core outcome IN’ rankings, and during the discussion, there were no objections, biddings to downgrade any of them or exclude them from the second round. ‘Perceptions affecting productivity’ was ranked to be left out from the second round. However, we did not exclude this outcome as the outcome concept included measures such as work–life balance and job-satisfaction, which were brought forward as highly important by several stakeholders during the discussion. Therefore, it was upgraded for inclusion during the discussion.

    Stage 4: return to employment

    The steering group discussed the disagreement on ‘return to work—workers’ perceptions, which were voted out by all stakeholder groups combined. The main argument against including it was whether the outcome actually measured work participation and whether it would be relevant to measure at the beginning and end of all types of interventions. However, this outcome was ranked as ‘definite core outcome IN’ by the employee representatives and some stakeholders from other groups were keen on including it in the second round. Therefore, it was not excluded. All versions of sickness absence outcomes received low ratings. Feedback from stakeholders suggested that this may be due to varying insurance schemes, how it is registered, lack of compensation of sick leave in many countries and in some cases punitive measures for being on registered sick leave. However, as there are interventions for which it is relevant to measure accumulated absence over time due to a health problem, and return to work may not be a relevant measure, the steering group decided to keep one absence outcome in the second round. We created a new outcome for the second round, with the aim of replacing sick leave terminology: ‘duration of absence from work due to a health problem over a period of time’. In addition, we added one outcome ‘having lost employment’ to this stage, as an indicator of workers who do not return to work at all after a period of health-related absence.

    Second Delphi round: final vote on core outcomes

    Results of the second round are presented in tables 4 and 5 and contains the COS for Work.

    Table 4

    Results of the second-round ratings presented for all stakeholders combined (overall) and for each stakeholder group separately

    Table 5

    COS for work

    Stage 1: initiating employment

    ‘Having any type of employment including self-employment’ as an outcome for interventions aiming to help unemployed people get work did not meet the consensus criteria to be included in the COS for Work.

    Stage 2: having employment

    ‘Any type of employment including self-employment’ received 87% consensus and was therefore included in the COS for Work

    Stage 3: increasing or maintaining productivity at work

    None of the outcomes in this stage received enough consensus to be included in the COS for Work. However, ‘work productivity loss’, ‘work ability’ and ‘work activity impairment’ received similar ratings to round one indicating they are seen as important but not critical for the COS for Work.

    Stage 4: return to employment

    Two outcomes received overall consensus for inclusion: (1) ‘return to work—proportion’ and (2) ‘time to return to work’. ‘Proportion of workers with sustainable return to work’ was voted as important but not critical by all stakeholders. ‘Workers’ perception on return to work’ received 90% agreement within the employee representatives group for inclusion but very low scores by other stakeholder groups. The ‘duration of absence from work due to health problem over a period of time’ also received low scores for inclusion (38% score between 7 and 9), making it a higher rating than all the sick leave outcomes from the first round—but nonetheless excluding any absenteeism outcome from consideration for the COS for Work.

    Discussion

    Summary of main findings

    Our international Delphi panel achieved consensus on three work participation outcomes for inclusion in COS for Work. As a minimum, all intervention studies addressing work participation should include ‘any type of employment including self-employment’ as an outcome. Intervention studies which include participants who are absent from work due to a health problem should include the outcomes ‘proportion of workers that return to work after being absent because of illness’ and ‘time to return to work’.

    Strengths

    We followed recommendations by the COMET initiative5 to design the Delphi study, as well as the COS-STAD:9 and the COS-STAR.10 For example, based on these standards we described the consensus definition a priori in a publicly available protocol. In addition, we used clearly communicated and specifically designed criteria for deciding which additional outcomes suggested by stakeholders would be suitable for COS for Work. Our SR provided us with a comprehensive overview of how work participation is measured and reported internationally. We were able to cluster all the outcome concepts we found in the literature into 24 outcomes. Working with a relatively low number of outcomes is a facilitating factor for having focused discussions with stakeholders and may have contributed to the low attrition rate.11 12

    The DelphiManager software was designed by the COMET initiative to conduct Delphi studies.

    We had a group of international stakeholders representing a wide range of expertise relevant for research in the field of OH. The online discussions provided valuable exchange of perspectives between the stakeholders. For instance, through sharing the information on the various social security systems it became clear why some outcomes would be less feasible to measure internationally.

    Limitations

    A limitation of the study is that we do not have core outcomes for all four work-participation stages. However, for some outcomes that did not meet the inclusion criterion, the consensus was still high and they are good candidates to be considered for measurement in studies (see table 6).

    Table 6

    Candidate outcomes

    Another limitation of this study is that we had difficultly recruiting employee and employer representatives. The employee group did not join the first online discussion. As our panel consisted mostly of researchers it is likely that views of the other four stakeholder groups were underrepresented. Nonetheless, as COS for Work is developed to be used by researchers we believe that it addresses issues researchers face at an elemental level. As we presented the work participation stages with relevant outcomes in order of most to least commonly measured6 the ‘initiating employment’ stage might have received less, and possibly not enough, attention.

    Most participants were from Europe and North America. This might have influenced which outcomes are seen as most relevant for studies conducted on these continents and the results might be less applicable to low-income and middle-income countries.

    Results compared with previous findings

    In terms of defining core outcomes for work participation most research to date has been done by the Outcome Measures in Rheumatology initiative.13 Their main focus is on worker productivity loss including outcomes such as absenteeism, presenteeism and costs, and focus on people with rheumatic diseases and musculoskeletal diseases.14 15 We consider that COS for Work is a suitable addition to include in studies which measure such productivity outcomes.

    Sustainable employment was proposed as a potential COS outcome by a stakeholder during the first Delphi round. While we did not find this outcome as a commonly measured work outcome in the literature6 it appears to be a highly important outcome from the employee perspective. Sustainable return to work is commonly measured in terms of having the ability to work for an extended period. However, employee representatives and some researchers understand sustainable employment in terms of quality of work, having decent work and a good work-life balance. Defining sustainability in terms of healthy work is in line with results of a study done among employees on how they perceive sustainable work.16

    Implications for practice and research

    In table 6, we present the candidate outcomes that did not meet the cut-off but were also highly ranked by stakeholders. These findings are not reported in the results section as the aim of the study was to determine core outcomes for work participation. However, there were several outcomes for which there was strong disagreement between the stakeholder groups. The reasons are discussed in the results section; the summary of the stakeholder discussions and ratings of the Delphi voting rounds. It is possible that the candidate outcomes did not reach the consensus criterion due to the background of the researchers (most researchers do not investigate outcomes relevant to help people with initiating employment). In addition, group discussion generally supported an outcome such as ‘sustainable employment’, but lack of consensus on the exact definition may have influenced low scores in the second Delphi voting round. Further research is needed to investigate which of these outcomes should be prioritised under which circumstances.

    In our next study we will define and evaluate how to best measure ‘having any employment’ and ‘return to work’ in the COS for Work. Use of COS for Work by researchers will greatly help with evidence synthesis for researchers thereby helping decision making for practice and policy.

    Data availability statement

    All data relevant to the study are included in the article or uploaded as online supplemental information.

    Ethics statements

    Patient consent for publication

    Ethics approval

    The research was conducted in accordance with the Declaration of Helsinki. The research proposal was submitted and approved by the Medical Ethical Committee of the Amsterdam UMC, Academic Medical Centre, Amsterdam, which judged that a comprehensive evaluation was not required since this study was not subject to the Netherlands’ Medical Research Involving Human Subjects Act (reference: W22_129 # 22.171). Participants gave informed consent to participate in the study before taking part.

    Acknowledgments

    We would like to thank all the stakeholders who participated in this study for contributing to the results for COS for Work, the investment of time and the lively discussions.

    References

    Supplementary materials

    Footnotes

    • Collaborators Delphi participants: Aasdahl, Lene (NO); Anema, Johannes (NL); Berkowitz, Debbie (US); Bethge, Matthias (DE); Bühler, Jonas (CTJH); Bültmann, Ute (NL); Böckerman, Petri (FI); Coons, Trevor (US); Curti, Stefania (IT); Crosse, Caroline (AU); de Boer, Angela (NL); Dorstyn, Diana (AU); Foster, Nadine (AU); Friberg, Emilie (SE); Gehanno, Jean Francois (FR); Godderis, Lode (BE); Goodson, Nicola (UK); Graveline, Christine (CA); Gross, Douglas (CA); Hara, Karen Walseth (NO); Hannu, Timo (FI); Hoff, Andreas (DK); Hensing, Gunnel (SE); Hegewald, Janice (DE); Hoorntje, Alexander (NL); Janssen, Svenja (NL); Lacaille, Diane (CA); Lam, Raymond (CA); Landsbergis, Paul (US); Luiza Comper, Maria (BR); Lytsy, Per (SE); Macfarlane, Gary (UK); Mantis, Steve (CA); Merry, Kohle (CA); Meyers, Alysha (US); Neupane, Subas (FI); Nygård, Clas-Hakan (FI); Oyeflaten, Irene Larsen (NO); Pingle, Shyam (IN); Prior, Yeliz (UK); Schaafsma, Frederieke (NL); Samant, Yogindra (NO); Shaw, William (US); Snyder, Alexis (US); Steenstra, Ivan (CA); Sturkenboom, Ingrid (NL); Suijkerbuijk, Yvonne (NL); Tsutsumi, Akizumi (JP); Urquhart, Donna (AU); van Ee, Ilse (NL); van Zaanen, Yvonne (NL); Yamaguchi, Sosei (JP); Zhang, Wei (CA); Walker Bone, Karen (UK).

    • Contributors MR, JH, JHV, RK, CTJH, ML, IM and SV contributed to the conception, the study design and the execution of the Delphi. MR and JH tested the DelphiManager software prior to the first Delphi round. MR and JHV analysed the data. MR and JH assessed data quality. MR drafted the first version of the manuscript and the minor revisions, and JH, JHV, RK, CTJH, ML, IM and SV contributed to the review and the editing process. JH is the guarantor of the publication.

    • Funding This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors. SV is supported by Versus Arthritis (grant numbers 20385, 20380) and the NIHR Manchester Biomedical Research Centre.

    • Competing interests None declared.

    • Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.