Article Text

Download PDFPDF

Participating in core outcome set development via Delphi surveys: qualitative interviews provide pointers to inform guidance
  1. Alice M Biggane1,2,
  2. Paula R Williamson1,
  3. Philippe Ravaud2,3,4,
  4. Bridget Young5
  1. 1 Biostatistics, University of Liverpool, Liverpool, UK
  2. 2 CRESS, INRA, INSERM, Université de Paris, Paris, France
  3. 3 Epidemiology, Columbia University, New York City, New York, USA
  4. 4 Centre d’Épidémiologie Clinique, AP-HP (Assistance Publique des Hôpitaux de Paris) Hôpital Hôtel Dieu, Paris, France
  5. 5 Health Services Research, University of Liverpool, Liverpool, UK
  1. Correspondence to Dr Alice M Biggane; abiggane{at}


Objectives To explore participants’ views of Delphi surveys in core outcome set (COS) development.

Study design and setting Patients and health professionals (n=24) from seven recently concluded COS studies that had involved a Delphi survey took part in semistructured qualitative interviews (telephone and email exchange). Interviews explored participants’ understanding of COS and their experiences of the Delphi survey. Analysis was thematic.

Results Several interviewees had previously participated in two or more COS or Delphi surveys. Those with multiple experiences of participation generally understood the purpose of COS and were satisfied with the Delphi survey. However, some interviewees who were first-time participants struggled to understand the purpose of COS and aspects of the Delphi survey, which limited their contribution and satisfaction with the study. Interviewees also differed in how they interpreted and subsequently used the written documentation provided to COS participants. Some interviewees wanted guidance regarding whose perspective to take into account when scoring outcomes and on how to use the scoring system. Interviewees reported being motivated to take part by the international and expert consensus aspects of the Delphi survey. A few interviewees reported experiencing either positive or negative emotional impacts arising from when they reviewed outcomes and stakeholder feedback.

Conclusion This study identifies important information that should be communicated to COS Delphi study participants. It also indicates the importance of communicating about COS Delphi studies in ways that are accessible and salient to participants, to enhance their experience of participation and make the process more meaningful for all.

  • Delphi survey
  • patient participation
  • core outcome set development

This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See:

View Full Text

Statistics from


  • Twitter @alice_biggane

  • Contributors All authors have read and approved the final version of this manuscript. AMB was the lead researcher on this project and was responsible for the preparation and drafting of the protocol, data collection and analysis and writing of this manuscript. PRW is a coinvestigator and contributed to project conception, design, protocol writing, analysis, writing and proofreading of this manuscript. PR is a coinvestigator and contributed to the methodology used in creating the sampling framework. BY is principal investigator on this project and is responsible for its design, protocol writing, analysis, writing and proofreading of this manuscript.

  • Funding This project is a part of a Methods in Research on Research (MiRoR) funded PhD, which is being undertaken by AB. MiRoR has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 676207.

  • Competing interests PRW chairs the Management Group of the COMET Initiative. BY is a member of the COMET People and Public Participation, Involvement and Engagement Working Group.

  • Patient consent for publication Not required.

  • Ethics approval Ethical approval was granted from Health and Life Sciences Committee on Research Ethics (human participants, tissues and databases) at The University of Liverpool, on 22/06/2017 (reference 1969). All interviewees were provided with full written information prior to interview commencement. All interviewees gave audio recorded or written informed consent prior to commencing the interview and were free to end the interview at any time without providing a reason.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data availability statement No data are available.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.