Article Text

Download PDFPDF

Defining predatory journals and responding to the threat they pose: a modified Delphi consensus process
  1. Samantha Cukier1,
  2. Manoj Lalu1,2,
  3. Gregory L Bryson1,2,
  4. Kelly D Cobey1,3,
  5. Agnes Grudniewicz4,
  6. David Moher1,3
  1. 1 Centre for Journalology, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
  2. 2 Anesthesiology and Pain Medicine, The Ottawa Hospital, Ottawa, Ontario, Canada
  3. 3 School of Epidemiology and Public Health, University of Ottawa Faculty of Medicine, Ottawa, Ontario, Canada
  4. 4 Telfer School of Management, University of Ottawa, Ottawa, Ontario, Canada
  1. Correspondence to Dr David Moher; dmoher{at}


Objective To conduct a Delphi survey informing a consensus definition of predatory journals and publishers.

Design This is a modified three-round Delphi survey delivered online for the first two rounds and in-person for the third round. Questions encompassed three themes: (1) predatory journal definition; (2) educational outreach and policy initiatives on predatory publishing; and (3) developing technological solutions to stop submissions to predatory journals and other low-quality journals.

Participants Through snowball and purposive sampling of targeted experts, we identified 45 noted experts in predatory journals and journalology. The international group included funders, academics and representatives of academic institutions, librarians and information scientists, policy makers, journal editors, publishers, researchers involved in studying predatory journals and legitimate journals, and patient partners. In addition, 198 authors of articles discussing predatory journals were invited to participate in round 1.

Results A total of 115 individuals (107 in round 1 and 45 in rounds 2 and 3) completed the survey on predatory journals and publishers. We reached consensus on 18 items out of a total of 33 to be included in a consensus definition of predatory journals and publishers. We came to consensus on educational outreach and policy initiatives on which to focus, including the development of a single checklist to detect predatory journals and publishers, and public funding to support research in this general area. We identified technological solutions to address the problem: a ‘one-stop-shop’ website to consolidate information on the topic and a ‘predatory journal research observatory’ to identify ongoing research and analysis about predatory journals/publishers.

Conclusions In bringing together an international group of diverse stakeholders, we were able to use a modified Delphi process to inform the development of a definition of predatory journals and publishers. This definition will help institutions, funders and other stakeholders generate practical guidance on avoiding predatory journals and publishers.

  • medical education & training
  • statistics & research methods
  • medical journalism

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.


  • Twitter @manojlalu, @dmoher

  • Contributors CRediT (Contributor Roles Taxonomy): Conceptualisation: KDC, ML, DM. Data curation: SC. Funding acquisition: ML, KDC, DM. Methodology: KDC, ML, AG, DM. Project administration: ML, GLB, DM. Supervision: DM. Validation: ML, GLB, AG, KDC, DM. Visualisation: SC. Investigation (data collection): SC. Writing (original draft): SC. Writing (review and editing): all authors.

  • Funding The Predatory Summit received funding from the President’s Fund, Canadian Institutes of Health Research (CIHR); Institute of Health Services and Policy Research, CIHR; Institute of Musculoskeletal Health and Arthritis, CIHR; Natural Sciences and Engineering Research Council of Canada (NSERC); Swiss National Science Foundation (grant no 174281); Social Sciences and Humanities Research Council of Canada (SSHRC); and Office of the Vice President of Research, University of Ottawa. DM is supported by a University Research Chair (University of Ottawa). ML is supported by The Ottawa Hospital Anesthesia Alternate Funds Association.

  • Competing interests None declared.

  • Patient consent for publication Not required.

  • Ethics approval The Ottawa Hospital Research Ethics Board (Ottawa, Ontario, Canada, 20180927-01H) ( approved the protocol of this study.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data availability statement Data are available in a public, open access repository: