You are here

  • Home
  • Archive
  • Volume 7, Issue 10
  • A scoping review protocol on the roles and tasks of peer reviewers in the manuscript review process in biomedical journals

Article Text

Article menu

Download PDFPDF

XML
Medical publishing and peer review
Protocol
A scoping review protocol on the roles and tasks of peer reviewers in the manuscript review process in biomedical journals
  1. Ketevan Glonti1,2,3,
  2. Daniel Cauchi4,
  3. Erik Cobo5,
  4. Isabelle Boutron2,3,
  5. David Moher6,
  6. Darko Hren1
  1. 1 Faculty of Humanities and Social Sciences, University of Split, Split, Croatia
  2. 2 METHODS Team, Epidemiology and Biostatistics Sorbonne Paris Cité Research Centre, UMR 1153, INSERM, Paris, France
  3. 3 Paris Descartes University, Paris, France
  4. 4 Ministry of Health, Valletta, Malta
  5. 5 Statistics and Operations Research Department, Barcelona-Tech,Universitat Politecnica Catalunya, Barcelona, Spain
  6. 6 Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
  1. Correspondence to Ketevan Glonti; kglonti{at}unist.hr

Abstract

Introduction The primary functions of peer reviewers are poorly defined. Thus far no body of literature has systematically identified the roles and tasks of peer reviewers of biomedical journals. A clear establishment of these can lead to improvements in the peer review process. The purpose of this scoping review is to determine what is known on the roles and tasks of peer reviewers.

Methods We will use the methodological framework first proposed by Arksey and O’Malley and subsequently adapted by Levac et al and the Joanna Briggs Institute. The scoping review will include all study designs, as well as editorials, commentaries and grey literature. The following eight electronic databases will be searched (from inception to May 2017): Cochrane Library, Cumulative Index to Nursing and Allied Health Literature, Educational Resources Information Center, EMBASE, MEDLINE, PsycINFO, Scopus and Web of Science. Two reviewers will use inclusion and exclusion criteria based on the ‘Population–Concept–Context’ framework to independently screen titles and abstracts of articles considered for inclusion. Full-text screening of relevant eligible articles will also be carried out by two reviewers. The search strategy for grey literature will include searching in websites of existing networks, biomedical journal publishers and organisations that offer resources for peer reviewers. In addition we will review journal guidelines to peer reviewers on how to perform the manuscript review. Journals will be selected using the 2016 journal impact factor. We will identify and assess the top five, middle five and lowest-ranking five journals across all medical specialties.

Ethics and dissemination This scoping review will undertake a secondary analysis of data already collected and does not require ethical approval. The results will be disseminated through journals and conferences targeting stakeholders involved in peer review in biomedical research.

  • Peer review
  • Competencies
  • Scoping review
  • Roles
  • Tasks
  • Biomedical publishing

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

https://doi.org/10.1136/bmjopen-2017-017468

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • The large volume of literature informing this scoping review ensures that a broad overview of the roles and tasks of peer reviewers in the manuscript peer review process within biomedical journals will be obtained

    • Another strength of this study is the inclusion of grey literature, including the review of journal guidelines.

    • As this is a scoping review, the quality of the evidence and risk of bias will not be evaluated.

    Background

    The publication of peer-reviewed articles in scientific journals has long been the cornerstone of science,1 and the primary means by which new research is documented and the outcomes disseminated.2 Manuscripts that are submitted for publication in scientific journals typically undergo a critical appraisal process by researchers from a similar field who are in the wider sense peers and colleagues — known as peer review — as part of a broader editorial process led by journal editors.3 However, the importance of peer reviewing within this process extends beyond purely academic concerns. Academic publishing lies at the interface between biomedical research and practice, having the potential to influence clinical decisions.4 5 Clinical decisions should be guided by the best evidence available, yet these can be misleading if they are based on incomplete or inaccurate information. Any process that influences the accuracy, quality, assessment and dissemination of clinical evidence may therefore have a direct impact on patient care.3 The editorial process within biomedical journals can thus be considered to be a ‘gatekeeper’ for scientific publications, consisting of the following steps:

    1. Editors consider the overall ‘fit’ of the research article to the journal, as well as suitability and relevance for the journal and its readership.6

    2. Selection of reviewers by the editors: within the traditional biomedical sphere, peer reviewers are typically invited by journal editors to review manuscripts on the basis of their apparent expertise, which is often gauged in terms of their article output in their respective research area.

    3. Editors communicate with both reviewers and authors and coordinate their interaction during peer review.

    Editors are responsible for taking an independent decision regarding the fate of the manuscript (ie, whether it is accepted for publication or not).7 However, it has been suggested that journal editors are not entirely independent in their assessment of an article’s suitability for publication once it has undergone peer review. Research indicates that editors give considerable weight to reviewers’ recommendations on whether to reject or accept a manuscript.8 This may in part be due to the fact that core competencies for scientific editors in biomedical research have not yet been formally established,9 and most scientific editors of biomedical journals do not receive formal training.10 This is also the case for the majority of reviewers. Despite a significant proportion of reviewers perceiving that they need guidance and formal training on how to conduct a peer review,11 most are not trained in how to write a reviewers’ report. Instead, reviewing is often a skill learnt through feedback received on their own submitted manuscripts.12 Furthermore, since it is rare for reviewers to receive feedback on their own reviewer reports, it is difficult for them to know whether their reviews are of good quality.13

    Although journals, authors and reviewers widely support peer review as the primary tool for evaluating research outputs in biomedical research,11 14 15 there is concurrently a broad consensus across scientific disciplines that the peer review process may be flawed.12 16 17 A growing body of literature has identified several potential problems, including misjudgement by editors, and biased, inconsistent or inadequate reviewing by reviewers.17

    Over the years, there have been various attempts to improve the quality of peer reviewer reports in biomedical science. A recent systematic review evaluating the impact of interventions aimed at improving the quality of peer review of randomised controlled trials (RCTs) for biomedical publications concluded that there is a need to clarify the roles and tasks of peer reviewers as a step forward in quality improvement of peer reviewing.18 Within the biomedical field, the apparent roles and tasks of peer reviewers are closely related to the structural properties of the editorial process itself. For example, some — but not all — journals require peer reviewers to assess novelty and/or clinical relevance of articles in addition to assessing scientific rigour. Journals also differ with regard to their expectations of how a reviewer report should be written. Some journals encourage reviewers to follow a specific structure in their reporting, whereas other journals prefer free text. Journals also differ in their request for peer reviewer recommendations regarding whether an article should be accepted for publication in the journal or not.

    These differences may influence quality of peer review reporting, and thus quality of the peer review process across journals. An RCT aimed at determining the effects of training peer reviewers found only a slight positive impact on the quality of peer review. After receiving training, the quality of the peer reviewers’ reports as measured by the ‘Review Quality Instrument’ — which assesses the extent to which a reviewer has commented on five aspects of a manuscript (importance of the research question, originality of the paper, strengths and weaknesses of the method, presentation, interpretation of results) and on two aspects of the review (constructiveness and substantiation of comments) — was deemed to have improved overall. However, peer reviewers in the study failed to detect all major errors that were introduced to the articles under review.19 At the same time, a major criticism of this study was that reviewers do not necessarily think that their task is to find all major errors in an article.20 This dissonance was also reflected in a recent study that showed that the most important tasks in peer review, as perceived by peer reviewers evaluating RCTs, were not congruent with the tasks most often requested by journal editors in their guidelines to reviewers.21

    These differences clearly illustrate the need to clarify the roles and tasks of peer reviewers. Thus far, this has only been somewhat explored, to a limited extent, for RCTs21 but not for other study designs.

    The primary objective of this research is to determine the specific roles and tasks of peer reviewers as depicted in biomedical research. The wider purpose of this research is to inform and facilitate the future development of a set of core tasks that should be carried out by peer reviewers. This will contribute to improvements in the quality of peer reviewer reports, and ultimately of the biomedical scientific literature in general.

    Methods

    A scoping review was considered to be the most suitable approach to responding to the broad aim of this study. In contrast to systematic literature reviews that aim to answer specific questions, scoping reviews have been described as a process of producing a broad overview of the field.22–24 In addition to published biomedical journal articles, grey literature will also be searched because it is likely that most of the information being sought (ie, descriptions of the roles of peer reviewers) would be found in calls for reviewers on journal websites, and guidance documents which would not generally be captured in a traditional review of published research. This approach has been previously adopted by authors of a study that aimed to identify competencies of scientific editors of biomedical journals.9 We used the Preferred Reporting Items for Systematic Reviews and Meta-analysis for Protocols (PRISMA-P) to draft this protocol.25

    This scoping review will use the methodological framework proposed by Arksey and O’Malley,22 as well as the amendments made to this framework by Levac et al 26 and by the Joanna Briggs Institute.27 The framework consists of six consecutive stages: (1) identifying the research question, (2) identifying relevant studies, (3) study selection, (4) charting the data, (5) collating, summarising and reporting results, and (6) consultation. Each stage is discussed in further detail below.

    Stage 1: identifying the research question

    Arksey and O’Malley suggest an iterative process for developing one or more research questions. In the first stage two research questions have been identified based on gaps in the literature:

    1. What are the expected roles of peer reviewers in the editorial peer review process in biomedical journals?

    2. What tasks are peer reviewers are expected to perform for biomedical journals?

    Given that some overlap between the terms ‘roles’ and ‘tasks’ is expected, we defined ‘roles’ as referring to the overarching nature of peer reviewers’ function, whereas ‘tasks’ refer more specifically to actions that fulfil these roles.

    These questions might be refined, or new ones added, as the authors gain increasing familiarity with the literature.

    Stage 2: identifying relevant studies

    A comprehensive search strategy will be developed in order to identify relevant literature, underpinned by key inclusion criteria (see box). These are based on ‘Population–Concept–Context (PCC)’ framework recommended by the Joanna Briggs Institute for scoping reviews,27 which has roots in the PICO (population, intervention, comparator and outcome) framework commonly used to focus clinical questions and develop systematic literature search strategies.28

    Box

    Inclusion criteria

    P—Population = journal editors, publishers, peer reviewers, (corresponding) authors in biomedical journals and organisations that offer (educational) resources and training to peer reviewers in the biomedical field

    C—Concept = articles with specific focus and/or statements mentioning roles, tasks and competencies pertaining to the role of peer reviewers in the journal editorial process

    C—Context = the review will include all study designs, as well as book chapters, editorials and commentaries from the biomedical field; there will be no date and language restrictions

    Exclusion criteria

    Studies referring to peer review that is not related to manuscript peer reviewing in biomedical journals (eg, grant peer review, professional performance review, peer review of teaching and so on) will be excluded.

    Search strategy

    The electronic literature search strategy will follow the three-step process recommended by the Joanna Briggs Institute.27 The first step consisted of an initial preliminary search of at least two online databases relevant to the topic. This was undertaken for MEDLINE (via Ovid) using the ‘peer review, research’ medical subject headings and ‘peer review’ keyword in the Cochrane Library, resulting in 2,017 studies in the Cochrane Library and 13,717 in MEDLINE. In the second step, we will closely review potentially relevant text words in the titles and abstracts of the most pertinent papers in order to compile a list of terms that can be used to inform our search strategy. Index terms used to describe the articles will also be included. This list will be combined with search strategies from existing scoping and systematic reviews on peer review9 21 29 to develop database-specific search strategies.

    The Peer Review of Electronic Search Strategies 2015 Guideline statement will be used to guide the electronic literature search strategies.30 These will be further refined in collaboration with a health sciences librarian. Subsequently, the following databases will be searched: Cochrane Library, Cumulative Index to Nursing and Allied Health Literature, Educational Resources Information Center, EMBASE (via Ovid), PsycINFO (via Ovid), MEDLINE (via Ovid), Scopus and Web of Science. The search strategy for MEDLINE can be found in online supplementary appendix 1.

    Supplemental material

    There will be no time or language restrictions. The authors involved in this protocol are in command of the following languages: Catalan, Croatian, English, French, German, Italian, Russian and Spanish. Relevant articles identified in any other language will be translated.

    In the third and last step, reference lists of included studies, as well as websites of journals which display a strong interest in peer review, as evidenced by numerous publications on the topic (such as the Journal of the American Medical Association (JAMA), Nature and Science), will be hand-searched using keywords related to peer review, as outlined in the MEDLINE strategy to identify any additional literature.

    The search strategy for grey literature will include searching in websites of existing networks (ie, EQUATOR Network, New Frontiers of Peer Review), biomedical journal publishers (ie, BMJ Publishing Group, Elsevier, Springer Nature, Taylor & Frances, Wiley) and organisations that offer resources for reviewers (including educational courses, eg, those provided by Cochrane31 and Publons32). Relevant blogs, newsletters (ie, The METRICS Research Digest33), surveys and reports of authors/reviewer workshops will also be considered.

    In addition we will review journal guidelines to peer reviewers on how to perform the manuscript review.

    The guidelines will be searched for statements around the roles and tasks of peer reviewers.

    Journals will be selected using 2016 journal impact factor (Thomson Reuters Journal Citation Reports-Science Citation Index Expanded). We will identify and assess the top five, middle five and lowest-ranking five journals across the medical specialties recognised in the Directive 2005/36/EC of the European Parliament and of the Council of 7 September 2005 (on the recognition of professional qualifications).

    It is expected that some journals may directly communicate their instructions to peer reviewers via email or through their submission systems, rather than through publicly available instructions. In order to obtain the content of such instructions for examination, we will contact the editor-in-chief and/or managing editor of the identified journals to request details of any ‘direct to reviewer’ guidance.

    Stage 3: study selection

    Following the execution of the search strategy, the identified records (titles and abstracts) will be collated in a reference manager for de-duplication. The final unique set of records will be imported into a systematic review paper manager that facilitates independent screening and logs disagreements between reviewers.

    The study selection process will be implemented over two stages. The first stage will involve the screening of titles and abstracts by two reviewers (KG and DC) to determine each article’s eligibility for full-text screening based on a priori inclusion criteria. The second stage of the selection process will consist of retrieving the full text of all potentially eligible articles, which will also be independently screened. Disagreements between reviewers regarding eligibility will be resolved by a third member of the research team (DH). Data will also be extracted independently by KG and DC.

    We expect that some of the grey literature might subsequently be published elsewhere in the indexed literature. This will be accounted for by cross-checking authors’ names across grey literature and index literature results in order to identify potential duplicates.

    An adapted version of the PRISMA flow diagram will be used to report final numbers in the resulting study publication once the review is completed. Reasons for exclusion will be recorded at the full-text review stage.

    Stage 4: charting the data

    A draft charting form (see table 1) has been developed at the protocol stage to aid the collection and sorting of key pieces of information from the selected articles. It will be pilot-tested and refined during the full-text screening to capture detailed information on each study. The information from research-based and non-research-based publications will be collected in separate extraction forms. Additional categories that may emerge during data extraction will be added accordingly.

    View this table:
    Table 1

    Draft data charting form

    Another form will be developed for the extraction of information from the journal guidelines to peer reviewers. In addition to the general and specific descriptions of expectations and competencies of peer reviewers, variations according to journals and their peer review models (such as single-blind peer review, double-blind peer review, open peer review, postpublication peer review) and whether peer reviewers have to provide specific recommendations (ie, no revision, minor revision, major revision, reject) will be noted.

    Stage 5: collating, summarising and reporting the results

    In order to create a useful summary of the data, we will combine all expectations and competency-related statements retrieved from all sources.

    The general and specific descriptions of expectations and competencies of peer reviewers extracted from the different sources will be combined and de-duplicated, producing a list of unique statements. These will subsequently be organised into emerging categories. While the primary goal is to extract roles and tasks of peer reviewers, additional items related to particular abilities, knowledge, training and skills will also be extracted.

    A checklist for reporting scoping reviews—the ‘Preferred Reporting Items for Systematic Reviews and Meta-Analysis: extension for Scoping Reviews (PRISMA-ScR)’—is currently under development.34 If published by the time the scoping review is complete, the PRISMA-ScR will be used.

    Stage 6: consultation

    This final stage refers to consultation with stakeholders in the field of peer review to inform and validate findings from the scoping review. This has also been shown to be a knowledge translation activity and an important step in scoping reviews.35

    Journal editors will be consulted to explore their views and perspectives on the roles and tasks of peer reviewers. Results will be presented in detail in separate research papers.

    Dissemination

    To the best of our knowledge this scoping review is the first attempt to systematically identify the roles (overarching nature of the work) and tasks (specific actions carried out to fulfil these roles) of peer reviewers involved in the manuscript review process in biomedical journals.

    As a standalone research piece, it will primarily be helpful to determine and highlight the different perspectives around the roles and tasks of peer reviewers, and will be relevant to a variety of audiences including editors, peer reviewers and authors. It will also inform the consequent consultation with stakeholders, with the aim of developing a taxonomy of peer reviewers’ roles and tasks leading to the development of a set of core competencies for peer reviewers of biomedical journals. The study findings could further be used by journal editors to review their instructions to peer reviewers and develop/update training courses for peer reviewers.

    Acknowledgments

    The authors would like to thank Dr Dario Sambunjak and David Blanco de Tena-Dávila for providing advice during the writing of this protocol.

    References

    1. 1.↵
      2. Kronick DA
      . Peer review in 18th-century scientific journalism. JAMA 1990;263:1321.doi:10.1001/jama.1990.03440100021002
      OpenUrlCrossRefPubMedWeb of Science
    2. 2.↵
      2. Harley D
      . Scholarly communication: cultural contexts, evolving models. Science 2013;342:80–2.doi:10.1126/science.1243622
      OpenUrlAbstract/FREE Full Text
    3. 3.↵
      2. Jefferson T ,
      3. Wager E ,
      4. Davidoff F
      . Measuring the quality of editorial peer review. JAMA 2002;287:2786.doi:10.1001/jama.287.21.2786
      OpenUrlCrossRefPubMedWeb of Science
    4. 4.↵
      2. Moher D ,
      3. Glasziou P ,
      4. Chalmers I , et al
      . Increasing value and reducing waste in biomedical research: who’s listening? Lancet 2016;387:1573–86.doi:10.1016/S0140-6736(15)00307-4
      OpenUrlCrossRefPubMed
    5. 5.↵
      2. Kassirer JP ,
      3. Campion EW
      . Peer review. Crude and understudied, but indispensable. JAMA 1994;272:96–7.
      OpenUrlCrossRefPubMedWeb of Science
    6. 6.↵
      2. Scarfe WC
      . Translation of the scientific method … peer review. Oral Surg Oral Med Oral Pathol Oral Radiol Endodontology 2010;109:485–7.doi:10.1016/j.tripleo.2010.02.001
      OpenUrl
    7. 7.↵
      2. Manchikanti L ,
      3. Kaye AD ,
      4. Boswell MV , et al
      . Medical journal peer review: process and bias. Pain Physician 2015;18:E1–14.
      OpenUrlPubMed
    8. 8.↵
      2. Kravitz RL ,
      3. Franks P ,
      4. Feldman MD , et al
      . Editorial peer reviewers' recommendations at a general medical journal: are they reliable and do editors care? PLoS One 2010;5:e10072.doi:10.1371/journal.pone.0010072
      OpenUrlCrossRefPubMed
    9. 9.↵
      2. Galipeau J ,
      3. Barbour V ,
      4. Baskin P , et al
      . A scoping review of competencies for scientific editors of biomedical journals. BMC Med 2016;14:16.doi:10.1186/s12916-016-0561-2
      OpenUrlCrossRefPubMed
    10. 10.↵
      2. Moher D ,
      3. Altman DG
      . Four proposals to help improve the medical research literature. PLoS Med 2015;12:e1001864.doi:10.1371/journal.pmed.1001864
      OpenUrlCrossRefPubMed
    11. 11.↵
      2. Mulligan A ,
      3. Hall L ,
      4. Raphael E
      . Peer review in a changing world: An international study measuring the attitudes of researchers. J Am Soc Inf Sci Technol 2013;64:132–61.doi:10.1002/asi.22798
      OpenUrlCrossRef
    12. 12.↵
      2. Smith R
      . Peer review: a flawed process at the heart of science and journals. J R Soc Med 2006;99:178–82.doi:10.1258/jrsm.99.4.178
      OpenUrlCrossRefPubMedWeb of Science
    13. 13.↵
      2. Tite L ,
      3. Schroter S
      . Why do peer reviewers decline to review? a survey. J Epidemiol Community Health 2007;61:9–12.doi:10.1136/jech.2006.049817
      OpenUrlAbstract/FREE Full Text
    14. 14.↵
      2. Shattell MM ,
      3. Chinn P ,
      4. Thomas SP , et al
      . Authors' and editors' perspectives on peer review quality in three scholarly nursing journals. J Nurs Scholarsh 2010;42:58–65.doi:10.1111/j.1547-5069.2009.01331.x
      OpenUrlCrossRefPubMed
    15. 15.↵
      2. Godlee F ,
      3. Jefferson T
      . Peer review in health sciences. BMJ Books London 2003.
    16. 16.↵
      2. Wager E ,
      3. Jefferson T
      . Shortcomings of peer review in biomedical journals. Learn Publ 2001;14:257–63.doi:10.1087/095315101753141356
      OpenUrl
    17. 17.↵
      2. Lee CJ ,
      3. Sugimoto CR ,
      4. Zhang G , et al
      . Bias in peer review. J Am Soc Inf Sci Technol 2013;64:2–17.doi:10.1002/asi.22784
      OpenUrl
    18. 18.↵
      2. Bruce R ,
      3. Chauvin A ,
      4. Trinquart L , et al
      . Impact of interventions to improve the quality of peer review of biomedical journals: a systematic review and meta-analysis. BMC Med 2016;14:85.doi:10.1186/s12916-016-0631-5
      OpenUrl
    19. 19.↵
      2. Schroter S ,
      3. Black N ,
      4. Evans S , et al
      . Effects of training on quality of peer review: randomised controlled trial. BMJ 2004;328:673.doi:10.1136/bmj.38023.700775.AE
      OpenUrlAbstract/FREE Full Text
    20. 20.↵
      PubPeer - What errors do peer reviewers detect, and does training improve their ability to detect them? [Internet]. https://pubpeer.com/publications/54CDC9161958EBE471CC7A633C236B (accessed 20 Nov 2016).
    21. 21.↵
      2. Chauvin A ,
      3. Ravaud P ,
      4. Baron G , et al
      . The most important tasks for peer reviewers evaluating a randomized controlled trial are not congruent with the tasks most often requested by journal editors. BMC Med 2015;13:158.doi:10.1186/s12916-015-0395-3
      OpenUrlPubMed
    22. 22.↵
      2. Arksey H ,
      3. O’Malley L
      . Scoping studies: towards a methodological framework. Int J Soc Res Methodol 2005;8:19–32.doi:10.1080/1364557032000119616
      OpenUrlCrossRef
    23. 23.↵
      2. Armstrong R ,
      3. Hall BJ ,
      4. Doyle J , et al
      . “Scoping the scope” of a cochrane review [Internet]. http://jpubhealth.oxfordjournals.org (accessed 7 Dec 2016).
    24. 24.↵
      2. Moher D ,
      3. Stewart L ,
      4. Shekelle P
      . All in the family: systematic reviews, rapid reviews, scoping reviews, realist reviews, and more. Syst Rev 2015;4:183.doi:10.1186/s13643-015-0163-7
      OpenUrl
    25. 25.↵
      2. Shamseer L ,
      3. Moher D ,
      4. Clarke M , et al
      . Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ 2015;349:g7647.doi:10.1136/bmj.g7647
      OpenUrlAbstract/FREE Full Text
    26. 26.↵
      2. Levac D ,
      3. Colquhoun H ,
      4. O’Brien KK
      . Scoping studies: advancing the methodology. Implement Sci 2010;5:69.doi:10.1186/1748-5908-5-69
      OpenUrlCrossRefPubMed
    27. 27.↵
      Joanna Briggs Institute. The joanna briggs institute reviewers’ manual 2015. Methodol JBI Scoping Rev Joanna Briggs Inst JBI 2015.
    28. 28.↵
      2. Richardson WS ,
      3. Wilson MC ,
      4. Nishikawa J , et al
      . The well-built clinical question: a key to evidence-based decisions. ACP J Club 1995;123:A12–13.
      OpenUrlPubMed
    29. 29.↵
      2. Larson BP ,
      3. Chung KC
      . A systematic review of peer review for scientific manuscripts. Hand 2012;7:37–44.doi:10.1007/s11552-012-9392-6
      OpenUrlCrossRefPubMed
    30. 30.↵
      2. McGowan J ,
      3. Sampson M ,
      4. Salzwedel DM , et al
      . PRESS Peer Review of Electronic Search Strategies: 2015 Guideline statement. J Clin Epidemiol 2016;75:40–6.doi:10.1016/j.jclinepi.2016.01.021
      OpenUrl
    31. 31.↵
      Free Online Course on Journal Peer Review. Cochrane eyes and vision [Internet]. eyes.cochrane.org/free-online-course-journal-peer-review (accessed 7 Feb 2017).
    32. 32.↵
      Home [Internet]. Publons. http://home.publons.com/ (accessed 3 Apr 2017).
    33. 33.↵
      Research Digest | METRICS [Internet]. http://metrics.stanford.edu/resources/research-digest (accessed 21 Feb 2017).
    34. 34.↵
      Reporting guidelines under development. The EQUATOR network [Internet]. http://www.equator-network.org/library/reporting-guidelines-under-development/ (accessed 21 Dec 2016).
    35. 35.↵
      2. Tricco AC ,
      3. Lillie E ,
      4. Zarin W , et al
      . A scoping review on the conduct and reporting of scoping reviews. BMC Med Res Methodol 2016;16:15.doi:10.1186/s12874-016-0116-4
      OpenUrl

    Footnotes

    • Contributors All authors have made substantive intellectual contributions to the development of this protocol. KG, DH, EC, IB and DM jointly conceived the idea for the project. DC, EC, IB, DM and DH contributed to the study design and development of research questions. KG conceptualised the review approach and led the writing of the manuscript. DH led the supervision of the manuscript preparation. All authors provided detailed comments on earlier drafts and approved this manuscript.

    • Funding This project was supported by the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement no 676207. The funders had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.

    • Competing interests None declared.

    • Provenance and peer review Not commissioned; externally peer reviewed.