Introduction Paediatric systematic reviews differ from adult systematic reviews in several key aspects such as considerations of child tailored interventions, justifiable comparators, valid outcomes and child sensitive search strategies. Available guidelines, including PRISMA-P (2015) and PRISMA (2009), do not cover all the complexities associated with reporting systematic reviews in the paediatric population. Using a collaborative, multidisciplinary structure, we aim to develop evidence-based and consensus-based PRISMA-P-C (Protocol for Children) and PRISMA-C (Children) Extensions to guide paediatric systematic review protocol and completed review reporting.
Methods and analysis This project's methodology follows published recommendations for developing reporting guidelines and involves the following six phases; (1) establishment of a steering committee representing key stakeholder groups; (2) a scoping review to identify potential Extension items; (3) three types of consensus activities including meetings of the steering committee to achieve high-level decisions on the content and methodology of the Extensions, a survey of key stakeholders to generate a list of possible items to include in the Extensions and a formal consensus meeting to select the reporting items to add to, or modify for, the Extension; (4) the preliminary checklist items generated in phase III will be evaluated against the existing evidence and reporting practices in paediatric systematic reviews; (5) extension statements and explanation and elaboration documents will provide detailed advice for each item and examples of good reporting; (6) development and implementation of effective knowledge translation of the extension checklist, and an evaluation of the Extensions by key stakeholders.
Ethics and Dissemination This protocol was considered a quality improvement project by the Hospital for Sick Children's Ethics Committee and did not require ethical review. The resultant checklists, jointly developed with all relevant stakeholders, will be disseminated through peer-reviewed journals as well as national and international conference presentations. Endorsement of the checklist will be sought simultaneously in multiple journals.
- systematic review
- reporting guideline
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
The methods chosen for the development of PRISMA-P-C (Protocol for Children) and PRISMA-C (Children) extensions are based on evidence-based principles of reporting guideline development.
The simultaneous development of a reporting guideline for both protocol and reports of paediatric systematic reviews will ensure that relevant items in the protocol (PRISMA-P-C) are reflected in the report (PRISMA-C).
Identification of paediatric systematic reviewers from published reports for the Delphi survey will help in identifying an unbiased selection of participants than the project steering committee could provide alone.
The involvement of various stakeholders in guideline development will ensure that a wide range of perspectives are captured and will help maximise the impact and implementation of the guideline by relevant stakeholders.
Systematic reviews and meta-analyses are considered the highest level in the hierarchy of scientific evidence and are of fundamental importance in decision-making by healthcare providers and policymakers. Systematic reviews may also identify the need for further research to establish evidence in a particular population or a subset of the population. In order to maximise the potential use of synthesised evidence, there had been repeated calls for transparent and consistent reporting of the systematic review.1–3 The Preferred Reporting Items in Systematic Review and Meta-Analysis (PRISMA 2009)4 and PRISMA-P(Protocol-2015)5 statements were developed to provide guidance on key elements needed for optimal reporting of systematic reviews and meta-analyses and their protocols, respectively, in order to maximise the completeness of reporting, transparency and replicability of such studies. An evaluation of the impact of endorsement of the PRISMA statement by specialty journals showed a significant increase in completeness of reporting and methodological quality of systematic reviews in those journals.6 Although the PRISMA statement was designed to improve the completeness of reporting of systematic reviews and meta-analyses, there are still other areas, for example, network,7 equity8 and individual patient data9 studies, that were not fully addressed by the original statement, resulting in PRISMA extensions in these areas.
Rationale for ‘newborn and child specific’ extension of PRISMA
Paediatric systematic reviews differ from adult systematic reviews in several key aspects. Some key issues identified relate to age-specific growth and developmental stages of the patients, newborn and child-tailored interventions. Since placebo response rates in drug trials appear to be higher in children compared to adults,10 ,11 consequently pooled response rates are higher in children than in adults with similar conditions.12 The synthesis of evidence from trials into paediatric systematic reviews is impaired by the use of outcome measurement instruments that are neither qualified nor validated in paediatric subpopulations.13 Paediatric systematic reviews have also been reported to be weak in terms of the comprehensiveness in their search to identify primary studies.2 Consequently, search filters have been developed to ensure comprehensiveness of paediatric search terms.14–16 Other studies have used search hedges that cover concepts using terms, that is, neonates, infants, adolescents, harvested from standard term indices to identify more potential relevant articles.17 Furthermore, for systematic reviews with a mixed adult and paediatric population, statistical analyses need to consider subgroup analyses according to targeted paediatric age groups to examine differences in intervention effects.18 These paediatric-specific methodological considerations play a role throughout the design, conduct and reporting of paediatric systematic reviews to permit adequate interpretation. The currently available guidelines, including PRISMA-P (2015) and PRISMA (2009), do not cover the complexities associated with reporting (protocols for) systematic reviews in the paediatric population. Hence, systematic reviews relating to newborns and/or children, including those with a mixed adult and paediatric population, require modified and additional standards for reporting items.
The need for paediatric-specific items in reporting guidelines is also evident from a recent international Consensus meeting on Standard Protocol Items for Randomized Trials in Children (SPIRIT-C) and Consolidated Standards of Reporting Trials in Children (CONSORT-C) held in Toronto in 2014, which agreed on 8 and 14 ‘pediatric-specific’ extension items, respectively, for the design and conduct (SPIRIT-C) and reporting (CONSORT-C) of paediatric clinical trials.19 At the same meeting, a call was made for guidance to enable scientists to improve the conduct and reporting of systematic reviews in newborn and child health. Our goal is therefore to develop evidence-based reporting guidelines for child relevant systematic review protocols and reports in order to improve the transparency, quality and quantity of child relevant systematic reviews.
Our primary objectives are: (1) to develop evidence-based and consensus-based PRISMA-P-C (Protocol for Children) and PRISMA-C (Children) checklist items to guide paediatric systematic review protocol development and completed review reporting and (2) to develop and launch a knowledge translation and implementation strategy that encompasses education, dissemination, endorsement and implementation of the final PRISMA-P-C and PRISMA-C checklists and accompanying guidance documents by key stakeholders.
Definition and scope of newborn and child relevant systematic reviews
PRISMA-P-C and PRISMA-C have adopted the same definition of a ‘systematic review’ and ‘protocol’ as PRISMA-P5 and PRISMA.4 A systematic review collates all relevant evidence that fits prespecified eligibility criteria to answer a specific research question. It uses explicit, systematic methods to minimise bias in the identification, selection, synthesis and summary of relevant studies. A protocol is a document that presents an explicit plan for a systematic review and details the rationale and a priori methodological and analytical approaches for the review. The PRISMA-P-C and PRISMA-C checklists will be applicable to paediatric systematic reviews with or without a meta-analysis; and for systematic reviews of randomised controlled trials and/or observational studies.
Phase I—project launch
A steering committee, who are also the authors of the current article, is comprised of paediatric systematic review authors, methodologists and guidelines developers from leading research institutions (Child Health Evaluation Sciences, and Centre for Global Child Health, The Hospital for Sick Children; Ottawa Hospital Research Institute (OHRI), Alberta Research Centre for Health Evidence (ARCHE), Canada; Stanford University, USA; NHMRC Clinical Trials Centre, University of Sydney, Australia; Cochrane Child Health Field; Cochrane Childhood Cancer Group; Cochrane Neonatology Group). An experienced librarian from the Hospital for Sick Children, Toronto with expertise in developing search strategies for such methodological systematic reviews will be added to the steering committee. The selection of the steering committee members was based on their extensive publication of paediatric systematic reviews and leadership role in systematic review methodology. The steering committee will manage the project via face-to-face (video conferencing) online meetings to discuss and finalise key steps of the guideline development process. They will also help recruit participants for the Delphi survey and Consensus meeting.
Phase II—review of evidence and compilation of paediatric-specific topics
On the basis of the results of the scoping review that identified a need for paediatric extensions of PRISMA and PRISMA-P, a preliminary list of paediatric-specific methodological issues will be compiled which may require detailed guidance to enhance the quality and consistency of reporting of paediatric systematic review protocols and reports. Furthermore, items that are relevant to paediatric systematic reviews will also be identified from the SPIRIT-C and CONSORT-C checklists. The two preliminary checklists for PRISMA-P-C and PRISMA-C will then be evaluated against the existing evidence and reporting practices in paediatric systematic reviews. The proposed knowledge synthesis will be completed using a recommended methodology for systematic review. The search strategy will be adopted from tested search filters developed for ‘systematic review’, ‘pediatric’ and ‘protocol’.14 The Cochrane Database of Systematic Review and Database of Abstracts of Reviews of Effects (DARE) databases will be searched from January 2010 to December 2014. The reason for limiting the search from 2010 and beyond is because the steering committee decided to review the quality of evidence following the publication of the PRISMA statement in 2009.4 The titles and abstracts will be screened for the following eligibility criteria: (1) a child-relevant systematic review (as per the definitions provided in box 1); (2) published in the English language; (3) not a commentary or editorial. A random sample of 300 paediatric systematic reviews will be included for this evidence synthesis. The screening of full text will continue until the desired sample size is achieved. We anticipate a limited number of published paediatric systematic review protocols; therefore, all the identified protocols that meet the inclusion criteria will be included. Data will be extracted on: (1) the characteristics of the review; (2) whether the review fulfilled the reporting criteria identified in the proposed items; (3) examples of good reporting.
Scope of newborn and child relevant systematic reviews with examples
A newborn and/or child relevant systematic review meets one or more of the following criteria:
A systematic review with an intended population of children only (0–18 years of age). Examples: “Late (>7 days) inhalation corticosteroids to reduce bronchopulmonary dysplasia in preterm infants”20 and “The effect of β-blocker therapy on progressive aortic dilation in children and adolescents with Marfan's syndrome: a meta-analysis”.21
A systematic review with an intended population including both children and adults. Examples: “Addition of long-acting β2-agonists to inhaled steroids vs higher dose inhaled steroids in adults and children with persistent asthma”22 and “Micronutrient supplementation in children and adults with HIV infection”23
A systematic review of family-based interventions intended to improve the health and well-being of children. Examples: “Group-based parent-training programmes for improving emotional and behavioural adjustment in children from birth to 3 years old”24 and “Parent-only vs parent-child (family-focused) approaches for weight loss in obese and overweight children: a systematic review and meta-analysis”.25
A systematic review of interventions in pregnancy with objectives to measure outcomes in the neonate. Examples: “Hepatitis B vaccination during pregnancy for preventing infant infection”26 and “Routine iron/folate supplementation during pregnancy: effect on maternal anaemia and birth outcomes”.27
Phase III—consensus process
The PRISMA-P-C and PRISMA-C guideline development will involve two streams of consensus activities as follows:
Meetings of the steering committee: Steering committee meetings will be held regularly throughout the project to achieve high-level decisions on the content and methodology of the paediatric extensions of PRISMA guidelines. Following the synthesis of evidence in phase II, a formal meeting will be held with the steering committee to discuss each topic that requires further guidance. A further meeting will be held following a survey (described below) in which items will be discussed for which strong objection for their omission or inclusion has been received.
Survey: An electronic survey of international experts in systematic reviews will lead to the preliminary list of potential paediatric extension items for conducting (PRISMA-P-C) and reporting (PRISMA-C) paediatric systematic reviews. Survey methodology has been used as an initial step of guideline development in other guideline extensions, such as PRISMA-IPD9 and PRISMA-Equity.8 Survey participants will be identified through the editorial boards of Cochrane Child Health, Cochrane Neonatal Group, leading systematic reviewers in the child health field, editorial boards of leading paediatric and other journals and through networks of our steering committee members. Potential survey participants will be invited by email to complete a web-based survey. The survey will remain open for 3 weeks. Eligibility criteria for survey participation will include a combination of experience in paediatric clinical research and systematic reviews or guideline development. In the survey, each item will be rated as ‘omit’. ‘possible’. ‘desirable’. or ‘essential’ to include in the final checklists.29 The ranked items will then be divided into three groups. Group I will contain items with the highest rankings (rated as ‘essential’ by ≥70% participants or ‘essential or desirable’ by ≥90%), and these items will be included for a discussion in the Consensus meeting. Group II will contain items with moderate rankings (‘essential’ or ‘desirable’ by ≥80–<90%) and will be further discussed by the Steering Committee members for their inclusion or exclusion in the Consensus meeting. Group III will contain items with low rankings (ie, <80% ‘essential’ or ‘desirable’, or >70% ‘omit’ or ≥85% ‘possible’ or ‘omit’), and these items will be removed and will not be discussed further. Participants will have the opportunity to suggest new items that will be considered by the Steering Committee members to decide whether they should be discussed at the Consensus meeting. In addition, participants will be given an opportunity to comment on each item's wording or provide general comments on its concept. We considered the need for several (usually three) rounds of the Delphi survey as unnecessary, as a similar multiround Delphi survey exercise was recently undertaken for the development of SPIRIT-C (Children) and the concepts and feedback on paediatric specific items were already captured by experts in paediatric research and other stakeholders such as journal editors. The feedback for SPIRIT-C items was further reviewed by the steering committee while identifying PRISMA-P-C and PRISMA-C relevant topics. However, a survey will establish its applicability to paediatric systematic reviews from the perspective of relevant end users such as paediatric systematic reviewers, clinicians and methodologists.
Consensus meeting: A Consensus development meeting will be held to reach consensus regarding the minimum items required in a paediatric extension of PRISMA-P-C and PRISMA-C. The Cochrane Colloquium will provide the ideal venue to host this Consensus meeting, since this annual meeting is attended by systematic reviewers, representatives from Cochrane and Prospective Register of Systematic Reviews (PROSPERO), and end users of paediatric systematic reviews such as patients and clinicians, funders, methodologists, guideline developers and journal editors, allowing them to gather under one umbrella for scientific exchange regarding systematic reviews and their methodology, as well as giving them the opportunity to develop further methods. Hence, the Cochrane Colloquium will facilitate the meeting of our goals and objectives to gather a wide range of stakeholders for the Consensus process.
Each item of the checklist will be discussed in the context of evidence synthesised through the systematic review and results of the Delphi surveys. The voting process will follow methods used in previous Consensus meetings of guideline development. A preliminary round of voting will take place for each candidate item. Each item will be presented sequentially and debated in the light of the results from the Delphi survey and a summary of literature findings. Votes will be carried out anonymously using an online m-clicker voting system. In order to reach consensus, a classification scheme for selecting items to include in the checklists will be used, similar to the one used in developing the original PRISMA checklist. Briefly, a candidate item will be included within the final checklist if ≥80% of voters agree on its inclusion. Items with ≤20% votes for inclusion will be excluded from the final checklist. For items that do not reach consensus through the preliminary votes, round table discussions will be held, whereby participants will be given the opportunity to express their points of view in support of or against the inclusion of the item of interest. Discussions will be followed by a second round of voting with the same qualification criteria for inclusion. An experienced moderator not directly involved in this project (to allow unbiased facilitation of the consensus process) will facilitate the meeting.
Phase IV—write up
Following the Consensus meeting, the proposed checklists for PRISMA-P-C and PRISMA-C will be reviewed by the project Steering Committee to draft final checklists using concise, unambiguous and comprehensive wording, taking into account any comments obtained in the Delphi survey and the Consensus meeting regarding the wording of the items. Guideline documents will be written, separately for PRISMA-P-C and PRISMA-C, including a statement and an explanation and elaboration document that will provide detailed advice for each item and examples of good reporting in paediatric systematic review protocols and reports, respectively. The systematic review from phase III will provide empirical evidence about the relevance and rationale to support paediatric specific reporting items of a systematic review. Results from this review will also provide an evidence base of studies about good reporting practice cited in an accompanying explanation and elaboration document. Drafts of the statements and the explanation and elaboration manuscripts will be circulated to Consensus meeting participants to ensure that the documents accurately represent the decisions made during the meeting and provide examples of good reporting for specific items.
A survey of paediatric systematic review authors will be conducted to introduce them to the new items in PRISMA-P-C and PRISMA-C, establish the extent to which they had historically addressed those items in their own systematic reviews, and gather feedback on the usefulness of the extension items, including facilitators and barriers of its use. The survey participants who were initially recruited for phase III of the project will be invited again to respond to this evaluation survey. In addition, new authors will be identified through the database of corresponding authors maintained by Cochrane Child Health.
Phase VI—integrated knowledge translation and implementation
PRISMA-P-C and PRISMA-C's potential for impacting clinical care in children can only be realised with an effective knowledge translation (KT) and implementation plan. The Steering Committee has been carefully selected to include principal knowledge users who will participate in all stages of the research process. Furthermore, a KT and dissemination plan will be developed and launched during the Consensus meeting that encompasses education, dissemination and endorsement by various key stakeholders. A Knowledge Translation Planning Template30 will be followed to develop a KT plan for building awareness and understanding of the guideline (KT goals) with identified knowledge users (eg, researchers, funders, journal editors). Active involvement of partners will be achieved by bringing representatives together from diverse international stakeholder groups in the development of the checklists, keeping them engaged throughout the development and evaluation process, and providing them with an active role in the strategic planning of actions to amplify the impact of PRISMA-P-C and PRISMA-C. Beyond translating the guidelines, evidence-based implementation strategies and processes will be developed to encourage its use.
A special session will be held in the Cochrane Colloquium to disseminate the meeting findings. All known (Cochrane) systematic reviewers who are active in child health will be invited to attend this KT meeting. In addition to disseminating knowledge about the need for a newborn and child extension of PRISMA and the method involved in developing this extension, attendees of this session will be invited to offer feedback on the checklist items and facilitators and barriers of its uptake. The goal of the dissemination plan is to maximise awareness, understanding and use of the PRISMA extensions when reporting protocols and results of paediatric systematic reviews. The potential KT strategies that have been used and proved successful in other guideline development processes such as CONSORT, SPIRIT and PRISMA will be used. These include open access publication and endorsement of the guideline in multiple journals including targeted paediatric journals, endorsement by funding agencies and systematic review registration portals such as PROSPERO, presentations at conferences and other meetings, webinars, short (eg, 5 min) youtube videos explaining each extension item with examples, and a dedicated website that will facilitate feedback about the guideline by end users. The findings will also be shared with the WHO guideline development group and experts dealing with Child and Adolescent health interventions and action plans. The final checklists will be copyrighted by the PRISMA-P-C and PRISMA-C Groups under the Creative Commons License.
The methods employed in developing the PRISMA-P-C and PRISMA-C checklists and the accompanying explanation and elaboration documents are based on best practice and evidence-based principles which are widely used in developing reporting guidelines.28 The selection of the Steering Committee will ensure that systematic reviewers, guideline developers and knowledge users with leadership roles in paediatric systematic reviews have actively participated throughout the project. The active recruitment of key stakeholder groups in the Delphi survey and the Consensus Meeting will ensure that a wide perspective is captured and will facilitate endorsement and implementation of the guidelines, hence maximising their impact. Moreover, in accordance with the EQUATOR network recommendations, consensus on the checklist items will be achieved through an iterative process involving a combination of the Delphi survey and Consensus meeting, thereby minimising potential bias associated with less structured Consensus methods. The gathering of partners, health researchers and knowledge users in the Consensus meeting will also lead to new and improved collaboration of stakeholders involved in paediatric systematic reviews, including funders, regulators and journal editors. A systematic review informing the checklist item, with examples of best reporting practice, will ensure that evidence-based practical guidance is available to facilitate its implementation. By employing a validated framework of knowledge translation, we will enable active engagement of key stakeholders by assigning leading roles in the knowledge translation process for their respective stakeholder groups.
Potential challenges and mitigation strategies
A key challenge is maximising both the breath and the depth of this work to enhance comprehensiveness and rigour, while ensuring the timely completion of tasks. We anticipate 2 years for the completion of this project (May 2015–April 2017) and the final PRISMA-P-C and PRISMA-C statements and E&E will be published in summer 2017. We have engaged a broad team of co-investigators and collaborators in paediatric systematic reviews and reporting guideline development who will provide support in all aspects of this project such as early critical review of the research findings. We will rely on our experience in conducting evidence synthesis for reporting guideline development such as CONSORT-C and SPIRIT-C.19 Though the current project examines in-depth reporting features of paediatric systematic reviews, on the basis of our intimate knowledge of the subject matter, we are confident that the systematic review can be completed in a timely and efficient manner. Another challenge is ensuring integrated and end of project knowledge translation of new evidence generated by the synthesis and Delphi survey. Our ongoing collaborations with our knowledge users, which comprised the network of our steering committee as well as the potential Delphi participants, who were the authors of a recently published paediatric systematic review, will ensure that the scope meets their decision-making needs and expectations, while adhering to timelines and deliverables. Our team has previously completed several successful collaborative projects with diverse stakeholders, and will be a highly effective team. Finally, implementation of the new reporting standard by paediatric systematic reviewers in their future studies may present challenges. Through our involvement of key research leaders and by engaging diverse stakeholders and collaborators, we hope to disseminate to a large audience in a timely and effective manner.
The resultant PRISMA-P-C and PRISMA-C statements and explanation and elaboration documents will help authors write clear protocols and reports of paediatric systematic reviews and create a framework for reviewers and funders to assess publications and protocols. These checklists will be applicable to both Cochrane and non-Cochrane paediatric systematic reviews involving newborns and children. These checklists will also provide a tool for training students and researchers on paediatric systematic review methodology. Furthermore, end users of the systematic review, such as paediatricians, policymakers and other decisionmakers, will be able to evaluate systematic review validity and applicability in their evidence-based decision-making process, thereby increasing the uptake of relevant evidence and ultimately improving child health outcomes.
Contributors MK conceived the study and made substantial contributions to the design of the manuscript and acquisition of data. MK and MO participated in the design and coordination of the drafting of the manuscript, as well as in the analysis and interpretation of data. MK, MO, LA, LH, RS and DC-I have been involved in the drafting of the manuscript or its critical revision for important intellectual content. All authors read and approved the final manuscript.
Funding MO is supported by the Hospital for Sick Children Investigator award. LH is supported by a New Investigator Salary Award from the Canadian Institutes of Health Research. The funding for the consensus meeting has been approved by the CIHR Planning and Dissemination Grant number 345296.
Competing interests LA is a co-convener of the Cochrane Prospective Meta-analysis Methods Group, a member of the Cochrane Individual Participant Data Meta-analysis Methods Group, an author on many (Cochrane and non-Cochrane) paediatric systematic reviews and a member of the PRISMA-IPD extension working group; RS is the Coordinating Editor of the Cochrane Neonatal Group and is President and Director of Clinical Trials of the Vermont Oxford Network; ZAB was a member of PRISMA-Equity extension. There are no other competing interests to declare by the authors.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement De-identified data from the Delphi survey will be available to the researcher on request.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.