Development of a core outcome set for lower limb orthopaedic surgical interventions in ambulant children and young people with cerebral palsy: a study protocol

Introduction Musculoskeletal deformities and gait deviations are common features in ambulatory cerebral palsy (CP). Deformity correction through lower limb orthopaedic surgery is the standard form of care aimed at improving or preserving motor function. Current research on CP care does not always take into account individual patients’ expectations and needs. There is a wide range of outcome domains and outcome measures used to assess outcome from treatment. This can lead to reporting bias and make it difficult to compare and contrast studies. A core outcome set (COS) would enhance the efficiency, relevance and overall quality of CP orthopaedic surgery research. The aim of this study is to establish a standardised COS for use in evaluating lower limb orthopaedic surgery for ambulatory children and young people with CP. Methods/analysis A set of outcomes domains and outcome measures will be developed as follows: (1) a qualitative evidence synthesis to identify relevant outcomes from children and young people and family perspective; (2) a scoping review to identify relevant outcomes and outcome measures; (3) qualitative research to explore the experience of key stakeholders; (4) prioritisation of outcome domains will be achieved through a two-round Delphi process with key stakeholders; (5) a final COS will be developed at a consensus meeting with representation from key stakeholder groups. Ethics and dissemination Ethical approval for this study was granted in the UK by the Oxfordshire Research Ethics Committee B (REC reference 19/SC/0357). Informed consent will be obtained from participants taking part in the qualitative research and Delphi process. Study findings will be published in an open access journal and presented at relevant national and international conferences. Charities and associations will be engaged to promote awareness of the project COS results. Trial registration number COMET registration: 1236. PROSPERO registration number CRD42018089538.


INTRODUCTION
Third paragraph • Correct typo "…limited consensus has been reached about what outcomes should bmeasured in CP" • Change the "International Classification of Function (ICF)" to its complete name of the classification International Classification of Functioning, Disability and Health (ICF) • Section referring to the ICF Core Sets for CP needs revisions. This sentence "However, this core set is a generic one for the CP population and does not specifically address specific interventions in specific population groups" does not reflect the purpose of the ICF Core Sets for CP. I suggest the following: There are five ICF Core Sets for CP, one comprehensive and four brief sets. The ICF Core Sets for children and young people with CP offer service providers and stakeholders an age-appropriate framework to explore functioning and disability for assessment, treatment, evaluation, and policy purposes in a global context. Specifically, the ICF Core Sets for CP standardize what should be measured and reportedadopting the ICF biopsychosocial model. However, the ICF Core Sets do not include specific sets of interventions for this population. • The last sentence referring to the GMFCS seems out of place. The purpose of this classification system is to describe the gross motor function of children and youth with cerebral palsy based on their self-initiated movement -mainly sitting, walking, and wheeled mobility. The GMFCS does not address outcomes or what to measure in CP. Use CP consistently throughout the manuscripts, e.g. last paragraph of introduction CONCEPTUAL FRAMEWORK • Provide references here … The ICF-CY …has been used extensively in CP research. (REFERENCES) • The ICF has two parts with two components each, please revise this statement "The framework provides three main domains: body function, activity, and participation" • Correct typo CF-CY METHODS AND ANALYSIS Study design • Please add an estimated timeline for the completion of both phases Establishing a steering committee • As the output of the project is to develop international standards, are you considering international representation of members in the steering committee? • Also, consider moving the information regarding the phases of the study under this subheading to a different subheading as the information does not belong here Phase 1systematic review of domains • What is the research question for this systematic review? • Study selection and criteria: the following criteria needs further clarification "(3) the study explored the experience of children and young people with CP and their family of lower limb orthopaedic surgery" it is not clear at this point what outcomes are you looking for, the experience during the surgical period?, after surgery?, the outcomes of a surgical intervention? Measured using a standardized tool/s? Themes related to goals for surgical interventions?
• ICF linking rulesyou might want to use the revised version of the linking rules published in March 2019 published in https://www.ncbi.nlm.nih.gov/pubmed/26984720 You will need to update Table 2, there are 10 rules proposed now.
Phase 2 -Scoping review • Search strategy needs revisions as key terms are missing, for example the search terms described in this section do not include the term "OUTCOME", or tools or measures, or "GAIT", you should maximize the search strategy, you might want to consultant an experienced librarian to help with this task due to the importance of this step • Can you provide a rationale for conducting two systematic reviews, you could run a search for domains and outcomes including qualitative and quantitative studies.
Phase 3 -Qualitative study • What is the research question that you want to answer with this study?
• Inclusion criteria: you might want to add a criteria that include "lower limb surgical intervention" as you are creating a COS for this population Phase 4 -Delphi process • Participants: you are inviting children to participate in this twostage scoring exercise, how are you planning to present the information to this group? • Also, the output of this project is meant to be international, are you planning to include international stakeholders? The recruitment strategy is not described for this study.
Phase 5 -Consensus meeting • Recruitment strategy of panel members? • International representation? • Describe the procedure for determining how outcomes will be added/combined/dropped from consideration during the consensus process • Describe how missing data will be handled during the consensus process Additional comments: • Include potential limitations to the proposed studies and successful completion of proposed project • Timeline of the total project • Figure 1. Please check misspellings in the main boxes • Sources of funding? • ICF-CY used in first study, are you planning to use the ICF in the other studies as well?

REVIEWER
Toby Smith University of East Anglia, UK I have a position within NDORMS, University of Oxford, UK.

GENERAL COMMENTS
Abstractclear presentation of the approach taken. Registration with COMET was acknowledged.
Strengths and limitationsappropriate and well-reasoned.
Introduction -Population clearly defined and sufficient context provided.
Acknowledgement of merits of Core Outcome Sets on research synthesis and efficiency justified. Further clarification on why this core outcome set is warranted over that cited in Page 4-5 would be beneficial. There is suggestion of a requirement for one related to 'motor function' however this message could be more clearly articulated. I am not convinced based on the current text why focusing on lower-limb orthopaedic surgery is required and the current text questions whether this is too narrow and the same COS would be relevant for pre-and post-surgery i.e. those with motor challenges. Based on the stage of this study, I would suggest that revising the text to clarify this would be beneficial. This has the potential to be really valuable and therefore please clarify/sell-this further to the reader.
Methodsreporting by COS-STAP which is appreciated. Wider stakeholder group which is a huge benefit.
Overall the methods are well-presented, clearly documenting and justifying the approaches taken. I am unsure about the merit of Table 2. Could this not just be cited rather than presented in the text? Justification for the narrow search period (Jan 2016 to June 2019) would be helpful for the scoping review? Why has this been undertaken when important studies may been available before 2016?
There is limited feed-forward of the work packages into the qualitative Phase 3 study. I would suggest that these findings from Phase 1 and 2 may be helpful to inform the topic guides for Phase 3. Should this be considered? Further detail is required in Table 1 as to who the Health Professionals are. Could the team please provide the professions they expect to approach i.e. surgeons, play therapists, physiotherapists, occupational therapists, orthotists etc etc. This would be really helpful.
The Phase 4 Delphi is appropriately reported however please provide further information on who will be completing this, where it will be sent to, who the target recruitment sites are, what approaches are made to improve the international reach of the Delphi. This is important.
Phase 5 needs further clarification. Phase 4 will inform what domains should be included. However the matching of domains to outcomes is a challenging process and the current text requires further detail. There is a suggestion that outcomes previously associated with the selected domains will be linked and their feasibility/psychometric properties will be evaluated against the COSMIN checklistthis process warrants its own Phase in reporting so the reader can appreciate how this is actually undertaken. There is currently insufficient information here. Furthermore, where will the stakeholder meetings take place. Are these all in Oxford? How generalisable/transferable are the views of this group?

Ethics and Disseminationappropriate
Limitationsthere is currently insufficient acknowledgement of the issue of generalisability/transferability of the COS internationally. The searches are based on English language publications. The consensus meetings are 'presumably' in Oxford, the Delphi reach feels UK base (but may not be). COS should be international in their importance and relevance. The current approach feels very Oxford-centric. This should be acknowledged somewhere and/or strategies highlighted to the reader as to where this has been negated. Comment 3: Section referring to the ICF Core Sets for CP needs revisions. This sentence "However, this core set is a generic one for the CP population and does not specifically address specific interventions in specific population groups" does not reflect the purpose of the ICF Core Sets for CP. I suggest the following: There are five ICF Core Sets for CP, one comprehensive and four brief sets. The ICF Core Sets for children and young people with CP offer service providers and stakeholders an age-appropriate framework to explore functioning and disability for assessment, treatment, evaluation, and policy purposes in a global context. Specifically, the ICF Core Sets for CP standardize what should be measured and reportedadopting the ICF biopsychosocial model. However, the ICF Core Sets do not include specific sets of interventions for this population.

Reviewer 1 Introduction
Response: Thank you for this, we have adjusted the text as suggested. 'There are five ICF Core Sets for CP, one comprehensive and four brief sets. The ICF Core Sets for children and young people with CP offer service providers and stakeholders an age-appropriate framework to explore functioning and disability for assessment, treatment, evaluation, and policy purposes in a global context. Specifically, the ICF Core Sets for CP standardize what should be measured and reportedadopting the ICF biopsychosocial model. However, the ICF Core Sets do not include specific sets of interventions for this population.' p.4-5 Comment 4: The last sentence referring to the GMFCS seems out of place. The purpose of this classification system is to describe the gross motor function of children and youth with cerebral palsy based on their self-initiated movement -mainly sitting, walking, and wheeled mobility. The GMFCS does not address outcomes or what to measure in CP.
Response: We agree with this point and the text was deleted accordingly. 'Since CP is a heterogeneous condition, a classification based on motor function was recommended to better define clinical groups based on lower limb functional ability (e.g., Gross Motor Function Classification System).' Deleted since its 'out of place' 'Therefore, the ICF-CY framework taxonomy will be used as a basis of the development of COS.' p.6 Methods and Analysis Comment 9: Study design, please add an estimated timeline for the completion of both phases Response: We have added the timeline for each phase in figure 1 and highlighted in text under 'study design' section 'A brief overview of our study design, including estimated time frames, is highlighted in Figure. 1.' p.6 Comment 10: Establishing a steering committee, As the output of the project is to develop international standards, are you considering international representation of members in the steering committee?
Response: An International steering committee will be considered 'An international steering committee will be formed' p.7 Comment 15: Search strategy needs revisions as key terms are missing, for example the search terms described in this section do not include the term "OUTCOME", or tools or measures, or "GAIT", you should maximize the search strategy, you might want to consultant an experienced librarian to help with this task due to the importance of this step Response: Thank you for this point. As we updated a previous review, we maintained the original review search terms exactly the same as instructed by the Cochrane Review recommendation: 'a review update should be re-conducted using the same methods as the original review.' Therefore, similar databases and key search terms will be replicated. Text was added under the main heading 'phase 2: Scoping Review' to clarify this point with reference to the Cochrane recommendation and the original review. 'According to the Cochrane Review recommendations, a review update should be re-conducted using the same methods as the original review, 35 therefore, this review was in line with the methodology of the original scoping review. 7 ' p.12 Comment 16: Can you provide a rationale for conducting two systematic reviews, you could run a search for domains and outcomes including qualitative and quantitative studies.
Response: Thank you for your point. While both reviews addressed a similar overall research purpose and question with respect to their methodological differences, we felt that conducting two reviews separately is favourable for the following reasons: 1. The two reviews target two different specific perspectives which complement each other. In the Qualitative review, we aimed to look at the children and parent experiences of surgical outcomes, while the scoping review would reflect the health professionals' perspective whilst identifying outcome measures that are used. 2. The qualitative review is considered a novel part of the study, as there is no such review in the CP orthopaedic surgical literature. In this review we included all studies from database inception to date, while the scoping review is an updated version of a previous one and covers a relatively short period of time.
3. We felt that a mixed methods review would complicate the description of the methodology and findings. Comment 20: Also, the output of this project is meant to be international, are you planning to include international stakeholders? The recruitment strategy is not described for this study.
Response: We are planning to recruit an international panel. We have clarified this further in our recruitment strategies under subheading '4.3: Recruitment' and have listed relevant international stakeholders in Table 1 'Stakeholder involvement'. 'Different strategies will be followed to identify the potential panel, as shown in table 3. Clinical academics and healthcare professionals will be invited through professional societies. Specific invitations to Authors of relevant references identified through the qualitative evidence synthesis and scoping review will be targeted. Snowballing techniques will be used to ensure a representative sample of international researchers and clinicians are invited. Children and family recruitment will be initiated through the clinical care team at Nuffield Orthopaedic Centre (Oxford, UK) and will expand nationally and internationally to include patient and parent organisations and charities. p.17-18 Response: We have added information about recruitment of the panel members. This is detailed under subheading '5.1 Recruitment' 'A consensus meeting will be hosted for the purpose of finalising the core outcome set. The meeting will include approximately 20 panel members. Representatives from all stakeholder groups, representing as much geographical, ethnic, demographic and cultural diversity as possible (as recommended by COMET) will be invited at this stage. Ten of those participants will be randomly selected from the Delphi survey participants and the study steering group. Face-to-face and remote access to the meeting will be available'. p.20 Comment 23: Describe the procedure for determining how outcomes will be added/combined/dropped from consideration during the consensus process Response: We have added information about how outcomes will be finalised. This is detailed under subheading '4.4 Delphi survey'. 'At the meeting, Delphi survey results will serve as the basis for the discussion and development of the final COS to be agreed across stakeholder groups. Across all stakeholder groups, any outcome categorised as 'consensus in' will be proposed to be included in the final COS, while any outcome categorised as 'consensus out' will be excluded. The panel members will electronically vote to accept this proposal or suggest outcomes that warrant further discussion.
Outcomes that are differently categorised by different stakeholder groups and those categorised as 'no-consensus' will be discussed individually. A second round of voting will be used to agree the final COS.' p.20-21 Comment 24: Describe how missing data will be handled during the consensus process Response: We anticipate 2 different types of missing data; first whereby a participant does not response to the second round of the Delphi survey and second, whereby a participant does not respond to a particular item on the Delphi survey (for whatever reason). In order to minimise the risk of participants not responding to the second round of the Delphi survey we will send out regular reminder emails. In order to take into account participants who do not respond to a particular Delphi survey item we will summarise the distribution of scores and to calculate the median and interquartile range (IQR) for each Delphi survey item separately. As such the denominator for each Delphi survey item will be the number of participants completing each specific item; rather than the number of participants completing the Delphi survey overall. We have clarified this in the text in subheading '4.4 Delphi survey' and '4.5 Analysis'.
'Each round will be open to the panel for four weeks and reminder emails will be sent at 2-week intervals in order to maximize follow up rates' p.19 (4.4. Delphi Survey) 'Descriptive statistics will be undertaken using SPSS software to summarise the distribution of scores and to calculate the median and interquartile range (IQR) for each Delphi survey item. The denominator for each Delphi survey item will be the number of participants completing that item; rather than the number of participants completing the Delphi survey overall (i.e. a participant may choose not to score a particular Delphi item for whatever reason). p.19-20 (4.5. Analysis)

Additional comments
Comment 25: Include potential limitations to the proposed studies and successful completion of proposed project Response: A new main heading 'Discussion' was added after 'Ethics and Dissemination'. 'Strengths and limitations of this study: To our knowledge, this is the first study on the development of a core outcome set for lower limb orthopaedic surgery in ambulant children with CP. This study employed a well-established and widely used method developed by the COMET Initiative. Involving patients in COS development has become common practice to ensure the relevance of the proposed COS to all stakeholders. In this protocol, children and families will be directly engaged with the COS development through participation in the steering committee, the interviews, the Delphi process and the consensus meeting.
This study includes a comprehensive search for potentially relevant outcomes through qualitative evidence synthesis, a scoping review and interviews with stakeholders' groups. This process will be conducted by at least two independent researchers ensuring identification of all potential outcomes. This will provide a comprehensive list of all pertinent outcomes for the Delphi survey.
As the comprehensive search for outcomes focuses on the English literature and on interviews with English speaking stakeholders any outcomes available in the non-English literature may be omitted. Free-text fields will be included in the Delphi survey to allow participants to suggest any additional outcomes that they consider important. The potential imbalance between national and international participants may represent a limitation of the study. Although, the proposed COS development will aim to reach international consensus, it is possible that most participants will be recruited from the UK, which may affect the wider generalisability of the COS findings.' p.23

Introduction
Comment 3: Population clearly defined and sufficient context provided.
Response: Thank you for your comment.
Comment 4: Acknowledgement of merits of Core Outcome Sets on research synthesis and efficiency justified.
Response: Thank you for your comment.
Comment 5: Further clarification on why this core outcome set is warranted over that cited in Page 4-5 would be beneficial. There is suggestion of a requirement for one related to 'motor function' however this message could be more clearly articulated. I am not convinced based on the current text why focusing on lower-limb orthopaedic surgery is required and the current text questions whether this is too narrow and the same COS would be relevant for pre-and post-surgery i.e. those with motor challenges. Based on the stage of this study, I would suggest that revising the text to clarify this would be beneficial. This has the potential to be really valuable and therefore please clarify/sell-this further to the reader.
Response: Thank you for this point. We have revised the text in last paragraph of the introduction.
'Lower limb orthopaedic surgery is a common intervention in the management of ambulatory children with CP. However, significant variation in the outcomes collected and reported remains a challenge. The post-surgical emotional and physical challenge that this treatment imposes on children and their families is significantly bigger than any other gait improvement intervention in this population. It is therefore of the highest importance to ensure that the surgical aims and the expected outcomes are of relevance to children and their families. Developing a COS representing all stakeholders for this specific intervention would represent the first step in this direction.' p.5 Method Comment 6: Reporting by COS-STAP which is appreciated. Wider stakeholder group which is a huge benefit.
Response: Thank you for your comment.
Comment 7: Overall the methods are well-presented, clearly documenting and justifying the approaches taken.
Response: Thank you for your comment.
Phase 1: Qualitative Evidence Synthesis Comment 8: I am unsure about the merit of Table 2. Could this not just be cited rather than presented in the text?
Response: We adjusted the text in the revised manuscript. 'the ICF linking rule will be used. 33 Each line of text will be coded according to its meaning and will be linked to the most precise ICF-CY code. If the content of a code not explicitly named in the ICF-CY category, the 'not defined', 'not covered', and 'health condition' category will be applied.' p.10 Phase 2: Scoping Review Comment 9: Justification for the narrow search period (Jan 2016 to June 2019) would be helpful for the scoping review? Why has this been undertaken when important studies may been available before 2016?
Response: We have justified conducting an updating scoping review over a short period of time as follows: 'Although, previous scoping reviews for studies published between 1990 and 2015 were identified, 7, 34 updating the review was important to ensure that recently published outcome domains and outcome measures were identified. This was important particularly because, in recent years, researchers and healthcare professionals have become more aware of patient priorities and have acknowledged the value of patient-reported outcomes.' p.12 Phase 3: Qualitative study Comment 10: There is limited feed-forward of the work packages into the qualitative Phase 3 study. I would suggest that these findings from Phase 1 and 2 may be helpful to inform the topic guides for Phase 3. Should this be considered?
Response: We entirely agree with this point and have now addressed it under the subheading '3.5 Data collection'.
'The interview topic guide will be formed and shaped by the findings of phases 1 and 2. For example, participants will be asked to reflect on the identified outcome domains from the reviews.' p.15 Comment 11: Further detail is required in Table 1 as to who the Health Professionals are. Could the team please provide the professions they expect to approach i.e. surgeons, play therapists, physiotherapists, occupational therapists, orthotists etc etc. This would be really helpful.
Response: Further categories of health professionals were added in table 1. p.7 Phase 4: Delphi process Comment 12: The Phase 4 Delphi is appropriately reported however please provide further information on who will be completing this, where it will be sent to, who the target recruitment sites are, what approaches are made to improve the international reach of the Delphi. This is important.
Response: Thank you for this point.
With regard to the first part of your comment, "who will be completing the Delphi survey": we have added clarification under the subheading '4.2 Selection of panel members and sample size' 'The eligibility criteria in table 1 will be used as a guide in the selection of the Delphi panel members.' p.17 With regard to the second part, "further information about recruitment": we have added a subheading '4.3 Recruitment' with the following explanation of our proposed recruitment strategies. 'Different strategies will be followed to identify the potential panel, as shown in table 3. Clinical academics and healthcare professionals will be invited through professional societies. Specific invitations to Authors of relevant references identified through the qualitative evidence synthesis and scoping review will be targeted. Snowballing techniques will be used to ensure a representative sample of international researchers and clinicians are invited. Children and family recruitment will be initiated through the clinical care team at Nuffield Orthopaedic Centre (Oxford, UK) and will expand nationally and internationally to include patient and parent organisations and charities.' p.17-18 Phase 5: Consensus meeting Comment 13: Phase 5 needs further clarification. Phase 4 will inform what domains should be included. However the matching of domains to outcomes is a challenging process and the current text requires further detail. There is a suggestion that outcomes previously associated with the selected domains will be linked and their feasibility/psychometric properties will be evaluated against the COSMIN checklistthis process warrants its own Phase in reporting so the reader can appreciate how this is actually undertaken.
Response: Thank you for this important comment. Section 'Phase 6: Selection of outcome measures' was updated. 'Phase 6: Selection of outcome measures: After the development of a COS, it is recommended to identify a set of measurements, the "outcome measures" that would be used to evaluate the selected outcomes. 44 In order to establish a core outcome measures set, a four-step process will be followed: (1) conceptual considerations (scope); (2) identifying existing outcome measures; (3) quality assessment of the identified outcome measures and (4) generic recommendations for the selection of outcome measures for a core outcome set.
The conceptual considerations of the proposed core outcome measures set will be associated with the study scope. Accordingly, all available outcome measures used in clinical research following lower limb orthopaedic surgery in ambulant CP will be considered for the core set. An international perspective on the subject will be captured by involving stakeholders from the study's international steering committee and consensus panel members.
Previous reviews on this field 7, 34 alongside the scoping review (phase 2) will be used as a starting point to identify currently used outcome measures. The quality assessment of each outcome measure identified will be determined by the available systematic reviews using the Consensus-based Standards for selection of health measurement Instruments (COSMIN). This will assess the psychometric properties of outcome measures that have been used in CP clinical studies. For example, two recent systematic reviews, using a modified COSMIN method to assess gait-related outcome measures in CP will be employed to choose suitable, high-quality outcome measures. 45,46 A consensus meeting with a panel of health professionals will subsequently be organised to establish appropriate outcome measures for each outcome domain identified during the COS development.
Members will be asked to recommend one high-quality outcome measure per core outcome domain. If no adequate outcome measures exist for a specific core outcome, this will be acknowledged, and recommendation will be made for future development of an adequate high-quality outcome measure.' p.21-22 Comment 14: There is currently insufficient information here. Furthermore, where will the stakeholder meetings take place. Are these all in Oxford? How generalisable/transferable are the views of this group?
'Strengths and limitations: … As the comprehensive search for outcomes focuses on the English literature and on interviews with English speaking stakeholders, any outcomes available in the non-English literature may be omitted. Free-text fields will be included in the Delphi survey to allow participants to suggest any additional outcomes that they consider important. The potential imbalance between national and international participants may represent a limitation of the study. Although, the proposed COS development will aim to reach international consensus, it is possible that most participants will be recruited from the UK, which may affect the wider generalisability of the COS findings.' p.23

GENERAL COMMENTS
Thank you for the opportunity to review the revised version of this manuscript. The authors have incorporated the recommendations raised in the first review, as such; the protocol is very informative, and the flow of the paper has significantly improved. In addition, the methodology proposed for each phase is much easier to follow. However, there are some minor comments that need to be addressed as follows: o Figure 1. Please check misspellings in the main boxes "measurs" o References: #3 is incomplete, there is no journal o References #33 and #40 are the same reference with different numbersplease review and keep the correct citation o Please carefully review the complete list of references to meet the journal's citation style o Table 2 -ICF linking rulesthe authors incorporated the revised version of the linking rules as suggested -for the upcoming phases of the project https://www.ncbi.nlm.nih.gov/pubmed/26984720 However you need to update the content of Table 2 or change the reference #33 on this table, as I pointed out before, there are 10 rules proposed in 2019 (reference #33) but you currently show 8 rules in your table.
Therefore, the paper should be considered for publication after minor revisions. Kind regards.

Toby Smith
University of East Anglia, Norwich REVIEW RETURNED 31-Jan-2020

GENERAL COMMENTS
The authors have addressed my earlier comments. I have nothing to add.

VERSION 2 -AUTHOR RESPONSE
Reviewer 1 Comment 1: the protocol is very informative, and the flow of the paper has significantly improved. In addition, the methodology proposed for each phase is much easier to follow.