Assessment of the methodological quality of local clinical practice guidelines on the identification and management of gestational diabetes

Objective Gestational diabetes is the most common metabolic disorder of pregnancy, and it is important that well-written clinical practice guidelines (CPGs) are used to optimise healthcare delivery and improve patient outcomes. The aim of the study was to assess the methodological quality of hospital-based CPGs on the identification and management of gestational diabetes. Design We conducted an assessment of local clinical guidelines in English for gestational diabetes using the Appraisal of Guidelines for Research and Evaluation (AGREE II) to assess and validate methodological quality. Data sources and eligibility criteria We sought a representative selection of local CPGs accessible by the internet. Criteria for inclusion were (1) identified as a guideline, (2) written in English, (3) produced by or for the hospital in a Western country, (4) included diagnostic criteria and recommendations concerning gestational diabetes, (5) grounded on evidence-based medicine and (6) accessible over the internet. No more than two CPGs were selected from any single country. Results Of the 56 CPGs identified, 7 were evaluated in detail by five reviewers using the standard AGREE II instrument. Interrater variance was calculated, with strong agreement observed for those protocols considered by reviewers as the highest and lowest scoring based on the instrument. CPG results for each of the six AGREE II domains are presented categorically using a 5-point Likert scale. Only one CPG scored above average in five or more of the domains. Overall scores ranged from 91.6 (the strongest) to 50 (the weakest). Significant variation existed in the methodological quality of CPGs, even though they followed the guideline of an advising body. Specifically, appropriate identification of the evidence relied on to inform clinical decision making in CPGs was poor, as was evidence of user involvement in the development of the guideline, resource implications, documentation of competing interests of the guideline development group and evidence of external review. Conclusions The limitations described are important considerations for updating current and new CPGs.

1. The main problem with this paper is that most of the guidelines that are evaluated are too parochial and empirical to be of use to any caregiver of pregnant women-in any hospital in the world. Currently, the major valid GDM guidelines are of a) The International Association of the Diabetes and Pregnancy Study Groups, IADPSG-2010 (accepted by WHO, ADA, ADIPS, FIGO amongst many others-and the most popular), b) The National Institute for Health and Care Excellence, NICE-2015, c) Canadian diabetes association, CDA-2013. In North America, Carpenter &Coustan, C&C andNational Diabetes Data Group, NGGD-1979 (accepted by ACOG/ADA) are very popular. 2. The authors recommend two guidelines of acceptable quality based on their assessment tool: CDA and New Zealand guidelines, NZ. Both these guidelines are insular and 'expertopinion driven' without any research backing. The CDA raised the HAPO study threshold 1.75 risk to 2.0 odds-risk ratio without universal consensus (unlike IADPSG). Furthermore, CDA-2013 guidelines are a hybrid using the 50-g screening and 75-g OGTT which is validated by no major study. The NZ guidelines are worse for 2 reasons: a) they were based on the now defunct ADPIPS criteria and b) their 2-hr 75-g OGTT value was raised from 8.0 mmol/l of ADIPS to 9.0 mmol/l to save 'New Zealand's stretched funds.' In short it was an economic decision. Furthermore, the NZ guidelines have not evolved using the latest research.
3. The NICE-2015 guidelines used by the authors were based on 'economics' rather than 'research.' Thus, many major hospitals, despite the clout of NICE, shy away from their GDM guidelines. 4. Due to the authors' restrictions in criteria selection (reject CPGs before 2013), the authors eliminated one of the most popular guidelines vogue in USA (C&C)which has stood the test of time.
Minor comments: 1. The authors make a good point that weakest link in the chain of practice guidelines is implementing the guideline. Thus, the recommendation of the latest FIGO guidelines to be 'practical.' 2. The author's reference 1 has been retracted. Please replace by an alternate one. 3. The references should be cut to less than 20.
In short, none of the guidelines suggested by the authors are not validated by any research. Their suggested guidelines may be meet all the criteria of AGREE II, but their conclusions would be a hard sell to any hospital. Somehow, this paper would have great value if the authors had evaluated the current (WHO-2013, C&C, ACOG, NICE-2015, CDA-2013 vogue and more acceptable guidelines using their instrument. (1) Page2Line4: "clinical practice guidelines (CPG)" should be rewritten as "clinical practice guidelines (CPGs)".

REVIEWER
(2) Page2Line10: "the Appraisal of Guidelines for Research and Education (AGREE II)" should be rewritten as "the Appraisal of Guidelines for Research and Evaluation (AGREE II)".
(4) Page3Line17: "6-to-11-fold" should be rewritten as "6to 11fold". (5) Page4Line37: "the expense of repeat or inappropriate treatments" should be rewritten as "the expense of repeating inappropriate treatments". (6) Page5Line4: "23 questions" would be better if written as "23 items". (7) Page5Line52: "clinical midwife research fellow" should be rewritten as "a clinical midwife research fellow". (8) Page7Line26: "treating staff" should be expressed in another word, like "clinician", for it isn't a customary usage. (9) Page11Line15: "treating clinicians" should be expressed in another way, reason mentioned above. (10) Page2Line29: "Prior to the introduction of insulin in 1923 maternal and neonatal …", there ought to be a comma after "in 1923", that is, "Prior to the introduction of insulin in 1923, maternal and neonatal …", (11) Page5Line49: "To increase the reliability of the appraisal the AGREE II protocol user manual…", there ought to be a comma after "the appraisal", that is, "To increase the reliability of the appraisal, the AGREE II protocol user manual…".
Overall, the paper is well-done Thank you 2.
No more than two CPGs were selected from any single country. How were they selected in the case of the presence of more than two?
The selection process is described in the paper under the heading Guideline Selection. We have now addressed the comment regarding selection of 2 CPGs per country where more than 2 met the criteria at the end of that section. In the methods we have stated (page 5 line 26) "Where more than two from any one country met the requirements, the ones having the most recent updates were used." We have also added a new section in the discussion (page 11 line 18) on limitations of the study "There are a number of study limitations to this study. Only two selected CPGs were selected from tertiary hospitals and from only 4 English-speaking countries; furthermore, CPGs before 2013 not considered, even though in many settings they are still used in current clinical practice. Nonetheless, the range of AGREE II scores would suggest we captured the range of available guidelines thus confirming that there is wide variation in local guidelines."

3.
The introduction section of the manuscript is too long. It should be shortened.
Content has been revised and shortened. Overall the paper has been reduced from 6839 to 6593 words, a total reduction of 246 with 237 of that in the content of the introduction.

4.
One of the main limits of the review is having selected CPGs only from tertiary hospitals and from some Countries (UK, New Zealand, Australia, Canada, and the United States of America). CPGs before 2013 were also rejected. These aspects should be recognized as study limitations. We have added a notation to the Strengths and Limitations section.' "Exclusion of CPGs prior to 2013 and limiting to the most recent two located in each country means that while those CPGs being reviewed are the most up-to-date, some well-known older guidelines were excluded." We have also discussed this under study limitations in the discussion; see comments reviewer 1 (2).

5.
At the end of page 11 there is written: "it should be noted that none of the six advising bodies listed in Table 4...". In table 4 there are 7 guidelines. Please check.
Authors should acknowledge that some CGPs on diabetes includes recommendations for diabetes in pregnancy. These are not included in the review, this limiting the number of considered CGPs Addressed. The authors are only addressing the diagnosis and management of gestational diabetes (GDM) not "diabetes in pregnancy" which includes pre-existing T1 and T2 diabetes.
This has been made clearer in the Abstract, Introduction and Methods sections. In the discussion we have added the sentence referring to limitations of the study (page 11line 22) "We restricted our evaluation to GDM and did not include the management of pre-existing diabetes; this narrowed down the available guidelines for review." Reviewer 2: 1.
The main problem with this paper is that most of the guidelines that are evaluated are too parochial and empirical to be of use to any caregiver of pregnant women-in any hospital in the world.
The reviewer has a valid point and indeed we are not recommending the adoption of any one of the guidelines reviewed but pointing out the limitations of guideline development and the areas that can be improved using the AGREE II protocol. As stated in the paper the authors reviewed local guidelines that are IN USE in hospitals in countries in which the authors have clinical experience, and which operate a centralised health service. Rather than review the overarching international guidelines (ADIPS, IADPSG, NICE etc.) which have already been extensively reviewed and analysed in the common literature, we sought to evaluate the guidelines as they are being prescribed locally within tertiary hospitals. We have also added under study limitations in the discussion the following sentences (page 11 line 24) "An alternate approach would have to evaluate all the international guidelines, however, in a tertiary setting they are interpreted locally leading to the guidelines used in the clinical setting, that was the subject of our review. Finally, the AGREE II protocol does not assess the evidence base of the clinical content of the guideline, nor its implementation. Thus, a guideline reviewed may score highly, independent of the local or national adaptation of the international guideline but does not indicate the clinical quality of the decision making or the evidence or how it was adapted for local needs." 2.
The authors recommend two guidelines of acceptable quality based on their assessment tool: CDA and New Zealand guidelines, NZ. Both these guidelines are insular and 'expert-opinion driven' without any research backing.
Our response is covered above in answer to Reviewer 2 Q1the purpose of the AGREE II protocol is not to identify the guideline with the best content but to analyse the process used to formulate the guideline using the AGREE II tool. Furthermore Domain 3 of the AGREE II protocol, regarding the rigour and evidence for a CPG, is only one of the areas that AGREE II evaluates. CDA-2013 scored well across all Domains in the opinions of the reviewers. NZ-ADHB scored "average" in this Domain, but maintained a better score across the other Domains than the other CPGs reviewed. NZ-ADHB also presented the evidence that they relied upon and explained decisions taken in formulating their recommendations, allowing readers and clinicians to evaluate for themselves the recommendations provided.

3.
The NICE-2015 guidelines used by the authors were based on 'economics' rather than 'research.' Thus, many major hospitals, despite the clout of NICE, shy away from their GDM guidelines.
Again, this is a valid point and indeed all international guidelines have their drawbacks leading to the confusion in the field as to which should be adopted. Unfortunately for the user of the service, affordability does play a role in the development of the guideline and its parochial interpretation.

4.
Due to the authors' restrictions in criteria selection (reject CPGs before 2013), the authors eliminated one of the most popular guidelines vogue in USA (C&C)which has stood the test of time.
We respect the reviewer's opinion. In the limitations section of the discussion we have therefore added… "CPGs before 2013 not considered" …even though in many settings they are still used in current clinical practice. Nonetheless, the range of AGREE II scores would suggest we captured the range of available guidelines thus confirming that there is wide variation in local guidelines."

5.
The authors make a good point that weakest link in the chain of practice guidelines is implementing the guideline. Thus, the recommendation of the latest FIGO guidelines to be 'practical.' No response 6.
The author's reference 1 has been retracted. Please replace by an alternate one.
The article itself remains available on the publisher's database and web archive. In any event the article was not retracted for any factual error or omission of the authors, it was retracted simply because, in the assessment of an editor, the content extensively overlapped with a number of prior publications.
The reference has been updated and the new reference 1 is : Metzger, B. and D. Coustan. (1998). Proceedings of the Fourth International Work-shop-Conference on Gestational Diabetes Mellitus. Diabetes Care, 21, B1-B167.

7.
The references should be cut to less than 20.
We have reduced the references from 56 to 36 8. In short, none of the guidelines suggested by the authors are not validated by any research.
We agree with the reviewer that rarely are locally adapted guidelines entirely evidence based and have been adapted by local considerations. The AGREE II study evaluates this under Domain 3 of the AGREE II protocol that specifically addresses this issue as part of assessing the rigour of development. All the CPGs reviewed identified which international diagnostic and treatment guidelines they evaluated for inclusion (eg IADPSG, NICE, ADIPS).