Article Text

Original research
Clinical guidelines of UTIs in children: quality appraisal with AGREE II and recommendations analysis
  1. Binhui Zhu1,2,
  2. Yali Liu3,
  3. Hui Wang1,4,5,
  4. Fan Duan1,2,
  5. Lan Mi1,2,
  6. Ying Liang1,4,5
  1. 1Department 2 of Nephrology, Beijing Children's Hospital, Capital Medical University, National Center for Children's Health, Beijing, China
  2. 2School of Pediatrics, Capital Medical University, Beijing, China
  3. 3Center for Clinical Epidemiology and Evidence-Based Medicine, Beijing Children's Hospital, Capital Medical University, National Center for Children's Health, Beijing, China
  4. 4Beijing Key Laboratory for Pediatric Chronic Renal Diseases and Blood Purification, Beijing Children's Hospital, Capital Medical University, National Center for Children's Health, Beijing, China
  5. 5Key Laboratory of Major Diseases in Children, Ministry of Education, Beijing Children's Hospital, Capital Medical University, National Center for Children's Health, Beijing, China
  1. Correspondence to Dr Hui Wang; wanghui{at}bch.com.cn

Abstract

Objective To explore the current Chinese and English guidelines of urinary tract infection (UTI) in children and provide a summary of the recommendations of the guidelines.

Methods An electronic search was conducted on databases, including Pubmed, SinoMed, Wangfang Data, CHKD,VIP, NICE, WHO, GIN and Medliveto retrieve data of the clinical practice guidelines on UTI from the establishment of the database to June 2020. Four assessors assessed the quality of guidelines using the Appraisal of Guidelines for Research & Evaluation II (AGREE II) and evaluated the specific recommendations in guidelines.

Results (1) Nine guidelines including two from the USA (AAP and A guideline for the inpatient care of children with pyelonephritis) and the remaining from EAU/ESPU, SINEPE, KHA-CARI, CPS, ISPN, NICE and CMA-CSP were explored. (2) The AGREE II evaluation demonstrated higher scores of UTI guidelines in terms of ‘scope and purpose’ (72.99%±11.19%) and ‘clarity of presentation’ (75.62%±7.75%), whereas the average scores were lower in the aspect of ‘stakeholder involvement’ (35.49%±14.41%), ‘rigour of development’ (37.05%±10.05%), ‘applicability’ (37.75%±11.98%) and ‘editorial independence’ (43.06%±48.14%). The average scores of the guidelines were as follows: SINePe (72.57%), CMA-CSP (62.96%), EAU/ESPU (59.61%), AAP (56.86%), NICE (47.54%), CPS (40.93%), KHA-CARI (38.86%), ISPN (38.63%) and A guideline for the inpatient care of children with pyelonephritis (34.72%). (3) All the selected guidelines basically reached a consensus on urine sample retention methods in older children, the antibiotic treatment course and renal and bladder ultrasonography application but lacked a conclusion on the determination of urine culture results, the choice of voiding cystourethrography and Tc-99mdimercaptosuccinicacid, and antibiotic prophylaxis.

Conclusion There remains a need to improve the quality of guidelinesfor UTI in clinical practice. Existing controversies on the current guidelines of UTI in some recommendations warrant further exploration to provide more evidence on formulating more unified and practical guidelines in the future.

Ethics and dissemination No ethical approval is required for this research, as it did not include patients or patient data.

  • Urinary tract infections
  • Protocols & guidelines
  • Clinical governance
  • Paediatric nephrology
  • EPIDEMIOLOGY
  • Paediatric urology

Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

  • To our knowledge, this is the first study that evaluates the quality of guideline of urinary tract infections (UTIs) in children by Appraisal of Guidelines for Research & Evaluation II (AGREE II).

  • Selection bias exists in this study because only guidelines published in English and Chinese were included.

  • AGREE Ⅱ tool does not account for the relative importance of the six domains.

  • AGREE II does not provide high-quality and minimum quality rating criteria.

Introduction

Urinary tract infections (UTIs) are a common clinical infectious disease, especially in infants, with approximately 7% incidence in infants with fever.1 The lack of specific clinical signs and symptoms largely contributes to missed and misdiagnosis of UTI in infants. Current evidence views UTI in childhood as an indicator of numerous renal diseases. UTI, for instance, has been revealed to be the first symptom in 30% of congenital anomalies of the kidney and urinary tract (CAKUT).2 3 Recurrent UTIs and CAKUT can certainly cause renal scars, long-term hypertension, some of which potentially progresses to chronic kidney diseases (CKD) and may have a detrimental effect on the quality of life of children. Therefore, it is imperative to practice early diagnosis, standardised treatment and prevention of relapses to reduce renal scar formation and prevent the occurrence of CKD.

The end of the 20th century has seen many countries issuing and/or updating the guidelines of UTIs in children both aimed to guide decision making among clinicians and reduce waste of resources. A consensus was made in 2007 on the diagnosis and management of UTI by the Chinese expert. However, after decades of development, the paediatric branch of the Chinese Medical Association in 2017 issued evidence-based guidelines of UTIs in children based on the latest research evidence.

In 2009, Appraisal of Guidelines for Research & Evaluation II (AGREE II) was published as a revised version of the original AGREE instrument, and the tool has presently been translated into multiple languages and is receiving wide application to assess the quality of guideline development.4

The present study explores the current UTI guidelines in children, evaluates the quality of guideline by AGREE II and compares and contrasts the selected recommendations of the guidelines aiming to provide a scientific basis for clinical diagnosis and treatment.

Methods

Inclusion and exclusion criteria

Inclusion criteria

  1. The literature includes the diagnosis and treatment of UTI.

  2. The type of literature is clinical guideline/consensus/norm.

  3. The applicable object of the guidelines is limited to children.

  4. The latest version of a set of guidelines that has been updated multiple times.

  5. A complete guideline text, published in English or Chinese.

Exclusion criteria

Guideline interpretations or translations and traditional Chinese Medicine guidelines were excluded, as well as any documents that were not guidelines (such as reviews and other guideline quality appraisal).

Literature search strategy

The following electronic databases will be searched: PubMed, SinoMed (The Chinese biomedical literature database), China Wanfang Digital Database, VIP database, China Hospital Knowledge Database (CHKD), NICE, WHO, GIN and Medlive, from database inception time to June 2020. The search terms included: urinary tract infection, guideline, practice guidelines as the topic, guidelines, guidance, recommendation, consensus, child, preschool, infant, adolescent, newborn, pediat, youth, toddle, teen, boy, girl and baby. Search strategy is in (online supplemental material 1).

Literature screening and data extraction

Two reviewers (BZ and FD) independently screened the articles according to inclusion and exclusion criteria by reading through the title and abstract. A third reviewer (LY) was consulted in the event of disagreement to reach a consensus. The reviewers used Microsoft office 2019 to extract the basic information from the selected guidelines, including title, guideline developers, country, target population, the tool of evidence quality and strength, and number of references.

Quality assessment

The research group comprised four clinicians experienced in paediatric nephrology and had been trained by an expert in evidence-based medicine, and they independently evaluated guidelines. The selected guidelines were independently using the AGREE II, which comprised 23 items in six domains: ‘Scope and purpose’, ‘Stakeholder involvement’, ‘Rigour of development’, ‘Clarity of presentation’, ‘Applicability’ and ‘Editorial independence’. Each item was rated on a seven-point scale from 1 (strongly disagree) to 7 (strongly agree). A scaled domain percentage score was calculated, according to the AGREE II methodology, as follows: Embedded Image. Only the scores in each domain were evaluated because the user manual for AGREE II does not provide high-quality and minimum quality rating criteria.

To resolve discrepancies between the four assessors, a method was used from a previous study: intraclass correlation coefficient (ICC) was calculated to assess inter-rater reliability, ICC values greater than 0.75 demonstrated acceptable stability.5 6

Comparison of recommendations

Recommendations on the diagnosis and treatment of UTIs in children reported in the selected guidelines were extracted and presented in comparative tables focusing on possible gaps and common messages.

Data analysis

Descriptive statistics were conducted in order to characterise the recommendation content. For quantitative data and the guidelines basic information, the statistical analysis was performed using Microsoft Office 2019, and ICCs were calculated by SPSS V.22.0.

Ethics and dissemination

No ethical approval is required for this research, as it did not include patients or patient data.

Results

Guideline selection

Of the 37 full texts retrieved and screened, nine articles were included in this study, including the USA7 8 (two articles), Europe9 (one article), Italy10 (one article), Australia11 (one article), Canada12 (one article), India13 (one article), UK14 (one article) and China15 (one article). The selected guidelines comprised nine medicine societies (AAP, EAU, ESPU, SINePe, KHA-CARI, CPS, ISPN, NICE, and CMA-CSP). Figure 1 (flow diagram of literature retrieval) shows the guideline screening process. Table 1 shows the basic information of the selected guidelines.

Table 1

The basic information of the selected guidelines

Figure 1

Flow diagram of literature retrieval. CHKD, China Hospital Knowledge Database.

Quality assessment

The four assessors independently evaluated the selected guidelines, and the scores are in (online supplemental material 2). Inter-rater reliability of the four assessors total scores was assessed using ICC (table 2). ICC values for all the nine guidelines were greater than 0.75 (0.787~0.925), providing evidence of high consistency in the assessment results between the four assessors. Table 3 illustrate the scores of the AGREE II quality assessment.

Table 2

Intraclass correlation coefficient (ICC)

Table 3

Standardised scores of each domain by AGREE II of guidelines

Scores of the six domains

  1. Scope and purpose: this domain yielded an average score of 72.99%±11.19%. NICE demonstrated the highest score in this domain at 91.76%, while KHA-CARI showed the lowest score at 54.17%.

  2. Stakeholder involvement: this domain yielded an average score of 35.49%±14.41%. SINePe demonstrated the highest score in this domain at 91.76%, while the USA guideline for the inpatient care of children with pyelonephritis showed the lowest score at 12.5%.

  3. Rigour of development: this domain yielded an average score of 37.05%±10.05%. CMA-CSP demonstrated the highest score in this domain at 54.17%.

  4. Clarity of presentation: this domain yielded an average score of 75.62%±7.75%. AAP demonstrated the highest score in this domain at 54.17%.

  5. Applicability: this domain yielded an average score of 37.75%±11.98%. CMA-SINePe demonstrated the highest score in this domain at 54.17%.

  6. Editorial independence: this domain yielded an average score of 43.06%±48.14%. Four guidelines AAP, EAU/ESPU, SINePe and CMA-CSP, scored more than 90%.

Scores of the nine guidelines

The average score for selected guidelines (from high to low) were as follows: SINePe (72.57%), CMA-CSP (62.96%), EAU/ESPU (59.61%), AAP (56.86%), NICE (47.54%), CPS (40.93%), KHA-CARI (38.86%), ISPN (38.63%) and the guideline for the inpatient care of children with pyelonephritis (34.72%).

Summary of recommendations

Similarities

  1. A clean voided midstream urine sample is the preferred method for UTI diagnosis for toilet-trained children.

  2. The choice of antibiotic should be based on resistance patterns of urinary pathogens, and the course of treatment of antibiotics is generally 3–4 days for lower urinary tract infection and 7–14 days for upper UTI.

  3. Renal and bladder ultrasonography (RBUS) is recommended for all children with febrile UTI (except NICE), but voiding cystourethrography (VCUG) and Tc-99mdimercaptosuccinicacid (DMSA) are not regarded as routine examinations.

  4. Antibiotic prophylaxis is not routinely recommended after the first febrile UTI, excepting high-grade VUR.

Differences

  1. Urine collection method for non-toilet-trained children: AAP, EAU/ESPU, ISPN and the guideline for pyelonephritis in the USA are more inclined to bladder catheterisation (BC) and suprapubic aspiration (SPA); other guidelines (except CMA-CSP) recommend urinary bag or clean voided urine (CVU).

  2. The guidelines (except NICE) recommend that the positive urinalysis be dependent on the urine collection method. Table 4 illustrates the specific results.

  3. The recommendations for imaging evaluation slightly differ. Generally, the guidelines of AAP, EAU/ESPU, KHA-CARI, SINePe and CPS do not recommend for the sequence of VCUG and DMSA, while ISPN, NICE, CMA-CSP and the guideline for pyelonephritis in the USA tend to prioritise DMSA examination and give different opinions based on the age of the children. Table 4 illustrates the specific results.

  4. There is no consistent regulation on the dose and course of antibiotic prophylaxis (EAU/ESPU, SINePe, KHA-CARI, CPS, ISPN and CMA-CSP), in particular, AAP and NICE failed to address this aspect. Table 4 illustrates the specific results.

Table 4

Summary of recommendations

Discussion

Quality appraisal of the guidelines for UTIs in children with AGREE II

Nine guidelines have been analysed in this study. The quality appraisal with AGREE II demonstrated that four guidelines (SINePe, CMA-CSP, EAU/ESPU and AAP) exhibited scores greater than 50%. Among the six fields of the AGREE II tool, the scores of domain 1 ‘scope and purpose’ and domain 4 ‘clarity of presentation’ were >70%, while the scores of the other four fields were lower than 50%.

Of note, the rigour of development better reflected the quality of the guidelines. We reported low scores of guidelines in domain 3 (rigour of development), which is consistent with the previous findings by Chen et al.16 In the present study, the Chinese guideline exhibited the highest score in domain 3, particularly because it describes the evidence selection criteria and the search methods for the evidence. Moreover, EAU/ESPU, SINePe and KHA-CARI describe the evidence selection criteria and the methods for formulating the recommendations. There is a previous suggestion that guidelines be updated every 3–5 years.17 The publishing period in the present study is between 2010 and 2019, and five guidelines (ISPN, CPS, KHA-CARI, EAU/ESPU and the guideline for the inpatient care of children with pyelonephritis) had not been updated for more than 5 years.

SINePe demonstrated the highest applicability score because it considered the obstacles in the application of recommendations. For example, this guideline does not recommend a new imaging technique—contrast-enhanced voiding urosonography (ceVUS)—because it is time consuming, expensive and not available on a large scale, despite its high specificity, sensitivity and safety in VUR diagnosis. Other guidelines demonstrated low scores in domain 5 because they omit facilitators, barriers and potential resource implications of its application. These data strongly recommend the need to develop new guidelines, involving health economists, who can fully consider the cost-effectiveness of recommendations, to improve the applicability of the guidelines.

The development of many guidelines is funded by bodies, such as governments, professional associations and pharmaceutical companies. There should be an explicit statement that these bodies have not influenced the recommendations. In the present study, AAP, EAU/ESPU, SINePe and CMA-CSP guidelines provided statements regarding the source of funding and competing interests. Other guidelines did not provide such statements. In future, its importance for guideline developers to clarify their editorial independence to improve credibility of guidelines.

Comparison of recommendations in the selected guidelines

The selected guidelines reached a consensus in urine collection methods for toilet-trained children, RBUS application and treatment courses. However, there is no consensus on the standard of positive urine culture results, the choice of VUCG and DMSA and antibiotic prophylaxis.

Moreover, the criteria for determining the positive or negative results of urine culture varied across urine collection methods. Previously in 2016, a study reported 80% bacterial counts in children with UTI greater than 50 000 CFU/mL18; notably, AAP adopted this criterion. However, because low colony counts can be indicative of a UTI in some circumstances, EAU/ESPU outlines 1000–50000 CFU/mL as the criteria of BC. Swerkersson et al’s study reported similar results whereby 19% bacterial counts lower than 104 CFU/mL were found in the first diagnosis of UTI infants by SPA,19 which is in agreement with CPS.

Regarding the choice for imaging, no uniform opinions have been relayed on the order of VCUG and DMSA. Currently, two approaches, the ‘top-down’ method (DMSA scan and, if positive, VCUG) and the ‘down-top’ method (VCUG and, if positive, DMSA scan), have been described. Our analysis revealed that NICE, CMA-CSP, ISPN and the guideline for pyelonephritis in the USA tend to emulate the ‘top-down’ method, which can allow for early kidney damage assessment. Previous evidence indicates that UTI and renal scarring can occur in patients without VUR, and many renal scars are associated with the fetal period, therefore, may present renal dysplasia.20 In addition, DMSA demonstrates an upstanding predictive ability for high-grade VUR (99% sensitivity), which is why the ‘top-down’ method not only can decrease patient discomfort caused by invasive examinations but also save medical costs.21 Contrarily, the VUR-based ‘down-top’ method is strongly associated with renal scar formation and UTI recurrence. As such, early diagnosis and treatment of VUR are imperative in preventing further kidney damage. It is of particular note that the guideline of the American College of Radiology in 2017 recommends this method but does not recommend DMSA regardless of children’s age.22 Overall, in clinical practice, whether to apply the ‘top-down’ or ‘down-top’ approach should decide based on actual situation of patients.

SINePe guideline outlined a new imaging technique—ceVUS—that shows the ureter and bladder in real-time using ultrasound contrast agents. Compared with VCUG, it has the advantages of real-time imaging, radiation free and favourable safety profile. Mounting evidence shows that ceVUS is highly sensitive (80%–100%) and accurate (77%–86%) in the diagnosis of VUR, and the diagnosis agreement between VCUG and ceVUS is greater than 70%.23–27 EFSUMB (European Federation of Societies for Ultrasound in Medicine and Biology) guideline and Chinese expert consensus about VUR recommend the application of ceVUS in the following situations: (1) first examination for VUR in girls; (2) follow-up examinations for VUR in girls and boys after conservative or surgical therapy; and (3) screening high-risk patients for reflux.28 29 In this view, ceVUS holds promise as a preferred choice for VUR screening in the future.

For antibiotic prophylaxis, most selected guidelines (except NICE and AAP) recommend its application for high-grade VUR, but no consensus has been made on the regulation of the dose and course. SINePe suggests one-quarter to one-third of the treatment dose and duration of prophylaxis is 12‐24 months in girls and 6‐12 months in boys; however, ISPN has a different view that antibiotic prophylaxis can be until 1 year old for VUR Ⅰ–Ⅱ and 5 years old for VUR Ⅲ–Ⅳ. Meanwhile, we cannot ignore some dissents of benefit and risk assessment of antibiotic prophylaxis. Some researchers revealed that long-term antibiotic prophylaxis plays no or insignificant role in preventing UTI recurrence but increases the risk of resistant strains significantly.30 31 These data suggest the need to discuss and tailor the decision to use antibiotic prophylaxis in an individualised fashion.

Study limitations

First, selection bias exists in this study because only guidelines published in English and Chinese were included. Second, we may have missed some guidelines because of the limitation of the search strategy. Third, AGREE Ⅱ tool has some limitations, for example, it does not account for the relative importance of the six domains, domain 3 (rigour of development) is considered of equal importance to other domains. There is a need to carefully consider the recommendations if the scores for domain 3 are low.

In conclusion, although the current guidelines of UTI in children can be adopted in clinical practice, their qualities are uneven. As such, there is a whole lot of room for improvement, especially in the areas of the rigour of development, applicability, editorial independence and stakeholder involvement. Additionally, the existing controversial opinions warrants continued exploration to provide powerful evidence.

Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information.

Ethics statements

Patient consent for publication

Ethics approval

Not applicable.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • Contributors HW is the guarantor for the article. BZ, YL and HW conceptualised and designed the study, carried out the initial analysis and drafted the initial manuscript. BZ, FD, LM and YL used Appraisal of Guidelines for Research & Evaluation II (AGREE II) instrument to evaluate the retrieved guidelines. YL trained four reviews on the use of AGREE II instrument. All authors approved the fnal manuscript as submitted and agree to be accountable for all aspects of the work.

  • Funding Funding this study is supported by Beijing Municipal Science & Technology Commission (No. Z191100006619062) and Special Foundation for National Science and Technology Basic Research Program of China (2019FY101200).

  • Competing interests None declared.

  • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.