Article Text

Download PDFPDF

Original research
Quality appraisal of clinical guidelines for venous thromboembolism prophylaxis in patients undergoing hip and knee arthroplasty: a systematic review
  1. Yu Wang1,
  2. Li-Yun Zhu1,
  3. Hai-Bo Deng2,
  4. Xu Yang3,
  5. Lei Wang4,
  6. Yuan Xu3,
  7. Xiao-Jie Wang5,
  8. Dong Pang6,7,
  9. Jian-Hua Sun8,
  10. Jing Cao1,
  11. Ge Liu9,
  12. Ying Liu10,
  13. Yu-Fen Ma11,
  14. Xin-Juan Wu1
  1. 1Department of Nursing, Chinese Academy of Medical Sciences - Peking Union Medical College, Peking Union Medical College Hospital, Beijing, China
  2. 2Department of Cardiac Surgery, Chinese Academy of Medical Sciences - Peking Union Medical College, Peking Union Medical College Hospital, Beijing, China
  3. 3Department of Orthopedic Surgery, Chinese Academy of Medical Sciences - Peking Union Medical College, Peking Union Medical College Hospital, Beijing, China
  4. 4Department of Vascular Surgery, Chinese Academy of Medical Sciences - Peking Union Medical College, Peking Union Medical College Hospital, Beijing, China
  5. 5Department of Breast Surgery, Chinese Academy of Medical Sciences - Peking Union Medical College, Peking Union Medical College Hospital, Beijing, China
  6. 6School of Nursing, Peking University, Beijing, China
  7. 7Evidence-Based Nursing: A Joanna Briggs Institute Centre of Excellence, Peking University Health Science Centre, Beijing, China
  8. 8Intensive Care Unit, Chinese Academy of Medical Sciences - Peking Union Medical College, Peking Union Medical College Hospital, Beijing, China
  9. 9Department of Neurological Surgery, Chinese Academy of Medical Sciences - Peking Union Medical College, Peking Union Medical College Hospital, Beijing, China
  10. 10Department of General Surgery, Chinese Academy of Medical Sciences - Peking Union Medical College, Peking Union Medical College Hospital, Beijing, China
  11. 11Outpatient Department, Chinese Academy of Medical Sciences - Peking Union Medical College, Peking Union Medical College Hospital, Beijing, China
  1. Correspondence to Dr Xin-Juan Wu; wuxinjuan{at}sina.com; Dr Yu-Fen Ma; yumafen{at}126.com

Abstract

Introduction Venous thromboembolism (VTE) occurs in up to 40%–80% of patients after hip and knee arthroplasty. Clinical decision-making aided by guidelines is the most effective strategy to reduce the burden of VTE. However, the quality of guidelines is dependent on the strength of their evidence base. The objective of this article is to critically evaluate the quality of VTE prevention guidelines and the strength of their recommendations in VTE prophylaxis in patients undergoing hip and knee arthroplasty.

Methods Relevant literature up to 16 March 2020 was systematically searched. We searched databases such as Web of Science, PubMed, EMBASE, Cumulative Index of Nursing and Allied Health Literature, China National Knowledge Infrastructure and WanFang and nine guidelines repositories. The identified guidelines were appraised by two reviewers using the Appraisal of Guidelines for Research and Evaluation II and appraised the strength of their recommendations independently. Following quality assessment, a predesigned data collection form was used to extract the characteristics of the included guideline.

Results We finally included 15 guidelines. Ten of the included guidelines were rated as ‘recommended’ or ‘recommended with modifications’. The standardised scores were relatively high in the domains of Clarity of Presentation, and Scope and Purpose. The lowest average standardised scores were observed in the domains of Applicability and Stakeholder Involvement. In reference to the domains of Rigour of Development and Editorial Independence, the standardised scores varied greatly between the guidelines. The agreement between the two appraisers is almost perfect (intraclass correlation coefficients higher than 0.80). A considerable proportion of the recommendations is based on low-quality or very-low-quality evidence or is even based on working group expert opinion.

Conclusions In summary, the majority of the recommendations are based on low-quality evidence, and further confirmation is needed. Furthermore, guideline developers should pay more attention to methodological quality, especially in the Stakeholder Involvement domain and the Applicability domain.

  • thromboembolism
  • quality in healthcare
  • hip
  • knee
http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

  • Our research critically evaluated the quality of guidelines for prevention of venous thromboembolism (VTE) in patients undergoing elective hip and knee arthroplasty and the strength of their recommendations in VTE prophylaxis.

  • Two appraisers used Appraisal of Guidelines for Research and Evaluation II, an assessment with methodological rigour and reliability, to appraise the quality of included guidelines and resolved any discrepancies by discussion.

  • Our search strategy was also reproducible; however, because of language or publication restrictions, there may be a language barrier.

Introduction

Total knee arthroplasty (TKA) and total hip arthroplasty (THA) are widely regarded as effective treatment options for patients with joint failure, which can help alleviate pain and improve function.1–3 Despite considerable advances in surgical and anaesthetic techniques, patients undergoing TKA and THA are at high risk of venous thromboembolism (VTE), manifesting as deep vein thrombosis or pulmonary thromboembolism.4 VTE is a severe postoperative complication, which commonly occurs in 40%–80% of patients undergoing THA and TKA.5 VTE is a potentially preventable medical condition that can prolong hospital stays and increase mortality.6 Despite the cost-effectiveness of THA and TKA, in-hospital cost and rehabilitation cost associated with hospital-acquired VTE place significant burdens on global healthcare systems.7

Using evidence-based VTE programmes can improve practice outcomes while reducing the physical, psychological, social and economic burden on individuals, families and countries. Clinical practice guidelines (CPGs) enable health professionals and patients to make the best decisions about treatment or care for a particular condition or situation and reduce waste. However, the quality of a CPG is dependent on the strength of its evidence base.8 As such, there is a need to evaluate CPGs to assess their quality. Therefore, we undertook this systematic review to evaluate the quality of the CPGs and the strength of their recommendations in VTE prophylaxis.

Methods

Objectives

The purpose of this systematic review is to critically appraise the quality of VTE prevention guidelines specific to the patients after THA and TKA. The Appraisal of Guidelines for Research and Evaluation II (AGREE II) tool was used. We wrote this study following Preferred Reporting Items for Systematic Review and Meta-Analysis 2009 statement9 (see online supplemental table 1).

Data sources and search strategy

Academic databases, including Web of Science, PubMed, EMBASE, Cumulative Index of Nursing and Allied Health Literature, and Chinese databases (China National Knowledge Infrastructure and WanFang), were searched from inception until 16 March 2020. The search strategy was tailored to the requirements of each database. Searching of reference lists from identified papers was carried out along with forwarding citation searching using Google Scholar. All searches were saved in each database and imported into EndNote (V.X9; Clarivate Analytics), where duplicates were removed. To supplement our database searches, we also searched guidelines repositories, including CPG Infobase: Clinical Practice Guidelines (Canadian Medical Association), the Guidelines International Network, the National Health and Medical Research Council—Australian Clinical Practice Guidelines, the National Institute for Health and Care Excellence (NICE), the National Guideline Clearinghouse, Scottish Intercollegiate Guideline Network, New Zealand Guidelines Group, BMJ Best Practice and Chinese guidelines repository (YiMaiTong). Details of the searches are provided in online supplemental appendix 1.

Eligibility criteria

A complete list of inclusion and exclusion criteria is detailed in table 1.

Table 1

Inclusion and exclusion criteria

Data screening and extraction

Two reviewers used prespecified eligibility criteria to screen titles and abstracts. Articles that met the above inclusion and exclusion criteria were included for a second full-text screen. Conflicts were resolved through discussion or the involvement of a third reviewer. Reasons for exclusion were documented in a tabular format (online supplemental appendix 2). Data extraction was then performed independently using a standardised data extraction form developed based on AGREE II.10

Quality assessment of CPGs

To evaluate the quality of pre-existing guidelines selected for guideline adaptation, two reviewers graded each guideline according to AGREE II.11 This instrument consists of 23 items organised into six domains. AGREE II also includes two overall assessment items for overall judgements of the practice guideline. Online supplemental appendix 3 provides a brief description of each domain.

The 23-item AGREE II tool uses a seven-point agreement scale from 1 (strongly disagree) to 7 (strongly agree).10 Standardised scores for each domain were computed as (X/Y) ×100%, where X = obtained score−minimum possible score and Y = maximum possible score−minimum possible score.10 As defined by AGREE II, we considered a CPG as ‘recommended’ if it scored above 50% on ≥4 domains, as ‘recommended with modifications’ if it scored above 50% on 3 domains and as ‘not recommended’ if it scored less than 50% on ≥4 domains.

Before the quality appraisal using AGREE II, two reviewers completed an Online Training Tool12 and performed calibration exercises to clarify the eligibility criteria. Following training, the two reviewers independently applied AGREE II criteria to eligible CPGs using the My AGREE PLUS online platform.13 Our team met regularly to resolve any discrepancies in the quality appraisal. We used intraclass correlation coefficients (ICCs) to measure the agreement between the two assessors’ assessment of quality (AGREE II) of included CPGs. The results were interpreted as follows: 0.00, poor agreement; 0.00–0.20, slight agreement; 0.21–0.40, fair agreement; 0.41–0.60, moderate agreement; 0.61–0.80, substantial agreement; and 0.81–1.00, almost perfect agreement.14

Results

The electronic database search retrieved 4808 citations. We retrieved and assessed the full texts of 42 promising reports, and among these, we excluded 32 (figure 1). The guidelines repositories search retrieved 327 citations, of which 317 full texts were excluded (figure 2). In total, 15 guidelines were included in the final analysis, and the detailed characteristics are shown in table 2. These CPGs were published between 2006 and 2019. Most of the CPGs were developed in the USA (n=3),15–17 with the remaining coming from China (n=1),18 the UK (n=1),19 France (n=1),20 Poland (n=1),21 Malaysia (n=1),22 Korea (n=1),23 Italy (n=1),24 Scotland (n=1)25 and Southern Africa (n=1),26 or from Asia (n=1),27 Europe (n=1)28 or International (n=1).29 Information sources regarding where CPGs were obtained are shown in online supplemental appendix 4.

Table 2

Characteristics of CPGs regarding VTE prevention in patients undergoing THA or TKA

Figure 1

Search strategy for library databases (final search undertaken on 16 March 2020). CPGs, clinical practice guidelines; CINAHL, Cumulative Index of Nursing and Allied Health Literature; WOS, Web of Science; CNKI, China National Knowledge Infrastructure.

Figure 2

Search strategy for guideline repositories (final search undertaken on 16 March 2020). CPGs, clinical practice guidelines.

Two assessors appraised each CPG. The AGREE II domain scores of each guideline are presented in table 3. Detailed scoring of each AGREE II item under each domain is presented in online supplemental appendix 5. Online supplemental figure 1 shows a radar chart of the results of the guideline appraisal. The quality of the evaluated guidelines showed significant variability. The standardised scores ranged from 50% to 100% in the Scope and Purpose domain, and all CPGs scored above 50%. The standardised scores in the Stakeholder Involvement domain ranged from 3% to 89%, with 6 of 15 CPGs scoring above 50%. The standardised scores in the Rigour of Development domain ranged from 16% to 98%, with 8 of 15 CPGs scoring above 50%. The standardised scores in the Clarity of Presentation domain ranged from 42% to 100%, with only one CPG scoring below 50%. The standardised scores in the Applicability domain ranged from 4% to 94%, with only 2 of 15 CPGs scoring above 50%. The standardised scores in the Editorial Independence domain ranged from 0% to 92%, with 8 of 15 CPGs scoring above 50%. Per the quality assessment tool used in this review, 6 of the 15 included CPGs were judged to be ‘recommended’. There is an almost perfect agreement between two appraisers, with the ICC ranging from 0.875 to 0.955.

Table 3

AGREE II scaled domain scores of CPGs for VTE prevention in patients undergoing THA or TKA

Table 4 shows the levels of evidence for recommendations of VTE prevention in patients undergoing THA or TKA, as reported in the included CPGs. There are four CPGs developed using the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) system to rank recommendations.15 16 23 28 Comparatively, four CPGs were developed based on expert opinion.18 26 27 29 Despite unanimous agreement in the recommendations for providing pharmacological and/or mechanical prophylaxis, early or delayed prophylaxis, and extended duration of prophylaxis, details disagree on the pharmacological and mechanical prophylaxis choice, time of early or delayed prophylaxis, and duration of prophylaxis. The American College of Chest Physicians (ACCP) 2012 guidelines,16 European Society of Anaesthesiology (ESA) 2017 guidelines28 and French Society for Anaesthesiology and Intensive Care (FSAIC) 2006 guidelines20 recommended low-molecular-weight heparin (LMWH) as a preference pharmacological prophylaxis choice, whereas direct oral anticoagulants (DOACs) were recommended in the American Society of Hematology (ASH) 2019 guidelines.15 An extended duration of thromboprophylaxis of 35 days in patients undergoing THA and 14 days in patients undergoing TKA seemed to be the primary choice.16 18 19 21 26 In terms of improving CPG implementation, patient/family education, type of anaesthesia, risk assessment and bridging therapy, we observed little recommendations with very low quality. The recommendations from each CPG that are informed in table 4 are detailed in online supplemental appendix 6. Online supplemental appendix 7 shows an explanation of the different evidence levels used across included CPGs.

Table 4

Levels of evidence for recommendations of VTE prevention in patients undergoing THA or TKA as reported in included CPGs

Discussion

To our knowledge, this is the first systematic quality appraisal of CPGs for VTE prevention in patients undergoing THA and TKA. Finally, 15 guidelines were recognised. Generally, the quality of 67% (10/15) of included guidelines was acceptable and evaluated as ‘recommended’ or ‘recommended with modifications’. The included CPGs were consistent in the recommendations, whereas they used different classification systems in indicating the levels of evidence. The data availability of trials and the timing of approval by regulatory agencies may also explain some differences in the preferred pharmacological prophylaxis (such as LMWH or DOACs). It is worth noting that a considerable proportion of the recommendations is based on low-quality or very-low-quality evidence or is even based on working group expert opinion, representing uncertain clinical significance. Therefore, high-quality randomised controlled trials are needed to support the evidence and potentially improve the cost-effectiveness of treatment.30 Notably, in terms of patient/family education and improving CPG implementation, very few strong recommendations were identified, indicating a lack of robust evidence. These findings would explain why VTE prophylaxis is still not routinely administered as guideline recommended in most hospitals.31 32

The standardised scores varied between different domains. In the Scope and Purpose domain and the Clarity of Presentation domain, the standardised scores were relatively high. In reference to the Rigour of Development domain and Editorial Independence domain, the standardised scores varied considerably between the CPGs. Our results are consistent with the results of other CPG quality appraisal focusing on different clinical topics.33 34 Marked improvements in CPG development methodology over the past decade may have a role in explaining the variance scores. Moreover, guideline development should be carried out according to the formulated plan, such as the WHO Guideline Development Handbook.35 It is also recommended to report methodological details for clinical guideline development based on AGREE II.36

We found that the domains of Stakeholder Involvement and Applicability were marked with the lowest standardised scores, which may be factors influencing implementation. Stakeholder involvement focuses on gaining support from a strong collaborative multidisciplinary network and obtaining the needs of all the potential users.37 Indeed, a multidisciplinary approach to VTE prevention involving key stakeholders is essential for putting recommendations into practice.19 However, only three CPGs included patients and their representatives in guideline development.15 19 25 Evidence-based medicine highlights the importance of patient-centred communication.38 Patient values and preferences should be taken into account, and the pros and cons of these options should be discussed with the patient.39 Therefore, guideline developers should consider the involvement and engagement of patients and the public in future CPG updates.

Guideline applicability is exceptionally critical for implementation. However, there is a lack of consensus on how CPG should be done in practice. Only two CPGs appraise the barriers and facilitators to guideline implementation and provide strategies to improve guideline uptake.16 19 Putting recommendations into practice is always challenging. Examples of multiple evidence-based implementation strategies for preventing VTE have been evaluated, such as computerised reminder systems, education, audit and feedback, and distribution of guidelines.40–44 Two published Cochrane systematic reviews have reported the interventions for implementing thromboprophylaxis in hospitalised patients at risk of VTE.45 46 We call researchers to add the Improve CPG Implementation domain as one of the pillars in guideline development.

This review has some strengths and weaknesses. First, our search strategy was developed with an experienced senior librarian. Our search strategy was also reproducible, as required by systematic reviews of published work. However, because of language or publication restrictions, we may miss some CPGs. Second, the CPGs we included range from 2006 (FSAIC) to 2019 (ASH and NICE). CPGs that are ‘recommended’ based on the AGREE II scoring could be obsolete if the CPGs are derived from outdated evidence. Therefore, some caution is warranted here. Finally, two appraisers used AGREE II, an assessment with methodological rigour and reliability, to appraise the quality of included guidelines and resolved any discrepancies by discussion. Although the appraisers were inexperienced in guideline evaluation, all had completed the AGREE II online training. Besides, the team members met weekly online to discuss progress and problems. And six of our group members have attended the Joanna Briggs Institute (JBI) evidence-based medicine training courses.

Conclusions

In summary, the majority of the recommendations are based on inadequate evidence quality, and further confirmation is needed. Furthermore, guideline developers should pay more attention to methodological quality, especially in the Stakeholder Involvement domain and the Applicability domain. Finally, improving CPG implementation and sustainability should also be carefully considered in CPG development.

References

Footnotes

  • Contributors YW, LYZ, YX, XJW, JHS, HBD, YFM and XJW designed the study, critically appraised the guidelines and collected the data. XY, LW, JC, GL and YL collected the data. DP interpreted the data. YW, LYZ, YX, XJW and JHS wrote the first draft. XY, LW, JC, DP, GL, YL and HBD conducted the systemic review and revised the manuscript. All authors contributed to subsequent versions and approved the final manuscript.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests None declared.

  • Patient consent for publication Not required.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data availability statement All data relevant to the study are included in the article or uploaded as supplementary information. The data set supporting the conclusions of this article is included in the article.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.