Article Text
PDF
Abstract
Objective To systematically summarise acupuncture-related Clinical Practice Guidelines (CPGs)’s clinical and methodological characteristics and critically appraise their methodology quality.
Design We summarised the characteristics of the guidelines and recommendations and evaluated their methodological quality using the Appraisal of Guidelines Research and Evaluation II (AGREE II) instrument.
Data sources Nine databases were searched from 1 January 2010 to 20 September 2020.
Eligibility criteria for selecting studies We included the latest version of acupuncture CPGs, which must have used at least one systematic review addressing the benefits and harms of alternative care options to inform acupuncture recommendations.
Data extraction and synthesis Reviewers, working in pairs, independently screened and extracted data. When there are statistical differences among types of CPGs, we reported the data by type in the text, but when not, we reported the overall data.
Results Of the 133 eligible guidelines, musculoskeletal and connective tissue diseases proved the most commonly addressed therapeutic areas. According to the AGREE II instrument, the CPG was moderate quality in the domain of clarity of scope and purpose, clarity of presentation, the rigour of development, stakeholder involvement and low quality in editorial independence, and applicability. The study identified 433 acupuncture-related recommendations; 380 recommended the use of acupuncture, 28 recommended against the use of acupuncture and 25 considered acupuncture but did not make recommendations. Of the 303 recommendations that used Grading of Recommendations Assessment, Development and Evaluation to determine the strength of recommendations, 152 were weak recommendations, 131 were strong recommendations, of which 104 were supported by low or very low certainty evidence (discordant recommendations).
Conclusion In the past 10 years, a large number of CPGs addressing acupuncture interventions exist. Although these guidelines may be as or more rigorous than many others, considerable room for improvement remains.
- complementary medicine
- health economics
- public health
Data availability statement
Data are available on reasonable request. The full dataset and statistical code will be available on reasonable request by emailing YZ.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This systematic survey includes a comprehensive search of eligible clinical practice guidelines (CPGs) and systemic and explicit eligibility criteria.
This study used the Appraisal of Guidelines Research and Evaluation II instrument to evaluate the methodological of eligible CPGs.
This study investigated the utilisation of the Grading of Recommendations Assessment, Development and Evaluation in CPGs.
This study documented recommendations discordant with respect to evidence strength and certainty.
Limitations include the exclusion of guidelines not supported by any systematic reviews; our findings, therefore, do not apply to the lowest methodological quality guidelines.
Introduction
With more than 3000 years of history, acupuncture is one of the widely used complementary and alternative therapies.1 2 Among the 192 WHO member states, 183 have used acupuncture.3 In Norway, 34%–64% of doctors recommend acupuncture to their patients.4 The increasing acupuncture application in practice has led to growing demands for clinical practice guidelines (CPGs), with many countries and academic organisations including acupuncture in their CPGs.5 6
Trustworthy CPGs base their recommendations on systematic reviews (SRs) assessing the benefits and harms of alternative care options, rate the certainty of the evidence and grade the strength of recommendations.7 8 CPGs help clinicians, managers and policy-makers select the best available evidence to support decision making. Despite the existing large number of acupuncture-related CPGs, the clinical and methodological status of CPGs, including recommendations, remain underexplored.
This systematic survey summarises acupuncture-related CPGs’ clinical and methodological characteristics and critically assesses their methodology quality.
Methods
Definition
Acupuncture and acupuncture point
In this study, acupuncture is used in a broad sense refers to interventions that use any stimulation on acupuncture points, including manual acupuncture, electric acupuncture (electroacupuncture), acupressure, moxibustion, warm needling, fire needling, transcutaneous electrical nerve, laser acupuncture, microsystem acupuncture, thread-embedding therapy, medicine acicula, point injection, acupoint paste; magnetic acupuncture, blood-letting therapy, acupotomy.9
Acupuncture point: the point in humans where stimulation and manipulation are performed in acupuncture therapies.10
Types of CPG
This study divides eligible CPGs into three types: acupuncture-specific CPG (all recommendations include acupuncture interventions), traditional and complementary medicine (T&CM) CPG (all recommendations include T&CM interventions), and Comprehensive CPG (all recommendations include conventional interventions, and some include acupuncture interventions).
Acupuncture recommendation
We define acupuncture recommendations as recommendations (including for, against or considered but did not make recommendations) in which authors considered acupuncture as a treatment or prophylactic (eg, prevent nausea and vomiting after chemotherapy) option.
Conventional medicine
Conventional medicine is defined as pharmacological, and other non-pharmacological used in conventional medicine systems to treat, prevent disease, or restore, correct, or modify physiological function.11
Outcome classification
We categorised outcomes into symptoms (eg, pain, insomnia); function (eg, poststroke motor function); surrogate outcomes (eg, blood pressure); quality of life (eg, short Form 36 survey); morality and major morbid events (eg, myocardial infarction).12
Eligibility criteria
Eligible CPGs met all the following criteria: (1) title or abstract included keywords ‘CPG’ or ‘guideline’ or ‘guidelines’ or ‘guidance’; If the CPG had no abstract, introduction, scope, purpose, rationale, background or objectives presented at least one of these keywords; (2) full text included ‘acupuncture’ or ‘acupressure’ or ‘transcutaneous electrical nerve’ ; (3) the CPG used at least one SRs addressing the benefits and harms of alternative care options to inform recommendations; (4) the CPG addressed patient prevention or treatment; (5) the CPG included at least one recommendation addressing acupuncture and (6) the latest version of a CPG when multiple versions exist. This study excluded health technology assessment.
Literature search
LL, HZ and YZ developed the search strategy. The keywords included guideline, recommendation, advice, opinion, acupuncture, transcutaneous electrical nerve stimulation, acupressure, etc (detail in online supplemental material 1). Two reviewers (ZC and XS) systematically searched databases including MEDLINE, EMBASE, Trip medical database, the Guidelines International Network, Agency for Healthcare Research and Quality, National Institute for Health and Care Excellence, Canadian Medical Association’s CPGs database, the New Zealand Ministry of Health, China National Knowledge Internet (CNKI), Wanfang Data, China Biology Medicine, and China Science and Technology Journal Database from 1 January 2010 to 20 September 2020. Without language restriction, we searched subject terms and free words. The reviewers managed records using EndNote (V.X9.0).
Supplemental material
CPGs selection, data extraction and analysis
Reviewers, working in pairs, independently screened and extracted data. After title and abstract screening, reviewers retrieved and evaluated potentially eligible guidelines’ full texts, resolving disagreement by discussion or consultation with a third reviewer (XS). To collect data, we designed a structured data extraction table in Excel.
Reviewers extracted two types of information from each CPG: (1) CPG level information, for example, region of the publication, types of CPG, target users, and diseases or conditions investigated, (2) recommendation level information, for example, strength and direction of the recommendation, intervention (eg, types of acupuncture and acupoints), comparators included in the recommendation, outcomes informing the recommendation.
To compare the variables across different types of CPGs, we used the χ2 and, when necessary, Fisher’s exact test. When there are statistical differences among types of CPGs, we reported the data by type in the text, but when not, we reported the overall data. We showed all data in the tables. This study conducted the analyses using Statistical Package for the Social Sciences (SPSS, V.22) and set the significance level at 5% (p<0.05).
Quality appraisal
The study used the Appraisal of Guidelines Research and Evaluation II (AGREE-II) instrument, which includes 23 appraisal criteria (items) organised within six domains: scope and purpose, stakeholder involvement, the rigour of development, clarity of presentation, applicability, and editorial independence, to assess the quality of CPGs. Reviewers rated each item on a seven-point scale from strongly disagree (score 1) to strongly agree (score 7). We calculated the scaled domain score (higher score indicates better quality) according to AGREE II (the ‘obtained score’ was the sum of the appraisers' scores per item)13: (Obtained score − Minimum possible score) / (Maximum possible score − Minimum possible score). Prior to quality appraisal, eight reviewers received training in AGREE II with repeated assessment until the chance-corrected agreements were ≥0.6 for each pair of reviewers in each domain. Two reviewers (ZC and XT) independently assessed each CPG, and when the discussion failed to resolve the disagreement, a third reviewer (XS) arbitrated the case.
AGREE II instrument does not set minimum domain scores or patterns of scores across domains to differentiate between high quality and low quality guidelines. As previously suggested, we considered domain and overall scores under 50% to indicate low quality, >50% as moderate quality and >80% as high quality.14–16
Patient and public involvement
No patient involved.
Results
Literature search
Our initial search yielded 8943 citations, of which 3122 were excluded due to duplicate publication and 4848 deemed ineligible. Of the 973 full texts, we excluded 840 reports for the following reasons: 793 did not include any recommendation regarding acupuncture, 28 were not CPGs, 16 had unavailable full texts and 3 were duplicate publications. The 133 CPGs deemed eligible, including 433 acupuncture recommendations (figure 1).
The Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram. CPG, clinical practice guideline.
Characteristics of included CPGs and recommendations
Table 1 and online supplemental material 2 present eligible CPGs’ summary characteristics.
Supplemental material
online supplemental material 2 and 3 are reversed, please change them.Characteristics of 133 included CPG in the study
The organisations that conducted CPGs were from North America (49, 36.8%), Asia (44, 33.2%), Europe (31, 23.3%), Oceania (6, 4.5%) and international organisations (eg, WHO) (3, 2.2%). Guideline category proved comprehensive in 89 (66.9%), acupuncture-specific CPGs in 31 (23.3%) and T&CM specific in 13 (9.8%). Sixty-nine (51.9%) used Grading of Recommendations Assessment, Development and Evaluation (GRADE) to assess the certainty of the evidence, and 59 (44.4%) used GRADE to assess the strength of recommendation.
Figure 2 shows CPGs and recommendations’ therapeutic areas distribution, quantity, and methodological rigour. Areas most frequently addressed proved musculoskeletal and connective tissue diseases (39, 29.3%), neurological disorders (21, 15.8%) and oncology (11, 8.3%). CPGs seldom addressed endocrine, nutritional and metabolic, ophthalmology, cardiovascular diseases, respiratory diseases and dermatology (one per therapeutic area).
A multidimensional plot for acupuncture guidelines and recommendations’ quantity, methodological rigour and distribution of therapeutic areas. Each pie represents a therapeutic area. (1) Pie size: number of CPGs (larger=more studies). (2) Pie colour: direction of recommendation (green: recommend for, red: recommend against; pink: did not make a recommendation). CPG, clinical practice guideline.
Methodological quality of CPGs
Table 2 presents CPGs AGREE II rating. Of all CPGs, domains evaluated by AGREE Ⅱ scored from the highest to lowest include: ‘scope and purpose,’ ‘clarity of presentation,’ ‘rigour of development,’ ‘stakeholder involvement’, ‘editorial independence,’ and ‘application.’ The first four domains proved moderate quality (ie, 50% or higher rating), and the last two domains deemed low quality (ie, scored less than 50%).
The methodological quality of 133 CPGs (%)
For comprehensive CPGs, ‘clarity of presentation’ (75.2%), ‘scope and purpose’ (74.0%), ‘editorial independence(53.4%)’ and ‘rigour of development’ (51.6%)proved moderate quality; meanwhile ‘stakeholder involvement(49.8%)’, and ‘application(32.1%)’ proved lower quality. For T&CM CPGs, ‘clarity of presentation’ (68.2%), and ‘scope and purpose’ (66.0%) were moderate quality; meanwhile ‘editorial independence’ (48.7%), ‘rigour of development’ (45.9%), ‘stakeholder involvement’ (32.7%) and ‘application’ (22.4%) were lower quality. For acupuncture-specific CPGs, ‘scope and purpose’ (95.2%) was high quality; and ‘clarity of presentation’ (79.8%), ‘rigour of development’ (71.3%), and ‘stakeholder involvement’ (67.8%) were moderate quality; meanwhile ‘application’ (35.4%) and ‘editorial independence’ (33.2%) were lower quality.
The low rating in the ‘application’ domain reflects inadequate consideration of resource use (52, 39.1%) and monitoring/auditing standards. The absence of a conflict-of-interest (COI) statement (61, 45.9%) and lack of explicit explanation on how COIs are considered (58,43.6%) from CPGs developers led to the low rating in the ‘editorial independence’ domain. In addition to item 5 in AGREE II, whether guideline developers considered patients’ values and preferences, we evaluated whether the CPG explicitly explained how they considered it. The results showed that 43.6% reported considering patients' values and preferences, of which only less than 10.5% explicitly explained how they were considered (online supplemental 3 table A).
Supplemental material
Acupuncture recommendations specific analyses
Evidence supporting acupuncture recommendations
Table 3 shows the evidence that supported acupuncture recommendations. Type of study design used to inform acupuncture recommendations from the most to the least frequently are randomised controlled trial and controlled clinical trial, SR and meta-analysis, observational study, narrative review, expert consensus and Chinese classic texts/textbook. Of all acupuncture recommendations, 314 (72.5%) used GRADE to assess the certainty of evidence. Low or very low certainty evidence supported over 80% of the 314 acupuncture recommendations.
Evidence supports acupuncture recommendations
Direction, strength and rationale of the recommendations
Of the 433 acupuncture recommendations, 380 (87.8%) recommended or suggested using acupuncture, 28 (6.4%) recommended or suggested against, and 25 (5.8%) did not make recommendations. Of the 303 (70.0% of all recommendations in the guidelines) recommendations that used GRADE to determine the strength of recommendations, 152 (50.2%) were weak recommendations, and 131 (43.2%) were strong recommendations, of which 104 (79.4%) were supported by low or very low certainty evidence (we will refer to this situation as discordant recommendations15 (table 4).
Direction and strength of acupuncture recommendations
online supplemental 3 table B shows the rationale when making acupuncture recommendations. For all CPGs, treatment benefits are the main reason for recommending or against acupuncture (264, 61.0%). Forty-one (36.3%) comprehensive CPGs consider the certainty of evidence as an important issue in making recommendations, while T &CM and acupuncture-specific CPGs seldom consider certainty (1, 0.4%). Less than 10% of all CPGs reported patients’ values and preferences, cost, adverse events and feasibility as part of their rationale for making acupuncture recommendations.
Patient characteristics in acupuncture recommendations
Acupuncture recommendations report patient information poorly. Of all acupuncture recommendations, only 16 (8.7%) reported the severity of the disease. Seventeen (15.0%), 3 (4.5%), and 101 (39.8%) of the comprehensive, T & CM and acupuncture-specific CPGs reported the stage of the disease (online supplemental 3 table C).
Acupuncture interventions in the recommendations and remarks
Acupuncture recommendations and remarks reported limited details regarding acupuncture intervention (table 5). Comprehensive CPGs most frequently include manual acupuncture and acupressure in acupuncture recommendations. T&CM and acupuncture-specific CPGs most frequently include manual and electro-acupuncture in acupuncture recommendations. Most acupuncture recommendations in all CPGs focus on stand-alone usage of acupuncture therapies, and only 99 (22.9%) recommendations use acupuncture as a combination/ adjunct therapy. Moreover, almost no recommendations reported the acupuncture points selection except for acupuncture-specific CPGs (206, 81.1%).
Details of acupuncture interventions in the recommendations and remarks
Comparators in acupuncture recommendations
Approximately three-quarters of the CPGs recommendations (323, 74.6%) did not report the comparators (online supplemental 3 table D). Among all CPGs recommendations that reported the comparators, comprehensive CPGs most often compared acupuncture with no acupuncture (12, 10.6%), T&CM CPGs most often compared acupuncture with non-pharmacological therapy (22, 33.3%), and acupuncture-specific CPGs most often compared acupuncture with conventional medicine (32, 12.6%).
Outcomes summarised to inform acupuncture recommendations
Among all acupuncture recommendations, 130 (30.0%) did not summarise the outcomes used to inform recommendations: 43/113, 38/66 and 49/254 were in comprehensive, T&CM and acupuncture-specific CPGs. Of the remaining 382, from the most to the least summarised types of outcomes are: symptoms (253, 58.4%), function (86, 19.9%), quality of life (29, 6.7%), surrogate outcomes (12, 2.8%), mortality (1, 0.2%) and major morbid events (1, 0.2%) (online supplemental 3 table E).
Discussion
This study identified 133 CPGs and 433 recommendations addressing acupuncture intervention published between 2010 and 2019 that most frequently addressed musculoskeletal and connective tissue diseases (29.3%), neurological disorders (15.8%), obstetrics, gynaecology, and women’s health (9.7%), and oncology (9.0%) (figure 2). Guidelines overwhelmingly recommended in favour of acupuncture (87.8% of recommendations), with the remainder split more or less evenly between recommendations against or no recommendation. Of the 303 (70.0%) recommendations that used GRADE to determine the strength of recommendations, 152 (50.2%) were weak recommendations, 131 (43.2%) were strong recommendations, of which 104 (79.4%) were supported by low or very low certainty evidence (discordant recommendations). According to the AGREE II instrument, included CPGs proved moderate quality in ‘scope and purpose,’ ‘clarity of presentation,’ ‘rigour of development’ and ‘stakeholder involvement,’ but low quality in ‘editorial independence’ and ‘applicability’ domains.
Strengths and limitations
Our study, the most comprehensive that systematically evaluated CPGs addressing acupuncture interventions, has several strengths. First, this systematic survey includes a comprehensive search of eligible CPGs, systemic and explicit application of eligibility criteria, duplicate and independent study selection and data extraction, and rigorous methodological assessment of CPGs using the AGREE-II instrument. Second, our research investigated the utilisation of the GRADE approach in CPGs and recommendations. Moreover, we documented recommendations discordant with respect to strength and certainty. Third, this survey identified the missing aspect in AGREE II that most acupuncture-related CPGs did not explicitly report how they consider patients’ values and preferences, hindering guidelines’ applicability.
The limitation of our study includes: (1) we did not include Japanese and South Korean guideline databases, and (2) we excluded guidelines not supported by any SRs—thus, results do not apply to the lowest methodological quality guidelines.
Relation to other studies
Jeremy Y Ng .et al17 investigated the quantity and quality of T&CM (eg, herbal medicine, acupuncture, chiropractic and osteopathic manipulation) guidelines in MEDLINE, EMBASE and CINAH, etc. This survey examined a few selected interventions in T & CM and only included three acupuncture-related guidelines. Guo LH et al18 evaluated the methodological quality of acupuncture CPGs conducted by the China Institute of Acupuncture and Moxibustion. Consistent with our results, both articles identified ‘applicability’ and ‘editorial independence’ as Acupuncture-specific CPGs ’main limitations, but they omitted non-Chinese CPGs. Guo Y et al6 conducted a systematic survey to summarise the diseases and disorders most commonly treated with acupuncture in CPGs and concluded that acupuncture recommendations might be limited to painful conditions. They only searched the National Guideline Clearinghouse database, which might explain their failure to present the broad picture of acupuncture guidelines that our survey describes. None of the three studies analysed whether patients’ values and preferences were explicitly considered, and only Wu et al summarised whether CPGs used the GRADE approach.
Implications
Acupuncture guidelines could improve by consistently supporting recommendations with SRs, explicitly considering patients’ values and preferences, providing additional details regarding patients’ characteristics, acupuncture interventions/definition, comparators and outcomes summarised.
Despite available evidence and the wide application of acupuncture as an adjunct therapy in practice (eg, in combination with conventional medicine),19–22 acupuncture recommendations often do not address this question. Future guidelines should much more frequently address this issue.
The GRADE approach perceives strong recommendations supported by low or very low certainty evidence (ie, discordant recommendations) as questionable and often inappropriate.23 Guidelines addressing acupuncture intervention should better understand the GRADE approach, include panels that support the GRADE approach’s optimal usage, and collaborate with well-trained methodologists to avoid discordant recommendations.24 25 Both financial and intellectual COIs can impact panels and guideline developers’ judgement and the recommendations.26 27 Proper management of COIs can mitigate bias related to it and increase CPGs’ credibility. CPGs addressing acupuncture should report guideline’ funding sources, declare and manage all guideline developers and panel members’ COIs considering available literature.28 29
Conclusion
In the past 10 years, many CPGs addressing acupuncture interventions covering various diseases and conditions exist. Although these guidelines may be as or more rigorous than many others, considerable room for improvement remains, particularly, guideline developers should better manage COIs, explicitly considering patients values and preferences and resources utilisation, using more SRs to support recommendations and avoiding discordant recommendations.
Data availability statement
Data are available on reasonable request. The full dataset and statistical code will be available on reasonable request by emailing YZ.
Ethics statements
Patient consent for publication
Ethics approval
This study does not involve human participants.
Acknowledgments
We thank Hao wen, Peiming zhang, Xiaojing Wei, Yong Jiang, Haoyu Zhang, Chen Chen, and Yiming Chen all from the Guangzhou University of Chinese Medicine, for contributing to the literature screening, data extraction and visualisation.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
XT and XS contributed equally.
Contributors YQZ, LL, HZ, XJ and NX codeveloped the protocol for this systematic survey. LL, HZ and YQZ developed the search strategy. XS, ZC, YF, LL, RD, PZ, YX, SC, FG and JF conducted the screening of studies, data extraction and critical appraisal. YQZ, XT and XS reviewed each stage of study selection and calibrated findings. XT, XS, YQZ, LL and HZ wrote the manuscript’s first draft. All authors assisted in the interpretation and write-up of results. All authors approved the final manuscript prior to submission. The corresponding author attests that all listed authors meet authorship criteria and that no others have been omitted. YQZ and XJ are the guarantors of this paper.
Funding This study was supported by the Innovation Team and Talents Cultivation Program of the National Administration of Traditional Chinese Medicine (ZYYCXTD-C-202004); the special project of “Lingnan Modernization of Traditional Chinese Medicine” within the 2019 Guangdong Provincial Research and Development Program (2020B1111100008); the Project of First Class Universities and High-level Dual Discipline for Guangzhou University of Chinese Medicine; and the National Natural Science Foundation of China (82174527); Graduate Research Innovation Project of Guangzhou University of Chinese Medicine; The belt and road' International Cooperation Project supported by China Academy of Chinese Medical Sciences (project number: GH201901).
Disclaimer The funders had no influence on study design, data collection, analysis, decision to publish, or manuscript preparation.
Competing interests All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare funding for the submitted work as described above; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.