Article Text
PDF
Abstract
Objective To provide a route map regarding systematic reviews (SRs) of acupuncture therapies that will meet two goals: (1) to identify areas in which more or better evidence is required and (2) to identify acupuncture applications that, although proven effective, remain underused in practice, and thus warrant more effective knowledge dissemination.
Eligibility criteria We included SRs that conducted meta-analyses (MAs) of randomised controlled trials (RCTs) for this overview.
Information sources We searched for SRs without language restrictions from January 2015 to November 2020 in four Chinese electronic databases and Epistemonikos database. And we also searched for newly published RCTs that were eligible for selected best SRs in PubMed, Medline, Cochrane Central Register of Controlled Trials, Embase and four Chinese electronic databases from its lasted search dates to November 2020.
Synthesis of results We reanalysed the selected MAs if new primary studies were added. We used random-effect model to calculate the overall effect.
Results Our search identified 120 SRs published in the last 5 years addressing acupuncture therapies across 12 therapeutic areas and 77 diseases and conditions. The SRs included 205 outcomes and involved 138 995 participants from 1402 RCTs. We constructed 77 evidence matrices, including 120 SRs and their included RCTs in the Epistemonikos database. Seventy-seven SRs represented the effect estimate of acupuncture therapies. Finally, we system summarised the areas of possible underutilisation of acupuncture therapies (high or moderate certainty evidence of large or moderate effects), and the areas of warranting additional investigation of acupuncture therapies (low or very low certainty evidence of moderate or large effects).
Conclusion The evidence maps and overview of SRs on acupuncture therapies identified both therapies with substantial benefits that may require more assertive evidence dissemination and promising acupuncture therapies that require further investigation.
- HEALTH SERVICES ADMINISTRATION & MANAGEMENT
- COMPLEMENTARY MEDICINE
- THERAPEUTICS
Data availability statement
Data are available upon reasonable request. The full dataset and statistical code will be available upon reasonable request by emailing Nenggui Xu.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This study was the first evidence map for acupuncture therapies across all therapeutic areas to our knowledge.
This study includes a comprehensive search of eligible systematic reviews and randomised controlled trials and explicit eligibility criteria.
This study use of both a combination of evidence mapping and an overview approach provides readers with both a broad perspective of the evidence landscape and in-depth information on the certainty of evidence and the effect size on patient-important outcomes.
This study, in-depth collaboration with the Epistemonikos foundation, makes it possible for readers to have an overview of evidence and access the primary studies.
The limitation of this review is that we excluded studies investigating the effect of acupuncture as an adjunct therapy, and some diseases/conditions may be omitted.
Introduction
Clinicians and patients worldwide now make wide use of acupuncture, a form of traditional medicine.1 According to a 2013 WHO report,2 103 of the WHO’s member countries have approved the use of acupuncture. According to a 2013 survey conducted by the World Federation of Acupuncture-moxibustion Societies,3 183 (91%) of the 202 countries surveyed use acupuncture, while 178 (93%) of the 192 member countries of the United Nations have acupuncture practices, and 59 (31%) have partial or full insurance coverage.
Based on the extensive application of acupuncture in practice, in recent years4 numerous systematic reviews (SRs) have explored the effects of acupuncture therapies. Despite the mass of evidence, acupuncture practice and related policies practice in different jurisdictions vary, including overutilisation or underutilisation.5 Cultural, societal and awareness factors may result in research waste.
Highlight possible overutilisation or underutilisation or both compared with Western clinicians; Chinese clinicians treat many more diseases and conditions using acupuncture therapies.6 Insurance in Western countries covers only some conditions, for example, low back pain, at the healthcare system level compared with that in Eastern countries (eg, South Korea, China).7 Reviews of existing evidence may clarify areas in which overutilisation exists (highlighting the need for either deimplementation or further research) and where underutilisation exists (highlighting the need for more effective knowledge dissemination).
Evidence mapping is one of the latest methods for synthesising evidence using a user-friendly format with visual graphics and or searchable databases. Evidence mapping integrates, analyses, refines and presents the evidence quickly and conveniently,8 in the process helping define future research and knowledge dissemination priorities.8 This relatively novel method for the overview of SRs is particularly suitable for areas where a large number of SRs exist and overlap across diseases and conditions. To date, no evidence maps or overview of SRs summarising evidence regarding acupuncture therapies exist.
We, therefore, developed a route map guiding the acupuncture global research agenda to either further investigate promising acupuncture therapies or implementing more assertive evidence dissemination on areas where acupuncture demonstrated substantial benefits. In this article, we present: (1) the characteristics of the included SRs from multiple dimensions, including populations, interventions, comparisons and outcomes; (2) the most up-to-date treatment effect of acupuncture therapies in all conditions supported by randomised controlled trial (RCT)-based evidence along with the corresponding certainty of evidence.
Methods
Definition
Acupuncture
We used the WHO’s definition of acupuncture, as follows9: Acupuncture literally means to puncture with a needle. However, acupuncture may also involve the application of other kinds of stimulation to certain points. We included any type of commonly used acupuncture that stimulates certain points with needles, lasers, electricity or pressure. The specific types of acupuncture therapies included in this manuscript were manual acupuncture, electro-acupuncture, body needling, ear (auricular) acupuncture, scalp acupuncture, laser acupuncture, transcutaneous electrical nerve stimulation, and acupressure. Forms combined with moxibustion or medication, such as warm needling, acupoint injection or hydro-acupuncture, were excluded.
Manual acupuncture10: the most commonly seen acupuncture therapies worldwide. A practitioner uses thin, solid, metallic needles to penetrate the skin on a series of acupoints and then performs gentle and specific stimulation techniques. The concept of manual acupuncture emphasises the non-electrified stimulation method.
Electroacupuncture: additional electric current stimulation following the needles insertion.
Body needling11: a type of manual acupuncture performed on the trunk and limbs of the body, excluding the acupoints on the scalp and ears.
Ear (auricular) acupuncture10: acupuncture at the points located on the auricle, also called auriculoacupuncture.
Scalp acupuncture10: acupuncture at the specific lines located on the scalp.
Laser acupuncture10: a variant of acupuncture in which needling is replaced by laser irradiation on the points.
Transcutaneous electrical nerve stimulation (TENS)10: a therapy that uses low voltage electrical current to provide pain relief.
Acupressure12: a practitioner use hand, elbow or various devices to apply physical pressure to acupuncture points or trigger points.
Patient-important outcome
A patient-important outcome was defined as an outcome for which one would answer ‘yes’ to the following question: ‘If patients knew that this outcome was the only thing to change with treatment, would they consider receiving this treatment is associated with side effects or cost?’13 Some patient-important outcomes are shown below:
Mortality
All-cause mortality
Disease-specific mortality
Morbidity
Cardiovascular major morbid events
Other major morbid events (eg, loss of vision, seizures, fractures)
Onset/recurrence/relapse of cancer and other chronic diseases (eg, chronic obstructive pulmonary disease exacerbation, symptomatic diabetes)
Renal failure requiring dialysis
Hospitalisation, medical and surgical procedures (eg, pacemaker placement, cardioversion and revascularisation)
Symptomatic infections
Dermatologic/rheumatologic disorders
Quality of life/functional status (eg, failure to become pregnant, failure to nurse/breastfeed, depression)/specific symptoms (eg, pain, nausea)
Disease and condition
The term disease broadly refers to any condition that impairs the body’s normal functioning. The term condition is a synonym for medical state, which describes an individual patient’s current state from a medical standpoint, such as postoperative pain, postoperative ileus after abdominal surgery, cancer-related pain and breast cancer-related lymphoedema.14
Eligibility and inclusion criteria
Type of study
We included SRs that conducted meta-analyses (MAs) of RCTs for this overview. An eligible SR fulfilled the following criteria:
Reported a search in at least one electronic database.
Reported at least one criterion for the inclusion of studies.
Reported an effect estimate for at least one patient-important outcome.
We excluded overviews of SRs, narrative reviews, protocols of SRs or studies published prior to January 2015.
Type of participants
There were no restrictions on the type of participants. To classify diseases and conditions, we used the taxonomy established in the Living Overview of Evidence platform15 built on the International Classification of Diseases, 10th revision (ICD-10), with modification.
Type of interventions
We included SRs summarising at least one of the following comparisons:
Acupuncture versus no intervention/waiting list.
Acupuncture versus sham/placebo.
Acupuncture versus standard of care/usual care.
Acupuncture versus western medicine.
Acupuncture versus other interventions (such as psychotherapy, rehabilitation).
We excluded SRs with the control group receiving Traditional Chinese Medicine-related therapies, such as acupuncture, moxibustion, scraping, cupping, bloodletting, acupoint catgut embedding, massage, Chinese herbal medicine and tai chi.
Type of outcomes
We included SRs reporting at least one patient-important outcome.
Database and search
We searched for SRs without language restrictions from January 2015 to November 2020 in four Chinese electronic databases (Chinese National Knowledge Infrastructure (CNKI), Chinese Biomedical Literature Database (CBM), WANFANG Database and Chinese Scientific Journal Database (VIP)), and Epistemonikos database (https://www.epistemonikos.org/). The Epistemonikos database includes SRs from 10 electronic databases (PubMed, Embase, Cochrane Database of SRs, Cumulative Index to Nursing and Allied Health Literature, PsycINFO, Latin American and Caribbean Health Sciences, Database of Abstracts of SRs of Effectiveness, Campbell Library, Joanna Briggs Institute Database and EPPI-Centre Library).
We also searched for newly published RCTs that were eligible for selected best SRs (from its lasted search dates to November 2020) in PubMed, Medline, Cochrane Central Register of Controlled Trials, Embase, CNKI, CBM, WAN FANG Database and VIP.
Selection of best SRs for effect estimates and certainty of evidence assessment
Two reviewers (SG and HW) independently screened titles and abstracts to identify eligible SRs. The same reviewers retrieved and independently evaluated the full text of potentially eligible SRs for final inclusion and addressed disagreements through discussion. If a consensus could not be reached, a third (LL) resolved the conflict.
When multiple SRs existed for the same disease or condition, we selected the best SR to provide the most up-to-date effect estimate and its corresponding certainty of evidence. To select the best SR, we applied the following selection criteria in the order listed:
For each clinical question, we selected the most recent SR.
If the publication years of SRs were consistent or close, we selected the SR that contained the most RCTs or the Cochrane SR.
If the primary studies included in the SRs completely overlapped, that is, the primary studies were consistent in quantity and content, we selected the highest quality SR using ‘A Measurement Tool to Assess SRs’ (AMSTAR) 2. If the primary studies partially overlapped or did not overlap, we updated the MA with all eligible primary studies. We updated identified SRs when newly published RCTs were eligible to be included.
Selection of newly published RCTs
For all best SRs, two reviewers (SG and HW) independently screened for newly published RCTs that might be eligible. A third author resolved any unresolved disagreements between the reviewers.
Data collection and analysis
Data extraction
For each condition, two reviewers independently extracted data from all SRs using standardised forms, including the following: study ID; first author; publication year; country; therapeutic areas (eg, periprocedural care or oncology) classified by the modified ICD-10 classification; the number of participants; the number of included primary studies; outcomes; interventions and comparison. In addition, for the selected best SR, reviewers extracted information on the effect sizes and related 95% CIs. To ensure consistency, prior to the data extraction, we conducted calibration exercises. Discrepancies in the extracted data were resolved by discussion; if needed, a third author arbitrated.
For newly published RCTs that are eligible for the best SRs, we extracted the following information: study ID; first author; year of publication; the number of participants; interventions; comparisons; result data; and patient-important outcomes that matched the selected SRs.
Quality assessment
Quality of the SRs
We assessed the methodological quality of all SRs using the AMSTAR 2 tool; items 2, 4, 7, 9, 11, 13 and 15 were considered critical items. We categorised the overall quality of the SRs as high (meeting the criteria of all items or not meeting the criteria of only one non-critical item), moderate (not meeting the criteria of more than one non-critical item), low (not meeting the criteria of one critical item) and critically low (not meeting the criteria of more than one critical item).16 Two independent reviewers performed each quality assessment, with discrepancies adjudicated by a third reviewer.
Certainty of evidence
We assessed the certainty of evidence for patient-important outcomes in the best SRs using the grading of recommendations assessment, development and evaluation (GRADE) methodology through GRADEpro Guideline Development Tool. GRADE classified the certainty of evidence as high, moderate, low or very low. If SRs used GRADE, we used their certainty ratings. If they did not use GRADE, we assessed the certainty of evidence.
Data synthesis
If newly published RCTS were updated in the SRs, we recalculate the pooled estimates in each MA. We used random-effect model to calculate the overall effect.
Evidence mapping
For all SRs and included RCTs, assisted with mapping, we visualised the overall and geographical distribution of evidence corresponding to therapeutic areas, acupuncture therapies’ effect size and certainty of evidence.
For the geographical distribution of all included SRs, we presented a geographical information system map. For the best SRs, we used a bubble plot to display therapeutic areas, their corresponding SRs, RCTs and the type of acupuncture tested.
For patient-important outcomes with large or moderate effect in the best SRs, we used evidence figures to show the effect size and corresponding certainty of evidence. For the pooled effect estimate of dichotomous outcomes, we presented the risk ratio (RR) and absolute risk reduction (ARR) along with the 95% CIs. We calculated the ARR by multiplying the median of risks observed in control groups by the pooled RR and then presenting the result in terms of the anticipated increase or decrease in patients experiencing the effect per 1000 patients treated. For continuous outcomes, we presented the standardised mean difference (SMD) along with its 95% CI. When the best SRs reported other effect estimates, we converted all to SMD or RR and classified the effect size as ‘small’, ‘moderate’ or ‘large.’ For the relative effect of dichotomous outcomes, we chose RR of ‘2.0 or 0.5’ as the reference for small and large effects, respectively (RR≥2.0 or RR≤0.5 as large, 0.5<RR<2.0 as small). For continuous outcomes, we used SMD of 0.4 and 0.717 as the reference for small, moderate and large effects (SMD<0.4 as small, 0.4≤SMD＜0.7 as moderate, ≥0.7 as large).
Through a tabular approach, a matrix of evidence displayed the clusters of SRs with corresponding included RCTs. For every disease or condition where acupuncture SRs exist, we created a matrix consists of all SRs and their corresponding RCTs.18 If newly published RCTs were eligible for any existing SR, we added the new RCTs to the matrices. We aggregated all matrices into an online repository on the Epistemonikos database.
Patient and public involvement
No patients were involved in selecting the research question, conducting the study, or interpreting the results, this study solely focused on patient-important outcomes.
Results
Results of literature search and study selection
We retrieved 6122 citations. After removing 2695 duplicates and screening 3427 titles/abstracts, we evaluated the full text of 614 articles, of which 120 proved eligible. Figure 1 presents the study selection process. Figure 2 shows the distribution of the country of publication represented by the first author’s affiliation. For each disease or condition, we used the best SR for each question to construct the bubble plot (figure 3) and evidence figures (figures 4 and 5, and online supplemental appendix 1). Online supplemental appendix 2 provides reasons for exclusion at the full-text screening stage.
Supplemental material
Supplemental material
Flow chart of the publication-selection process. CBM, Chinese Biomedical Literature Database; CNKI, Chinese National Knowledge Infrastructure.
Country distribution of the first authors affiliation in all included systematic reviews.
Bubble plot. Note: each pie represents a therapeutic area. Pie size: number of randomised controlled trials (RCTs) (larger=more studies).
Large or moderate effects with high or moderate certainty evidence. ARR, absolute risk reduction; CI, confidence interval; C, control group; CoE, certainty of evidence assed using the Grading of Recommendations Assessment, Development and Evaluation approach; certainty of evidence assed using the Grading of Recommendations Assessment, Development and Evaluation approach; EA, electro-acupuncture; MA: manual acupuncture; NO, number; NRT, nicotine replacement therapy; RCT, randomised controlled trials; RH, rehabilitation; RR, relative risk; relative risk; SA, sham acupuncture; SC, standard care; SMD: standard mean difference; T, treatment group; WM: western medicine. Theasterisk(*) was used to present the effect estimate when a lower value indicated a better outcome.
Large effects with low or very low certainty evidence. AA, auricular acupuncture; ARR, absolute risk reduction; BEI: behavioral and educational interventions; C, control group; CI, confidence interval; CoE, certainty of evidence assed using the Grading of Recommendations Assessment, Development and Evaluation approach; EA, electro-acupuncture; MT music therapy; MA, manual acupuncture; NI, no intervention; NO, number; RCT, randomised controlled trials; RH rehabilitation; RR, relative risk; SA, sham acupuncture; SC, standard care; SMD, standard mean difference; T, treatment group; TENs, transcutaneous electrical nerve stimulation; UC, usual care; WM, western medicine. The asterisk(*) was used to present the effect estimate when a lower value indicated a better outcome.
Characteristics of included SRs
Online supplemental appendix 3 and table 1 shows the characteristics of the 120 included SRs and MAs published between 2015 and 2020, of which SRs published in 2019 accounted for the highest proportion (n=28, 23.3%), followed by those published in 2018 (n=26, 21.7%). The first authors of the included SRs are from China (n=98, 81.7%), South Korea (n=9, 7.5%), Australia (n=5, 4.2%), the USA (n=4, 3.3%), Germany (n=2, 1.7%), the UK (n=1, 0.8%) and Iran (n=1, 0.8%) (figure 2). The included studies involve 12 therapeutic areas (periprocedural care, oncology, neurology, connective tissue diseases, ear, nose, and throat disorders, eye disorders, gastrointestinal disorders, genitourinary disorders, mental health, nutrition and metabolic disorders, obstetrics, gynaecology, and women’s health, and pregnancy or intended pregnancy) and 77 diseases or conditions. The reviews include 1402 RCTs that enrolled 138 995 unique participants. The number of RCTs included in each SR ranges from 1 to 68, and the sample sizes range from 81 to 7618 (figure 3 provides further details). Typical reviews apply the Cochrane risk of bias tool (n=98, 81.7%), do not use GRADE (n=97, 80.8%) and do not report the time point at which outcomes are measured (n=86, 71.7%).
Supplemental material
Methodological quality of included SRs
Figure 6 summarises the methodological quality of the 120 included SRs. Of all 120 SRs, 119 (99.2%) SRs are rated as having ‘low’ or ‘critically low’ quality. Failure to provide a list of excluded studies and explain any modifications of previously published protocols are the main reasons for the quality being rated low or critically low.
Methodological quality: evaluation results of each A Measurement Tool to Assess Systematic Reviews question of included studies. N, no; PY, partial yes; Y, yes.
Evidence map: bubble plot
We included 77 best SRs to construct the bubble plot. The bubble plot (figure 3) visualised the evidence distribution across 12 therapeutic areas in which we presented the number of SRs, their included RCTs and the total number of participants. Of the 77 reviews, neurology is the content area most frequently represented (33.8%), with a number of areas in the vicinity of 10% (oncology, connective tissue diseases, gastrointestinal disorders, mental health, obstetrics, gynaecology and women’s health), and others 5.2% or less (periprocedural care, genitourinary disorders, eye disorders, ear, nose, and throat disorders, nutrition and metabolic disorders, pregnancy or intended pregnancy). The number of RCTs included in each review ranges from 1 to 40. RCTs in only three areas (neurology, connective tissue diseases, mental health) enrol more than 16 000 participants. Manual acupuncture proves the most frequently used intervention (54.1%), following by electroacupuncture (20.2%) and body needling (5.5%).
Evidence figure
We included 77 best SRs to construct the evidence figures (figures 4 and 5, and online supplemental appendix 1).
Effective interventions at risk of underutilisation
Large effect with moderate certainty evidence
Therapies directed at improvement in functional communication of patients who had poststroke aphasia, reduction in myofascial pain,and increased lactation success rate within 24 hours postdelivery demonstrated probable large effects (RR≥2.0; SMD≥0.7). Details of comparisons and effect estimates are in figure 4.
Moderate effect with high or moderate certainty evidence
Therapies directed at relief of neck pain and shoulder pain demonstrated moderate effects (0.4≤SMD＜0.7, high certainty evidence).
Therapies directed at a reduction in the severity of vascular dementia symptoms, relief of fibromyalgia-related pain and non-specific low back pain and improvement of allergic rhinitis nasal symptoms demonstrated probable moderate effects (0.4≤SMD＜0.7, moderate certainty evidence). Details of comparisons and effect estimates are in figure 4.
Promising but unproven interventions warranting further study
Large effect with low or very low certainty evidence
Low-certainty evidence
Therapies directed at the reduction in the length of first flatus and defecation in postoperative ileus after abdominal surgery, reduction in pain of poststroke shoulder–hand syndrome, increase in motor function and reduction in spasm symptoms of patients who had poststroke spastic hemiplegia, improvement in sleep quality of patients with insomnia disorder, patients with haemodialysis and menopausal women, reduction in neurogenic pain, improvement in the symptoms of autism spectrum disorder in children, reduction in the severity of migraine pain, improvement in quality of life of migraine patients, reduction in the severity of carpal tunnel syndrome pain and tic symptoms in patients with tic disorder, reduction in sciatica pain, improvement in quality of life and constipation symptoms of patients with functional dyspepsia, reduction in the severity of pain and voiding symptoms and improvement in quality of life of patients with chronic prostatitis, increase in abstinence rate for smoking in short and long terms, reduction in severity of opioid craving and depression in opioid use disorder, reduction in body mass index in patients with obesity and improvement in success rate of lactation within 24 and 72 hours after delivery, improvement in the overall symptoms of chronic pelvic pain syndrome, reduction in severity of pain and menstrual symptoms of dysmenorrhoea, and improvement in pregnancy rate among infertile women may result in a large effect (RR≥2.0; SMD≥0.7). Details of comparisons and effect estimates are in figure 5.
Very low-certainty evidence
Therapies directed at relief of postoperative pain on movement or cough and aromatase inhibitor-induced arthralgia in breast cancer, improvement in quality of life of patients with breast cancer, increase in motor function and pain in poststroke shoulder–hand syndrome, increase in sleep quality of patients with insomnia and menopausal women, reduction in frequency of migraine symptoms, reduction in the severity of carpal tunnel syndrome pain, improvement in function of patients with hip osteoarthritis, relief of pain caused by postmenopausal osteoporosis, reduction in the severity of dizziness in Meniere’s disease, improvement in symptoms of xerophthalmia, reduction in the risk of peptic ulcer recurrence, improvement in symptoms of gastroparesis, reduction in the risk of urinary tract infection recurrence, reduction in the severity of chronic prostatitis, reduction in the severity of depression in chronic kidney disease, opioid use disorder, and in postpartum depression, reduction in the severity of alcohol craving in alcohol use disorder, improvement in functional status in posttraumatic stress disorder, decrease in mental fatigue in chronic fatigue syndrome, reduction in frequency of hot flash symptom and improvement in the symptoms of menopausal women, and relief of pain during labour may result in a large effect (RR smaller than 0.5; SMD greater than 0.7). Details of comparisons and effect estimates are in figure 5.
Moderate effect with low or very low-certainty evidence
Low-certainty evidence
Therapies directed at a reduction in the severity of hot flashes after breast cancer surgery, reduction in the severity of nausea and vomit in patients with cancer, increase in motor function and activities of daily living of poststroke spastic hemiplegia, improvement in the symptoms of paediatric autism spectrum disorder and sleep quality in patients with fibromyalgia syndrome, relief of non-specific low back pain, increase in complete spontaneous bowel symptom of functional constipation patients, improvement in symptoms of patients with functional dyspepsia, reduction in alcohol withdrawal symptoms in alcohol use disorder patients, reduction in the severity of depression in patients with depression, and reduction in hot flash frequency in menopausal women may result in a moderate effect (0.4≤SMD＜0.7). Details of comparisons and effect estimates are in online supplemental appendix 1.
Very low-certainty evidence
Therapies directed at relief of postoperative resting pain and postoperative (total knee arthroplasty) pain at 24 hours, increase in sleep quality of patients with insomnia, improvement in cognitive function of patients with mild cognitive impairment, relief of pain, and improvement in quality of life of patients with hip osteoarthritis, reduction in the severity of constipation symptoms of functional constipation patients, improvement in constipation symptoms of patients with functional dyspepsia, and improvement in the severity of depression and sleep quality in patients with chronic kidney disease may result in a moderate effect (0.4≤SMD＜0.7). Details of comparisons and effect estimates are in online supplemental appendix 1.
MA supplementation and updating
We updated three selected SRs with five RCTs published after the search date of the selected SRs.
Matrices of evidence
We produced 77 matrices of evidence clusters of SRs with corresponding included RCTs. Among all matrices, about three-quarters (n=56, 72.73 %) contained one SR, the rest (n=21, 25.97%) included two or more SRs in which simple obesity had the largest number of SRs (n=8). We digitalised this portion on Epistemonikos(https://www.epistemonikos.org/).
We presented two examples for the digitised matrices: Acupuncture for non-specific low back pain: (URL: https://www.epistemonikos.org/matrixes/60654c866ec0d61dc0b9e0d4) (figure 7);
Digitised matrix: acupuncture for non-specific low back pain.
Acupuncture for allergic rhinitis: (URL: http://www.epistemonikos.org/matrixes/606553857aaac81f38258f0f) (figure 8).
Digitised matrix: acupuncture for Rhinitis.
Online supplemental appendix 4 has the links to the remaining 75 matrices.
Supplemental material
Discussion
Principal findings
Our evidence mapping and overview of SRs included 120 SRs across 12 therapeutic areas and 77 diseases or conditions, including 138 995 participants from 1402 RCTs. Neurological conditions proved the most frequently studies area; connective tissue, mental health, obstetrics, gynaecology, and women’s health also proved frequent areas of study. While ear, nose, and throat disorders, eye disorders, nutrition, and metabolic disorders had the fewest SRs (figure 3), the conditions and outcomes in which acupuncture is at highest risk of underutilisation include the following: lactation within 24 hours after delivery, poststroke aphasia, myofascial pain (pain relief), vascular dementia, neck, shoulder, and non-specific low-back pain, fibromyalgia syndrome, allergic rhinitis (figure 4).
The conditions and outcomes in which acupuncture therapies showed promising effect yet warranting further research include the following: peptic ulcer, urinary infection, smoking cessation, lactation within 24 and 72 hours after delivery and so on (figure 5 and online supplemental appendix 1).
Strengths and limitations of the study
Our study, the first describing an acupuncture evidence map, has several strengths. First, this is the first evidence map for acupuncture therapies across all therapeutic areas to our knowledge. Second, we had clear, practical objectives: to identify therapies with large or moderate effects supported by high or moderate certainty evidence at risk of underutilisation and therapies suggesting large or moderate effects supported by low or very low-quality evidence warranting further investigations. Third, our use of both a combination of evidence mapping and an overview approach provides readers with both a broad perspective of the evidence landscape and in-depth information on the certainty of evidence and the effect size (large, moderate, and small) on patient-important outcomes.19 Fourth, by only including patient-important outcomes and summarising the results with additional information on the absolute rather than the relative effect size, our study adds more value to support clinical and health system decision-making than the minimal contextualised approach adopted by most SRs or overviews. Fifth, the digitalisation of available evidence provides a repository and a roadmap for readers’ further usage. Readers not only have an overview of evidence but also have access to the primary studies. Our in-depth collaboration with the Epistemonikos foundation makes this possible. Finally, we have made efforts to minimise bias in every step of this project. For the literature search, an experienced librarian developed the search strategy and performed the comprehensive literature search. All screening and data extraction were performed independently and in duplicate. These rigorous processes increased the credibility of our study.
Our study also has several limitations. First, we excluded studies investigating the effect of acupuncture as an adjunct therapy (eg, acupuncture combined with Western medicine or rehabilitation). Second, despite the non-randomised studies overwhelmingly providing only low-certainty evidence, excluding observational studies might limit the conclusion of our evidence map. Third, most SRs did not specify the follow-up time of the reported outcomes. We did not retrieve all of the follow-up times corresponding to the outcomes from the original 1402 RCTs but reported the information provided in the SRs. Fourth, a large number of low or critically low-quality included SRs warrant caution when interpreting our research.
Fifth, since this study only included the SRs from January 2015 to November 2020, some useful SRs with high quality might be omitted. Finally, and perhaps most important, we provided only candidates for possible underutilisation. The interventions in which high or moderate quality evidence had demonstrated large or moderate effects are only candidates for underutilisation—identification of areas in which more assertive dissemination efforts may prove beneficial will require utilisation data.
Comparison with other studies
Current overviews of SRs on acupuncture therapies20–22 covered a limited number of diseases. Janz and McDonald conducted a narrative review to identify existing evidence on acupuncture’s efficacy in 122 conditions23 using literature search, study selection and evidence extraction methods that were neither systematic nor transparent. Du et al’s conducted a narrative review and a national survey among 524 acupuncturists and identified 400 diseases and conditions treated by acupuncture.19 24 Their project’s objective was to summarise the landscape of current practice in China rather than provide a systematic and comprehensive view of the evidence map for acupuncture therapies.
Implications for practice and policy
The areas in which high or moderate certainty evidence demonstrates the large or moderate-sized effects of acupuncture therapy mandate its widespread use. Patients, clinicians, guideline developers, health policy-makers and payers can use the digitalised repository on Epistemonikos to support point-of-care decision-making, to produce additional evidence summaries (eg, develop clinical practice guidelines or policy briefs), or to make decisions at the health system level (eg, recommendations, implementation decisions and reimbursement decisions).
Implications for research
Trialists can review diseases and conditions with high/moderate-certainty evidence before conducting new research to reduce research waste (see figure 4 for more details). Granting agencies can use our research to assess the need and relevance for grant applications.
Ensuring that clinicians and patients are making optimal use of acupuncture therapies with substantial effects demonstrated by high or moderate certainty evidence will require surveys of the use of these therapies in practice. Should such surveys demonstrate underutilisation, concerted efforts to remedy the underutilisation will be required. We identified a large number of outcomes in various diseases and conditions with large/moderate effect sizes and low/very low-certainty evidence, for example, peptic ulcer, urinary infection, smoking cessation, lactation within 24 and 72 hours after delivery and so on. These areas represent potentially fruitful targets for future clinical trials. Grant agencies can consider setting up or encouraging research in the areas mentioned above. Trialists will need to address the above areas when conducting future research. Trialists, systematic reviewers and grant agencies can use the digitalised repository on Epistemonikos to assist in the design of RCTs (eg, fine-tuning the research questions, selecting the outcomes) and SRs.
Conclusion
The evidence maps and overview of SRs on acupuncture therapies identified both promising acupuncture therapies that require further investigation and therapies with demonstrated substantial benefits that may require more assertive evidence dissemination.
Data availability statement
Data are available upon reasonable request. The full dataset and statistical code will be available upon reasonable request by emailing Nenggui Xu.
Ethics statements
Patient consent for publication
Ethics approval
Not applicable.
Acknowledgments
We thank Jigang Luo, Bing Deng, Juan Huang, Xiaoting Yan, Peiming Zhang, Yuyuan Tang, Ruihua Duan, Fen Gong, and Huishan Chen, all from the Guangzhou University of Chinese Medicine, for contributing to the literature screening and data extraction.
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @radagabriel
LL, YZ, SG and HW contributed equally.
Contributors LL, YZ, SG and HW, as co-first authors, contributed equally to this work and they wrote the manuscript’s first draft. C-ZT and N-GX, as co-corresponding authors, contributed equally to this work. LL and YZ conceived the study. SG, HW, XT and ZZ designed the search strategy. SG and HW performed the literature search. SG, HW, XT and JcZ screened the studies for eligibility. SG, HW, RC, YD, XW, WL, LW performed the data extractions. LW, GR, CA and CV digitalised the matrices on Epistemonikos. GG, C-ZT and N-GX were responsible for the manuscript editing and manuscript review. All the authors interpreted the data, critically revised the manuscript for important intellectual content, and gave the version’s final approval. LL and N-GX are guarantors. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.
Funding This study was funded by: the Innovation Team and Talents Cultivation Program of This study was funded by: the National Administration of Traditional Chinese Medicine (ZYYCXTD-C-202004); the special project of “Lingnan Modernization of Traditional Chinese Medicine” within the 2019 Guangdong Provincial Research and Development Program (2020B1111100008); the Project of First Class Universities and High-level Dual Discipline for Guangzhou University of Chinese Medicine; and the National Natural Science Foundation of China (82174527). The funders had no influence on study design, data collection, analysis, decision to publish, or manuscript preparation. The funders had no influence on study design, data collection, analysis, decision to publish, or manuscript preparation.
Map disclaimer The depiction of boundaries on this map does not imply the expression of any opinion whatsoever on the part of BMJ (or any member of its group) concerning the legal status of any country, territory, jurisdiction or area or of its authorities. This map is provided without any warranty of any kind, either express or implied.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.