Article Text

Original research
Factors Associated with the Magnitude Of acUpuncture treatment effectS (FAMOUS): a meta-epidemiological study of acupuncture randomised controlled trials
  1. Wei-Juan Gang1,2,
  2. Wen-Cui Xiu1,2,
  3. Lan-Jun Shi1,2,
  4. Qi Zhou3,
  5. Rui-Min Jiao1,2,
  6. Ji-Wei Yang1,2,
  7. Xiao-Shuang Shi1,2,
  8. Xiao-Yue Sun1,2,
  9. Zhao Zeng4,
  10. Claudia M Witt5,
  11. Lehana Thabane6,
  12. Ping Song7,
  13. Long-Hui Yang7,
  14. Gordon Guyatt3,6,
  15. Zhi-Yun Zhang1,
  16. Heng-Cong Li1,
  17. Jing-Tao Shi1,
  18. An-Li Chen1,
  19. Zheng-Yang Qu1,
  20. Ling Zou1,
  21. Dong-Xiao Mou1,
  22. Xiao-Yu Wang1,
  23. Qing-Quan Yu1,
  24. Li-Zhen Chen1,
  25. Yu-Ting Huang1,
  26. Tiago V Pereira8,
  27. Jason Chambers9,
  28. Cameron Ho10,
  29. Layla Bakaa11,
  30. Kevin Loniewski12,
  31. Kyle Tong6,
  32. Jaryd Tong13,
  33. Jared E Dookie14,
  34. Jie-wei Zhu13,
  35. Malini Hu15,
  36. Yujin Suk16,17,18,
  37. Kay Wu19,
  38. Luciane Cruz Lopes20,
  39. Julia White10,
  40. Tayler A Buchan3,
  41. Lauren Giustti Mazzei21,
  42. Maíra Ramos Alves22,
  43. Mariana Del Grossi Moura23,
  44. Cristiane De Cássia Bergamaschi20,
  45. Jing Meng24,
  46. Cynthia Chan25,
  47. Flávia Blaseck Sorrilha23,
  48. Xiang-Hong Jing1,2,
  49. Yu-Qing Zhang1,3,26
  1. 1 Institute of Acupuncture and Moxibustion, China Academy of Chinese Medical Sciences, Beijing, China
  2. 2 China Centre for Evidence-Based Traditional Chinese Medicine, China Academy of Chinese Medical Sciences, Beijing, China
  3. 3 Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada
  4. 4 Library of Guangzhou University of Chinese Medicine, Guangzhou, China
  5. 5 Institute for Complementary and Integrative Medicine, University Hospital Zurich and University of Zurich, Zurich, Switzerland
  6. 6 Department of Medicine, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada
  7. 7 China Academy of Chinese Medical Sciences, Beijing, China
  8. 8 Department of Health Sciences, University of Leicester, Leicester, UK
  9. 9 Schulich School of Medicine, Western University, London, Ontario, Canada
  10. 10 Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, Ontario, Canada
  11. 11 Department of Psychology, McMaster University, Hamilton, Ontario, Canada
  12. 12 Faculty of Health, York University, Toronto, Ontario, Canada
  13. 13 University of Toronto, Toronto, Ontario, Canada
  14. 14 Schulich School of Medicine and Dentistry,Western University, Ontario, Canada, Western University, London, Ontario, Canada
  15. 15 Department of Medicine, University of Toronto, Toronto, Ontario, Canada
  16. 16 Centre for Discovery in Cancer Research (CDCR), McMaster University, Hamilton, Ontario, Canada
  17. 17 Department of Biochemistry and Biomedical Sciences, McMaster University, Hamilton, Ontario, Canada
  18. 18 Michael G DeGroote School of Medicine, McMaster University, Hamilton, Ontario, Canada
  19. 19 Department of Medical Imaging, University of Toronto, Toronto, Ontario, Canada
  20. 20 Pharmaceutical Sciences Graduate Course, University of Sorocaba, São Paulo, Brazil
  21. 21 Physical Therapy Course, University of Sorocaba, São Paulo, Brazil
  22. 22 Psychology Course, University of Sorocaba, São Paulo, Brazil
  23. 23 University of Sorocaba, São Paulo, Brazil
  24. 24 Obstetric & Gynaecology department, Dunedin Hospital, Health, Dunedin, New Zealand
  25. 25 Department of Family Medicine, Faculty of Medicine, University of Ottawa, Canada, Canada
  26. 26 Nottingham Ningbo GRADE Centre, University of Nottingham Ningbo China, Ningbo, China
  1. Correspondence to Dr Xiang-Hong Jing; xhjingt66{at}163.com

Abstract

Objective To identify factors and assess to what extent they impact the magnitude of the treatment effect of acupuncture therapies across therapeutic areas.

Data source Medline, Embase, Cochrane Central Register of Controlled Trials, China National Knowledge Infrastructure, Wanfang Database, VIP Database, and China Biology Medicine disc, between 2015 and 2019.

Study selection The inclusion criteria were trials with a total number of randomised patients larger than 100, at least one patient-important outcome and one of two sets of comparisons.

Data analysis The potential independent variables were identified by reviewing relevant literature and consulting with experts. We conducted meta-regression analyses with standardised mean difference (SMD) as effect estimate for the dependent variable. The analyses included univariable meta-regression and multivariable meta-regression using a three-level robust mixed model.

Results 1304 effect estimates from 584 acupuncture randomised controlled trials (RCTs) were analysed. The multivariable analyses contained 15 independent variables . In the multivariable analysis, the following produced larger treatment effects of large magnitude (>0.4): quality of life (difference of adjusted SMDs 0.51, 95% CI 0.24 to 0.77), or pain (0.48, 95% CI 0.27 to 0.69), or function (0.41, 95% CI 0.21 to 0.61) vs major events. The following produced larger treatment effects of moderate magnitude (0.2–0.4): single-centred vs multicentred RCTs (0.38, 95% CI 0.10 to 0.66); penetration acupuncture vs non-penetration types of acupuncture (0.34, 95% CI 0.15 to 0.53); non-pain symptoms vs major events (0.32, 95% CI 0.12 to 0.52). The following produced larger treatment effects of small magnitude (<0.2): high vs low frequency treatment sessions (0.19, 95% CI 0.03 to 0.35); pain vs non-pain symptoms (0.16, 95% CI 0.04 to 0.27); unreported vs reported funding (0.12, 95% CI 0 to 0.25).

Conclusion Patients, clinicians and policy-makers should consider penetrating over non-penetrating acupuncture and more frequent treatment sessions when feasible and acceptable. When designing future acupuncture RCTs, trialists should consider factors that impact acupuncture treatment effects.

  • complementary medicine
  • statistics & research methods
  • epidemiology

Data availability statement

All data relevant to the study are included in the article or uploaded as online supplemental information.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

  • This study included a comprehensive search, independent and duplicated screening and data extraction, rigorous data analysis and interpretation by multidisciplinary researchers.

  • This study focused on patient-important outcomes and chose the independent variables considering literature, clinicians, and patients’ perspectives.

  • This study constructed a robust three-level mixed model multivariable analysis to adjust for multiple variables to reduce the potential bias and used Cramer’s V and the weighting approach of robust regression to deal with the collinearity and substantial amount of outlier and influential values.

  • The multivariable analyses excluded important independent variables such as practitioners’ experience due to poor reporting.

  • Including extremely imbalanced variables (eg, country, trial registered) limits the generalisability of the study results.

Introduction

Acupuncture is one of the most used and researched interventions under the integrative medicine umbrella.1–4 By 2014, the total number of acupuncture randomised controlled trial (RCT) has increased dramatically and accounted for 20.3% of all acupuncture studies.5 Since 2010, over 1000 acupuncture RCTs were published annually, with the total number exceeding 10 000 to date.6

Acupuncture’s treatment effect varies largely across trials.7 8 Efforts to determine factors associated with effect size in acupuncture RCTs have reported conflicting findings. For example, Vickers et al reported that, in studies of chronic pain, penetrating sham vs non-penetrating and non-needle sham control showed larger treatment effects.9 However, other studies reported that the effect of acupuncture in pain studies was unrelated to the type of sham acupuncture.10 11 Some found the total number of acupuncture treatments,11–13 frequency of treatment sessions14 and acupuncture type (manual acupuncture vs electroacupuncture)14 were significant factors of the treatment effect whereas others did not.9 15 The reason may be related to little data variation,15 small number of included studies,12 14 and variation of the clinical areas and settings investigated.10 11 16

To improve acupuncture RCTs’ design, and optimise acupuncture interventions’ clinical effectiveness, we conducted this meta-epidemiological study, including acupuncture RCTs published between 2015 and 2019 across therapeutic areas and outcomes, and explored the factors of acupuncture’s treatment effects. We aim to (1) identify factors regarding patient, acupuncture, comparator, outcome and methodology that impact the magnitude of the treatment effect of acupuncture therapies and (2) explore to what extent the factors impact the treatment effect across therapeutic areas.

Methods

Definitions

We define acupuncture therapies based on the WHO definition: Acupuncture literally means to puncture with a needle. However, there may also involve the application of other kinds of stimulation to certain points.17 The study addressed commonly used acupuncture modalities, including manual acupuncture, electroacupuncture (electro-acupuncture), laser acupuncture, transcutaneous electrical acupoint stimulation (TEAS), acupressure, traditional body needling, ear (auricular) acupuncture and scalp acupuncture.

We define sham acupuncture as an intervention with a minimal treatment effect designed to blind patients as they received real acupuncture.18 Often sham acupuncture includes ‘placebo’ needles with a blunt collapsing tip that does not penetrate the skin, real acupuncture but inserted at non-acupuncture points or true acupuncture points but not targeting the intended disease. Non-needle sham can be detuned lasers, deactivated transcutaneous electric nerve stimulation devices or less pressure on acupuncture points.

We define a patient-important outcome as one in which the patient would be interested, despite the risk, burden or cost, were it the only outcome to improve with an intervention.19

To differentiate from individual outcomes (eg, dysphagia), we define a construct as a category of patient-important outcomes (eg, functional status).

We define a therapeutic area as a class of related diseases or conditions based on modified International Classification of Diseases 11th Revision (ICD-11) criteria (eg, Neurology). In this study, the classification of the therapeutic areas targeted diseases or conditions for which patients seek acupuncture treatment. For example, if an acupuncture RCT investigated post-stroke depression, we would classify the RCT into ‘Mental health’ rather than ‘Neurology’.

Literature search

In collaboration with clinical and methodological experts, a medical information specialist developed a search strategy that included PubMed, Embase, the Cochrane Central Register of Controlled Trials, and 4 Chinese databases, including China National Knowledge Infrastructure (CNKI), Wanfang Database, VIP Database for Chinese Technical Periodicals (VIP) and China Biology Medicine disc (CBM). We searched acupuncture RCTs published from 2015 January to 2019 December with no language restrictions. The detailed search strategy is presented in online supplemental eAppendix 1.

Supplemental material

Eligibility criteria

Eligible studies fulfilled the following inclusion criteria:

  • RCT defined by authors.

  • Reported at least one of two sets of comparisons: acupuncture vs no intervention, sham acupuncture or waiting list; or acupuncture plus other interventions vs other interventions with or without sham acupuncture. The other interventions must be conventional medical treatment and identical in both intervention and control groups.

  • Reported at least one patient-important outcome.

  • Randomised over 100 individuals.

  • Appeared in a peer-reviewed journal publication in any language.

We excluded conference abstracts, letters, commentaries, editorials, protocols, non-human trials, cluster RCTs, n-of-1 trials, cost–utility studies, secondary analyses of RCTs, reviews and meta-analyses, RCTs in which control groups received any traditional Chinese medicine related therapies (eg, acupuncture, moxibustion, scraping, cupping, bloodletting, acupoint catgut embedding, massage, Chinese herbal medicine) and studies in which tables and text reported contradictory results on the selected outcomes.

Study selection

We exported Chinese citations to Endnote V.X9.0 and English citations to a web-based software (https://collaboratron.epistelab.com/) for eligibility screening. To conduct, independently and in duplicate, title and abstract and full-text screening, a team of 16 Chinese and 22 English reviewers worked in pairs using standardised forms with detailed instructions. To ensure screening quality, reviewers participated in a calibration exercise prior. If needed, reviewers resolved disagreements through discussion or arbitrated by a third party.

Generation and ranking of the factors that impact treatment effect

We first, through the literature review and consultation with acupuncturists, generated a list of potential factors that might be associated with the magnitude of effect resulting in 13 methodological factors and 26 clinical factors. To ensure our list was comprehensive, and to rank the importance of the factors, we conducted an online survey using Wenjuanxing (www.wjx.cn) among a global panel (n=27) composed of acupuncture trialists, acupuncturists, surgeons, trial methodologists, patients and statisticians. The survey results added seven factors, and we finally included 46 factors (online supplemental eAppendix 2) in the meta-regression analyses.

Data extraction

We classified patient-important outcomes into six constructs box 1.

Box 1

Classification of constructs

  1. Mortality.

  2. Major events include morbid events (eg, incidence of myocardial infarction,fracture, stroke), recurrence (eg, the recurrence of facial spasm) or fertilisation-related events (eg, live birth rate).

  3. Pain (eg, low back pain).

  4. Non-pain symptoms (eg, nausea and vomiting).

  5. Quality of life (eg, health-related quality of life).

  6. Functional status (eg, dysphagia).

To select outcomes, we first extracted all patient-important outcomes, classified them into the six constructs (box 1), and then, within constructs, classified each outcome into therapeutic areas (we will refer to these as subconstructs). For example, for the non-pain symptoms construct, reviewers classified nausea and vomiting into ‘gastroenterology’. We retained the subconstructs, including 30 studies or more.

Within each construct/subconstruct, for each outcome, we calculated the number of studies reporting the outcome. If one study reported multiple outcomes within the same subconstruct, we extracted the more frequently reported outcome across all studies. When studies reported the same outcome measured by different instruments, we selected the most frequently reported instrument for that outcome across all studies.

If the above process excluded either the primary outcome or the first patient-important outcome in the result, in addition to the outcomes selected through that process, we also included the first patient-important or primary outcome reported in the result section.

For multiple-arm RCTs, we considered only those comparisons that met eligibility criteria. For RCTs with multiple follow-up times, we selected the outcome both at the end of treatment and at the longest follow-up time in which the loss to follow-up rate was 20% or less.

Following a calibration exercise, a team of 10 reviewers, working in pairs, independently extracted data and resolved discrepancies through discussion. If they could not reach a consensus, an arbiter resolved the conflict.

For outcome selection, three pairs of reviewers reviewed all included studies selecting outcomes. After completing the outcome selection and discussing as necessary to come to an agreement, reviewers extracted data on the preselected outcomes.

For each trial, reviewers extracted the number of randomised and analysed participants, data on all factors and recorded the selected outcomes’ effect estimates. Risk of bias was assessed using the Cochrane Collaboration tool.20 For dichotomous outcomes, we collected the number of events and for continuous outcomes, point and associated variabilities, ranges and directions. To extract data from figures in which the data were unavailable in the text or tables, we used GetData Graph Digitizer V.2.25 (by Mark Mitchell) software.

Statistical analysis

Depending on the data distribution, we summarised data using means and SD, or medians and IQRs. For statistical tests, we used a threshold p value of 0.05 to indicate a statistical significance. To combine the outcomes from different measurement scales, we applied the standardised mean difference (SMD). A positive SMD indicated a beneficial effect. The variance of SMD21 was given by

Embedded Image

where n1 and n2 were the sample sizes of the acupuncture therapies group and the control group, respectively. For the dichotomous outcome, by the method of Hasselblad and Hedges,21 22 we converted the calculated log OR to SMD using

Embedded Image

where π is the mathematical constant (approximately 3.14159). The variance of SMD was obtained by

Embedded Image

We initially considered 46 variables (online supplemental eAppendix 2) to investigate factors that might influence the SMD among the RCTs. However, 26 variables were excluded from the multivariate analysis because they were missing in more than 90% of the studies (online supplemental eAppendix 3). To detect possible multicollinearity, we calculated the Cramer’s V statistics23 24 (ranges 0–1) between every pair of the variables using a threshold of 0.70. When excessive collinearity existed, we excluded those variables from the regression analysis (online supplemental eAppendix 3).

To account for the heterogeneity between the studies and the dependency of the multiple outcomes within a study, we used a meta-regression in three-level random-effects mixed model25–27 to simulate the sampling variation for each effect size (level one), variation over outcomes within a study (level two), and variation over studies (level 3). The dependent variable was the SMD of the acupuncture therapies. The independent variables were the study level factors treated as fixed effects.

We had three different specifications in conducting the analyses. The first specification was an empty model with no independent variables to test heterogeneity of effect sizes at the study and outcome levels. The second specification (primary analysis) was a multivariable analysis that estimated the effects of the multiple independent variables associated with the SMD. To ensure sufficient power for the estimation, we determined the number of independent variables included in the model by applying the rule of 10 observations per variable. If no enough sample would contain all independent variables, a hierarchical list of variables was used to determine the priority of entry into the model. The third specification was a univariable analysis with a single factor each time.

To limit the influence of outliers and provide the resistant (stable) results, we incorporated the robust regression approach28 to the three-level random-effects mixed model for the analysis and used the difference of the least-squares means of the SMDs (or the difference of adjusted SMDs) to indicate the effect of a factor. We used 0.2 and 0.4 as the thresholds to name small, moderate and large (<0.2 as small, 0.2–0.4 as moderate,>0.4 as large) for the effect.

We conducted all the analyses in SAS, V.9.4.

Patient and public involvement

The online survey on potential factors involved empirical data and input from a global panel that included patients.

Results

The search yielded 169 406 studies, of which 6530 proved eligible. We retrieved and screened the full texts, excluded 5946 ineligible studies, and finally included 584 studies (figure 1).

Figure 1

Study selection flow diagram.

Characteristics of included studies

The 584 eligible studies published between 2015 and 2019 reported 1304 effect estimates that met our relevance criteria. Online supplemental eTables 1.1, 1.2 and 1.3 show the basic and clinical characteristics (classification of acupuncture treatment frequency, duration and the total number of treatments provided in online supplemental eAppendix 4), and risk of bias of included studies, respectively. Over 90% of the trials (n=540, 92.5%) were conducted in China. Of the 584 studies, 444 (76%) tested traditional Chinese acupuncture, and 313 (53.6%) used manual acupuncture. Acupuncture was the add-on intervention in 564 studies (96.8%), and 542 studies (92.8%) used other interventions as control. Some variables were important but poorly reported and thus excluded from the multivariable analysis.

Included RCTs had a high risk of bias. For example, over 90% of the RCTs were labelled as inadequate or probably inadequate allocation concealment (n=536, 91.8%); close to 90% of the trials did not report any allocation concealment approaches (524, 89.7%).

The extent of the heterogeneity of the acupuncture’s treatment effect when compared with sham or no acupuncture control (unconditional model-specification 1)

We applied a robust mixed model without exploratory variables to examine the effect sizes’ variations at study and outcome levels and observed significant heterogeneity (p<0.0001). This finding provided a basis for the multivariable analysis to further explore the influencing factors of heterogeneity.

Assessment on factors influencing acupuncture treatment effect (multivariable analysis-specification 2)

Of the 46 factors, 20 met our criterion of <10% of missing (retained at least 526 studies or 1174 outcomes) factor data. The Cramer’s V assessments for multicollinearity assessment further excluded publication language, journal impact factors, trial registration, therapeutic areas and blinding of participants due to the high association with other independent variables (Cramer’s V statistic >0.7, online supplemental eAppendix 3); thus resulted in 15 variables that were eventually included in the analysis (online supplemental eAppendix 5).

The multivariable analysis, including 1133 effect estimates from 508 studies, identified 5 significant factors: type of outcome, acupuncture type, frequency of treatment sessions, number of centres and funding availability (table 1).

Table 1

Multivariable meta-regression analysis

Compared with major events outcomes, effects proved larger in quality of life (large magnitude, difference of adjusted SMDs 0.51, 95% CI 0.24 to 0.77; p<0.001), pain (large magnitude, 0.48, 95% CI 0.27 to 0.69; p<0.001), function (large magnitude, 0.41, 95% CI 0.21 to 0.61; p<0.001) and non-pain symptoms (moderate magnitude, 0.32, 95% CI 0.12 to 0.52; p<0.001). Compared with non-pain symptoms, effects proved larger in pain (small magnitude, 0.16, 95% CI 0.04 to 0.27; p=0.01). Single centre, compared with multicentre, was associated with moderately larger effects (0.38, 95% CI 0.10 to 0.66; p=0.01). Penetration acupuncture (ie, manual acupuncture and electroacupuncture), compared with non-penetration type of acupuncture (ie, laser acupuncture, TEAS and acupressure), was associated with moderately larger effects (0.34, 95% CI 0.15 to 0.53; p<0.001). High frequency acupuncture treatment sessions, compared with low frequency, was associated with larger effects of small magnitude (0.19, 95% CI 0.03 to 0.35; p=0.02). Compared with reported funding, effects proved larger of small magnitude in studies that did not report funding (0.12, 95% CI 0 to 0.25; p=0.03) (figure 2, online supplemental eTable 2)

Figure 2

Forest plots of significant factors in the multivariable analysis. SMD, standardised mean difference.

Assessment on factors influencing acupuncture treatment effect (univariable analysis: specification 3)

Univariable analysis for independent variables excluded from the multivariable analysis

In univariable analysis, of 31 independent variables excluded from the multivariable analyses, 17 were statistically significant factors (table 2). However, these significances may be attributed to extremely large sample sizes and/or the absence of the other strong predictors in the model.

Table 2

Univariable meta-regression analysis

Online supplemental eTable 3 presents the effect sizes of significant factors impacting acupuncture’s effect in univariable analysis (excluded from multivariable analysis).

Significant factors in multivariable versus univariable analyses

Of the 15 independent variables, multivariable analysis proved five significant factors associated with the magnitude of effect; in contrast, univariable analysis proved 14 (table 2).

Discussion

Principal findings

We conducted a meta-epidemiological study including 1304 effect estimates from 584 RCTs. Our robust three-level mixed multivariable analyses identified five significant factors that impacted the magnitude of the acupuncture effect. Acupuncture produced the largest treatment effect on quality-of-life, followed by function, pain, non-pain symptoms and major events. Penetration acupuncture induced a larger effect than non-penetration acupuncture. High-frequency acupuncture sessions, single-centred acupuncture RCTs and acupuncture RCTs that did not report funding are associated with larger effects.

Strengths and limitations of the study

This study is the first three-level multivariable meta-epidemiological analysis that included the largest number of RCTs across all therapeutic areas, exploring factors associated with acupuncture’s treatment effect. Hence, the rigorous study provided robust results on critical design factors for acupuncture trialists to consider when designing future RCTs. This study provided a favourable type of acupuncture and treatment regimen for patients, clinicians and policy-makers to achieve acupuncture’s maximum treatment effect for clinical and health system decisions. Our study has several strengths. First, our study is highly patient-centred and clinically relevant. To ensure the conclusion from our study is the most pertinent for healthcare decision making, we included only patient-important outcomes. We consulted a group of international clinicians, researchers and patients when choosing the independent variables.

Second, we constructed a robust three-level mixed model multivariable analysis to adjust for multiple variables to reduce the potential bias raised from the univariable analysis. To deal with the collinearity and substantial amount of outlier and influential values in our datasets, we used Cramer’s V and the weighting approach of robust regression.

Third, our study has a high methodological rigour. We worked with an experienced medical librarian to develop a systematic and exhaustive search strategy. Teams of reviewers then screened and extracted data independently and in duplicate, with third-party adjudication of disagreement.

Our study has several limitations. First, we used a cut-off value of 0.7 in Cramer’s V statistics to identify collinearity, and when applicable, dropped the less important independent variable. Others might find a cut-off of 0.7 being too stringent and therefore left out too many independent variables from the multivariable model. Second, acupuncture RCTs poorly reported the risk of bias and acupuncture techniques related factors. Thus, we could not include some important independent variables such as practitioners’ experience in the multivariable analyses. Finally, some factors (eg, country, trial registered) distributed extremely imbalanced, limiting the results' generalisability.

Comparison with other studies

Previous studies9–15 typically performed univariable analyses in a small number of studies (5 to 39 trials) and identified 15 significant factors, including 10 clinical, 1 methodological and 4 other factors. Although our univariable analyses confirmed all these factors, the multivariable analyses identified only five significant factors.

An individual patient data meta-analysis (IPDMA) on chronic pain trials found the total number of acupuncture treatments was a significant factor9 15 and more treatment sessions were associated with better effects when comparing acupuncture to no acupuncture controls. Meta-regression studies also revealed the same results.11–13 However, due to a considerable amount of studies that didn't report the number of treatment sessions, we could not include total number of acupuncture treatment sessions in our multivariable analysis.

One study suggested treatment frequency as a significant predictor for tension-type headaches (more frequent treatment, larger effects)14 while others did not.9 15 In our multivariable analyses, the frequency of treatment sessions proved a significant factor. Some studies included homogeneous treatment frequency9 15 whereas others included varied frequency, leading to different findings.

For the type of sham acupuncture, the IPDMA9 15 reported that compared with non-penetrating and non-needle sham, penetrating needle sham associated with a larger effect. In contrast, a systematic review10 found no association between the type of sham and acupuncture’s treatment effect. Similarly, our multivariable analyses did not identify the type of sham as a significant factor.

Implications for practice and research

When feasible and acceptable, patients, clinicians and policy-makers should consider using penetrating over non-penetrating types of acupuncture with more frequent treatment sessions.

Identifying significant factors for acupuncture’s treatment effect in trials has important implications for future trials design and conducting secondary analyses. When trialist collaboration designs an acupuncture trial: (1) they should follow Consolidated Standards of Reporting Trials29 and STandards for Reporting Interventions in Clinical Trials of Acupuncture30 reporting guidelines, especially for those that might impact the treatment effect (random sequence generation and allocation concealment, acupuncture technique related information, practitioners related information, and the source of funding); (2) consider the quality of life outcome more often; (3) carefully choose the type of acupuncture, frequency of treatment sessions, choice of single or multicentre as those impact the treatment effect. When exploring factors associated with acupuncture’s treatment effect, researchers should use multivariable analyses over univariable analyses to avoid confounding variables caused biases. Researchers can further investigate factors excluded from multivariable analyses (eg, practitioners’ expertise).

Data availability statement

All data relevant to the study are included in the article or uploaded as online supplemental information.

Ethics statements

Patient consent for publication

Acknowledgments

We thank the global panel, including Zhisun Liu, Baoyan Liu, Hui Zheng, Lee Myeong Soo, Tae-Hun Kim, Caroline Smith, Kim L Bennell, Jun Mao, Lixing Lao, Michael E Wechsler, Karen J Sherman, Andrew J Vickers, Emily Vertosick, Benno Brinkhaus, Klaus Linde, Cummings Mike, Anna Kim, Jiani Wu, Yan Liu, Mohit Bhandari, Philip J Devereaux, and Jianping Liu for ranking the importance of a list of factors, and Jun Mao, Lixing Lao, Klaus Linde and Dawn Richards for discussing the paper’s content at the Society of Acupuncture Research 2021 International Research Conference, and Daniel Pérez Rada for supporting the online screening system.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • X-HJ and Y-QZ contributed equally.

  • Contributors XHJ, YQZ and WJG had the idea and designed the study. GG was involved in designing the study. YQZ, WJG, and ZZ designed the search strategy. WJG, WCX, LJS, RMJ, JWY, XSS, XYS, ZYZ, HCL, JTS, ALC, ZYQ, LZ, DXM, XYW, QQY, LZC, YTH, TVP, JC, CH, LB, KL, KT, JT, JED, JZ, MH, YS, KW, LCL, JW, TAB, LGM, MRA, MDG, CDCBM, JM, CC and FB screened abstracts. WJG, WCX, LJS, RMJ, JWY, XSS, XYS, ZhYZ, HCL, JTS, ALC, ZYQ, LZ, DXM, XYW, QQY, LZC and YTH screened full texts. WJG, WCX, LJS, RMJ, JYW, XSS, and XYS extracted data. WCX coordinated the reviewers’tasks. QZ proposed the analysis plan and analyzed the data. LT reviewed and confirmed the statistical analysis plan. WJG, YQZ and QZ drafted the manuscript, with revision from all authors. YQZ and GG substantially revised the manuscript. XHJ is the guarantor. The corresponding author attests that all listed authors meet authorship criteria and that no others have been omitted.

  • Funding This research was supported by the China Academy of Chinese Medical Sciences (No. CI2021A03503, GH201901, 2020YJSZX-1) and the National Natural Science Foundation of China (No. 81973968).

  • Disclaimer The funders had no role in considering the study design, analysis, interpretation of data, writing of the report, or decision to submit the article for publication.

  • Competing interests All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare support from supported by China Academy of Chinese Medical Sciences (CACMS) Innovation Fund, the National Natural Science Foundation of China, the Fundamental Research Funds for the Central public welfare research institutes, and China Center for Evidence-Based Traditional Chinese Medicine for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.

  • Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Linked Articles