Article Text
PDF
Abstract
Introduction The pragmatic design has received much attention in the field of acupuncture clinical trials because of insufficient information about the specific effects of acupuncture. However, pragmatism in pragmatic acupuncture trials has not been comprehensively investigated. The PRECIS-2 tool was developed and has been gradually used to design pragmatic trials; therefore, we will apply the PRECIS-2 tool to investigate the pragmatism of pragmatic acupuncture trials in this study.
Methods and analysis In this systematic review, self-declared ‘pragmatic’ randomised clinical trials (RCTs) or protocols of self-declared ‘pragmatic’ RCTs investigating acupuncture will be searched and included to be reviewed. MEDLINE, EMBASE, the Cochrane Central Register for Controlled Trials, CINAHL, Allied and Complementary Medicine Database (AMED), China National Knowledge Infrastructure, VIP, WANFANG, Taiwan Periodical Literature Database, KoreaMed, KMbase, Research Information Service System, Oriental Medicine Advanced Searching Integrated System, CiNii and ClinicalTrials.gov for registered trials will be electronically searched from inception to March 2022. Protocols of published RCTs or secondary analysis of RCTs will be excluded. Additionally, no language restriction will be applied. Two authors will independently extract descriptive information and assess the pragmatism of pragmatic acupuncture trials using nine domains of the PRECIS-2 tool and one additional domain—control. Descriptive statistics will be reported for each domain and the overall score, and a one-sample t-test will be used to statistically analyse whether the score is greater than 3 (equally pragmatic and explanatory). The wheel diagrams of the nine domains of the PRECIS-2 tool will be used to demonstrate the pragmatism of the included studies.
Ethics and dissemination Ethical approval is not warranted as this study will obtain data from previously reported articles. The results will be disseminated through peer-reviewed journals and conferences.
PROSPERO registration number CRD42021236975.
- COMPLEMENTARY MEDICINE
- STATISTICS & RESEARCH METHODS
- Clinical trials
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This protocol will be the first to assess the pragmatism of self-declared pragmatic acupuncture trials.
The pragmatism of trials will be evaluated using PRECIS-2 tool.
Any type of acupuncture, including electroacupuncture and microsystem acupuncture, will be included.
Assessing the risk of bias and the quality of reporting of trials is not included in this protocol.
Trials with pragmatic intentions will be excluded unless they are self-declared as ‘pragmatic’ in titles, abstracts or manuscripts.
Introduction
Acupuncture is a treatment modality used in traditional East Asian medicine. It stimulates acupuncture points on the body with acupuncture needles to manage various symptoms and diseases.1 Scientific clinical trials have been conducted to assess the effect of acupuncture on several diseases and address the mechanism of acupuncture treatment.2–4 However, the specific efficacy and placebo effect of acupuncture have not been clearly revealed. Consequently, explanatory clinical trials have been unable to establish the exact efficacy of acupuncture.5 The methodology of acupuncture clinical trials has been continuously discussed, and researchers have tried to report reliable results for decision-makers.6 7 Alternatively, pragmatic acupuncture trials designed to evaluate the effectiveness of acupuncture treatments in real-world practice conditions have been conducted, and several studies have tried to show clinical benefits of acupuncture over conventional treatments even though the mechanism and specific efficacy could not be verified.8
To methodologically assess the pragmatism of trials, the PRagmatic-Explanatory Continuum Indicator Summary 2 (PRECIS-2) tool has been recently developed,9 and it consists of nine domains: eligibility criteria, recruitment, setting, organisation, flexibility (delivery), flexibility (adherence), follow-up, primary outcome and primary analysis. The tool has been shown to have sound reliability and validity,10 and when it is used retrospectively to assess clinical trials, one additional domain—control—has been suggested.11 Unfortunately, pragmatic acupuncture trials have not been comprehensively assessed with this tool and other tools to investigate the extent of their pragmatic design. In the field of acupuncture trials, the pragmatic design has received much attention; however, the assessment of relevance has not been studied. Therefore, this systematic review aims to investigate the methodological characteristics of pragmatic acupuncture trials using the PRECIS-2 tool and assess whether the trials are designed appropriately to be applied to the real-world environment.
Methods and analysis
Design
This study is a protocol of systematic review and follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Protocols guideline online supplemental file 1).12 The results will be a systematic review in accordance with the PRISMA guideline.13
Supplemental material
Inclusion criteria of the studies in this review
Types of studies
Randomised clinical trials (RCTs) and RCT protocols that state it is a pragmatic design, published until March 2022, will be searched and included in this study. The inclusion criteria are (1) RCTs and RCT protocols self-declared as ‘pragmatic’ in title, abstract or manuscript, and (2) RCTs and RCT protocols of interventions that include acupuncture treatment. The exclusion criteria are (1) protocols of RCTs already published with results, (2) secondary analyses of published RCTs and (3) studies that use the word ‘pragmatic’ not in a methodological manner.
Type of participants
We will include participants with all the possible conditions or diseases; however, healthy participants will be excluded unless the study is a prevention study.
Type of interventions
Any type of acupuncture including manual acupuncture, electroacupuncture and microsystem acupuncture such as auricular acupuncture and acupoints acupressure will be included. RCTs investigating complex interventions without acupuncture will be excluded.
Information sources and search strategy
MEDLINE, EMBASE, the Cochrane Central Register for Controlled Trials, CINAHL, Allied and Complementary Medicine Database (AMED), four Chinese databases (China National Knowledge Infrastructure, VIP, WANFANG and Taiwan Periodical Literature Database), four Korean databases (KoreaMed, KMbase, Research Information Service System, and Oriental Medicine Advanced Searching Integrated System), CiNii for Japanese literature and ClinicalTrials.gov for registered trials will be electronically searched from inception to March 2022. The research terms for each database are provided in online supplemental file 2. If necessary, appropriate articles will be manually retrieved. Additionally, no language restriction will be applied.
Selection of studies
Duplicates will be removed before the screening. After reviewing the titles and abstracts, JL and HL will first select the studies and collect the manuscripts of relevant articles. Next, after indexing, the two reviewers will independently review the manuscripts of the articles and include or exclude the articles based on the inclusion/exclusion criteria.
Data extraction and applying the PRECIS-2 tool to the included studies
General information about the studies, including the first author, publication year, country, intervention used in the experimental and control groups, and primary outcome measures will be extracted by JL and HL. The PRECIS-2 tool will be used to investigate the pragmatic characteristics of the included trials. Ten domains will be used, and two authors will first review 10% of the included articles and discuss the criteria. Based on the criteria, JL and HL will assess the other articles. The two reviewers will independently review the articles and discuss the scores of the PRECIS-2 tool for each article. The following descriptive information and rationale for the scores of 10 domains will be independently extracted and summarised: (1) eligibility criteria, (2) recruitment methods, (3) trial setting and number of centres, (4) organisational information—expertise and resources, (5) intervention delivery protocol and flexibility of the delivery, (6) methods to manage the adherence of participants, (7) follow-up features: the frequency and duration of follow-ups and additional data collection, (8) primary outcome measures, (9) primary analysis methods and (10) intervention in control groups. If there is inconsistency, a discussion will be held with Y-SK. If there is inconsistency after the discussion, JL’s and HL’s mean score will be used.
Basically, the PRECIS-2 tool consists of nine domains; however, Zwarenstein et al recommended one additional domain—control—when the PRECIS-2 tool is retrospectively applied to clinical trials in systematic reviews.11 We will, therefore, use nine domains based on the recommendation of the PRECIS-2 tool9 with the control domain. When the control group is sham-controlled and considered as completely explanatory, the score is 1, and when the control group involves usual care without any discipline of treatments and is considered as completely pragmatic, the score is 5. If there is uncertainty regarding a domain, the score will be left blank, as suggested by Loudon et al.9 The flow chart of this study is shown in figure 1.13
Flow chart of the study.
Data analysis plan
Scores for each domain and the overall score for each article will be summarised using descriptive statistics, including mean/median, measure of variance, interquartile range and percentage. The wheel diagrams of the nine domains of the PRECIS-2 tool will be used to show the extent of the domain’s pragmatic design (figure 2).9
Wheel diagram of the nine domains of the PRECIS-2 (PRagmatic-Explanatory Continuum Indicator Summary 2) tool.9
According to Loudon et al,9 for the domains of flexibility: delivery, flexibility: adherence and control, if there are more than two groups, each group needs to be scored separately. Therefore, we will score each group separately; however, when it comes to data analysis, we will use the score of the group that was more related to acupuncture, and if all groups are related to acupuncture, we will use the highest score to reflect the potential pragmatism of the trial. A one-sample t-test will be applied to statistically analyse whether the score is greater than three (equally pragmatic and explanatory), and p<0.05 (null hypothesis: the score is not greater than three) will be considered statistically significant.
Risk of bias assessment
Since this study is a systematic review on the methodology of trials using the PRECIS-2 tool and is not about the clinical outcome, risk of bias will not be assessed.
Patient and public involvement
Patients and the public are not directly involved in this study as we will use data from already published articles.
Ethics and dissemination
Since we will obtain data from already published articles, ethical approval is not required. We plan to publish the results of the study through peer-reviewed journals and conferences and share the findings with the relevant trialists and researchers.
Discussion
The aim of this systematic review is not to investigate the efficacy or effectiveness of interventions but to investigate the methodological issue of acupuncture clinical trials. To the best of our knowledge, this study is the first systematic review protocol to comprehensively deal with the pragmatic design of acupuncture clinical trials. Previous systematic reviews used PRECIS-2 evaluating interventions of integrative medicine14 and Chinese herbal medicine15; however, this protocol will primarily focus on acupuncture and include various diseases or conditions. Furthermore, Dal-Ré et al reported that some self-labelled pragmatic trials showed explanatory features,16 and Neta and Johnson argued using PRECIS-2 tool to enhance ‘real-world’ evidence.17 This protocol will estimate the status of self-declared pragmatic acupuncture trials using PRECIS-2 tool.
As Loudon et al reported,9 defining a trial as pragmatic or explanatory is on a continuum rather than dichotomous. Trials having a pragmatic intention could be explanatory in some respects. PRECIS-2 has been developed considering the characteristics, and the results of this study will show the summary of the characteristics of pragmatic acupuncture trials and the sufficient and deficient components of pragmatic design in acupuncture trials on the continuum. Based on these results, although we will not be able to suggest the clinical advantages or disadvantages of acupuncture, we will be able to suggest the proper direction for future pragmatic trials, which will clearly reveal the advantages or disadvantages of acupuncture in the future.
The limitations of this study are as follows: (1) This study will not assess the risk of bias and reporting quality. Two previous studies reported the risk of bias and reporting quality of included trials with the results of PRECIS-2 assessments.18 19 However, since this study will evaluate the methodological features of trials in terms of pragmatism rather than reporting the clinical effect of interventions or quality of trials, assessing the risk of bias and reporting quality would be non-essential. (2) Search terms that could mean pragmatic intention are not included in this study. Previously, two studies assessed ‘self-labelled’ or ‘self-declared’ pragmatic trials, and used additional search terms including ‘practical’, ‘comparative effectiveness’ or ‘naturalistic’16 19; however, this study will include trials self-declared as ‘pragmatic’ and other terms will not be included in the search strategy.
Ethics statements
Patient consent for publication
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
JL and HL contributed equally.
Contributors JL and HL contributed equally to this work. JL and HL conceptualised this study and drafted the manuscript. Y-SK supervised this study. All authors approved the publication of the protocol.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.