Objectives To examine variation in outcomes, outcome measurement instruments (OMIs) and measurement times in clinical trials of non-valvular atrial fibrillation (NVAF) and to identify outcomes for prioritisation in developing a core outcome set (COS) in this field.
Design This study was a systematic review.
Data sources Clinical trials published between January 2015 and March 2019 were obtained from PubMed, the Cochrane Library, Web of Science, Wanfang Database, the China National Knowledge Infrastructure and SinoMed.
Eligibility criteria Randomised controlled trials (RCTs) and observational studies were considered. Interventions included traditional Chinese medicine and Western medicine. The required treatment duration or follow-up time was ≥4 weeks. The required sample size was ≥30 and≥50 in each group in RCTs and observational studies, respectively. We excluded trials that aimed to investigate the outcome of complications of NVAF, to assess the mechanisms or pharmacokinetics, or for which full text could not be acquired.
Data extraction and synthesis The general information and outcomes, OMIs and measurement times were extracted. The methodological and outcome reporting quality were assessed. The results were analysed by descriptive analysis.
Results A total of 218 articles were included from 25 255 articles. For clinical trials of antiarrhythmic therapy, 69 outcomes from 16 outcome domains were reported, and 28 (31.82%, 28/88) outcomes were reported only once; the most frequently reported outcome was ultrasonic cardiogram. Thirty-one outcomes (44.93%, 31/69) were provided definitions or OMIs; the outcome measurement times ranged from 1 to 20 with a median of 3. For clinical trials of anticoagulation therapy, 82 outcomes from 18 outcome domains were reported; 38 (29.23%, 38/130) outcomes were reported only once. The most frequently reported outcome was ischaemic stroke. Forty (48.78%, 40/82) outcomes were provided OMIs or definitions; and the outcome measurement times ranged from 1 to 27 with a median of 8.
Conclusion Outcome reporting in NVAF is inconsistent. Thus, developing a COS that can be used in clinical trials is necessary.
- clinical trials, non-valvular atrial fibrillation
- systematic review
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Strengths and limitations of this study
This systematic review is the first to describe variation in outcomes, outcome measurement instruments and outcome measurement time reporting in clinical trials for non-valvular atrial fibrillation (NVAF).
The methodology is reproducible and transparent and has been assessed during a peer-review process.
English and Chinese databases were searched, and randomised controlled trials and observational studies were considered.
The aim of this review was to provide a list of outcomes for clinical trials of NVAF in traditional Chinese medicine, which is focused on Chinese herbal medicine therapy. Thus, clinical trials of surgery were not considered.
According to a systematic review, atrial fibrillation (AF) is the main contributor to many diseases, such as ischaemic heart disease, stroke, renal disease and peripheral arterial disease. In addition, AF usually results in major cardiovascular events, cardiovascular and all-cause mortality, and sudden cardiac death.1 Thus, treating AF is important.
There are different kinds of classifications for AF. According to the aetiology, AF can be classified as isolated AF, valvular AF, non-valvular AF (NVAF) and so on. NVAF refers to AF occurring without rheumatic mitral stenosis, mechanical/bioprosthetic or mitral valve repair.2 According to the characteristics and timing of AF onset, AF can be classified as first diagnosed AF, paroxysmal AF, persistent AF, long-standing persistent AF and permanent AF.3
Current evidence has shown that catheter ablation and drug therapy are beneficial for controlling heart rhythm, maintaining ventricular rate, and preventing thrombosis and stroke. However, the arrhythmogenic effects and risk of death after taking antiarrhythmic drugs cannot be ignored. With the increasing number of traditional Chinese medicine (TCM) clinical trials in treating AF, the efficacy and safety of TCM have been proven.4 However, there are some problems in these TCM clinical trials; for example, similar clinical trials reported different outcomes. Therefore, some trials cannot be included in systematic reviews/meta-analyses because of outcome reporting heterogeneity. In TCM clinical trials, the long-term outcomes, patient-reported outcomes and safety outcome reporting are limited; thus, these trials cannot provide appropriate evidence for TCM in treating AF. Developing a core outcome set (COS) may resolve these problems.
A COS is a minimum set that should be measured and reported in all clinical trials for a specific condition.5 According to the characteristics and advantages of TCM, we intend to develop a COS for TCM clinical trials for NVAF, with registered6 and published7 protocols.
According to the study protocol, conducting a systematic review is the first step in the development of a COS for NVAF to develop a long list of outcomes. In this research, we will report the results of the systematic review, including assessing the quality of outcome reporting and the quality of trials, as well as examining the variation in outcome reporting, outcome measurement instrument (OMI) reporting and measurement time point reporting.
In clinical trials and clinical practice, TCM, especially Chinese herbal medicine therapy is often used as an adjuvant therapy in internal medicine treatment; thus, obtaining a comprehensive list of outcomes for TCM clinical trials is difficult. In this systematic review, we focused on clinical trials of TCM, integrated medicine and Western medicine in internal medicine. The literature database included PubMed, the Cochrane Library, Web of Science, Wanfang database, the China National Knowledge Infrastructure and SinoMed. A literature search was conducted two times. The first search was conducted from January 2015 to June 2017, and the second search was conducted from May 2017 to March 2019. The search strategy for English databases is shown in online supplementary additional file 1.
According to the protocol, both randomised controlled trials (RCTs) and observational studies were considered. Patients with NVAF who accepted interventions including TCM or Western medicines were eligible. The required treatment duration or follow-up time was ≥4 weeks. For RCTs, the required number of participants was ≥30 in each group. For observational studies, the required number of participants was ≥50.
We excluded clinical trials that aimed to investigate the outcome of complications of NVAF, to assess the mechanism of drug action or pharmacokinetics, or for which full text could not be acquired.
Methodological quality has little influence on developing a long list of outcomes in the development of a COS. However, we excluded some studies with serious problems, such as a Jadad score of 0 for RCTs, contradictions in the research or the authors are in the institutions who do not have the ability to conduct RCTs in China.
Two reviewers (RQ and SH) independently assessed the titles and abstracts from searches. Then, the full texts of the potential articles were retrieved and assessed for further identification. Any disagreement was resolved by discussion or consulting the third investigator (HS).
Two reviewers (RQ and JH) independently extracted information. The information included the first author’s name, publication time, number of participants, country of authors (if the authors are from different countries, it was stated as ‘international’), interventions, comparisons, course of treatment, follow-up duration, outcomes, the definition of outcomes, OMIs and measurement time (intervention duration or follow-up time). Any disagreement was resolved by discussion or consulting the third investigator (HS).
In addition, we assessed the quality of outcome reporting according to the method used in other studies.8 9 There were six items; if the information of eligible studies completely meet the items, then 1 point was awarded. If this information did not meet or fully meet the items, then 0 point was awarded. If the outcome was objective, then the definition is unnecessary.
The items include the following:
Is the primary outcome clearly stated?
Is the primary outcome clearly defined so that another researcher would be able to reproduce its measurement? Where appropriate, this outcome should include a clear description of time points, the person measuring the outcome, how the outcome was measured (for example, tools and methods used) and where the outcome was measured.
Are the secondary outcomes clearly stated?
Are the secondary outcomes clearly defined?
Do the authors explain the use of the outcomes they have selected?
Are methods used to enhance the quality of outcome measurement (for example, repeated measurement, training) if appropriate?
The methodological quality was assessed according to the type of study. The Jadad score was used to assess the quality of RCTs,10 and the Newcastle-Ottawa Scale was used to assess the quality of cohort studies (CoSs).11 The tool developed by Canadian Institute of Health Economics can be used to assess the quality of case series studies.12
Two reviewers (RQ and JH) independently assessed the quality of outcome reporting and the methodological quality. Any disagreement was resolved by discussion or consulting the third investigator (HS).
Merging outcomes and grouping under outcome domains
Two researchers (RQ and CZ) merged the overlapping outcomes according to the definition of outcomes independently. If no definition was provided, they discussed and achieved consensus if necessary. For example, death, death from any cause, mortality, overall mortality, total mortality, all causes of death and all causes of mortality were aggregated as ‘all-cause mortality’.
The original list of outcomes from systematic review is usually very long and unwieldy,13 so researchers developed a taxonomy for outcome classification14 that included 38 outcome domains. Two researchers (RQ and CZ) grouped individual outcomes into the appropriate outcome domain together and achieved consensus.
The results were analysed by descriptive analysis.
Patient and public involvement
Patients and the public were not involved in the design or planning of the study. Patients will be involved in the larger study to develop the COS. Informed consent will be obtained from patients who will participate in the later research.
Characteristics of literature
In this systematic review, a total of 25 255 articles from Chinese and English databases were retrieved. After removing duplicates, there were 17 240 articles. By reading the titles and abstracts, ineligible articles were removed, and full texts for 1233 potential eligible articles were retrieved. A total of 1015 articles were removed for various reasons, and 218 articles were finally included. The flowchart of this systematic review is shown in figure 1.
In the included studies, 88 studies were for antiarrhythmic therapy, and 130 studies were for anticoagulant therapy. A total of 110 articles were in Chinese, and 108 articles were in English. Thirty articles were TCM clinical trials, and 188 were Western medicine clinical trials. Seventy-five articles were observational studies (including 66 CoS and 9 case series), while 143 articles were RCTs. The general characteristics of the included articles are shown in table 1.
The majority of RCTs were conducted in China. The USA had more CoSs than other countries did (figure 2). Because of the limited information provided in the articles, 35.32% (77/218) of the studies received 0 points for the quality of outcome reporting, and the majority were RCTs (figure 3). Compared with other countries, China had a much lower quality of outcome reporting (figure 4). The majority of RCTs were poor quality, while the majority of observational studies were high quality.
The list of outcomes
There are two main types of therapy for NVAF: antiarrhythmic treatment and anticoagulation treatment. Some differences exist in the outcome reporting between these therapies. This review shows the outcomes according to the type of interventions in the original study.
For clinical trials of antiarrhythmic therapy, 69 outcomes from 16 outcome domains were reported (table 2). Twenty-eight (31.82%, 28/88) outcomes were reported only once; the most frequently reported outcome was ultrasonic cardiogram, which was reported 39 times (44.32%, 39/88). None of the outcomes were reported more than 50 times. In the 16 outcome domains, 5 outcome domains (vascular outcomes, adherence/compliance, adverse events/effects; physical functioning; withdrawal from treatment) consisted of only one outcome. These outcomes were reported between 1 and 26 times, and the median outcome reporting time was 1. Cardiac outcomes consisted of the largest number of outcomes, including 22 outcomes. In cardiac outcomes, ultrasonic cardiogram (39 times), AF recurrence (36 times), conversion to sinus rhythm (26 times), heart rate (21 times) and blood pressure (20 times) were reported much more often than other outcomes.
For clinical trials of anticoagulation therapy, there were 82 outcomes from 18 outcome domains in the studies of anticoagulation therapy (table 3). Thirty-eight (29.23%, 38/130) outcomes were reported only once; the most frequently reported outcome was ischaemic stroke, which was reported 105 times (80.77%, 105/130). Only 5 (3.85%, 5/130) outcomes were reported more than 50 times. In the 18 outcome domains of anticoagulation therapy studies, 5 outcome domains (satisfaction/patient preference, withdrawal from treatment, global quality of life, economic and adverse events/effects) consisted of only one outcome. These outcomes were reported between 1 and 16 times, and the median outcome reporting time was 3. Blood and lymphatic system outcomes included the largest number of outcomes, which was 14 outcomes; the international normalised ratio (INR) was reported more frequently than other outcomes.
There were 24 duplicated outcomes between antiarrhythmic therapy and anticoagulation therapy. After removing duplicates, there were 127 outcomes. Figure 5 shows a summary of outcomes reporting times. Figure 6 shows the number of outcomes in different outcome domains in antiarrhythmic treatment trials. Figure 7 shows the number of outcomes in different outcome domains in anticoagulation treatment trials.
A large number of clinical trials did not provide definitions or OMIs. In the outcomes of antiarrhythmic treatment trials, 31 outcomes (44.93%, 31/69) were provided definitions or OMIs. Twenty-three (33.33%, 23/69) outcomes were provided one OMI or definition, seven (10.14%, 7/69) outcomes were provided two OMIs or definitions and one (1.45%, 1/69) outcome was provided three OMIs or definitions. Sinus rhythm maintenance had three different OMIs or definitions, which was higher than that of other outcomes. In the outcomes of anticoagulant therapy trials, 40 (48.78%, 40/82) were provided OMIs or definitions. Twenty-eight (35.37%, 28/82) outcomes were provided one OMI or definition, seven (8.54%, 7/82) outcomes were provided two OMIs or definitions and five (6.10%, 5/82) outcomes were provided three or more OMIs or definitions. Major bleeding had more definitions than other outcomes did.
In addition, there were many different measurement times for the same outcome. In the clinical trials of antiarrhythmic treatment, the outcome measurement times ranged from 1 to -14 times, and the median time was 3.Forty-three outcomes (62.32%, 43/69) had two or more measurement times. Heart rate and ECG outcomes had more measurement times than other outcomes did. In clinical trials of anticoagulant therapy, the outcome measurement times ranged from 1 to 56, with a median of 1.5; among these outcomes 41(50.00%, 41/82) had two or more measurement times. In addition, ischaemic stroke had more measurement times than other outcomes did.
This systematic review is the first to evaluate the quality of outcome reporting of clinical trials of TCM and western medicine for treating NVAF. The results showed variations in the outcome reporting, OMIs/outcome definitions and outcome measurement time reporting in different clinical trials. These problems may result in the exclusion of some studies from systematic reviews/meta-analyses due to the heterogeneity of outcomes or outcome measurements; thus, these studies cannot provide a higher level of evidence for clinical practice.
In clinical trials for NVAF, investment wastes also exist because approximately 1/3 of outcomes were reported only once in included trials of anticoagulation therapy and antiarrhythmic therapy. For example, conversion to sinus rhythm, which is important to the results of clinical trials of antiarrhythmic therapy, was reported by 29.55% (26/88) of articles. Some long-term outcomes, such as all-cause mortality and cardiovascular deaths, were reported in 12.50% (11/88) and 5.68% (5/88) of articles, respectively.
In addition, adverse events/effects were inadequately reported. In clinical trials of anticoagulant therapy, safety outcomes such as haemorrhage were grouped under vascular outcomes according to the degree of bleeding (such as major bleeding, clinically relevant non-major bleeding and minor bleeding). Then, only 12.31% (16/130) of the included articles reported other kinds of adverse events/effects. For clinical trials of antiarrhythmic therapy, only 29.55% (26/88) of the included articles reported adverse events/effects.
For all of the outcomes in the list, patient’ perspectives could not be identified sufficiently. For example, among all of the included 88 articles for antiarrhythmic therapy, none of them reported quality of life, while in all of the included 130 articles for anticoagulant therapy, only 4 of them reported quality of life.
There were 30 articles for clinical trials of TCM. TCM syndrome, which could reflect the characteristics of TCM, was reported only two times. A few other articles reported symptoms related to TCM syndrome. This phenomenon cannot reflect the characteristics and advantages of TCM.
After assessing the quality of outcome reporting and studies, the results showed that the majority of included trials had poor quality. Although the poor quality of studies may not influence the result of developing a long list of outcomes, the poor quality of outcome reporting made it difficult to extract sufficient information from the articles. The reasons for poor quality of studies and outcome reporting may be because most studies in China do not follow the Consolidated Standards of Reporting Trials (CONSORT) statement or observational studies reporting items. Moreover, the majority of journals in Chinese do not require studies to follow the CONSORT statement; thus, some studies provided limited information on key methodological issues. In addition, Chinese researchers prefer to report comprehensive outcomes rather than individual outcomes, and studies have reported only primary outcomes.
Only a small number of included studies provided OMIs or definitions, which made it difficult to assess the quality of outcome measures. Additionally, the variation in OMIs or definitions can make it impossible to conduct meta-analyses. In addition, selecting OMIs with good measurement properties is very important after developing a COS15 to ensure that reliability, validity and ethical standards are achieved.
The measurement time was much shorter in Chinese journals than in English journals. In general, long-term outcomes were usually reported in observational studies, while short-term outcomes were usually reported in RCTs. It is a challenge for a single trial to measure all of these outcomes in a meaningful way, especially an outcome such as mortality, which requires longer follow-up and a larger sample size.16 Therefore, recommending measurement times for different outcomes is important.
Developing a COS for NVAF may reduce the heterogeneity of outcome reporting in different clinical trials, so that clinical trials can be included in systematic reviews/meta-analyses to provide a higher quality of evidence for clinical practice. Moreover, if the majority of clinical trials can be included in systematic review, it may help reduce investment wastes. Reviewers can easily determine if publication bias is present when a COS is used. For TCM clinical trials, a COS may help improve the quality of studies if researchers report consensus outcomes, which may help improve the development of TCM.
RQ and JH contributed equally.
Contributors RQ, JC and HS contributed to the study design. RQ extracted data from articles and drafted the manuscript. SH, YH, TH, CZ and JH contributed to the data extracting and assessment. ML, JH, JC, YL, MG and HS revised the manuscript. All authors read and approved the final manuscript.
Funding This work was supported by the National Natural Science Foundation of China .
Competing interests None declared.
Patient consent for publication Not required.
Ethics approval Ethical approval has been granted by the Ethics Committee of Dongzhimen Hospital, Beijing University of Chinese Medicine (DZMEC-KY-2017–81).
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement The data are from published papers and do not include identifiable patient data.