You are here

  • Home
  • Archive
  • Volume 12, Issue 9
  • Develop an ADR prediction system of Chinese herbal injections containing Panax notoginseng saponin: a nested case–control study using machine learning

Article Text

Article menu
Medical management
Original research
Develop an ADR prediction system of Chinese herbal injections containing Panax notoginseng saponin: a nested case–control study using machine learning
  1. Xing-Wei Wu1,2,
  2. Jia-Ying Zhang3,
  3. Huan Chang1,
  4. Xue-Wu Song1,2,
  5. Ya-Lin Wen1,
  6. En-Wu Long1,2,
  7. Rong-Sheng Tong1,2
  1. 1Pharmacy, University of Electronic Science and Technology of China Sichuan Provincial People's Hospital, Chengdu, Sichuan, China
  2. 2Chinese Academy of Sciences Sichuan Translational Medicine Research Hospital, Chengdu, Sichuan, China
  3. 3Pharmacy, Chengdu First People's Hospital, Chengdu, Sichuan, China
  1. Correspondence to Dr Rong-Sheng Tong; 318004031{at}qq.com

Abstract

Objective This study aimed to develop an adverse drug reactions (ADR) antecedent prediction system using machine learning algorithms to provide the reference for security usage of Chinese herbal injections containing Panax notoginseng saponin in clinical practice.

Design A nested case–control study.

Setting National Center for ADR Monitoring and the Electronic Medical Record (EMR) system.

Participants All patients were from five medical institutions in Sichuan Province from January 2010 to December 2018.

Main outcomes/measures Data of patients with ADR who used Chinese herbal injections containing Panax notoginseng saponin were collected from the National Center for ADR Monitoring. A nested case–control study was used to randomly match patients without ADR from the EMR system by the ratio of 1:4. Eighteen machine learning algorithms were applied for the development of ADR prediction models. Area under curve (AUC), accuracy, precision, recall rate and F1 value were used to evaluate the predictive performance of the model. An ADR prediction system was established by the best model selected from the 1080 models.

Results A total of 530 patients from five medical institutions were included, and 1080 ADR prediction models were developed. Among these models, the AUC of the best capable one was 0.9141 and the accuracy was 0.8947. According to the best model, a prediction system, which can provide early identification of patients at risk for the ADR of Panax notoginseng saponin, has been established.

Conclusion The prediction system developed based on the machine learning model in this study had good predictive performance and potential clinical application.

  • adverse events
  • herbal medicine
  • toxicity

Data availability statement

Data are available upon reasonable request. Data are available upon reasonable request. Data may be obtained from a third party and are not publicly available. The first author (7190175@uestc.edu.cn) will share any publicly available data if requested by email.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

http://dx.doi.org/10.1136/bmjopen-2022-061457

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • To the best of our knowledge, this study was the first to develop an adverse drug reaction (ADR) prediction system for Chinese herbal injection containing Panax notoginseng saponin using machine learning.

    • Data of patients with ADR came from the National Center for Adverse Drug Reaction Monitoring, which is highly representative.

    • In order to obtain the best model, the data processing adopted 4 data filling, 5 data sampling, 3 variable selection methods and 18 machine learning algorithms were applied for model establishment.

    • The area under curve, accuracy, precision, recall rate and F1 value were used to evaluate the predictive performance of the model.

    • As the study population was all from southwest China, the results may be biased while the prediction system was applied in other medical institutions.

    Introduction

    Panax notoginseng saponins, as the main ingredients of Panax notoginseng (Buck.) F.H.Chen, has been widely used in the disease therapy of nervous system and cardiocerebral vascular system.1–4 High frequency of adverse drug reactions (ADR) in Chinese herbal containing Panax notoginseng saponin has received widespread attention. Among these ADR, about 69.57% were caused by injections, mainly manifested as drug eruption (50.5%), allergic reaction (20.4%) and anaphylactic shock (9.7%), which can be life-threatening in severe cases.5

    At present, ADR is mainly monitored by spontaneous reporting system, case–control study, cohort study, prescription event monitoring and centralised hospital monitoring system. However, most of these methods have obvious hysteresis. Therefore, there is an increasing need to develop an ADR antecedent prediction system to prevent the occurrence of ADR in Chinese herbal injections containing Panax notoginseng saponin.

    Machine learning, the core technology of artificial intelligence, is commonly used to build prediction models. In recent years, some prediction models for ADR have been established.6–10 Based on a clustering method for the postprocessing of association rules, Wei and Scott6 developed an application of stepwise association rule mining to identify the associations between vaccine and multiple adverse events. In addition, Imai et al10 used artificial neural networks to evaluate vancomycin-induced nephrotoxicity. However, small sample size, incomplete patient information and unsatisfactory predictive performance restrict the application of ADR prediction models in clinical practice. In view of these challenges, this study aimed to develop an ADR prediction system of Chinese herbal injections containing Panax notoginseng saponin based on machine learning algorithms and provide reference for clinical ADR management and prevention.

    Methods

    Data collection

    Patients with ADR who used Chinese herbal injections containing Panax notoginseng included in this study were from the National Center for Adverse Drug Reaction Monitoring reported by five hospitals in Sichuan Province from January 2010 to December 2018. Then, a nested case–control study was used to randomly match patients without ADR from the Electronic Medical Record system of the five medical institutions. The ratio of patients with ADR to those without ADR was 1:4. For multiple lab results, in order to facilitate clinical application, we selected the last results of patients before the usage of medication. And for multiple admissions, all patients were included according to their first admission.

    Data cleaning

    Variable assignment

    Binary-state variables were directly assigned values of 0 or 1. According to whether in the normal range, clinical laboratory variables were assigned values of 1, 2 and 3 (1, below the normal range; 2, within the normal range and 3, above the normal range).

    Column deletion

    Variables with missing data >90%, or a single category >90%, or the coefficient of variation <0.1 were deleted.

    Data filling

    There are four ways to data filling. No filling: retained the original data. Simple filling: missing data of continuous variables replaced by the mean or median and categorical variables by the mode. Random Forest (RF) filling: used the RF model to predict and replace the missing data directly. RF improve filling: ordered variables based on the number of missing data that were replaced by RF filling next.

    Data sampling

    No sampling: built models from the original data. Random over sampler: randomly replicated the data of fewer categories to match the sample size to that of more categories. Random under sampler: deleted the data of more categories to match the sample size to that of fewer categories. Synthetic minority oversampling technique (SMOTE) over sampler: synthesise new data from a small amount of original data. Borderline SMOTE over sampler: synthesise new data from borderline data.

    Variable selection

    No variable selection or use Lasso or Boruta for variable selection.

    Model establishment

    Through different data filling, data sampling and variable selection, 60 data sets were obtained. Eighteen machine learning algorithms, including AdaBoost, Bagging, Bernoulli Naïve Bayes, Decision Tree, Extra Tree, Gaussian Naïve Bayes, Gradient Boosting, K-Nearest Neighbour, Latent Dirichlet Allocation, Logistic Regression, Multinomial Naïve Bayes, Passive Aggressive, Quadratic Discriminant Analysis, RF, Stochastic Gradient Descent, Support Vector Machine, eXtreme Gradient Boosting and Ensemble Learning, were used to build models.

    The model establishment was as follows. The data were randomly divided into a training set and a test set by the ratio of 8:2. The training set was used to build models, and the test set was used to evaluate the predictive performance of the models. Ten-fold cross-validation on the training set was applied for internal validation of the model, and 200 Bootstrapping samples from the test set for the evaluation of the impact of different data processing methods or machine learning algorithms on model predictive performance. Ensemble learning models were developed by five machine learning algorithms with the largest area under curve (AUC) on each data set.

    Model evaluation

    We used the AUC, accuracy, precision, recall rate and F1 value to evaluate the predictive performance of the model. Five models with the largest AUC were compared, and the best model was selected to develop an ADR prediction system of Chinese herbal injections containing Panax notoginseng saponin. SHapley Additive exPlanations (SHAP) helped to explain the contribution of variables to the model.

    Sample size assessment

    To evaluate the influence of different sample sizes on model predictive performance, randomly extracted 10%, 20%, 30% to 100% subsets from the training set by Bootstrapping. The 10 subsets were used to establish models, respectively. Repeated the procedure 100 times and the AUC, calculated from the testing set, was used for sample size examination.

    Patient and public involvement

    Patients and/or the public were not directly involved in this study.

    Statistical analysis

    Categorical variables were expressed as counts and percentages and continuous variables as mean±SD. Analysis of variance will be used if the data were normally distributed and the variances were equal, otherwise, Kruskal-Wallis test will be used. p value <0.05 was considered statistically significant. Hypothesis testing and models building were implemented using the stats and sklearn packages in Python (V.3.8), respectively.

    Results

    Research population

    A total of 530 patients were enrolled in this study, of which 106 patients had ADR. The patients included 250 (47.17%) men and 280 (52.83%) women. The demographic and clinical characteristics of the patients are shown in online supplemental table 1.

    Data cleaning

    The results of 83 variables assignment are shown in online supplemental table 2. After the column deletion, 63 variables were included in the following study (online supplemental table 3). Then, four data filling methods were used for replacing the 1290 (3.86%) missing data. We used Lasso or Boruta for variable selection, and the results are shown in online supplemental table 3. Using four data filling, five data sampling and three variable selection methods for data processing, respectively, 60 data sets were obtained.

    Model establishment

    A total of 1080 prediction models were established by 18 machine learning algorithms and 60 data sets. The results of 10-fold cross-validation are shown in online supplemental table 4. Using 200 Bootstrapping samples from the test set to evaluate the impact of different data processing methods or machine learning algorithms on model predictive performance. The results showed that differences of model predictive performance exist by different data filling, data sampling, variable selection (table 1) and machine learning algorithms (table 2). The ensemble learning model had the best performance with an AUC of 0.793±0.083 (table 2).

    View this table:
    Table 1

    The effect of different data processing methods on model prediction performance (bootstrapping)

    View this table:
    Table 2

    The effect of different machine learning algorithms on model prediction performance (bootstrapping)

    Model evaluation

    The AUC, accuracy, precision, recall rate and F1 value were used to evaluate the performance of the model. The best five models were selected and model 1 had the best performance with an AUC of 0.9141 (table 3). The receiver operating characteristic curve of the five best models is shown in figure 1.

    Figure 1
    Figure 1

    ROC curve of the five best models. ROC, receiver operating characteristic.

    View this table:
    Table 3

    Predictive performance indicators of the five best models

    Model interpretation

    The importance of each variable to the final prediction model is shown in figure 2. The result showed that pretreatment serum levels, renal function, dermatoses, gender and age were the top five most important variables for the model. We used the SHAP value to explain the contribution of the variables to the model, and the SHAP value of the top 20 is shown in figure 3. This plot explains how high and low variable values were in relation to SHAP values. For the prediction model, the higher the SHAP value of a variable, the more likely ADR occurs.

    Figure 2
    Figure 2

    Importance matrix plot of each variable to the final prediction model. Variable names are shown in online supplemental table 2). X83, pre-treatment serum levels; X55, renal function; X25, dermatoses; X1, gender; X2, age; X29, dose; X62, low-density lipoprotein; X64, hypoproteinemia; X30, anti-infective agents; X82, pre-treatment indicators of carcinoma; X79, haemoglobin; X6, history of allergy; X16, respiratory diseases; X66, albumin/globulin; X78, red blood cell; X81, hypersensitive C reactive protein; X51, dermatology medication; X77, eosinophils; X13, Charlson comorbidity index (Score); X57, serum potassium.

    Figure 3
    Figure 3

    SHAP summary plot of the top 20 variables of the model. Red represents higher variable values, and blue represents lower variable values. Variable names are shown in online supplemental table 2). X83, pre-treatment serum levels; X55, renal function; X25, dermatoses; X1, gender; X2, age; X29, dose; X62, low-density lipoprotein; X64, hypoproteinemia; X30, anti-infective agents; X82, pre-treatment indicators of carcinoma; X79, haemoglobin; X6, history of allergy; X16, respiratory diseases; X66, albumin/globulin; X78, red blood cell; X81, hypersensitive C reactive protein; X51, dermatology medication; X77, eosinophils; X13, Charlson comorbidity index (Score); X57, serum potassium. SHAP, SHapley Additive exPlanations.

    Sample size assessment

    With the continuously increased size of sample data, the AUC values of the testing sets continued to increase, which shows a sufficient sample size included in this study (figure 4).

    Figure 4
    Figure 4

    Sample size validation. The vertical bars represent the 95% CI of AUC of ROC. AUC, area under curve; ROC, receiver operating characteristic.

    Develop an ADR prediction system for Panax notoginseng saponin

    According to the best model, a prediction system for the ADR of Panax notoginseng saponin has been developed and we had obtained the software copyright. The development of the ADR prediction system is shown in figure 5. The operation and output of the system are shown in figure 6.

    Figure 5
    Figure 5

    The development of ADR prediction system. ADR, adverse drug reaction; AUC, area under curve; DT, Decision Tree; ET, Extra Tree; FN. false negative; FP, false positive; KNN, K-Nearest Neighbour; RF, Random Forest; TP, true positive; TN, true, negative.

    Figure 6
    Figure 6

    The operation (A) and output (B) of the ADR prediction system. ADR, adverse drug reaction.

    Discussion

    Traditional Chinese medicine has been used for the prevention and treatment of diseases for centuries.11 In recent years, the application of Chinese herbal injections containing Panax notoginseng saponin has become more and more common in clinical practice, while ADR often causes concerns. Studies have shown that the Chinese herbal ingredients, traditional Chinese medicine preparation and combination medication are the important factors for the ADR of Chinese herbal injections containing Panax notoginseng saponin. Drug eruption (50.5%), allergic reactions (20.4%) and anaphylactic shock (9.7%) were the most common, and some cases were even life threatening.5 However, the ADR monitoring methods, including spontaneous reporting systems, prescription event monitoring and centralised hospital monitoring system, were all reported after the event and may even have data bias, under-reporting or repeated reporting. Therefore, the realisation of ADR prediction has important significance for preventing ADR of Chinese herbal injections containing Panax notoginseng saponin in clinical practice.

    In our study, a nested case–control study was performed for data collection. In order to obtain the best model, we used four data filling, five data sampling and three variable selection methods for data processing and combined 18 machine learning algorithms to establish 1080 ADR prediction models. By comparing the AUC, accuracy, precision, recall rate and F1 value of these models, the best one was selected to develop an ADR prediction system for the Chinese herbal injections containing Panax notoginseng saponin.

    In recent years, some ADR prediction models have been developed based on data mining,6–9 machine learning algorithms10 12–15 and statistical methods.16–18 Tangiisuran et al16 combined univariate analysis and multivariate binary logistic regression for the identification of clinical risk factors to develop an ADR risk model. The AUC of the model at the internal and external validation stage was 0.74 and 0.73, respectively, the sensitivity was 80% and 84%, and the specificity was 55% and 43%.16 Imai et al10 used artificial neural networks to predict the ADR risk and made an AUC of 0.83. Compared with other studies, the model established in our study had better predictive performance (accuracy was 0.8947, precision was 0.75, the recall rate was 0.6667 and AUC was 0.914). As missing data are common in clinical practice, the methods of data filling used in our study may be advantageous for the deal with imbalanced data in clinical real-world research. More importantly, the system developed by the best model was potentially convenient for clinical application because of its’ simple operation, fast calculation and high accuracy.

    It is worth noting that Hammann et al19 established a decision tree model based on the chemical, physical and structural properties of compounds for the prediction of ADR occurrence and the model had high predictive accuracy (78.9–90.2%). However, the model was difficult to interpret as it ignored the effect of pathological and physiological conditions and the combination medication on ADR. This made the model unlikely to be accepted by clinicians. In our study, we collected more than 80 factors including the patient’s pathophysiological characteristics, clinical laboratory results and medication conditions. Meanwhile, the critical predictors associated with the ADR were identified by the SHAP values. Although using the SHAP values as a generalised approach to identify the important clinical determinants of ADR caused by Chinese herbal injections containing Panax notoginseng saponin is not possible, it may help generate clinical hypotheses for some specific clinical events.

    The results of SHAP indicated that whether the patients have dermatoses will significantly affect the models’ predictive performance. Cutaneous ADR is one of the most common adverse reactions of Panax notoginseng, such as erythema multiforme, urticaria, severe erythema multiforme and acute generalised exanthematous pustulosis.20 21 Therefore, those patients with original dermatoses are more likely to have ADR after using Panax notoginseng. In addition, we found that age and gender are related to the occurrence of Panax notoginseng-induced ADR, which is consistent with the results reported by Yang et al.22

    This study had some limitations. First, the small sample size of this study might affect the model prediction performance. Second, as the study population was all from southwest China, the results may be biased while the prediction system was applied in other medical institutions. Finally, a prospective controlled trial is required to demonstrate the accuracy of the ADR prediction system.

    Data availability statement

    Data are available upon reasonable request. Data are available upon reasonable request. Data may be obtained from a third party and are not publicly available. The first author (7190175@uestc.edu.cn) will share any publicly available data if requested by email.

    Ethics statements

    Patient consent for publication

    Ethics approval

    This study was approved by the Ethics Committee of Sichuan Academy of Medical Sciences and Sichuan Provincial People’s Hospital (2017-11-01). Participants gave informed consent to participate in the study before taking part.

    References

    1. ↵
      1. Xie W,
      2. Meng X,
      3. Zhai Y, et al
      . Panax notoginseng saponins: a review of its mechanisms of antidepressant or anxiolytic effects and network analysis on Phytochemistry and pharmacology. Molecules2018;23:940.doi:10.3390/molecules23040940
      OpenUrl
    2. ↵
      1. Kim J-H
      . Pharmacological and medical applications of Panax ginseng and ginsenosides: a review for use in cardiovascular diseases. J Ginseng Res 2018;42:264–9.doi:10.1016/j.jgr.2017.10.004pmid:http://www.ncbi.nlm.nih.gov/pubmed/29983607
      OpenUrlPubMed
    3. ↵
      1. Yang F,
      2. Ma Q,
      3. Matsabisa MG, et al
      . Panax notoginseng for Cerebral Ischemia: A Systematic Review. Am J Chin Med 2020;48:1331–51.doi:10.1142/S0192415X20500652pmid:http://www.ncbi.nlm.nih.gov/pubmed/32907361
      OpenUrlPubMed
    4. ↵
      1. Qu J,
      2. Xu N,
      3. Zhang J, et al
      . Panax notoginseng saponins and their applications in nervous system disorders: a narrative review. Ann Transl Med 2020;8:1525.doi:10.21037/atm-20-6909pmid:http://www.ncbi.nlm.nih.gov/pubmed/33313270
      OpenUrlPubMed
    5. ↵
      1. Xiang Z,
      2. Qiao T,
      3. Xiao H, et al
      . The anaphylactoid constituents in Xue-Sai-Tong injection. Planta Med 2013;79:1043–50.doi:10.1055/s-0032-1328746pmid:http://www.ncbi.nlm.nih.gov/pubmed/23839821
      OpenUrlPubMed
    6. ↵
      1. Wei L,
      2. Scott J
      . Association rule mining in the US vaccine adverse event reporting system (VAERS). Pharmacoepidemiol Drug Saf 2015;24:922–33.doi:10.1002/pds.3797pmid:http://www.ncbi.nlm.nih.gov/pubmed/26045284
      OpenUrlPubMed
    7. ↵
      1. Harpaz R,
      2. DuMouchel W,
      3. Shah NH, et al
      . Novel data-mining methodologies for adverse drug event discovery and analysis. Clin Pharmacol Ther 2012;91:1010–21.doi:10.1038/clpt.2012.50pmid:http://www.ncbi.nlm.nih.gov/pubmed/22549283
      OpenUrlCrossRefPubMed
    8. ↵
      1. Sakaeda T,
      2. Tamon A,
      3. Kadoyama K, et al
      . Data mining of the public version of the FDA adverse event reporting system. Int J Med Sci 2013;10:796–803.doi:10.7150/ijms.6048pmid:http://www.ncbi.nlm.nih.gov/pubmed/23794943
      OpenUrlCrossRefPubMed
    9. ↵
      1. Kadoyama K,
      2. Kuwahara A,
      3. Yamamori M, et al
      . Hypersensitivity reactions to anticancer agents: data mining of the public version of the FDA adverse event reporting system, AERS. J Exp Clin Cancer Res 2011;30:93.doi:10.1186/1756-9966-30-93pmid:http://www.ncbi.nlm.nih.gov/pubmed/21970649
      OpenUrlCrossRefPubMed
    10. ↵
      1. Imai S,
      2. Takekuma Y,
      3. Kashiwagi H, et al
      . Validation of the usefulness of artificial neural networks for risk prediction of adverse drug reactions used for individual patients in clinical practice. PLoS One 2020;15:e0236789.doi:10.1371/journal.pone.0236789pmid:http://www.ncbi.nlm.nih.gov/pubmed/32726360
      OpenUrlPubMed
    11. ↵
      1. Liu S-H,
      2. Chuang W-C,
      3. Lam W, et al
      . Safety surveillance of traditional Chinese medicine: current and future. Drug Saf 2015;38:117–28.doi:10.1007/s40264-014-0250-zpmid:http://www.ncbi.nlm.nih.gov/pubmed/25647717
      OpenUrlCrossRefPubMed
    12. ↵
      1. Choudhury O,
      2. Park Y,
      3. Salonidis T, et al
      . Predicting adverse drug reactions on distributed health data using Federated learning. AMIA Annu Symp Proc 2019;2019:313–22.pmid:http://www.ncbi.nlm.nih.gov/pubmed/32308824
      OpenUrlPubMed
    13. ↵
      1. Liu X,
      2. Chen H
      . A research framework for pharmacovigilance in health social media: identification and evaluation of patient adverse drug event reports. J Biomed Inform 2015;58:268–79.doi:10.1016/j.jbi.2015.10.011pmid:http://www.ncbi.nlm.nih.gov/pubmed/26518315
      OpenUrlPubMed
    14. ↵
      1. Davis J,
      2. Costa VS,
      3. Peissig P, et al
      . Demand-Driven clustering in relational domains for predicting adverse drug events. Proc Int Conf Mach Learn 2012;2012:1287–94.pmid:http://www.ncbi.nlm.nih.gov/pubmed/25285329
      OpenUrlPubMed
    15. ↵
      1. Lee CY,
      2. Chen Y-PP
      . Prediction of drug adverse events using deep learning in pharmaceutical discovery. Brief Bioinform 2021;22:1884–901.doi:10.1093/bib/bbaa040pmid:http://www.ncbi.nlm.nih.gov/pubmed/32349125
      OpenUrlPubMed
    16. ↵
      1. Tangiisuran B,
      2. Scutt G,
      3. Stevenson J, et al
      . Development and validation of a risk model for predicting adverse drug reactions in older people during hospital stay: Brighton adverse drug reactions risk (BADRI) model. PLoS One 2014;9:e111254.doi:10.1371/journal.pone.0111254pmid:http://www.ncbi.nlm.nih.gov/pubmed/25356898
      OpenUrlPubMed
    17. ↵
      1. Clothier HJ,
      2. Lawrie J,
      3. Lewis G
      . SAEFVIC: surveillance of adverse events following immunisation (AEFI) in Victoria, Australia, 2018. Commun Dis Intell 2020:44.
    18. ↵
      1. Alvarez Y,
      2. Hidalgo A,
      3. Maignen F, et al
      . Validation of statistical signal detection procedures in eudravigilance post-authorization data: a retrospective evaluation of the potential for earlier signalling. Drug Saf 2010;33:475–87.doi:10.2165/11534410-000000000-00000pmid:http://www.ncbi.nlm.nih.gov/pubmed/20486730
      OpenUrlCrossRefPubMedWeb of Science
    19. ↵
      1. Hammann F,
      2. Gutmann H,
      3. Vogt N, et al
      . Prediction of adverse drug reactions using decision tree modeling. Clin Pharmacol Ther 2010;88:52–9.doi:10.1038/clpt.2009.248pmid:http://www.ncbi.nlm.nih.gov/pubmed/20220749
      OpenUrlCrossRefPubMed
    20. ↵
      1. Yan S,
      2. Xiong H,
      3. Shao F, et al
      . HLA-C*12:02 is strongly associated with Xuesaitong-induced cutaneous adverse drug reactions. Pharmacogenomics J 2019;19:277–85.doi:10.1038/s41397-018-0051-3pmid:http://www.ncbi.nlm.nih.gov/pubmed/30237582
      OpenUrlPubMed
    21. ↵
      1. Chen WJ,
      2. Kuang YY,
      3. JT L
      . Analysis on 13 cases of adverse drug reaction by Xuesaitong injection. Journal of North Pharmacy 2013;10:16–17.
      OpenUrl
    22. ↵
      1. Yang P,
      2. Qian N,
      3. Yao D
      . 62 cases of adverse reactions in Xuesaitong oral preparations. Chinese Medicine Modern Distance Education of China 2021;19:34–6.
      OpenUrl

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • Contributors X-WW, E-WL and R-ST were involved in the conception and design of the study. X-WW drafted the article. J-YZ, HC, X-WS and Y-LW analysed the data. E-WL and R-ST revised the manuscript. All authors gave final approval of the version to be published. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted. R-ST is the guarantor.

    • Funding This study was funded by the National Natural Science Foundation of China (Number 72004020), the Program of Science and Technology Department of Sichuan Province (Number 2021YJ0427), the Key Research and Development Program of Science and Technology Department of Sichuan Province (Number2021YFS0197 and Number 2019YFS0514), the Postgraduate Research and Teaching Reform Project of the University of Electronic Science and Technology of China (Number JYJG201919) and the Research Subject of Health Commission of Sichuan Province (Number 19PJ262).

    • Competing interests None declared.

    • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.