Article Text

Download PDFPDF

Original research
Development and validation of an individualised nomogram to predict mother-to-child transmission in pregnant women with syphilis in China: a retrospective cohort study
  1. Shuaixin Feng1,
  2. Shuang Gao2,
  3. Qian Wang3,
  4. Min Cai2,
  5. Zhaoqian Huang1,
  6. Ying Huang1,
  7. Yeting Hong1,
  8. Li Yuan2,
  9. Fenghua Liu2,
  10. Hongbo Jiang1,4
  1. 1 Department of Epidemiology and Biostatistics, School of Public Health, Guangdong Pharmaceutical University, Guangzhou, China
  2. 2 Guangdong Women and Children Hospital, Guangzhou, China
  3. 3 Department of Women Healthcare, Chinese Center for Disease Control and Prevention, Beijing, China
  4. 4 Institute for Global Health, University College London, London, UK
  1. Correspondence to Dr Hongbo Jiang; hongbojiang3{at}163.com; Dr Fenghua Liu; liushine2006{at}163.com

Abstract

Objectives The elimination of mother-to-child transmission (MTCT) of syphilis has been set as a public health priority. However, an instrument to predict the MTCT of syphilis is not available. We aimed to develop and validate an intuitive nomogram to predict the individualised risk of MTCT in pregnant women with syphilis in China.

Design Retrospective cohort study.

Setting Data was acquired from the National Information System of Prevention of MTCT of Syphilis in Guangdong province between 2011 and 2020.

Participants A total of 13 860 pregnant women with syphilis and their infants were included and randomised 7:3 into the derivation cohort (n=9702) and validation cohort (n=4158).

Primary outcome measures Congenital syphilis.

Results Among 13 860 pregnant women with syphilis and their infants included, 1370 infants were diagnosed with congenital syphilis. Least absolute shrinkage and selection operator regression and multivariable logistic regression showed that age, ethnicity, registered residence, marital status, number of pregnancies, transmission route, the timing of syphilis diagnosis, stage of syphilis, time from first antenatal care to syphilis diagnosis and toluidine red unheated serum test titre were predictors of MTCT of syphilis. A nomogram was developed based on the predictors, which demonstrated good calibration and discrimination with an area under the curve of the receiver operating characteristic of 0.741 (95% CI: 0.728 to 0.755) and 0.731 (95% CI: 0.710 to 0.752) for the derivation and validation cohorts, respectively. The net benefit of the predictive models was positive, demonstrating a significant potential for clinical decision-making. We have also developed a web calculator based on this prediction model.

Conclusions Our nomogram exhibited good performance in predicting individualised risk for MTCT of syphilis, which may help guide early and personalised prevention for MTCT of syphilis.

  • syphilis
  • risk factors
  • epidemiology
  • public health

Data availability statement

Data are available upon reasonable request.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

  • The predictive model was based on the pregnant women with syphilis who were reported by the Information System of Prevention of mother-to-child transmission of syphilis, which contains a sufficiently large number of samples in Guangdong province, China.

  • This is a passive surveillance system that relies on reporting by healthcare workers, which may result in low reporting due to under-detection, misclassification and under-reporting.

  • We only evaluated the predictive performance of the nomogram internally; an external validation is warranted, and any generalisation of these results should be made with caution.

Introduction

In 2007, the elimination of mother-to-child transmission (EMTCT) of syphilis was declared to be a public health priority by the WHO.1 In 2016, WHO estimated that there had been more than 350 000 adverse pregnancy outcomes and 204 000 deaths due to congenital syphilis (CS), highlighting barriers to achieving the EMTCT targets.2 In China, the National Syphilis Prevention and Control Plan (2010–2020) and the National Implementation Guidelines on Integrated prevention of mother-to-child transmission (MTCT) of HIV, syphilis and hepatitis B virus (HBV) were launched in 2010 and 2011, respectively.3 The incidence of CS decreased from 69.9 cases per 100 000 live births in 2013 to 11.87 in 2019.1 3 In Guangdong province, the incidence of MTCT of syphilis in Guangdong province dropping significantly from 22.71% in 2016 to 5.76% in 2020.4 In addition, the WHO has set a target of reaching 100 countries validated for EMTCT of syphilis by 2030,5 which will require further efforts.

Importantly, early diagnosis and adequate early penicillin treatment can cure the infected fetus before delivery.6 However, many pregnant women miss the optimal timing for screening and treatment to prevent adverse pregnancy outcomes caused by MTCT of syphilis.7 8 To date, relevant studies have only focused on identifying the associated factors without quantifying and validating for factors associated with MTCT of syphilis,8 9 and have neglected to predict the risk of MTCT for individuals. Information to help identify women who are most likely to transmit syphilis to their babies is limited, with very few studies focusing on the implementation of nomograms. A nomogram is a useful tool for visualising regression equations as it can establish scoring standards and obtain the corresponding risk probability of individuals based on the regression coefficients of all the independent factors.10 Predicting the probability of CS is beneficial to identify pregnant women who are at high risk. This allows early identification of women at higher risk of MTCT of syphilis and facilitates early individualised interventions. Therefore, we aimed to develop and validate an intuitive nomogram to predict the individualised risk of MTCT in pregnant women with syphilis as early as possible.

Methods

Participants

The Information System of Prevention of MTCT of Syphilis (IPMTCT) is a nationwide health facility-based case reporting system established in 2011. It has been used to monitor and evaluate the epidemics of maternal syphilis and CS in China.8 Pregnant women with syphilis and their children registered in the IPMTCT of Guangdong province between January 2011 and December 2020 were included in this study. Inclusion criteria were: (1) delivery between January 2011 and December 2020; (2) a definitive diagnosis or confirmed exclusion of CS by June 20219; and (3) the provision of informed consent. Exclusion criteria were: (1) twin or multiple pregnancies; (2) neonatal deaths; (3) HIV or HBV infection in the mother; or (4) incomplete records. For self-assessment, we used the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis checklist.11

Data collection

Based on a retrospective study, we collected the registration data of pregnant women with syphilis from the Guangdong IPMTCT including social demographic data (eg, age, ethnicity, registered residence, education, occupation and marital status), time of first antenatal care (ANC) visit, reproductive history (eg, gravidity, parity and the number of children), maternal syphilis information (eg, transmission route, toluidine red unheated serum test (TRUST) titre, timing of syphilis diagnosis, stage of syphilis and history of syphilis) and partner’s syphilis infection status.

Diagnosis of CS

Infants born with maternal syphilis were followed-up every 3 months from birth to 18 months until the diagnosis of CS was confirmed or excluded. A definitive confirmed diagnosis of CS required one or more of the following criteria: (1) a positive dark-field or fluorescent antibody test of body fluid; (2) a positive reactive treponemal antibody test of IgM at birth; (3) a neonatal non-treponemal titre more than fourfold the maternal titre before delivery and positive treponemal test; (4) a fourfold increase in non-treponemal titre and a positive treponemal test at any follow-up visit; or (5) a positive treponemal test after 18 months of age. An excluded case was defined as an infant with a negative treponemal test result at 6 months of age or later.12

Statistical analysis

The data set was randomly split into a derivation cohort (70%) and a validation cohort (30%). Differences between the two cohorts were compared using χ2 test. Because the prevalence of CS in the derivation cohort is relatively low (9.85%, 956/9702), which would affect the performance of the models, the synthetic minority oversampling technique13 was employed to balance the minority group and the newly generated data sets were used to build the models.

The least absolute shrinkage and selection operator (LASSO) regression is a common auto-variable selection technique. In our study, LASSO regression (based on the λ-1 SE)14 15 was used to screen out potential factors associated with CS, and multivariable logistic regression analysis was further used to identify factors associated with CS. The significant variables (p<0.05) in the multivariable logistic regression were used to construct a nomogram to predict the individualised risk for CS using the derivation cohort. We evaluated the discrimination of the nomogram using the area under the curve of the receiver operating characteristic (AUROC) curve with 95% CI, and the Youden Index was used to determine the optimal cut-off value. The 1000-times-bootstrapped resampling calibration curves were used to assess the agreement between the predicted risks and the actual outcomes.16 In addition, clinical decision curve analysis (DCA) reflected the net benefit of the model for women with syphilis.17 Data analyses were performed using R software (V.4.2.2; http://www.r-project.org) with the following packages: ‘DMwR’, ‘glmnet’, ‘foreign’, ‘rms’, ‘pROC’, ‘rmda’, ‘ggDCA’ (https://mirrors.sustech.edu.cn/CRAN/web/packages/available_packages_by_date.html). Discrete variables were presented as frequencies and proportions. All hypothesis tests were two-tailed with α=0.05.

Patient and public involvement

Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Results

Study population

A total of 13 860 pregnant women with syphilis and their infants were included (online supplemental figure S1). Of these, 1370 (9.88%) infants were diagnosed with CS, including 334 infants who were diagnosed with CS at birth and 1036 infants whose diagnosis of syphilis was confirmed during follow-up. No significant differences in the demographic and clinical characteristics were observed between the derivation and validation cohorts (table 1).

Supplemental material

Table 1

Demographic and clinical characteristics of pregnant women with syphilis in the derivation cohort and validation cohort n (%)

Factors associated with MTCT of syphilis

Ten potential predictors including age, ethnicity, registered residence, marital status, number of pregnancies, transmission route, time of syphilis diagnosis, time from first ANC to syphilis diagnosis, stage of syphilis and TRUST titre with no zero coefficients were selected as the predictive factors of CS by the LASSO regression model (online supplemental figure S2). These factors were subsequently included in the multivariable regression model.

Multivariable logistic regression analyses showed that older women were less likely to transmit syphilis to their child than younger women (those aged 20–34 vs <20 years: adjusted OR, aOR=0.43, 95% CI: 0.35 to 0.52; those aged ≥35 vs <20 years: aOR=0.44, 95% CI: 0.35 to 0.55). Pregnant women with syphilis from minority ethnicity (aOR=1.64, 95% CI: 1.39 to 1.92) or other provinces (aOR=1.30, 95% CI: 1.18 to 1.44) showed a higher risk of MTCT of syphilis compared with women from Han ethnicity and Guangdong province. Unmarried or cohabiting women were more likely to transmit syphilis than married participants (aOR=1.61, 95% CI: 1.41 to 1.84). Individuals with a greater number of previous pregnancies were less likely to transmit syphilis (2 vs 1: aOR=0.81, 95% CI: 0.71 to 0.93; ≥3 vs 1: aOR=0.78, 95% CI: 0.69 to 0.88). Women who acquired syphilis through an unknown transmission route were less likely to transmit syphilis than those who acquired it through sexual routes (aOR=0.68, 95% CI: 0.61 to 0.75). A higher risk was observed for women in whom the gap between first ANC and syphilis diagnosis was more than 4 weeks (aOR=1.17, 95% CI: 1.05 to 1.32). Women who were diagnosed in the third trimester (aOR=1.84, 95% CI: 1.58 to 2.15) or perinatal/postnatal period (aOR=1.99, 95% CI: 1.72 to 2.31) showed a higher risk of syphilis transmission than those diagnosed before pregnancy or early in pregnancy. Women with primary (aOR=2.51, 95% CI: 2.02 to 3.12) or unknown stage of syphilis (aOR=1.80, 95% CI: 1.57 to 2.07) showed a higher risk of syphilis transmission than those with latent syphilis. Finally, women with a higher baseline TRUST titre (≥1:8) had a higher risk of syphilis transmission than women with a lower titre (aOR=3.50, 95% CI: 3.15 to 3.88) (table 2).

Table 2

Predictors of mother-to-child transmission of syphilis based on multivariable logistic regression analysis

Nomogram construction and validation

A nomogram was developed on the basis of the multivariable logistic regression coefficients of all the predictive factors (figure 1). According to the values of the predictive factors, vertical lines can be drawn to the ‘Points’ axis for each of the 10 predictors and the corresponding score for the predictors was confirmed. The total points of the 10 predictors could be transformed in the same way as the risk probability of MTCT of syphilis. Thus, the clinicians could easily predict the risk probability and identify the high-risk women with syphilis. In the derivation and validation cohorts, the AUROC of the nomogram prediction model was 0.741 (95% CI: 0.728 to 0.755) and 0.731 (95% CI: 0.710 to 0.752), respectively (figure 2A,B). The actual predicted curves were in good agreement with the verified curve (figure 2C,D). The model demonstrated good calibration and discrimination in both the derivation and validation cohorts. Based on figure 2E,F, the net benefit of the predictive models was positive, demonstrating a significant potential for clinical decision-making. To make it easier for clinicians to use, we have also developed a web calculator based on this prediction model, which can be found on the website https://solcool.shinyapps.io/dynnomapp/ (accessed on 20 March 2023). By entering relevant patient data for each of the 10 variables, the web calculator could automatically output the probability of MTCT for individuals in a second, enabling the early identification of high-risk individuals.18 Based on the optimal cut-off score of 121.7 or predicted probability of 0.283, participants were divided into the low-risk (score <121.7 or probability <0.283) and high-risk (score ≥121.7 or probability ≥0.283) groups.

Figure 1

Nomogram for predicting mother-to-child transmission of syphilis. MTCT, mother-to-child transmission; TRUST, toluidine red unheated serum test titre.

Figure 2

(A) Receiver operating characteristic (ROC) of derivation cohort. (B) ROC of validation cohort. (C) Calibration curves of derivation cohort. (D) Calibration curves of validation cohort. (E) Decision curve of derivation cohort. (F) Decision curve of validation cohort.

Discussion

A nomogram is a useful tool for visualising the individualised risk for MTCT of syphilis, which allows early identification of pregnant women who are at higher risk and facilitates early individualised interventions. To our knowledge, quantitative nomograms for predicting the individualised risk for MTCT of syphilis have not been reported in the literature. In this study, using a large sample size retrieved from the National Information System for Prevention of MTCT, factors associated with MTCT were identified by multivariable logistic regression, and simple intuitive graphs were established to visualise the effect of each predictor of multivariable regression analysis, which should prove beneficial for early detection of women with syphilis who have a higher risk of MTCT of syphilis, allowing formulate individualised treatment regimens and improve patient outcomes. The internal validation analysis of our nomograms showed good discriminatory, and calibration capabilities. In addition, we found a good clinical applicability of our nomogram by DCA. We have developed a web calculator based on the predictive model to facilitate early identification of women at high risk of MTCT of syphilis at their first antenatal visit.

In our study, younger maternal age, later syphilis diagnosis, early stage of syphilis and higher baseline TRUST titre increased the risk of MTCT of syphilis, similar to what has been reported in previous studies.19–21 Higher titres of non-treponemal antibodies suggested early syphilis.22 Historical data showed that the risk of CS was 14 and 22-fold higher when the pregnant women had a higher non-treponemal antibody titre or were in the early stage of syphilis.19 Compared with older pregnant women, maternal syphilis in younger women was more likely to be early syphilis, which could increase the risk of MTCT.19 Late diagnosis of syphilis may be associated with late or no treatment, resulting in a higher risk of CS.22

In our study, unmarried or cohabiting marital status and a woman’s first pregnancy increased the risk of MTCT of syphilis. On the one hand, similar to younger women with syphilis, their maternal syphilis was more likely to be early syphilis, leading to a higher risk of MTCT. On the other hand, they are more likely to have a lower risk awareness and poor knowledge about syphilis and MTCT due to a lack of experience. Therefore, publicity and education about the stage of syphilis, and prevention of CS, along with free treatment services, should be strengthened to link women to primary or prenatal care (PNC) and increase their risk awareness and prompt them to seek better PNC and health services.8 22

Our study showed that ethnic minorities and pregnant women not registered in Guangdong were at higher risk of MTCT of syphilis. In Guangdong, more than 60 million migrants were reported in 2020. In particular, the Pearl River Delta had attracted large numbers of migrants due to its prosperous economy and well-developed transport system.23 24 Migrant populations are more likely to change their jobs, residences and telephone numbers, making follow-up more difficult. In addition, previous studies have found that migrant women tend to be socioeconomically disadvantaged in society, and that they have a higher prevalence of maternal syphilis and poorer adherence to treatment.8 25 26 In particular, these high-need women have later and fewer ANC visits, which may explain the higher risk of MTCT of syphilis among them.21 Therefore, it is necessary to expand access to ANC services for migrant women in order to promote the uptake of ANC services and adherence to treatment.22 Internet and text-message reminders for initial screening or retesting and financial incentive strategies have been proven effective in some populations.27 Rapid point-of-care tests, as an alternative testing approach, may help meet the needs of these high-risk populations.28 29

Our results suggested that there was a higher risk of MTCT among women who acquired syphilis through sexual transmission. They may be more afraid of the discrimination and stigma of acquired syphilis through sexual transmission, which could lead to the reluctance to be tested or treated.27 HIV self-testing could provide a rapid and private result of testing, which could contribute to syphilis testing among individuals who are unwilling to seek health services in clinical-based settings.27 Meanwhile, legal frameworks and policies should be established in China to address discrimination and stigma against women with syphilis.

Our study also demonstrated that pregnant women are at higher risk of MTCT of syphilis if the time from first ANC to syphilis diagnosis was more than 4 weeks. Screening and treatment are considered simple and cost-effective interventions for women with syphilis.20 WHO recommends that all pregnant women with syphilis should receive at least one dose of benzathine penicillin G in the first trimester.1 2 In China, all pregnant women were recommended to be screened at their first antenatal visit and could confirm their syphilis infection status within a month.8 12 They could then receive treatment for free once a definitive diagnosis of syphilis during pregnancy was made according to the IPMTCT guidelines.3 8 Unfortunately, more than 20% of participants in our study did not confirm their syphilis status until 4 weeks after their first ANC visit in our study. First, screening for syphilis may be delayed because of a lack of test kits in some clinics.2 Second, parts of the health staff might neglect the importance of screening tests for syphilis or provide limited information to pregnant women at their first ANC visit.2 8 27 Third, women with syphilis may mistrust the healthcare system or ignore testing because of stigma, cost or low awareness of syphilis.8 28 30 ANC is the best time to test and treat syphilis.21 According to a previous study, each week of delay in treatment was associated with a 2.82-fold increased risk of adverse pregnancy outcomes.3 Therefore, the mechanism of the test kit’s supply and referral system should be constructed to improve the timely diagnosis and treatment, especially emphasising the countermeasures for insufficient supply occurred.1 Publicity and education about the harms of undetected and untreated syphilis should be strengthened at women’s first ANC visit to increase their awareness of the risk of syphilis and their willingness to be tested and treated. Besides, one-stop services that include rapid syphilis testing and treatment at the first antenatal visit could contribute to a reduction in the number of CS cases.31 Meanwhile, the awareness of the importance of syphilis testing in pregnancy and counselling services in antenatal clinics should be further strengthened at all levels of healthcare.8

Our study has several limitations. First, our data were extracted from China’s Information System for Prevention of MTCT. This is a passive surveillance system that relies on reporting by healthcare workers, which may result in low reporting due to under-detection, misclassification and under-reporting. The gestational syphilis detection could be linked to socioeconomic and accessibility of healthcare services.32 Therefore, it is imperative to prioritise the provision of high-quality PNC services and improvements in surveillance and case reporting to reduce failures in reporting pregnant women with syphilis and CS. A previous study proposed a correction methodology in Brazilian gestational syphilis data, which may be used as a reference for application in other middle-income countries.32 Meanwhile, incomplete records would be inevitable. Therefore, training on information quality should be emphasised in the future. Second, some important information (eg, income, health insurance, behavioural characteristics, nutritional status and socio-psychological characteristics) that could contribute to the MTCT of syphilis was not available in the information system. Although treatment plays an irreplaceable role in preventing CS, we focus on the early prevention and timely treatment of MTCT of syphilis and developed an individualised nomogram to predict individualised risk for MTCT of syphilis at their first ANC visit. Third, loss to follow-up is common, as a series of serological tests should be evaluated up to 18 months before the confirmed or excluded diagnosis of syphilis can be made in infants born to women with syphilis, thus, more efforts should be made to strengthen the follow-up management in the future. Fourth, this study used a large sample of pregnant women with syphilis in Guangdong province over the past decade, which can better predict the risk of CS among children born to pregnant women with syphilis in Guangdong province and similar areas. However, due to geographical and economic differences between regions, any generalisation of these results should be made with caution. Finally, external validation and prospective study design are needed to further validate the results.

Conclusions

In our study, we found that younger age, minority ethnicity, residence registered outside Guangdong, unmarried/cohabiting marital status, the first pregnancy, sexual transmission route, late trimester syphilis diagnosis, longer time from first ANC to syphilis diagnosis, primary/unknown stage of syphilis and a higher baseline TRUST titre were independent risk factors for MTCT of syphilis. Based on these predictors, we developed a simple and reliable nomogram model that can predict the risk of MTCT of syphilis among pregnant women with syphilis and may serve as a potential tool to guide early and personalised prevention for MTCT of syphilis. Targeted interventions, including publicity and education to increase risk awareness of syphilis infection, address discrimination and stigma against syphilis, expand access to counselling, testing and treatment, improving the linkage to PNC and health services, should be tailored to facilitate early and timely diagnosis and treatment of pregnant women with syphilis, ultimately helping to validate the elimination of MTCT of syphilis.

Data availability statement

Data are available upon reasonable request.

Ethics statements

Patient consent for publication

Ethics approval

This study was approved by the Ethical Review Committee for Biomedical Research, School of Public Health, Guangdong Pharmaceutical University (No. 2022-02). Participants gave informed consent to participate in the study before taking part.

Acknowledgments

We thank all staff members at municipal and district Women and Children Hospital for data collection.

References

Footnotes

  • SF, SG and QW contributed equally.

  • Contributors HJ conceived, designed and supervised the study, and finalised the analysis. SG, MC and LY assisted in the literature search, data collection and analysis. SF conducted data analysis and wrote the drafts of the manuscript. SF, SG, QW, MC, ZH, YHu, YHo, LY, FL and HJ interpreted the findings, commented on and helped revise drafts of the manuscript. All authors reviewed, revised and approved the final report. HJ is the guarantor of the study.

  • Funding This work was supported by the Department of Education of Guangdong Province (Grant numbers 2022SFKC074, KTSCX060). The funding body was not involved in the design of the study and collection, analysis and interpretation of data for this manuscript.

  • Competing interests None declared.

  • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.