Article Text

Download PDFPDF

Original research
Evaluation of the WHO/CDC Syphilis Serology Proficiency Programme to support the global elimination of mother-to-child transmission of syphilis: an observational cross-sectional study, 2008–2015
  1. André O Hopkins1,
  2. Thuy Trinh2,
  3. Yetunde F Fakile3,
  4. Allan Pillay2,
  5. Melanie M Taylor4,5,
  6. Ellen Kersh2,
  7. Mary Kamb2
  1. 1 NCEZID/ DFWED, Centers for Disease Control and Prevention, Atlanta, Georgia, USA
  2. 2 Centers for Disease Control and Prevention, Atlanta, Georgia, USA
  3. 3 Laboratory Reference Research Branch, Centers for Disease Control and Prevention, Atlanta, Georgia, USA
  4. 4 Organisation mondiale de la Sante, Geneve, Switzerland
  5. 5 Centers for Disease Control and Prevention, Atlanta, Georgia, USA
  1. Correspondence to Dr André O Hopkins; ahopkins{at}cdc.gov

Abstract

Objectives Syphilis morbidity is high among pregnant women in lower income countries with limited laboratory capacity. We evaluated a long-standing global Syphilis Serology Proficiency Programme (SSPP) that supports testing quality in national reference laboratories to determine if participation affects congenital syphilis elimination strategies.

Design In this observational cross-sectional study, we calculated coverage on type, frequency and quality of syphilis testing reported by laboratories enrolled in the SSPP from 2008 to 2015. We used country-reported data to WHO on four congenital syphilis (CS) indicators and World Bank country economic data to compare coverage and completeness of reporting of indicators in lower income countries with and without an SSPP-enrolled laboratory.

Participants From 2008–2015, 78 laboratories from 51 countries participated in >1 SSPP evaluation; 56% were national reference laboratories, of which most (93%) participated for >3 years and 11 (22%) in all 24 cycles.

Results Median proficiency performance score was >95% regardless of test conducted. Of the 51 countries with an SSPP-enrolled laboratory, 22 (43%) were lower-income countries, of which 21 reported CS data during 2008–2015. Comparing CS data from 87 (90% of total) lower income countries with and without an SSPP-enrolled laboratory, countries with an SSPP-laboratory had stronger reporting on antenatal syphilis testing (p=0.04). For 2015, an estimated 74% of prenatal syphilis tests and 63% of positive tests reported to WHO from countries with an SSPP-enrolled laboratory.

Conclusion The SSPP has focused well on national reference laboratories, but has been only partially successful in recruiting laboratories from lower income countries. The finding that over half of syphilis infections in pregnant women living in countries with SSPP-enrolled laboratories suggests wide reach of the current quality assurance programme. However, reach could expand with focussed recruitment of laboratories from lower income countries.

  • diagnostic microbiology
  • public health
  • microbiology
  • protocols & guidelines
  • quality in health care
http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Strengths and limitations of this study

  • Labs that participated in the Syphilis Serology Proficiency Programme (SSPP) had high participation rates and performance scores but may reflect long-term participants who are familiar with the external quality assurance requirements.

  • Less than half of the labs enrolled in the SSPP are from lower income countries, which is a priority of the programme.

  • Improved and continued efforts will be made to increase the presence of the SSPP in lower income countries.

  • WHO-reported data may not be representative of screening and treatment practices throughout the country.

  • The SSPP is provided at no cost to participating countries and can be easily incorporated into a laboratory’s workflow.

Introduction

Syphilis, caused by the bacterium Treponema pallidum subspecies pallidum, is a common disease causing substantial global morbidity and mortality, especially from mother to child during pregnancy (ie, congenital syphilis (CS)).1 The estimated global maternal syphilis prevalence in 2016 was 0.69% (95% confidence interval: 0.57-0.81%).2 Adverse pregnancy outcomes caused by syphilis are preventable with early detection of cases in the pregnant mother with prompt treatment to cure infection and prevent further disease sequelae. However, syphilis diagnosis is challenging, requiring clinical suspicion and supportive serologic results—ideally paired treponemal and non-treponemal serologic tests.

In 2007, the WHO launched a global initiative to eliminate CS as a public health problem based on the pillars of (1) political commitment, (2) access to maternal and newborn health services, (2) universal screening and treatment of pregnant women and (4) surveillance and monitoring.3 Ensuring quality of syphilis testing in front-line, clinical laboratories around the world is an essential element of both the third pillar, ensuring syphilis infections are detected and treated promptly, and fourth pillar, ensuring syphilis cases are accurately counted. External quality assurance including proficiency testing is one tool verifying accuracy and reliability of laboratory testing. It is anticipated that national reference laboratories will, in turn, provide simple proficiency testing programmes to assure quality syphilis testing in underlying regional, district and other clinical facilities.

Syphilis Seroproficiency Testing Programme procedures

The Syphilis Seroproficiency Testing Programme (SSPP) is a voluntary, free-of-charge, external quality assurance programme for syphilis testing provided by WHO in collaboration with the Centers for Disease Control and Prevention (CDC) Division of STD Prevention (DSTDP) Laboratory Reference and Research Branch (LRRB)4. Laboratories are able to enrol through the WHO website, which provides details on eligibility and application requirements (online supplemental appendix I).

Supplemental material

Global congenital syphilis reporting to WHO

To monitor the global CS elimination initiative, since 2008, WHO has asked countries to report national-level data on four CS indicators through the existing Global AIDS Response Progress Reporting (GARPR) system.5 The four indicators are: proportion of pregnant women screened for syphilis; syphilis seropositivity among pregnant women; proportion of seropositive pregnant women treated for syphilis and CS case rate.

In this programme evaluation, we reviewed the implementation of the WHO/CDC SSPP from 2008 to 2015, focusing on its ability to reach national reference laboratories in lower income countries and the extent to which the programme has supported the global initiative for elimination of CS.

Methods

We used data provided by laboratories participating in the WHO/CDC SSPP from 1 January 2008 through 31 December 2015, starting in 2008 because it was the first full year of the global elimination of CS initiative in partnership with WHO, CDC and other organisations.6

Participating laboratories received a proficiency testing panel three times per year, which consisted of five vials of prepared human sera, containing treponemal and non-treponemal antibodies of various reactivity, with reactivity blinded to laboratory staff (figure 1). CDC laboratorians prepare the panels using commercially available human serum or plasma (Physician’s Plasma Alliance, Johnson City, Tennessee, USA) that previously screened negative for hepatitis B surface antigen, hepatitis C antibody and HIV-1/2 (serologically). Bulk quantities of either serum or plasma are processed to ensure uniformity and stability as previously described.7 Samples were aliquoted aseptically into glass vials, packaged according to United States Postal Services (USPS) guidelines and mailed expeditiously to laboratories with tracking to monitor delivery within a 3-week window.8

Figure 1

Flowchart depicting overall workflow and process of the CDC/WHO SSPP. CDC, Centers for Disease Control and Prevention.

Laboratories were advised to treat testing panels in the same manner as patient samples and perform all laboratory-based syphilis serologic tests offered to the public. Laboratorians completed a pre-formatted ‘results form’ (online supplemental appendix I) and submitted it electronically within 45 days of receipt. For each serologic test conducted, participating laboratories were scored based on results obtained from three United States Reference Laboratories (CDC, Atlanta; Georgia Public Health Laboratory, Atlanta; Texas Public Health Laboratory, Austin) as well as consensus results (reported by at least 75% of laboratories) of all participating laboratories, with a minimum acceptable score of 80%. Laboratories with lower than passing scores could access technical assistance from LRRB laboratorians through job aids, email or telephone calls to help procedures and interpretation. Laboratories in the SSPP were monitored to ensure that laboratories remain engaged in the programme, and considered to have ‘full participation’ if they returned results for all three panels provided in a given year, ‘partial participation’ if they returned results for two panels for the year and ‘low participation’ if they returned one panel in a given year.

Since its inception, the SSPP has evaluated both non-treponemal and treponemal tests: non-treponemal tests include the rapid plasma reagin (RPR), unheated serum reagin (USR) and Venereal Disease Research Laboratory (VDRL) assays. Treponemal tests, measuring specific antibodies to T. pallidum, include the T. pallidum particle agglutination (TPPA) test, T. pallidum haemaglutination assay (TPHA), fluorescent treponemal antibody absorption (FTA-ABS) test and ELISAs for T. pallidum IgG and IgM.9

For this evaluation, we used SSPP enrolment and evaluation data to assess involvement and extent of participation in the programme from 2008 through 2015, according to type of laboratory, region, country income level and mean test score by syphilis serology test. We reported on countries geographically according to WHO regional office as part of the African Region (AFR), Americas Region (AMR), European Region (EUR), Eastern Mediterranean Region (EMR), South East Asian Region (SEAR) or West Pacific Region (WPR).10 We also categorised countries where laboratories were located according to World Bank criteria regarding per capita income as a high-income country (HIC), upper-middle-income country (UMIC), lower-middle-income country (LMIC) or low-income country (LIC).11 According to the 2008 classification scheme, HICs are those with an annual per capita national income >US$ 11,905, UMICs are those with income from US$3856 to US$11 905, LMICs are those with income from US$976 to US$3855 and LICs are those with income < US$976.12 We grouped LICs and LMICs in this analysis, defining them as ‘lower income’ countries, and considered HICs and UMICs to be ‘higher income’ countries. We linked GARPR, available on the WHO Global Health Observatory data repository at http://apps.who.int/gho/data/node.main.A1357STI?lang=en,13 and SSPP participation data to estimate the extent to which the SSPP supported syphilis testing in pregnant women in ‘lower income’ countries from 2008 to 2015. We compared countries participating or not participating in the SSPP during the study interval on completeness of reporting on the four CS indicators. These were (i) syphilis testing coverage in pregnant women (number of countries reporting and median-year coverage); (2) syphilis treatment coverage in pregnant women who tested seropositive (number reporting and median-year coverage); (3) increase in testing coverage of at least 25% (requiring at least two data points) and (4) increase in treatment coverage of at least 25% (also requiring at least two data points). As the SSPP goal prioritises support to lower income countries with limited laboratory capacity, for this analysis, we excluded HICs and UMICs as well as territories or countries with populations less than 100 000. We used the combined GARPR and SSPP data to estimate the number and proportion of all maternal syphilis infections globally during 2015 that were detected in countries with laboratories supported by the SSPP. For countries not reporting data during 2015, we used data from the most recent year that was reported. We used the UNICEF and United Nations Population Fund (UNFPA) data by country to augment data on number and proportion of pregnant women in each country. Statistical analyses were done using Microsoft Excel 2013 (Microsoft, Redmond, Washington, USA), and comparisons were made using χ2 and the Mann-Whitney tests.

Patient and public involvement statement

This research was done without patient involvement. No patients or patient data were included in this study and patients were not invited to contribute to the writing or editing of this document for readability or accuracy.

Results

Participation in the SSPP

From 2008 to 2015, 78 laboratories representing 51 countries participated in at least one SSPP evaluation (online supplemental appendix II). Of the 51 countries, 11 (22%) had full participation over the study years and thus consistently submitted data across all study months. Among participating laboratories, 44 (56%) were national reference laboratories, 15 (19%) were hospital or clinic laboratories, 13 (17%) were university or regional laboratories and 6 (8%) were private clinical laboratory facilities. Among the 44 national reference laboratories, 41 (93%) were active for at least 3 years, and 10 (23%) were fully active over the entire 8-year study interval. During the final year of the study interval (2015), 36 national reference laboratories participated in the SSPP, of which 24 (64%) were fully active.

Supplemental material

The 51 countries with SSPP-participating laboratories represented five of the six WHO regions (online supplemental appendix II); EMR had no laboratories participating in the SSPP among its 21 total member states. By region, SSPP participation of member states was highest in AMR, with 21 (66%) countries participating of 32 total states. This was followed by SEAR with 5 (45%) of 11 member states participating; WPR with 7 (33%) of 21 member states; AFR with 9 (20%) of 46 member states and EUR with 9 (18%) of 50 member states participating in the SSPP. Among the 51 countries with SSPP-participating laboratories, 15 (29%) were HICs, 14 (28%) were UMICs, 13 (25%) were LMICs and 9 (18%) were LICs. Thus, ‘lower income’ countries made up 22 (43%) of the 51 countries currently participating in the SSPP. Most participating countries (n=35, 69%) had at least one laboratory active in the SSPP during all 8 years evaluated, and 46 (90%) had at least one laboratory active during at least three of the evaluation years. Participation of at least one laboratory for all 8 years was more common among higher income than lower income countries (41% vs 27%, p<0.001).

Type of testing by year

Over the 8-year interval, participating laboratories performed a total of six types of tests: RPR or USR, VDRL, TPPA or TPHA, ELISA, FTA-ABS and rapid treponemal tests (any type). For non-treponemal tests, RPR or USR were consistently performed by 54% of laboratories while the VDRL was performed by about 38%. Among treponemal tests, the TPPA or TPHA tests were performed by approximately 58% of the participating laboratories, ELISA by approximately 24% of laboratories and FTA-ABS by 33%. Due to the growing popularity of various treponemal RSTs on the market, as well as expanded WHO recommendations on the use of rapid tests,14 several rapid treponemal tests were accepted into the programme in 2014 with approximately 32% of participating laboratories providing results in that year (figure 2). Over the study interval, there was consistent use of the traditional laboratory-based syphilis tests, but a slight decline in the number of laboratories using the FTA-ABS and an increase in RST usage.

Figure 2

Types of syphilis tests performed by laboratories participating in the CDC/WHO Syphilis Serology Proficiency Programme, 2008–2015. Laboratories that participated in at least one of the three evaluations per year. CDC, Centers for Disease Control and Prevention; EIA, Enzyme Immunoassay; FTA-ABS, Fluorescent Treponemal Antibody-Absorption Test; RPR, Rapid Plasma Reagin; RST, Rapid Syphilis Test; TPPA, T. pallidum particle agglutinationassay; TPHA, T. pallidum hemagglutination test; USR, Unheated Serum Reagin test; VDRL, Venereal Disease Research Laboratory.

Mean scores by year

Among participating laboratories, mean scores were consistently well above 80% (minimal acceptable score) throughout the study interval, and usually higher than 95% (figure 3). A dip in performance for the RPR and VDRL was observed in 2014, corresponding to a change in the SSPP scoring criteria to make it more consistent with other external quality assurance programme (eg, completeness of reporting on test information such as expiration dates and lot numbers; consistency in titer grades and timeliness of reporting results). Nonetheless, the mean scores of participating laboratories remained greater than 85% for all tests performed in 2014, and scores improved in 2015.

Figure 3

Mean test score by syphilis test among laboratories participating in the CDC/WHO Syphilis Serology Proficiency Programme, 2008–2015. CDC, Centers for Disease Control and Prevention; EIA, Enzyme Immunoassay; FTA-ABS, Fluorescent Treponemal Antibody-Ab-sorption Test; RPR, Rapid Plasma Reagin; VDRL Venereal Disease Research Laboratory; RST, Rapid Syphilis Test; TPPA, T. pallidum particleagglutination assay; TPHA, T. pallidum hemagglutination test; USR, Unheated Serum Reagin test;

Completeness of SSPP and WHO data

From 2008 through 2015, 162 (84%) of 194 WHO member countries reported at least one CS indicator through GARPR, including 87 (90%) of the 97 lower income countries (LICs and LMICs). Lower income countries reported CS indicators more commonly than higher income countries, with 40 (93%) of 43 LICs and 47 (87%) of 54 LMICs reporting at least one congenital syphilis indicator compared with 40 (86%) of 46 UMICs and 35 (52%) of 67 HICs (p<0.001). Among the 162 countries reporting CS indicator data from 2008 to 2015, the syphilis positive percentage (seroprevalence) among pregnant women ranged from less than 0.5% to more than 5%, with countries in AFR and AMR reporting the highest seroprevalence rates in pregnant women (figure 4).

Figure 4

Countries participating in the CDC/WHO Syphilis Serology Proficiency Programme by antenatal seropositivity, 2015. CDC, Centers for Disease Control and Prevention.

WHO data on syphilis seroprevalence among antenatal care attendees were available for 142 countries. Of these, 39 countries (27%) participated in the SSPP. Among these 39 countries, 23 (59%) reported syphilis seroprevalence among pregnant women attending antenatal care was less than 1%, 13 (33%) reported from 1% to 4.9% (high) seroprevalence and 3 (8%) reported greater than 5% (very high) seroprevalence. Of the 103 countries not participating in the SSPP, 54% reported having antenatal seroprevalence less than 1%, 36% reported having seroprevalence between 1% and 4.9% and 10% reported seroprevalence greater than 5%. The average maternal syphilis seroprevalence among SSPP countries was 1.5% compared with 1.7% among the non-SSPP participating countries (p=0.17). During 2015, using data from 189 countries that reported to UNICEF or UNFPA surveillance, an estimated 130 068 538 pregnant women, or 68% of all pregnant women, lived in the 51 countries with SSPP laboratories. In all 189 countries, 114 610 291 (59%) pregnant women were tested for syphilis (WHO data); and 74% of these syphilis tests in antenatal women were done in the 39 countries with an SSPP-supported laboratory.

Reach of the SSPP in lower income countries

Focusing on the 97 countries that met criteria for categorisation as ‘lower income’ countries, 87 (90%) countries reported to WHO at least one CS indicator from 2008 through 2015, 10 countries did not report any CS indicator and 9 (10%) reported all four indicators. The 87 lower income countries included 37 countries (43%) from AFR, 13 (16%) from WPR, 10 (11%) from AMR and 9 each (10%) from SEAR, EUR and EMR. The 10 countries that did not report to WHO on CS indicator were located in AFR (2 countries), EMR (4 countries), SEAR (2 countries) and EUR (2 countries). We found no statistically significant differences in reporting on CS indicators among the 21 countries with a laboratory participating in the SSPP compared with the 66 countries without SSPP-participating laboratories (p=0.56). Compared with countries without SSPP-participating laboratories, countries with laboratories in the SSPP had more complete reporting on antenatal syphilis testing over multiple years (median, 4.0 years vs 3.0 years; p=0.04), and a non-significant tendency toward more complete reporting on treatment (median, 2.0 years vs 1.0 years; p=0.55), this was most notable in SEAR and WPR countries. SSPP participating countries also tended to be more likely to report a more than 25% increase in treatment coverage during the study interval, although this was not significant at p<0.05 (10% vs 3%, p=0.25). The SSPP participating countries did not differ from non-participating countries in their reporting of 25% increase in testing coverage (table 1).

Table 1

Reporting of congenital syphilis indicators on testing and treatment by lower income countries, by participation in the WHO-CDC Syphilis Serology Proficiency Programme (SSPP), 2008–2015

Among the 87 ‘lower income’ countries reporting CS indicator data to WHO, the 21 countries participating in the SSPP contributed an estimated 273 966 positive syphilis test results reported in pregnant women, or 63% of the positive results reported by all lower income countries.

Discussion

In this first evaluation of the extent to which this long-standing SSPP supports the WHO initiative for CS elimination, we found that participating laboratories had high participation rates and performance scores. The finding that more than half of participating laboratories were national reference laboratories is supportive of the SSPP priority of focusing on national laboratories that may be able to support syphilis testing in their country’s underlying clinical laboratories. However, the fact that the programme often supports laboratories located in higher income countries, with just under half (43%) located in the lower income countries targeted by the programme, indicates that more work is needed in attracting laboratories from priority countries. The consistently high scores of participating laboratories is important, but may be related to the fact that most laboratories were long-term participants and familiar with the external quality assurance requirements. Additionally, we found that more than half (63%) of the estimated syphilis infections in pregnant women during 2015 that occurred in lower income countries were detected in countries supported by at least one SSPP-participating laboratory. This finding suggests that the programme has good reach in supporting global elimination of CS, although this reach could be even wider if there were greater participation by national and regional reference laboratories from several large, lower-income countries in Africa and Latin America, where prevalence of syphilis in antenatal women is high.13

This programme evaluation has some limitations. The programme depends on laboratories reporting their test results based on usual practices; however it is believed that a few laboratories have used specialised kits only for proficiency testing rather than ‘routine testing kits’ while undergoing their quality assurance programmes in order to ensure high scores. Participating laboratories should treat the testing panel as clinical samples and use kits routinely used in the laboratory for proficiency testing. The SSPP is one tool used to verify accuracy and reliability of laboratory testing and the programme is not meant to be punitive, but informative. Tools must be developed that can objectively measure the performance of a laboratory outside of external proficiency testing. Additionally, we used data reported to WHO to estimate the extent to which the SSPP has supported the global CS elimination initiative; however, in some countries these data may not be representative of screening and treatment practices across the country.

External quality assurance contributes to laboratory capacity strengthening, and is also an essential component of laboratory evaluation for country-level review for consideration of validation of elimination of mother-to-child transmission of HIV and syphilis (EMTCT).15 The SSPP is one, relatively simple tool that can support laboratory quality assurance in lower income countries working towards validation of EMTCT. Advantages of the SSPP are that it is provided at no cost to countries and can be easily incorporated with their work flow as the external quality assurance is provided only three times per year. Further expansion of this programme to laboratories within low income and high syphilis morbidity countries could provide better access to high quality syphilis diagnostic capacity supporting CS elimination, as well as further support to global sexually transmitted infection (STI) surveillance strengthening. Additionally, technical support to countries on strategies to extend external quality assurance beyond the national reference laboratory would support further reach of this programme and quality of syphilis testing. These types of extensions of the SSPP might be accomplished through partnerships with WHO, CDC and other organisations that focus on improving the quality of laboratory testing. These types of laboratory data are important elements supporting countries in making more evidence-based decisions on control and elimination strategies.

References

Footnotes

  • Contributors All authors were involved in writing the manuscript. MK and EK were instrumental in the conceptualisation and design of the study. AP, MT and YF assisted with interpretation of the laboratory data. TT and AH performed data collection, statistical analysis, and drafting of the manuscript. The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.

  • Funding This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

  • Map disclaimer The depiction of boundaries on the map(s) in this article does not imply the expression of any opinion whatsoever on the part of BMJ (or any member of its group) concerning the legal status of any country, territory, jurisdiction or area or of its authorities. The map(s) are provided without any warranty of any kind, either express or implied.

  • Competing interests None declared.

  • Patient consent for publication Not required.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data availability statement All data relevant to the study are included in the article or uploaded as supplementary information. Additional data are available upon reasonable request.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.