METHODS

We conducted a cluster randomised controlled trial in the rural Hlabisa health district in KwaZulu-Natal, South Africa. The district has a population of approximately 220 000 served by 14 primary care clinics and a district hospital. Antenatal care is available 1 day each week at each clinic, and most women (95%) attend antenatal clinics at least once before delivery.¹⁸ The clinics in the district were divided into seven pairs, matched on uptake of antenatal care services, antenatal syphilis prevalence, and geographical similarities (for example, proximity to main roads) (fig 1). Within each pair, one clinic was randomised to receive the intervention of on-site syphilis testing complemented by confirmation in the reference laboratory, and the other to continue routine laboratory testing. An independent scientist randomised the clinics through toss of a coin; allocation was unblinded.

Participants

At each clinic, all pregnant women making their first antenatal visit answered a brief questionnaire to collect demographic information and pregnancy history. Blood specimens used in the study were taken during specimen collection for routine antenatal screening. Participants were women with a positive test for syphilis. All women testing positive for syphilis were given standard partner notification cards.

Intervention procedures

In the intervention clinics the rapid plasma reagin test (Macro-Vue RPR Card Test, Becton-Dickinson, USA) was performed on site by nursing staff. Nurse training at intervention clinics consisted of a 1 day workshop on the conduct of the on-site test. This included information on the logistics of the test, including specimen handling and maintenance, as well as supervised practice in conducting the test and reading test results. Continued monitoring of on-site test performance, as well as refresher training for new clinic staff, was provided at intervention clinics throughout the trial.

In the on-site test, cells were separated from plasma by standing for 30 minutes, and serum was mixed with RPR antigen using a battery powered rotator according to the manufacturer’s instructions. Clinic staff reported results as either positive or negative using colour photographs as a reference standard. Women whose on-site test was reported as positive received the first of three recommended weekly intramuscular injections of 2.4 mega units benzathine penicillin on the same day.^8,13 In addition, all samples from intervention clinics were sent to the provincial reference laboratory for routine RPR (quantitative) testing. Women who tested negative in the on-site test were counselled to return 2 weeks later to receive their laboratory results; if they tested positive only in the laboratory, these women were given their first penicillin injection when they returned. Women testing positive on site but negative in laboratory tests were still considered RPR positive and were counselled to complete their treatment.

Control procedures

The study procedures at control sites followed routine antenatal care practices in the district. Blood specimens drawn during the first antenatal visit in control clinics were sent to the provincial reference laboratory and all women were counselled to return 2 weeks later to receive their results. On returning, women with a positive result were counselled on the treatment regimen and received their first penicillin injection.

Follow up

After receiving the first treatment dose, all women who tested RPR positive (in either on-site or laboratory test) were counselled to return to the clinic for two follow up injections at weekly intervals. Pregnancy outcome data were determined from hospital and clinic records, or by home visits where necessary.

Definitions

The primary outcome of the trial, perinatal death, was defined as either stillbirth (born dead at or after 28 weeks’ gestation) or early neonatal death (death up to 7 days postpartum). Adequate treatment of syphilis was defined, according to provincial guidelines, as two or more doses of 2.4 mega units of benzathine penicillin, and inadequate treatment was defined as one dose or no treatment.^8,13 A positive syphilis test was defined by the on-site and/or laboratory RPR test results; participants who were missing test results were treated based on clinical judgment (in 33 of 7618 women, or <1%).

Sample size

The study was designed with 80% power of detecting a 50% reduction in the incidence of perinatal death among women with syphilis from 12% in the control clinics to 6% in the intervention clinics. Baseline data from this trial⁹ showed that syphilis prevalence was 9%, 68% of women received adequate treatment, 19% received no treatment, and overall perinatal mortality was 11%. To adjust for the clustering effect of the trial design, the coefficient of variation of the syphilis prevalence (CV = 0.28) in the Hlabisa antenatal clinics for 1996⁸ was used as a proxy for the between clinic variation of perinatal deaths. Using a significance level of 5% and an average clinic sample of 50 women, seven matched pairs were required and a recruitment period of 18 months was planned.

Analysis

Data were analysed using sas (SAS Institute, Cary, USA). The trial’s matched pair design was accommodated through the use of a random effects meta-analysis across clinic pairs.¹⁹ The risk difference or mean difference in outcome variables between the intervention and control clinics was calculated for each pair and a weighted average was calculated across pairs, with the weighting inversely proportional to the variance of the individual pairs. Weighted analyses were made necessary by imbalances between the cluster sizes across clinics in different pairs, and as zero proportions were recorded within some pairs, the risk difference was the necessary effect measure. All data analysis was by intention to treat.

The University of Natal research ethics committee granted ethical approval for the study. Consent to conduct the trial was provided by the Hlabisa District Health Service.

RESULTS

Recruitment started in the seven clinic pairs between October 1998 and February 1999, and ended in all pairs in January 2000. Of 7618 women seeking antenatal care (fig 2), syphilis results were available for 7134 (94%). Missing results were due largely to logistical problems in conducting the on-site test and transporting specimens to the provincial reference laboratory (for example, broken tubes, lost specimens), and were unlikely to be associated with syphilis serostatus.

Of the women with available test results, 793 (11.1%) were enrolled after testing RPR positive either on site or in the laboratory. Table 1 compares demographic characteristics and pregnancy histories of women attending intervention and control clinics. There was an imbalance in the numbers recruited between the two arms of the trial because of one very busy clinic in the intervention arm. Participants attending intervention clinics were slightly younger and of lower mean gravidity than those attending control clinics, but there were no differences in gestational age or previous pregnancy loss. In adjusted analyses (not shown), the baseline differences in age and gravidity did not alter substantially trial findings and as a result only unadjusted results are presented here.

Syphilis prevalence

The prevalence of laboratory confirmed syphilis was 7.5% and was similar in intervention and control clinics after accounting for the matched pair design of the study. RPR titre data were determined during the second half of the study period only, and were available for 346 women (55%) attending intervention clinics and 134 (75%) of those attending control clinics. The median titre was 1:4, and 20% of participants had titres of 1:16 or greater.

Validity of the on-site test

Of the 394 women who tested positive for syphilis in the laboratory, 246 also tested positive on site (on-site test sensitivity, 62%). Of the 4320 women who tested negative in the laboratory, 4138 tested negative on site (on-site test specificity, 96%; percentage agreement, 93%; κ = 0.56). The sensitivity of the on-site test increased to 83% among women with titres greater than 1:4.

Syphilis treatment

The average time from the first antenatal visit to the completion of syphilis treatment was 16 days shorter for women attending intervention clinics compared to those attending control clinics (95% CI: 11 to 21, p<0.001). Comparing intervention and control clinics (table 2), similar proportions received no treatment (20% v 19%) and adequate treatment (64% v 69%; adjusted risk difference, 0.3%; 95% CI: −10.4 to 10.9). At the intervention clinics, 63 of the 125 of women who required treatment (according to laboratory test results or clinical judgement) but received none were negative in the on-site test (50%). In contrast with this, only 17% of women who received at least one penicillin injection were negative in the on-site test (86 of 493), suggesting that a negative on-site test result was strongly associated with receiving no treatment (adjusted risk difference, intervention minus control, 28%; 95% CI: 20 to 36).

Pregnancy outcome

Pregnancy outcome data were available for 723 women (91%). Failure to collect outcome data was due primarily to movement outside of the district for delivery. In this case, women did not deliver at a facility in the district, and could not be traced at their home. Mobility, even during pregnancy, is very common in this setting, and this censoring is unlikely to be related to pregnancy outcome.

Of the women with pregnancy outcome data, 12 women in the intervention arm (2.1%) and five women in the control arm (3.1%) reported miscarriages; these were excluded from the perinatal mortality analysis. Table 3 shows the distribution of outcomes among viable pregnancies, for each clinic pair and overall. In all, 3.3% of pregnancies resulted in perinatal death among women attending intervention clinics, compared with 5.1% among women attending control clinics (adjusted risk difference, −0.9%; 95% CI: −4.4 to 2.7, p=0.31).

DISCUSSION

Our trial did not demonstrate a significant impact of implementing on-site syphilis testing in addition to laboratory testing on perinatal mortality and treatment rates, despite substantially reducing the delay to treatment. As such, we observed no benefit to implementing an on-site testing intervention in this setting, where relatively well functioning laboratory based syphilis testing services are already in place.

There are several possible reasons why our trial did not show an effect on perinatal mortality, the primary outcome. Firstly, the power of the trial to demonstrate the hypothesised effect was reduced by a lower than expected frequency of perinatal deaths in control clinics (11% at baseline v 5% during the trial), and by there being no increase in the proportion of women adequately treated for syphilis in the intervention clinics (68% at baseline⁹ and 64% in the trial). Although our baseline results suggested that a 50% reduction was possible, it is important to note that less dramatic reductions in perinatal mortality are also of public health relevance. Secondly, the intervention failed to treat a greater proportion of women at intervention clinics compared to control clinics. This appears to be due primarily to technical and logistical difficulties around use of the on-site test, and the relatively high quality of laboratory services provided in the control arm.

The on-site test is relatively complex, as serum needs to be separated from cells and mixed with antigen, and reading the result can be subjective and difficult. With high workloads and limited staffing, nurses at busy intervention clinics frequently had difficulty in conducting the on-site test, informing women of their infection, and initiating treatment. Additional problems were encountered in maintaining a regular supply of testing materials, particularly reagent and batteries for rotators. Even when test results were available, nurses frequently told us that some women left the clinic before receiving their results. These logistical problems in the conduct of the on-site test echo previous experiences from other developing country primary care settings.^5,10,14,20 The sum of these experiences suggests that the currently available technology for on-site testing is perhaps inadequate, both in terms of ease of use and in distinguishing weak positive results. Qualitative serology is generally highly sensitive and the materials used in the RPR test have been shown to be durable.²¹ However, the test requires considerable time to conduct adequately, and considerable experience to read accurately, particularly for low and moderate titres. It is not clear whether low titre RPR results indicate a substantially increased risk for perinatal mortality.²² However as the on-site test showed substantially higher sensitivity in detecting higher titre infections, it is likely that on-site testing represents an important approach for identifying high titre (and possible high risk) infections in settings without laboratory access.

Although there have been recent developments in syphilis laboratory diagnostics, there have been few advances in the technology of syphilis screening, particularly in tests which may be used in resource limited primary care settings. The difficulties we and others have encountered in on-site syphilis screening are in stark contrast with recent advancements in rapid HIV testing, in which the development of an inexpensive, easy to use, and accurate test has led to major improvements in the availability and feasibility of screening for HIV in developing countries.²³ Given the burden of unidentified and untreated syphilis infection on both perinatal and adult health, there is a clear need for advances in syphilis screening technology.

In addition, the control arm in this trial received services which may be of a relatively high quality compared to other resource limited healthcare settings. A range of factors contribute to the failure of laboratory based antenatal syphilis screening programmes. These include the failure of women to return for their test results, as well as logistical difficulties and delays in specimen collection, transport and testing, and the return of laboratory results. Here, on-site testing was evaluated in a context of relatively well functioning public sector laboratory services (at least in comparison with other settings in sub-Saharan Africa), and the nature of trial implementation may have also contributed to improvements to the existing mechanisms for laboratory based testing. In this light, the failure to demonstrate the impact of on-site testing may be attributed in part to the relatively high quality of services provided in the control arm. As a result, our null findings may not be generalisable to settings where laboratory services are less functional, or altogether non-existent.

In conclusion, our trial, while somewhat underpowered, does not support using on-site syphilis testing in addition to laboratory based testing. However, there still is a range of settings in the developing world where laboratory facilities for antenatal syphilis testing function poorly or do not exist at all.³ In such instances, on-site testing may represent an important intervention to improve antenatal care and reduce perinatal mortality.