Article Text
PDF
Abstract
Objective Seven recombinant viral capsid antigen-IgA (VCA-IgA) ELISA kits are widely used in China, but their diagnostic effects have not been evaluated. In this study, we evaluated whether the diagnostic effects of these kits are similar to those of the standard kit (EUROIMMUN, Lübeck, Germany).
Methods A diagnostic case–control trial was conducted with 200 cases of nasopharyngeal carcinoma (NPC) and 200 controls from NPC-endemic areas in southern China. The areas under the curve (AUCs), the sensitivities and the specificities of testing kits were compared with those of the standard kit. The test–retest reliability of each kit was determined by intraclass correlation coefficient (ICC). Their diagnostic accuracy in combination with Epstein-Barr virus nuclear antigen 1-IgA (EBNA1-IgA) was also evaluated in logistic models.
Results Three testing kits—BB, HA and KSB—showed diagnostic accuracy equal to that of the standard kit, with good performance in the AUCs (0.926–0.945), and no significant differences in sensitivity were found between early-stage and advanced-stage NPCs. ICCs exceeded 0.8. Three logistic regression models were built, and the AUCs of these models (0.961–0.977) were better than those of the individual VCA-IgA kits. All new models had diagnostic accuracy equal to that of the standard kit. New cut-off values of these three kits and their corresponding combinations for researchers to replicate and use in NPC early detection and screening in the future were provided.
Conclusions Three recombinant VCA-IgA kits—BB,HA and KSB—had diagnostic effects equal to those of the standard kit, and, in combination with EBNA1-IgA in logistic regression models, can be used in future screening for NPC.
- Nasopharyngeal carcinoma
- Epstein-Barr virus
- VCA-IgA
- Diagnostic effect
- Screening
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
This is the first study to carry out a comprehensive evaluation of recombinant commercial diagnostic viral capsid antigen-IgA (VCA-IgA) (ELISA) kits in China, and logistic models combining VCA-IgA with Epstein-Barr virus nuclear antigen 1-IgA were established.
New cut-off values for VCA-IgA kits and their corresponding combinations for researchers to replicate and use in nasopharyngeal carcinoma (NPC) early detection and screening in the future were provided.
All cases and controls were from NPC-endemic areas of southern China, and thus, these results might not be applicable to other populations.
Only 33 early-stage NPC cases were collected. Controls were recruited from rural area, but half of the NPC cases were from urban areas.
Cut-off values for NPC screening by means of these models described in this study must be verified in prospective mass screening.
Introduction
Nasopharyngeal carcinoma (NPC) is a common form of squamous cell carcinoma in southern China and Southeastern Asia. The annual incidence rate of NPC in southern China can reach 25 per 100 000 person-years, which is about 25-fold higher than in the rest of the world.1–4 NPC is a complex disease caused by a combination of Epstein-Barr virus (EBV), chronic infection, the environment and host genes in a multistep process of carcinogenesis, but until now there have been no effective preventive measures.5–9 Long-term survival rates differ substantially between patients with advanced-stage (stages III and IV) and early-stage NPC (stages I and II). Four-year survival rates of patients with early-stage NPC are 96.7% compared with 67.1% for those with advanced-stage NPC.5 Mass screening has become the most practical method for improved early detection in, and overall prognosis of, patients with NPC in the endemic areas.10 11
Serum antibodies against EBV-related antigens, especially IgA against viral capsid antigen-IgA (VCA-IgA), early antigen-IgA, EBV nuclear antigen 1 (EBNA1-IgA) and so on, remain elevated for an average of 38 months in the preclinical phase,9–14 and serological tests for these markers are simple and inexpensive.15–18 Therefore, since the 1970s, these tests have been used as screening markers for NPC in endemic areas. In our previous study, we evaluated the diagnostic performance of seven commercial EBV-related antibodies by ELISA and found EBNA1-IgA (Zhongshan Biotech, China) and VCA-IgA (EUROIMMUN, Lübeck, Germany) ELISA to be the top two seromarkers, with areas under the curve (AUC) of 0.95 (95%CI 0.93 to 0.97) and 0.94 (95%CI 0.92 to 0.97), respectively.16 We further verified that the combination of VCA-IgA and EBNA1-IgA outperformed any individual EBV seromarkers, with AUC up to 0.97 (95% CI 0.96 to 0.99).15–17 Thus, since 2011, the combination of VCA-IgA and EBNA1-IgA has been recommended as the standard tool for NPC screening in China.18
Nowadays, several kinds of commercial VCA-IgA kits based on recombinant peptides have been developed in China and are presently widely used for the early detection of and screening for NPC. However, their diagnostic performance for NPC alone and in combination with EBNA1-IgA has not been evaluated. In this study, we evaluated whether the effects of the NPC-diagnostic kits are comparable to those of the standard VCA-IgA kit and can be substituted for it. If so, we will further explore the combination diagnostic strategy with EBNA1-IgA for the early detection of and mass screening for NPC.
Methods
Study population
Serum specimens were continuously collected from 200 patients with NPC hospitalised in the Sun Yat-sen University Cancer Center (SYSUCC) from January 2013 to June 2013. These cases were histologically confirmed by biopsy, and the clinical stages were classified according to the 2009 Union for International Cancer Control criteria, including 33 patients with early-stag e NPC (stages I and II) and 167 with advanced-stage NPC (stages III and IV). The inclusion criteria included being between 30 and 59 years of age and residing in one of the six high-endemic provinces of southern China (Guangdong, Guangxi, Jiangxi, Hunan, Fujian or Hainan Province). Other information, including demographic data, smoking, drinking histories and family history of NPC, was collected by the physician in charge. All serum samples were collected before treatment.
The 200 healthy controls were randomly selected from among healthy people who participated in physical examinations at the Sihui Cancer Center (Sihui City, Guangdong Province, China) from July 2013 to September 2013 and were frequency matched with cases by age (5-year age groups) and gender. All participants completed a short questionnaire to record demographic data, smoking, drinking histories and family history of NPC and donated 3 mL of blood.
This study was approved by the Clinical Research Ethics Committee of the SYSUCC (YB2015-029-01), and all participants provided written informed consent.
Detection of serological EBV antibodies
Serum and buffy coat were separated <4 hours after collection and stored at −80°C before being tested. None of the specimens was haemolytic or repeatedly frozen more than twice. Seven recombinant VCA-IgA kits, the standard VCA-IgA kit (EUROIMMUN) and the standard EBNA1-IgA kit (Zhongshan) were tested (table 1).
Product information for eight brands of VCA-IgA kits and the EBNA1-IgA kit
All samples were renumbered and tested blindly by one technician according to the manufacturers’ instructions. Levels of antibodies were assessed by photometric measurement, which provided optical density (OD) values. Reference ODs (rODs) were obtained according to the manufacturers’ instructions by dividing OD values by a reference control. To investigate the test–retest reliability of each kit, 10% serum samples (40 samples) were randomly chosen and retested.
Statistical analysis
Demographic characteristics and NPC risk factors between cases and controls were compared by χ2 tests. The cut-off value of each single kit was defined with the largest Youden Indices (sensitivity+specificity−1) chosen from each receiver operating characteristic (ROC). The diagnostic efficacy of each kit was evaluated by AUC, and non-inferiority tests based on the bootstrap approach were performed to determine whether the AUCs of these recombinant testing kits were inferior to that of the standard kit (let Δ=0.05 be the predetermined clinically meaningful equivalence limit).19–22 The sensitivity and specificity of each kit were calculated and their 95% CIs were estimated by the methods of Simel et al 23 (The 95% CIs of sensitivities for early-stage groups were estimated by lookup table method for binomial distribution because the sample size was less than 50).24 Differences in sensitivities between early-stage and advanced-stage NPC with each kit were compared by χ2 tests (Fisher’s exact test and McNemar’s test will be specified while χ2 tests means Pearson’s χ2 test). Intraclass correlation coefficients (ICCs) were performed to determine test–retest reliability.
To prevent bias and study the virus factor only, we matched the baseline covariates (gender and age) and some of the important NPC risk factors (smoking, drinking and NPC history). Binary unconditional logistic regressions were used to establish formulae for VCA-IgA and EBNA1-IgA. The diagnostic efficacy of each formula was evaluated by sensitivity, specificity and AUC, compared with the standard formula, Logit p=−3.934+2.203 VCA-IgA (EUROIMMUN)+4.797 EBNA1-IgA. The cut-off p value in the corresponding logistic regression for distinguishing between NPC cases and controls was defined with the largest Youden Index chosen from each ROC. Two minimally acceptable false-positive rates (1-specificity), 3% and 7%, were used empirically to establish the cut-off p values for classifying different NPC risk subgroups.16 17
The non-inferiority tests were one sided, and p>0.05 was considered to be non-inferior. Other tests were two sided, and p<0.05 was considered to be statistically significant. Data were analysed by SAS V.9.2 and SPSS V.16.0 software.
Results
Baseline information
Baseline information on gender, age, smoking, drinking and NPC family history was comparable between cases and controls, and no statistically significant differences were found between them. Further, there were no statistically significant differences for these items between early-stage and advanced-stage cases (table 2).
Demographic characteristics of NPC cases and controls
The diagnostic accuracies of eight brands of VCA-IgA kits and the EBNA1-IgA kit
Table 3 shows that the AUCs of four kits—KSB, BB, BNV and HA—were as high as that of the standard VCA-IgA kit (AUC 0.942; 95% CI 0.920 to 0.964). The AUCs, in order, were 0.945 for KSB (95% CI 0.925 to 0.966), 0.940 for BB (95% CI 0.916 to 0.964), 0.936 for BNV (95% CI 0.911 to 0.961) and 0.926 for HA (95% CI 0.900 to 0.953). In addition, the AUCs of GBI, HK and ZS were lower than that of the standard kit. Furthermore, no significant differences were found between early-stage and advanced-stage NPC in the sensitivities of six kits (p>0.05), except for BNV (p=0.044).
The diagnostic accuracies of eight brands of VCA-IgA kits and the EBNA1-IgA kit in distinguishing between NPC cases and controls
The test–retest reliabilities of eight brands of VCA-IgA kits and the EBNA1-IgA kit
Ten per cent serum samples (40 samples) were randomly chosen and retested for calculation of the ICC of each brand of kit, VCA-IgA or EBNA1-IgA. The test–retest reliabilities of all kits were excellent (>0.75, excellent) according to Fleiss’s classification25 (table 4).
The test–retest reliabilities of eight brands of VCA-IgA kits and the EBNA1-IgA kit
The diagnostic accuracies of the combinations of VCA-IgA and EBNA1-IgA with logistic models
We chose three VCA-IgA kits with high AUCs, no differences in diagnoses for early-stage and advanced-stage NPC and excellent test–retest reliabilities, and then combined each with the EBNA1-IgA kit by logistic models. Three logistic regression models were established:
LogitP=−3.2323+0.8060 VCA-IgA (BB)+1.1044 EBNA1-IgA
LogitP=−2.7591+0.6380 VCA-IgA (HA)+1.0620EBNA1-IgA
LogitP=−2.6039+0.5312 VCA-IgA (KSB)+1.1673 EBNA1-IgA
In all these models, both VCA-IgA and EBNA1-IgA were statistically significant independent predictors of NPC risk (p<0.05), and the AUC of each combination was statistically significantly larger than that of each single VCA-IgA (p<0.05). The AUC of KSB increased from 0.945 (95% CI 0.925 to 0.966) to 0.964 (95% CI 0.947 to 0.981); BB increased from 0.940 (95% CI 0.916 to 0.964) to 0.977 (95% CI 0.963 to 0.991) and HA increased from 0.926 (95% CI 0.900 to 0.953) to 0.961 (95% CI 0.943 to 0.979) (figure 1).
ROCs for BB, HA, KSB, EUROIMMUN and their combination with EBNA1-IgA.
Table 5 shows the diagnostic accuracies of the three new combinations and the standard combination (Logit p=−3.934+2.203 VCA-IgA (EUROIMMUN)+4.797EBNA1-IgA) in distinguishing between NPC cases and controls. The AUCs of these three combinations were as high as that of the standard combination (AUC 0.970; 95% CI 0.956 to 0.985) (p<0.05). Furthermore, no statistically significant difference was found in the sensitivity of each combination between early-stage and advanced-stage NPC (p>0.05).
The diagnostic accuracies of the three new combinations and the standard combination in distinguishing between NPC cases and controls
We used two minimally acceptable false-positive rates (1-specificity) of 3% and 7% to define the high-risk and medium-risk cut-off values for the new combinations. The corresponding logistic regression p values were 0.707 and 0.232 for BB, 0.766 and 0.364 for HA and 0.831 and 0.384 for KSB, and the corresponding true-positive rates (sensitivities) were 88.0% and 93.5% for BB, 78.0% and 88.0% for HA and 79.0% and 87.5% for KSB.
Discussion
In our study, seven recombinant VCA-IgA kits were evaluated, and of these, KSB, BB and HA had diagnostic effects as good as those of the standard kit in terms of sensitivity, specificity and AUC. Combining VCA-IgA with EBNA1-IgA by logistic regression models increased the diagnostic accuracy of these three kits, and all combinations performed as well as the standard combination in sensitivity, specificity and AUC. This is the first study to carry out a comprehensive evaluation of recombinant commercial diagnostic VCA-IgA (ELISA) kits in China, and logistic models combining VCA-IgA with EBNA1-IgA were established. Furthermore, new cut-off values for these VCA-IgA kits and their corresponding combinations for researchers to replicate and use in NPC early detection and screening in the future were provided.
In this study, we first calculated the diagnostic performance of each brand of VCA-IgA kit. The AUC of the standard VCA-IgA kit (EUROIMMUN) was 0.942 (95% CI 0.920 to 0.964), which was consistent with the results from our previous studies and verified that the diagnostic performance of VCA-IgA was good and stable.16 17 We also found that the sensitivities, specificities and AUCs of three kits—KSB, BB and HA—were as high as those of the standard kit, and no significant differences in sensitivity were found between early-stage and advanced-stage NPC. Moreover, all test–retest reproducibilities were excellent (>0.75) and the coefficient of variations of difference values of test and retest results of all assays were shown in supplementary table 1. These results suggested that these three kits had equal diagnostic effects and can be substituted for the standard kit. The costs of these recombinant commercial diagnostic kits were only half that of the standard kit, making them more cost effective.
The EBV capsid antigen (VCA) is a late protein produced in the EBV lytic infection period. VCA contains a batch of capsid proteins, such as VCA-p18 (BFRF3), VCA-p23 (BLRF2), gp125/110 (BALF4) and so on, which have unique immune dominants and virus-specific antigenic domains. These domains contain several small peptide regions (epitopes) which can be combined to form a powerful diagnostic reagent for VCA-IgA.26 27 The capsid proteins in the EUROIMMUN kit28 were extracted from the pyrolysis products of human B lymphocytes (P3HR1 cell line) infected by EBV and contained a combined native capsid protein of EBV. We noticed that, in contrast to the standard kit with a combined native capsid protein, these testing kits contain primarily recombinant p18 capsid proteins (VCA-p18). VCA-p18 is a small capsid protein that contains several small peptide regions (epitopes) which can be combined to form a powerful diagnostic reagent for VCA-IgA antibody responses.27 Some researchers have reported that VCA-p18 is the major VCA antigen for IgA responses.27 29 Our study showed that the AUCs of these VCA-p18 recombinant kits were more than 0.85, and three of them had the same diagnostic effects as the standard kit, suggesting that, alth ough the manufacturing processes of some recombinant VCA-p18 kits still need to be improved, some of the recombinant kits can be substituted for the standard kit for NPC diagnosis.
As the serum antibody level (rOD) provides continuous data, the cut-off value for distinguishing between NPC cases and controls is critical for the early detection of and screening for NPC. A reasonable cut-off value can balance sensitivity and specificity. In the early detection of and screening for NPC, high sensitivity is required for the identification of high-risk individuals, and high specificity is required for reducing the rate of misdiagnosis and associated costs. According to the cut-off values provided by the kits’ instructions, the sensitivities of testing kits were always too low, whereas their specificities were always too high. For example, the sensitivity and the specificity of KSB are 0.780 and 0.925, respectively, suggesting that the old cut-off values should be adjusted (online supplementary table 2). We established new cut-off values for distinguishing between NPC cases and controls by Youden Indices, and then obtained reasonable sensitivities and specificities. After adjustment, the new cut-off value for KSB is 0.283, and the sensitivity and specificity are 0.890 and 0.875, respectively. Moreover, no differences were found between sensitivities and specificities of these three kits —KSB, BB and HA—and those of the standard kit. Due to the low percentage (<20%) of early stage in clinic, we can only collect 33 early-stage NPC participants in our study. Analysing the sensitivities by pooling early and late stage together only was not appropriate. So we did subgroup analysis and found there were also no statistically significant differences in the sensitivities of these three kits for early-stage and advanced-stage NPC. Furthermore, there were no differences between the early-stage sensitivities of these three kits and that of the standard kit too (0.202 for BB, 0.0672 for HA and 0.112 for KSB).
As for the standard VCA, we found that the combinations of VCA-IgA and EBNA1-IgA by logistic models increased the diagnostic accuracy for NPC from less than 0.946 to more than 0.961 in AUCs. Sensitivities and specificities also increased. For example, the sensitivity and specificity of BB increased from 0.865 and 0.920 to 0.935 and 0.955, respectively. VCA-IgA and EBNA1-IgA are two antibodies corresponding to EBV lytic-cycle proteins and latency gene products, respectively. Therefore, it is reasonable that host antibody responses for lytic-cycle and latency-associated EBV-related proteins can be complementary to each other in the diagnosis of NPC, and the combination of both could increase NPC diagnostic accuracy.11 30 Furthermore, these three new combinations had diagnostic effects in sensitivities (including subgroup analysis), specificities and AUCs equal to those of the standard combination, suggesting that the combinations of the three recombinant kits can be used for the early detection of and diagnostic screening for NPC. In this study, the control individuals came from NPC-endemic areas and belonged to a screening target population, so we attempted to define people at different risk levels by these new combinations for NPC screening. Compared with other common diseases, the NPC incidence rate in the screening target population was relatively low (about 50 per 100 000 person-years).2 10 31 Thus, it is important that the false-positive rate be small enough to avoid unnecessary fiberoptic endoscopy/biopsies and psychological stress for the NPC screening participants. Conversely, the true-positive rate (equal to sensitivity) should be acceptable.32 We used two minimally acceptable false-positive rates of 3% and 7% as the high-risk and medium-risk cut-off values, respectively,17 and the corresponding true-positive rates (sensitivities) for these three kits were 78.0%–88.0% and 87.5%–93.5%,5 respectively. If the baseline serologic results fulfilled the definition of high risk, the participants were referred for diagnostic examinations and different screening intervals were assigned to the high-risk, medium-risk and low-risk groups. The screening intervals for these groups are 1, 1 and 4 years, respectively.17
The study had some limitations. First, this study was a single-centre study and all cases and controls were from NPC-endemic areas of southern China (controls were from hospital); therefore, these results might not be applicable to other populations. Second, due to the low percentage (<20%) of early stage in clinic, we can only collect 33 early-stage NPC participants in our study period. But the phenomenon also indicated that most patients are typically not detected until NPC is in an advanced stage. Finding out such people was also very meaningful in real life. Third, controls were recruited from rural area, but half of the NPC cases were from urban areas (rural:urban=95:105). Although no evidence showed that there were different infection rates between rural and urban people, it might cause some other unknown bias. Fourth, it was a diagnostic trial in case–control design and new cut-off values of these new schemes for NPC screening from this study must be verified in prospective mass screenings.
Conclusions
Three recombinant VCA-IgA kits—BB, HA and KSB—had diagnostic effects equal to those of the standard kit. They can be substituted for the standard kit and their combinations could be used in the early detection of and screening for NPC.
References
Footnotes
Contributors S-MC and QL obtained the funding; RG, LW and L-FZ contributed to study conception and design; RG conducted experiments; LW, Y-FY, J-LD, S-HX, S-HC, JG, M-JY and C-L acquired or cleaned the data; RG analysed and interpreted the data; RG, S-MC and QL drafted or revised the manuscript. All authors critically revised the manuscript for important intellectual content. All authors read and approved the final version of the manuscript.
Competing interests None declared.
Patient consent Detail has been removed from this case description/these case descriptions to ensure anonymity. The editors and reviewers have seen the detailed information available and are satisfied that the information backs up the case the authors are making.
Ethics approval Sun Yat-sen University Cancer Center IRB.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Technical appendix, statistical code and dataset in main document are available from the corresponding author at caosm@sysucc.org.cn and liuqing@sysucc.org.cn.