Objective Although the values of soluble mesothelin-related peptides (SMRPs), including mesothelin and megakaryocyte potentiating factor, in serum and/or pleural fluid for diagnosing malignant pleural mesothelioma (MPM) have been extensively studied, the exact diagnostic accuracy of these SMRPs remains controversial. The purpose of the present meta-analysis is to update the overall diagnostic accuracy of SMRPs in serum and, furthermore, to establish diagnostic accuracy of SMRPs in pleural fluid for MPM.
Design Systematic review and meta-analysis.
Methods A total of 30 articles of diagnostic studies were included in the current meta-analysis. Sensitivity, specificity and other measures of accuracy of SMRPs in serum and pleural fluid for the diagnosis of MPM were pooled using random effects models. Summary receiver operating characteristic curves were used to summarise overall test performance.
Results The summary estimates of sensitivity, specificity, positive likelihood ratio, negative likelihood ratio and diagnostic OR were 0.61, 0.87, 5.71, 0.43 and 14.43, respectively, for serum and 0.79, 0.85, 4.78, 0.30 and 19.50, respectively, for pleural fluid. It was also found that megakaryocyte potentiating factor in serum had a superior diagnostic accuracy compared with mesothelin for MPM.
Conclusions SMRPs in both serum and pleural fluid are helpful markers for diagnosing MPM with similar diagnostic accuracy. The negative results of SMRP determinations are not sufficient to exclude non-MPM, and the positive test results indicate that further invasive diagnostic steps might be necessary for the diagnosis of MPM.
- RESPIRATORY MEDICINE (see Thoracic Medicine)
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/
Statistics from Altmetric.com
Strengths and limitations of this study
The studies included in this meta-analysis were methodologically satisfactory and their results were consistent and close.
The subjects in the control groups were very heterogeneous from one study to another.
Various cut-off points were used for distinguishing between malignant pleural mesothelioma and the other diseases.
Conference abstracts, letters to the editors, and non-English language studies were excluded.
Pathological types of malignant pleural mesothelioma were not specified in some studies.
Malignant pleural mesothelioma (MPM) is a highly aggressive almost uniformly fatal tumour primarily caused by exposure to asbestos.1 Current therapeutic options for MPM are limited and the prognosis is poor.2 When patients are treated with standard of care chemotherapy, cisplatin and an antifolate, median survival is approximately 1 year.3 ,4 Early diagnosis offers the best hope for a favourable prognosis; however, the early and reliable diagnosis of MPM is extremely difficult as only 5% of patients present with stage IA disease.5 ,6 There is therefore a critical need for reliable and non-invasive tools that shorten this diagnostic delay.
Many soluble markers, such as mesothelin family proteins, in serum or pleural fluid (PF) have been evaluated to facilitate the non-invasive diagnostic investigation for MPM.6 Mesothelin is a 40 kDa cell surface glycoprotein that is highly expressed in MPM, pancreatic cancers, ovarian cancers and some other cancers. It is synthesised as a precursor 69 kDa protein and forms two proteins, the membrane-bound mesothelin and a soluble 31 kD N-terminal fraction, megakaryocyte potentiating factor (MPF), also denominated ‘N-ERC/mesothelin’.7 Although mesothelin is bound to the cell membrane, a circulating form termed ‘soluble mesothelin’ has been reported to be related to abnormal splicing events leading to synthesis of a secreted protein and to an enzymatic cleavage from membrane-bound mesothelin.8
It has been well documented that soluble mesothelin-related peptides (SMRPs), including both soluble mesothelin and MPF, have been found in human serum and PF.9 ,10 The diagnostic accuracy of SMRP detections for MPM has been extensively studied, but the exact role of these detections needs to be elucidated. In 2010 we performed and published a first meta-analysis reporting the overall diagnostic accuracy of serum SMRPs for diagnosing MPM, and our results showed that serum SMRP determinations could play a role in the diagnosis of MPM.11 More recently, Hollevoet et al12 performed an individual patient data meta-analysis to evaluate serum SMRP levels for diagnosing MPM, and found that a positive test result at a high speciﬁcity threshold is a strong incentive to urge further diagnostic steps; however, the poor sensitivity of SMRPs limits its added value to early diagnosis of MPM. Since that time, many additional clinical studies determining the concentrations of SMRPs in serum and PF have been reported. We therefore performed the present meta-analysis to update the overall diagnostic accuracy of serum SMRPs and, furthermore, to establish the accuracy of PF SMRPs for diagnosing MPM.
Search strategy and study selection
MEDLINE (PubMed database) and EMBASE were searched for suitable studies up to 28 November 2013; no early date limit was applied. Search keywords included ‘soluble mesothelin-related peptides/SMRP’, ‘mesothelin’, ‘megakaryocyte potentiating factor/MPF’ and ‘mesothelioma’. Articles were also identified by use of the related articles function in PubMed. References of articles identified were further searched manually. Although no language restrictions were imposed initially, for the full-text review and final analysis our resources only permitted review of English articles. Conference abstracts and letters to journal editors were excluded because of the limited data presented in them.
A study was included in the meta-analysis when it provided SMRP values in serum and/or PF for both sensitivity and specificity of the diagnosis of MPM. Studies including at least 10 specimens were selected to be included in the meta-analysis, since very small studies may be vulnerable to selection bias. Publications with evidence of possible overlap of patients with other studies were discussed by AC, X-GJ and KZ and only the best quality study was used. Two reviewers (Z-HT and H-ZS) independently judged study eligibility while screening the citations. Disagreements were resolved by consensus.
Data extraction and quality assessment
The final set of English articles was assessed independently by two reviewers (AC and X-GJ). Data retrieved from the reports included author, publication year, study characteristics, participant characteristics, diagnostic methods, sensitivity and specificity data, cut-off value and methodological quality. All eligible studies were assessed for methodological quality using guidelines published by the Standards for Reporting Diagnostic Accuracy (STARD, maximum score 25) initiative13 (ie, guidelines that aim to improve the quality of reporting in diagnostic studies) and the Quality Assessment for Studies of Diagnostic Accuracy (QUADAS, maximum score 14) tool14 (ie, appraisal by use of empirical evidence, expert opinion and formal consensus to assess the quality of primary studies of diagnostic accuracy).
Standard methods recommended for meta-analyses of diagnostic test evaluations were used.15 Analyses were performed using two statistical software programs (Stata V.9; Stata Corporation; College Station, Texas, USA; and Meta-DiSc for Windows; XI Cochrane Colloquium; Barcelona, Spain). We computed the following measures of test accuracy for each study: sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR) and diagnostic OR (DOR).
The analyses were based on summary receiver operating characteristic (SROC) curves.15 ,16 The sensitivity and specificity for the single test threshold identified for each study were used to plot an SROC curve.16 ,17 A random effects model was used to calculate the average sensitivity, specificity and other measures across studies18 ,19 and χ2 and Fisher exact tests were used to detect statistically significant heterogeneity. Since publication bias is of concern for meta-analyses of diagnostic studies, we tested for the potential presence of this bias using funnel plots and the Egger test.20
After independent review, 62 publications determining concentrations of human SMRPs in serum and/or PF were considered to be eligible for inclusion in the meta-analysis. Of these publications, 32 were excluded (see online supplementary appendix 1), leaving 30 publications available for analysis of diagnostic accuracy of SMRPs. 21–50 Eleven publications from 12 studies21–31 were included in our previous meta-analysis11 and an additional 19 publications from 28 studies32–50 were added in the current meta-analysis.
Multiple ELISA kits were available for determining SMRP concentrations. Mesomark, which has been approved by the US Food and Drug Administration, was used to determine mesothelin in most studies and other mesothelin ELISA kits were used in the other four studies.21 ,30 ,31 ,34 Serum mesothelin concentrations were determined in 23 studies (22 articles)21 ,23–31 ,33 ,35 ,37–40 ,42 ,44 ,46 ,47 ,49 ,50 and serum MPF concentrations were determined in five studies22 ,30 ,32 ,37 ,46 (table 1). In the study by Scherpereel et al,23 the authors compared serum SMRP concentrations in patients with MPM with those in patients exposed to asbestos with benign pleural lesions and in patients with pleural metastasis of carcinomas separately using two different cut-off values; we therefore treated these research data as two independent studies. SMRP concentrations in PF were determined in 11 articles from 12 studies (mesothelin in 11 and MPF in 1) (table 2).23 ,34 ,36 ,41–47 ,49
On review, the studies showed large differences in the number of participants, clinical characteristics (especially histological subtypes of MPM and type of control groups) and reported diagnostic cut-off values of SMRPs (see online supplementary appendix 2). For serum SMRP studies, the average sample size was 265 (range 40–1086) and the subjects included 1562 patients with MPM and 5988 non-MPM. For PF SMRP studies, the average sample size was 126 (range 40–275) and the subjects included 460 patients with MPM and 1046 non-MPM.
In 21 publications the diagnosis of MPM was completely based on pathological findings in pleural biopsies, with or without positive cytological results while, in the remaining nine publications, some patients with MPM were diagnosed based only on the cytological ﬁndings. The quality of the study design and reporting of diagnostic accuracy of most studies were generally good since 26 of 30 publications had higher STARD scores (≥13) and 21 studies had higher QUADAS scores (≥10).
The funnel plots for publication bias showed asymmetry for serum SMRP studies (figure 1A) and evaluation of publication bias showed that Egger tests were significant for serum SMRPs (p=0.038). The funnel plots for publication bias also showed asymmetry for PF SMRP studies (figure 1B) and Egger tests showed that this was signiﬁcant for PF SMRPs (p=0.035). These results indicated a potential for publication bias for both serum and PF SMRP studies.
Figure 2A shows a forest plot of sensitivity and specificity for 28 serum SMRP studies in the diagnosis of MPM. The sensitivity ranged from 0.33 to 0.95 (pooled 0.61, 95% CI 0.58 to 0.63) while specificity ranged from 0.60 to 1.00 (pooled 0.87, 95% CI 0.86 to 0.88). It was also noted that PLR was 5.71 (95% CI 4.28 to 7.62), NLR was 0.43 (95% CI 0.38 to 0.50) and DOR was 14.43 (95% CI 9.98 to 20.87). χ2 values of sensitivity, specificity, PLR, NLR and DOR were 153.68, 460.32, 272.50, 143.64 and 142.07, respectively (all p<0.001), indicating a significant heterogeneity between studies.
Figure 2B shows a forest plot of sensitivity and specificity for 12 PF SMRP studies in the diagnosis of MPM. The sensitivity ranged from 0.70 to 1.00 (pooled 0.79, 95% CI 0.75 to 0.83) while specificity ranged from 0.65 to 0.95 (pooled 0.85, 95% CI 0.83 to 0.87). We also noted that PLR was 4.78 (95% CI 3.52 to 6.50), NLR was 0.30 (95% CI 0.24 to 0.36) and DOR was 19.50 (95% CI 12.14 to 31.33). χ2 values of sensitivity, specificity, PLR, NLR and DOR were 41.33 (p<0.001), 46.78 (p<0.001), 38.64 (p<0.001), 14.53 (p=0.205) and 23.49 (p=0.015), respectively, indicating some heterogeneity between studies.
The graphs of SROC curves for SMRP determinations showing sensitivity versus 1 − specificity from individual studies are shown in figure 3. The SROC curve for serum SMRPs was not positioned near the desirable upper left corner of the curve and the maximum joint sensitivity and specificity was 0.741 (SEM 0.029; figure 3A) while the area under the curve (AUC) was 0.806 (SEM 0.032). The maximum joint sensitivity and specificity of PF SMRP was 0.820 (SEM 0.022) while the AUC was 0.890 (SEM 0.021; figure 3B).
Thus, in total, the diagnostic performance of SMRPs in serum and PF was similar.
We first analysed the diagnostic values of serum mesothelin and MPF separately and the results are presented in table 3. Based on the comparisons of sensitivity, specificity, PLR, NLR, DOR and AUC, the diagnostic performance of serum MPF was superior to that of serum mesothelin.
Data from six studies21 ,22 ,24 ,28 ,30 ,31 were available for comparing the diagnostic accuracy of serum SMRPs in differentiating MPM from healthy control subjects, nine studies21 ,23 ,24 ,26 ,30 ,31 ,33 ,45 ,47 were available for differentiating MPM from other malignancies and eight studies21 ,23–25 ,28 ,30 ,33 ,40 were available for differentiating MPM from asbestos-exposed subjects. As shown in table 4, the values of sensitivity, specificity, PLR, NLR, DOR and AUC of SMRPs for discriminating between patients with MPM and healthy control subjects were quite acceptable, and were better than those for discriminating between patients with MPM and those with other cancers or asbestos-exposed people. By using serum SMRPs, it was more difficult to differentiate MPM from other cancers than from healthy controls or asbestos-exposed people.
Six studies23 ,34 ,36 ,41 ,42 ,48 provided the required data for comparing the diagnostic accuracy of PF mesothelin in differentiating MPM from other cancers and four studies36 ,41 ,42 ,48 provided data for differentiating MPM from benign pleural effusions (table 5). In total, the diagnostic accuracy of PF SMRP in differentiating MPM from other cancers was very similar to that of differentiating between MPM and benign pleural effusions.
The diagnosis of MPM is always challenging because (1) MPM may appear in patients up to 30–40 years after exposure to asbestos; (2) the clinical and imaging signs of MPM are non-specific; and (3) a definitive diagnosis, which relies on histology, can sometimes be very difficult to achieve, even with the use of immunohistochemistry.5 To date, no single marker or panel of soluble biomarkers is available for a clear diagnosis of MPM.51 ,52
In the present meta-analysis our results indicated that the pooled sensitivity of serum and PF SMRPs was 0.61 and 0.79, respectively, and their specificity was 0.87 and 0.85, respectively. These data indicated that the sensitivity and specificity of SMRPs in serum and PF were not as high as expected. SMRPs might be helpful in confirming (ruling in) MPM if the results are higher than the cut-off values. Thus, positive SMRP test results suggested that invasive diagnostic steps such as medical thoracoscopy might be necessary. On the other hand, the low sensitivity will not allow exclusion of non-MM patients even if the patients have mesothelin concentrations lower than the cut-off value. The associated poor sensitivity of SMRPs therefore clearly limits their added value to the diagnosis of MPM.
As previously described,11 SROC curves present a global summary of test performance and show the trade-off between sensitivity and specificity while DOR is a single indicator of test accuracy that combines the data from sensitivity and specificity into a single number. The results of our analyses based on SROC curves showed that the maximum joint sensitivity and specificity of serum and PF SMRPs were 0.741 and 0.820, respectively, while their AUCs were 0.806 and 0.890, respectively, indicating that the level of overall accuracy was not as high as expected. We also found that the pooled DORs of serum and PF SMRPs were 14.43, and 19.50, respectively, indicating that SMRPs seemed to be helpful in the diagnosis of MPM but they were not perfect.
Since SROC curves and DOR are not easy to interpret and use in clinical practice, and since PLR and NLR are considered more clinically meaningful,53 ,54 we further presented both PLR and NLR as our measures of diagnostic accuracy. If a value is >10 or <0.1, PLR or NLR generates large and often conclusive shifts from pre-test to post-test probability (indicating high accuracy).55 A PLR value of 5.71 with serum SMRPs suggests that patients with MPM have a nearly sixfold higher chance of being SMRP-positive compared with patients without MPM, and this was not high enough for the clinical purpose. On the other hand, the NLR value of serum SMRPs was found to be 0.43. If serum SMRP results were negative, the probability that the patient has MPM is 43%, which is not low enough to rule out MPM. Very similar results were found with PF SMRPs.
Although both mesothelin and MPF belong to mesothelin family proteins, we noted in the current meta-analysis that the overall diagnostic measures including sensitivity, specificity, PLR, NLR, DOR and AUC of serum MPF were better than those of serum mesothelin. MPF therefore had a superior diagnostic accuracy than mesothelin for MPM. We also noted that the diagnostic performance of serum SMRP for discriminating MPM from healthy control subjects was the best (although not as good as expected), followed by that for discriminating MPM from patients with other cancers or from asbestos-exposed people. In addition, the overall diagnostic accuracy of PF SMRP for differentiating MPM from other cancers was similar to that of differentiating MPM from benign pleural effusions.
Our meta-analysis had several limitations. First, exclusion of conference abstracts, letters to the editors and non-English language studies may have led to publication bias. Indeed, we observed a publication bias for both serum and PF SMRP studies. Publication bias may also be introduced by inflation of diagnostic accuracy estimates since studies that report positive results are more likely to be accepted for publication. Second, pathological types of MPM were not specified in three studies,35 ,42 ,50 and the epithelioid subtype of MPM was the most common pathological type in all studies, excluding the one reported by Creaney et al.25 In total, 69.9% (982/1404) of MPM were epithelioid subtype (range 29.9–100%). Analysis in terms of histological type has shown that SMRP levels are significantly higher in epithelioid subtype MPM than in other types.21 ,23 ,29 This could partly explain the rather low sensitivity of SMRPs in MPM diagnosis. Third, control populations were very heterogeneous from one study to another and various cut-off points were used for distinguishing between MPM and controls, other cancers or benign respiratory diseases, according to the best combination of sensitivity and specificity. These issues regarding accuracy of diagnosis could also lead to biased results.
It should be mentioned that, since our previous meta-analysis was published,11 the use of SMRPs in clinical practice has moved forward significantly.10 It has been recognised that SMRPs are diagnostic markers and also serve as markers of the disease course and response to treatment.56 ,57 The application of SMRPs in clinical practice in the near future may therefore be in monitoring the response to treatment rather than in guiding diagnostic decisions and risk assessment of asbestos-exposed populations.
In conclusion, the current evidence supports the view that SMRPs in both serum and PF are helpful markers for diagnosing MPM. The overall diagnostic performance of SMRPs in serum and PF was similar, and serum MPF had superior diagnostic accuracy compared with serum mesothelin. The negative results of SMRP determinations were not sufficient to exclude non-MPM whereas the positive test results might be helpful in confirming MPM.
Review history and Supplementary material
AC and X-GJ contributed equally.
Contributors H-ZS was responsible for the study conception and protocol design and wrote the first draft of the paper. AC and X-GJ were responsible for the overall study coordination under the supervision of H-ZS with contributions from KZ and Z-HT. All literature searching, abstract screening, study selection and data extraction was undertaken independently by AC and X-GJ with referral to KZ as a third reviewer as necessary. Assessment of methodological quality was also undertaken by Z-HT and H-ZS. All authors have read and approved the final version of the manuscript.
Funding This work was supported in part by a grant from the National Natural Science Foundation of China (No. 81270149) and in part by a grant from the 12th Five-Year National Science and Technology Program of Social Development, Ministry of Science and Technology, China (No. 2012BAI05B02).
Competing interests None.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.