Publications on diagnostic test evaluation in family medicine journals

Abstract

Search strategies for articles reporting on diagnostic test evaluations have been subjected to less research than those in the domain of clinical trials. We set out to develop an optimal search strategy for publications on diagnostic test evaluations in general, that could be added to keywords describing the specific diagnostic test at issue. Nine Family Medicine journals were searched from 1992 through 1995 for primary publications on diagnostic test evaluation by hand searching and a Medline search strategy published earlier. Additionally, new search strategies have been developed with stepwise logistic regression, using Mesh terms and free text words related to diagnosis and test evaluation as independent variables. Hand searching identified 75 primary publications on diagnostic test evaluation from a total of 2467 primary publications. The previously published search strategy had a sensitivity of 73%, a specificity of 94%, and a positive predictive value of 29%. The most accurate new search strategy had a sensitivity of 80.0% (60/75; 95% CI: 71.0–89.1), a specificity of 97.3% (2327/2392; 95% CI; 96.6–97.9%), a positive predictive value of 48% (95% CI: 40–56) and diagnostic odds ratio of 149. All four new strategies used the Mesh term “sensitivity and specificity” (exploded with the Mesh terms “predictive value” and “ROC”)and cumulatively added the text words “specificity,” “false negative,” “accuracy,” and “screening.” The search strategy using the Mesh term “sensitivity and specificity” (exploded) and the text words “specificity,” “false negative,” and “accuracy” has both higher sensitivity and specificity than the previously published strategy. The increase in specificity in three strategies reduces the absolute number of false-positive articles that have to be screened by 50–75%, compared to the number of false positives in the earlier strategy.

Introduction

With an increasing amount of scientific medical literature being published each year, it is almost impossible to keep abreast of the actual status of knowledge in any specific field [1]. Summarizing the evidence by systematically reviewing the available literature and pooling estimates in a meta-analysis, when possible and useful, is also undergoing rapid expansion [2]. In order to summarize the relevant evidence, the articles at issue have to be identified. The initial sources used in the identification of relevant publications are often bibliographical databases, such as Medline [3]. Several authors have stressed the importance of an accurate search strategy that incorporates both high sensitivity and high specificity 4, 5 for identifying relevant publications.

Search strategies for literature on clinical trials are available 3, 6, 7, 8, but search strategies for literature on the evaluation of diagnostic tests have been less well studied 9, 10. The strategies that have been published were developed for diagnostic publications, including both publications specifically evaluating diagnostic tests and publications on clinical diagnosis not focusing on the evaluation of specific diagnostic signs, symptoms, or tests 7, 9. Other search strategies have been developed for the evaluation of a specific diagnostic test [11]. Overall, sensitive searches tend to be weak in specificity, resulting in the selection of a large number of irrelevant publications. In view of this high number of false positives, we wondered if it would be possible to develop a more specific search strategy for selecting publications on diagnostic test evaluations without loosing sensitivity, compared to the earlier diagnostic search strategies. We were not interested in any specific field, but in a standard strategy that could be used in any field of interest, as is the case for the strategies developed for identifying clinical trials [3].