Objective: This study evaluated search strategies for finding high-quality studies on treatment and systematic reviews in PsycINFO.
Study design and setting: Sixty-four journals were hand searched at McMaster University. Methodologic criteria were applied to clinically relevant articles to identify "pass" and "fail" articles; 4,985 candidate terms were compiled, 7,463 combinations for therapy articles, and 5,246 combinations for reviews. Candidate search strategy results were compared with hand searches. The proposed strategies served as "diagnostic tests" for sound studies; the hand searches were the "gold standard." Sensitivity, specificity, precision, and accuracy were calculated.
Results: Of 716 treatment articles, 233 (32.5%) met criteria for scientific merit, and 58 (11.5%) of 506 review articles met criteria for systematic reviews. For treatment studies, combined terms had a peak sensitivity of 97.9% (specificity 52.2%). Maximum specificity was 97.7% (sensitivity 51.5%). Sensitivity and specificity were each 79% when optimizing both while minimizing their difference. For review articles, combined terms had a peak sensitivity of 81.0% (specificity 54.4%). Maximum specificity was 98.1% (sensitivity 51.7%). Sensitivity and specificity were each 65% when optimizing both while minimizing their difference.
Conclusions: Empirically derived search strategies can achieve high sensitivity and specificity for retrieving sound treatment studies and review articles from PsycINFO.