The definition of case is a core issue in psychiatric epidemiology. Psychiatric symptom screening scales have been extensively used in population studies for many decades. Structured diagnostic interviews have become available during recent years to give exact diagnoses through carefully undertaken procedures. The aim of this article was to assess how well the Hopkins Symptom Checklist-25 (HSCL-25) predicted cases by the Composite International Diagnostic Interview (CIDI), and find the optimal cut-offs on the HSCL-25 for each diagnosis and gender. Characteristics of concordant and discordant cases were explored. In a Norwegian two-stage survey mental health problems were measured by the HSCL-25 and the CIDI. Only 46% of the present CIDI diagnoses were predicted by the HSCL-25. Comorbidity between CIDI diagnoses was found more than four times as often in the concordant cases (case agreed upon by both instruments) than in the discordant CIDI cases. Concordant cases had more depression and panic/generalized anxiety disorders. Neither the anxiety nor the depression subscales improved the prediction of anxiety or depression. The receiver operating characteristic (ROC) curves confirmed that the HSCL-25 gave best information about depression. Except for phobia it predicted best for men. Optimal HSCL-25 cut-off was 1.67 for men and 1.75 for women. Of the discordant HSCL-25 cases, one-third reported no symptoms in the CIDI, one-third reported symptoms in the CIDI anxiety module, and the rest had symptoms spread across the modules. With the exception of depression, the HSCL-25 was insufficient to select individuals for further investigation of diagnosis. The two instruments to a large extent identified different cases. Either the HSCL-25 is a very imperfect indicator of the chosen CIDI diagnoses, or the dimensions of mental illness measured by each of the instruments are different and clearly only partly overlapping.