The reliability and validity of the Seasonal Pattern Assessment Questionnaire: a comparison between patient groups

Abstract

Background: The Seasonal Pattern Assessment Questionnaire (SPAQ) is a frequently used screening instrument in the research on Seasonal Affective Disorder (SAD). Nevertheless, studies on its reliability and validity are relatively scarce. In the present study the reliability and the contrast validity of the SPAQ are investigated. Methods: SAD patients, selected by means of a clinical interview, non-seasonal depressed out-patients, non-depressed out-patients, and a control group, are contrasted to estimate the discriminating power of the SPAQ. Also, the reliability and factor structure of the seasonality and the climate subscales are investigated. To study food intake the Seasonal Food Preference Questionnaire (SFPQ) was developed. Results: The SAD criterion of the SPAQ shows good specificity (94%), but a low sensitivity (44%). Discriminant analysis shows sufficient ability to classify subjects (81% correctly classified). The Global Seasonality Scale has a good internal consistency. It consists of two factors, a psychological factor and a food factor. The SFPQ is sensitive for carbohydrate intake by SAD patients. Limitations: Most SAD patients had received treatment and completed the SPAQ while they were not depressed, which may have influenced the sensitivity. Conclusions: The SPAQ is not sensitive enough to be considered a diagnostic instrument for SAD. Nevertheless, it is accurate enough to be used as a screenings instrument. The only false positives were found in the depressive group. The accuracy of prevalence Figs. can be improved by completion of the SPAQ in the summer months, combined with the completion of a depression scale.

Introduction

It is remarkable that one instrument, i.e., the Seasonal Pattern Assessment Questionnaire (SPAQ; Rosenthal et al., 1987) has such a prominent place in the study of Seasonal Affective Disorder (SAD), the more so since data on the psychometric qualities of the SPAQ are relatively scarce. The SPAQ has two objectives. The original purpose of the SPAQ was to study a number of characteristics of SAD like seasonal variation of mood, food intake and weight gain, seasonal sleep duration, and sensitivity to weather conditions. Although not developed as such, the SPAQ was introduced as a diagnostic or screening instrument for SAD by Kasper et al. (1989a), who were the first to formulate the SPAQ criteria for SAD. Since 1989 the SPAQ has been widely used as a screening instrument. In fact, all over the world the prevalence figures of SAD are based on the ‘Kasper criteria’ of the SPAQ (Mersch et al., 1999a).

Nevertheless, in only a few prevalence studies responders who met the criteria of the SPAQ for SAD were interviewed. Kasper et al. (1989a) concluded that the SPAQ underestimated the number of subjects that met the clinical criteria of SAD as established in an interview (50% false negatives, 0% false positives), while Magnusson (1996), evaluating the results of the Magnusson and Stefansson (1993) study, found that the clinical interview and the SPAQ reached approximately the same number of SAD subjects. The SPAQ in the latter study reached a sensitivity of 94% and a specificity of 73%. Michalak et al. (2001) interviewed 66 SAD cases detected by the SPAQ in a prevalence study on 1999 residents of the United Kingdom. No less than 55% of the SAD cases were false positives. No false negatives were detected in 23 interviewed non-SAD cases. In a study by Raheja et al. (1996) the SPAQ only missed three out of 47 SAD patients showing a very good sensitivity (94%). Since only SAD cases were studied, the specificity could not be assessed. At a follow-up assessment 5–8 years later the sensitivity was 74%, while the specificity was 46%.

There is accumulating evidence that the ‘Kasper criteria’ of the SPAQ lead to overestimation of the percentage of SAD cases in the general population. In the Michalak et al. (2001) study the prevalence of SAD was estimated by the SPAQ as 5.3% while a DSM-IV interview estimated the prevalence at 2.4%. In a population survey in the USA Blazer et al. (1998) studied SAD in a sample of 8098 subjects by means of a structured interview. Only 0.4% of the respondents met the criteria for SAD. This figure is considerably lower than the mean prevalence figure of SAD in North America (6.2%), based on the SPAQ criteria (Mersch et al., 1999a). Two studies investigated the prevalence in Canada, both using a clinical interview and the SPAQ in a telephone survey. Results of the Levitt et al. (2000) study on a sample of 781 respondents showed that the SPAQ detected almost twice as many SAD subjects (5.0 versus 2.9%) as a clinical interview based on the DSM-IV (American Psychiatric Association, 1994). In a second study on 1605 respondents designed to test the latitude hypothesis of SAD (Levitt and Boyle, 2002), the SPAQ overestimated the mean prevalence three to four times, compared to a ‘golden-standard’ clinical interview (7.4 versus 1.9%). So, validity studies that have been performed show mixed results and do not unequivocally support the confidence in the SPAQ’s validity suggested by its widespread use (Mersch, 2001).

Another way to establish the validity is to study the ability of the SPAQ to discriminate between different groups (e.g., SAD and sub-SAD subjects, different patient groups and normal control groups), composed in a way other than by means of SPAQ criteria (contrast validity). Regretfully, the few studies that have compared different patient groups with the SPAQ (Kasper et al., 1989b, Thompson et al., 1988, Hardin et al., 1991) fail to provide data on the sensitivity and specificity of the SAD criteria of the SPAQ.

In the present study the ratings on the SPAQ of SAD patients were compared with non-seasonal depressed outpatients, non-depressed outpatients and a control group of healthy volunteers. The main objective was to establish the validity of the SAD criteria of the SPAQ. A secondary goal was to establish the reliability and factor structure of the seasonality scale (a replication of the Magnusson et al., 1997 study) and the climate scale. Also, the differences between the groups with respect to atypical SAD symptoms will be studied.