Original article
Reliability of the 2005 Middle School Youth Risk Behavior Survey

https://doi.org/10.1016/j.jadohealth.2006.07.008Get rights and content

Abstract

Purpose

To evaluate the reliability of the middle school version of the 2005 Youth Risk Behavior Survey (MSYRBS) questionnaire.

Methods

A convenience sample of 232 Midwestern seventh and eighth grade middle school students completed the MSYRBS questionnaire twice in a 2-week period (14 days apart). The MSYRBS questionnaire, which queries a variety of health risk behaviors, was administered in a manner that preserved anonymity but allowed Time 1 and Time 2 matching. This was accomplished by using two questionnaire scantrons coded with the same unique number, and destroying all used materials to ensure that each participant was matched with their code. Kappa statistics were calculated for individual questions and group characteristics using SAS.

Results

The mean kappa was 62.6% and the median was 66.5%. Kappa statistics for each item ranged from −2.4% (injection drug use) to 83.8% (suicide contemplation). Negative kappa values were found for two items that had extremely small cell sizes. Kappas did not differ by gender, grade, or race. Based on nonoverlapping confidence intervals, there were no items that had significantly different prevalence estimates at Time 1 vs. Time 2. Nine items (24.3%) and one category (alcohol-drugs) had kappas below 61.0%.

Conclusions

This preliminary study suggests that the reliability of the MSYRBS is high over time. A number of items should be further examined to determine whether they should be amended or omitted from future versions of the MSYRBS. Further research with larger and more diverse samples is recommended, potentially involving children as content experts.

Section snippets

Sample

During the spring of 2005, a convenience sample of 232 middle school students in grades seven and eight were selected from two public school districts in this Midwestern state. Schools were selected to participate in the study based on previous work with the authors in curriculum deliberation and program evaluation. Each school was given $250 to assist with the recruitment of students for this study. Of the 402 students selected to participate in the study, 249 (62%) completed the questionnaire

Results

Data were analyzed similarly to Brener et al [5] in the examination of the reliability of the 1999 YRBS questionnaire on high school students. The mean kappa for all items was 62.6%, and the median kappa was 66.5%. Kappas ranged from −2.4% to 83.8% (Table 2). Qualitative values were assigned based on groupings described by Landis and Koch [7]. Results suggest that 75.7% of items can be described as having at least “substantial” reliability (kappa ≥ 61%), and 89.2% of items had at least

Discussion

Approximately 90% of the MSYRBS items had at least “moderate” reliability and nearly 76% of the items had “substantial” reliability in this preliminary study. In addition, no items were determined to display significantly different Time 1 vs. Time 2 prevalence estimates. However, items pertaining to cocaine use, steroid use, and injection drug use displayed poor reliability. A likely explanation for the negative kappa values for the steroid and injection drug items is the inconsistency due to

Acknowledgment

This research was funded by a grant from the Committee for Faculty Research at Miami University, Oxford, Ohio.

References (16)

  • N.D. Brener et al.

    Reliability of the Youth Risk Behavior Survey questionnaire

    J Adolesc Health

    (2002)
  • H.R. Rockett et al.

    Validation of a youth/adolescent food frequency questionnaire

    Prev Med

    (1997)
  • L.J. Kolbe

    An epidemiological surveillance system to monitor the prevalence of youth behaviors that most affect health

    J Health Educ

    (1990)
  • L.J. Kolbe et al.

    Overview of the Youth Risk Behavior Surveillance System

    Public Health Rep

    (1993)
  • M.P. Sussman et al.

    The Youth Risk Behavior Surveillance System: updating policy and program applications

    J Sch Health

    (2002)
  • N.D. Brener et al.

    Reliability of the Youth Risk Behavior Survey questionnaire

    Am J Epidemiol

    (1995)
  • J.D. Klein et al.

    Improving adolescent health care surveillance

  • J.R. Landis et al.

    The measurement of observer agreement for categorical data

    Biometrics

    (1977)
There are more references available in the full text version of this article.

Cited by (71)

  • Effort–reward Imbalance at Work, Parental Support, and Suicidal Ideation in Adolescents: A Cross-sectional Study from Chinese Dual-earner Families

    2017, Safety and Health at Work
    Citation Excerpt :

    Adolescents indicating “about every month” or more frequently were considered to have suicidal ideation. This latter decision was based on information available from previously conducted studies, in particular the USA Youth Risk Behavior Survey [49], the European School Survey Project on alcohol and other drugs [50], and the Global School-based Student Health Survey [51]. In addition, information on fathers and mothers (age, employment, education, and self-rated health), and adolescents (age, sex, grade, smoking, alcohol drinking, physical activity, family wealth, and self-rated health) was collected (Table 1).

  • Development of Alcohol and Drug Use in Youth With Manic Symptoms

    2017, Journal of the American Academy of Child and Adolescent Psychiatry
View all citing articles on Scopus
View full text