Abstract
Using data collected as part of the second pilot administration of Your First College Year (YFCY), a national survey of first-year college students, this study was designed to examine both response rates and nonresponse bias across four survey administration groups: paper-only, paper with web option, web-only with response incentive, and web-only without response incentive. Findings indicate that response rates vary by mode of administration. Moreover, predictors of response differed by administration group. Results are discussed in light of the recent surge of interest in online survey research.
Similar content being viewed by others
REFERENCES
Antons, C. M., Dilla, B. L., and Fultz, M. L. (1997). Assessing student attitudes: Computer vs. pencil. Paper presented at the 37th Annual Forum of the Association for Institutional Research, Orlando, FL, May 18–21, 1997.
Astin, A. W. (1993). An empirical typology of college students. J. Coll. Stud. Dev. 34: 36–46.
Bradburn, N. M. (1992). Presidential address: A response to the non-response problem. Public Opin. Q. 56: 391–398.
Carini, R. M., Hayek, J. C., Kuh, G. D., Kennedy, J. M., and Ouimet, J. A. (2001). College student responses to web and paper surveys: Does mode matter? Paper presented at the 41st Annual Forum of the Association for Institutional Research. Long Beach, CA, June 4, 2001.
Caslyn, R. J., and Winter, J. P. (1999). Understanding and controlling response bias in need assessment studies. Eval. Rev. 23: 399–417.
Clayton, R. L., and Werking, G. S. (1998). Business surveys of the future: The World Wide Web as a data collection method. In: Couper, M. P., Baker, R. P., Bethlehem, J., Clark, Z. F., Martin, J., Nicholls II, W. L., and O'Reilly, J. M. (eds.), Computer Assisted Survey Information Collection, John Wiley & Sons, New York, pp. 543–562.
Couper, M. P. (2000). Web surveys: A review of issues and approaches. Public Opin. Q. 64: 464–494.
Creswell, J. W. (1994). Research Design: Qualitative and Quantitative Approaches, Sage Publications, Thousand Oaks.
de Leeuw, E., and Heer, W. (2002). Trends in household survey nonresponse: A longitudinal and international comparison. In: Groves, R. M., Dillman, D. A., Eltinge, J. L., and Little, R. J. A. (eds.), Survey Nonresponse, John Wiley & Sons, New York, pp. 41–54.
Dey, E. L. (1997). Working with low survey response rates: The efficacy of weighing adjustments. Res. Higher Educ. 38: 215–227.
Dillman, D. A. (1991). The design and administration of mail surveys. Annu. Rev. Sociol. 17: 225–249.
Dillman, D. A. (2000). Mail and Internet Surveys: The Tailored Design Method (2nd Ed.), John Wiley & Sons, New York.
Fraenkel, J. R., and Wallen, N. E. (1993). How to Design and Evaluate Research in Education (2nd Ed.), McGraw Hill, New York.
Gjestland, L. (1996). Net? Not yet. Mark. Res.: Mag. Manage. Appl. 8: 26–29.
Goree, C. T., and Marszalek, J. F. (1995). Electronic surveys: Ethical issues for researchers. J. Coll. Stud. Affair. 15: 75–79.
Grandy, J. (1998). Response bias in a survey of Asian-American and white science and engineering students. J. Women Minor. Sci. Eng. 4: 1–13.
Hamilton, J. C. (December 30, 1999). The ethics of conducting social-science research on the Internet. The Chronicle of Higher Education, p. B6.
Handwerk, P. G., Carson, C., and Blackwell, K. M. (2000). Online vs. paper-and-pencil surveying of students: A case study. Paper presented at the 40th Annual Forum of the Association for Institutional Research, Cincinnati, OH, May 21–24, 2000.
Hutchison, J., Tollefson, N., and Wigington, H. (1987). Response bias in college freshman's responses to mail surveys. Res. Higher Educ. 26: 99–106.
Johnson, L. C., Beaton, R., Murphy, S., and Pike, K. (2000). Sampling bias and other methodological threats to the validity of health survey research. Int. J. Stress Manage. 7: 247–267.
Kehoe, C. M., and Pitkow, J. E. (1996). Surveying the territory: GVU's five WWW user surveys. W3J 1: 77–84.
Kelley, D. L. (1999). Measurement Made Accessible, Sage Publications, Thousand Oaks.
Krosnick, J. A. (1999). Survey research. Annu. Rev. Psychol. 50: 537–567.
Matz, C. M. (1999). Administration of web versus paper surveys: Mode effects and response rates. Master's Research Paper, University of North Carolina, Chapel Hill.
Newburger, E. C. (2001). Home computers and Internet use in the United States: August 2000. U.S. Census Bureau, Washington, DC. Report retrieved from the World Wide Web at http://www.census.gov/prod/2001pubs/p23-207.pdf.
Palmquist, J., and Stueve, A. (1996). Stay plugged into new opportunities. Mark. Res.: Mag. Manage. Appl. 8: 13–15.
Sax, L. J., Ceja, M., and Teranishi, R. T. (2001). Technological preparedness among entering freshmen: The role of race, class, and gender. J. Educ. Comput. Res. 24: 1–21.
Schmidt, W. C. (1997). World Wide Web survey research: Benefits, potential problems, and solutions. Behav. Res. Methods Instrum. Comput. 29: 274–279.
Schuman, H. (2002). Sense and nonsense about surveys. Contexts 1: 40–47.
Smith, C. B. (1997). Casting the net: Surveying an Internet population. J. Comput.-Mediat. Commun. 3: 77–84.
Smith, M. A., and Leigh, B. (1997). Virtual subjects: Using the Internet as an alternative source of subjects and research environment. Behav. Res. Methods Instrum. Comput. 29: 496–505.
Smith, T. (1995). Trends in nonresponse rates. Int. J. Pub. Opin. Res. 7: 157–171.
Smith, T. W. (1983). The hidden 25 percent: An analysis of nonresponse on the 1980 General Social Survey. In: Singer, E., and Presser, S. (eds.), Survey Research Methods: A Reader, The University of Chicago Press, Chicago, pp. 50–68.
Steeh, C. (1981). Trends in nonresponse rates. Public Opin. Q. 45: 40–57.
Stinchcombe, A. L., Jones, C., and Sheatsley, P. (1981). Nonresponse bias for attitude questions. Public Opin. Q. 45: 359–375.
Tomsic, M. L., Hendel, D. D., and Matross, R. P. (2000). A World Wide Web response to student satisfaction surveys: Comparisons using paper and Internet formats. Paper presented at the 40th Annual Meeting of the Association for Institutional Research, Cincinnati, OH, May 21–24, 2000.
Underwood, D., Kim, H., and Matier, M. (2000). To mail or to Web: Comparisons of survey response rates and respondent characteristics. Paper presented at the 40th Annual Forum of the Association for Institutional Research, Cincinnati, OH, May 21–24, 2000.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Sax, L.J., Gilmartin, S.K. & Bryant, A.N. Assessing Response Rates and Nonresponse Bias in Web and Paper Surveys. Research in Higher Education 44, 409–432 (2003). https://doi.org/10.1023/A:1024232915870
Issue Date:
DOI: https://doi.org/10.1023/A:1024232915870