Skip to main content
Log in

Assessing Response Rates and Nonresponse Bias in Web and Paper Surveys

  • Published:
Research in Higher Education Aims and scope Submit manuscript

Abstract

Using data collected as part of the second pilot administration of Your First College Year (YFCY), a national survey of first-year college students, this study was designed to examine both response rates and nonresponse bias across four survey administration groups: paper-only, paper with web option, web-only with response incentive, and web-only without response incentive. Findings indicate that response rates vary by mode of administration. Moreover, predictors of response differed by administration group. Results are discussed in light of the recent surge of interest in online survey research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

REFERENCES

  • Antons, C. M., Dilla, B. L., and Fultz, M. L. (1997). Assessing student attitudes: Computer vs. pencil. Paper presented at the 37th Annual Forum of the Association for Institutional Research, Orlando, FL, May 18–21, 1997.

  • Astin, A. W. (1993). An empirical typology of college students. J. Coll. Stud. Dev. 34: 36–46.

    Google Scholar 

  • Bradburn, N. M. (1992). Presidential address: A response to the non-response problem. Public Opin. Q. 56: 391–398.

    Google Scholar 

  • Carini, R. M., Hayek, J. C., Kuh, G. D., Kennedy, J. M., and Ouimet, J. A. (2001). College student responses to web and paper surveys: Does mode matter? Paper presented at the 41st Annual Forum of the Association for Institutional Research. Long Beach, CA, June 4, 2001.

  • Caslyn, R. J., and Winter, J. P. (1999). Understanding and controlling response bias in need assessment studies. Eval. Rev. 23: 399–417.

    Google Scholar 

  • Clayton, R. L., and Werking, G. S. (1998). Business surveys of the future: The World Wide Web as a data collection method. In: Couper, M. P., Baker, R. P., Bethlehem, J., Clark, Z. F., Martin, J., Nicholls II, W. L., and O'Reilly, J. M. (eds.), Computer Assisted Survey Information Collection, John Wiley & Sons, New York, pp. 543–562.

    Google Scholar 

  • Couper, M. P. (2000). Web surveys: A review of issues and approaches. Public Opin. Q. 64: 464–494.

    Google Scholar 

  • Creswell, J. W. (1994). Research Design: Qualitative and Quantitative Approaches, Sage Publications, Thousand Oaks.

    Google Scholar 

  • de Leeuw, E., and Heer, W. (2002). Trends in household survey nonresponse: A longitudinal and international comparison. In: Groves, R. M., Dillman, D. A., Eltinge, J. L., and Little, R. J. A. (eds.), Survey Nonresponse, John Wiley & Sons, New York, pp. 41–54.

    Google Scholar 

  • Dey, E. L. (1997). Working with low survey response rates: The efficacy of weighing adjustments. Res. Higher Educ. 38: 215–227.

    Google Scholar 

  • Dillman, D. A. (1991). The design and administration of mail surveys. Annu. Rev. Sociol. 17: 225–249.

    Google Scholar 

  • Dillman, D. A. (2000). Mail and Internet Surveys: The Tailored Design Method (2nd Ed.), John Wiley & Sons, New York.

    Google Scholar 

  • Fraenkel, J. R., and Wallen, N. E. (1993). How to Design and Evaluate Research in Education (2nd Ed.), McGraw Hill, New York.

    Google Scholar 

  • Gjestland, L. (1996). Net? Not yet. Mark. Res.: Mag. Manage. Appl. 8: 26–29.

    Google Scholar 

  • Goree, C. T., and Marszalek, J. F. (1995). Electronic surveys: Ethical issues for researchers. J. Coll. Stud. Affair. 15: 75–79.

    Google Scholar 

  • Grandy, J. (1998). Response bias in a survey of Asian-American and white science and engineering students. J. Women Minor. Sci. Eng. 4: 1–13.

    Google Scholar 

  • Hamilton, J. C. (December 30, 1999). The ethics of conducting social-science research on the Internet. The Chronicle of Higher Education, p. B6.

  • Handwerk, P. G., Carson, C., and Blackwell, K. M. (2000). Online vs. paper-and-pencil surveying of students: A case study. Paper presented at the 40th Annual Forum of the Association for Institutional Research, Cincinnati, OH, May 21–24, 2000.

  • Hutchison, J., Tollefson, N., and Wigington, H. (1987). Response bias in college freshman's responses to mail surveys. Res. Higher Educ. 26: 99–106.

    Google Scholar 

  • Johnson, L. C., Beaton, R., Murphy, S., and Pike, K. (2000). Sampling bias and other methodological threats to the validity of health survey research. Int. J. Stress Manage. 7: 247–267.

    Google Scholar 

  • Kehoe, C. M., and Pitkow, J. E. (1996). Surveying the territory: GVU's five WWW user surveys. W3J 1: 77–84.

    Google Scholar 

  • Kelley, D. L. (1999). Measurement Made Accessible, Sage Publications, Thousand Oaks.

    Google Scholar 

  • Krosnick, J. A. (1999). Survey research. Annu. Rev. Psychol. 50: 537–567.

    Google Scholar 

  • Matz, C. M. (1999). Administration of web versus paper surveys: Mode effects and response rates. Master's Research Paper, University of North Carolina, Chapel Hill.

    Google Scholar 

  • Newburger, E. C. (2001). Home computers and Internet use in the United States: August 2000. U.S. Census Bureau, Washington, DC. Report retrieved from the World Wide Web at http://www.census.gov/prod/2001pubs/p23-207.pdf.

    Google Scholar 

  • Palmquist, J., and Stueve, A. (1996). Stay plugged into new opportunities. Mark. Res.: Mag. Manage. Appl. 8: 13–15.

    Google Scholar 

  • Sax, L. J., Ceja, M., and Teranishi, R. T. (2001). Technological preparedness among entering freshmen: The role of race, class, and gender. J. Educ. Comput. Res. 24: 1–21.

    Google Scholar 

  • Schmidt, W. C. (1997). World Wide Web survey research: Benefits, potential problems, and solutions. Behav. Res. Methods Instrum. Comput. 29: 274–279.

    Google Scholar 

  • Schuman, H. (2002). Sense and nonsense about surveys. Contexts 1: 40–47.

    Google Scholar 

  • Smith, C. B. (1997). Casting the net: Surveying an Internet population. J. Comput.-Mediat. Commun. 3: 77–84.

    Google Scholar 

  • Smith, M. A., and Leigh, B. (1997). Virtual subjects: Using the Internet as an alternative source of subjects and research environment. Behav. Res. Methods Instrum. Comput. 29: 496–505.

    Google Scholar 

  • Smith, T. (1995). Trends in nonresponse rates. Int. J. Pub. Opin. Res. 7: 157–171.

    Google Scholar 

  • Smith, T. W. (1983). The hidden 25 percent: An analysis of nonresponse on the 1980 General Social Survey. In: Singer, E., and Presser, S. (eds.), Survey Research Methods: A Reader, The University of Chicago Press, Chicago, pp. 50–68.

    Google Scholar 

  • Steeh, C. (1981). Trends in nonresponse rates. Public Opin. Q. 45: 40–57.

    Google Scholar 

  • Stinchcombe, A. L., Jones, C., and Sheatsley, P. (1981). Nonresponse bias for attitude questions. Public Opin. Q. 45: 359–375.

    Google Scholar 

  • Tomsic, M. L., Hendel, D. D., and Matross, R. P. (2000). A World Wide Web response to student satisfaction surveys: Comparisons using paper and Internet formats. Paper presented at the 40th Annual Meeting of the Association for Institutional Research, Cincinnati, OH, May 21–24, 2000.

  • Underwood, D., Kim, H., and Matier, M. (2000). To mail or to Web: Comparisons of survey response rates and respondent characteristics. Paper presented at the 40th Annual Forum of the Association for Institutional Research, Cincinnati, OH, May 21–24, 2000.

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Linda J. Sax.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Sax, L.J., Gilmartin, S.K. & Bryant, A.N. Assessing Response Rates and Nonresponse Bias in Web and Paper Surveys. Research in Higher Education 44, 409–432 (2003). https://doi.org/10.1023/A:1024232915870

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1024232915870

Navigation