Skip to main content
SpringerLink
Log in

Non-response in student surveys: The Role of Demographics, Engagement and Personality

  • Published:
Research in Higher Education Aims and scope Submit manuscript

Abstract

What causes a student to participate in a survey? This paper looks at participation across multiple surveys to understand survey non-response; by using multiple surveys we minimize the impact of survey salience. Students at a selective liberal arts college were administered four different surveys throughout the 2002–2003 academic year, and we use the number of surveys participated in to understand how student characteristics such as demographics, engagement and Holland personality type affect cooperation. We find that survey respondents are more likely to be female and socially engaged, less likely to be on financial aid, more likely to be an investigative personality type and less likely to be an enterprising personality type.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. L. L. M. Adams D. Gale (1982) ArticleTitleSolving the quandry between questionnaire length and response rate in educational research Research in Higher Education 17 IssueID3 231–240

    Google Scholar 

  2. InstitutionalAuthorNameAmerican Association for Public Opinion Research (2004) Standard definitions: Final dispositions of case codes and outcome rates for surveys KS Lenexa

    Google Scholar 

  3. A. L. Aviv J. M. Zelenski L. Rallo R. J. Larsen (2002) ArticleTitleWho comes when: Personality differences in early and later participation in a university subject pool Personality and Individual Differences 33 487–496

    Google Scholar 

  4. M. R. Barrick M. K. Mount R. Gupta (2003) ArticleTitleMeta-analysis of the relationship between the Five-Factor Model of personality and Holland’s Occupational types Personal Psychology 56 46–74

    Google Scholar 

  5. Bradburn, N. M. (1978). Respondent burden. Paper presented at the The American Statistical Association, Proceedings of the Survey Research Methods Section.

  6. M. Brennan J. Hoek (1992) ArticleTitleThe behavior of respondents, nonrespondents, and refusers across mail surveys Public Opinion Quarterly 56 IssueID4 530–535

    Google Scholar 

  7. H. Cooper A. H. Baumgardner A. Strathman (1991) ArticleTitleDo students with different characteristics take part in psychology experiments at different times of the semester? Journal of Personality 59 109–127

    Google Scholar 

  8. M. P. Couper (1997) ArticleTitleSurvey introductions and data quality Public Opinion Quarterly 61 IssueID2 317–338

    Google Scholar 

  9. M. P. Couper M. W. Traugott M. J. Lamias (2001) ArticleTitleWeb survey design and administration Public Opinion Quarterly 65 250–253

    Google Scholar 

  10. S. D. Crawford M. P. Couper M. J. Lamias (2001) ArticleTitleWeb surveys: Perceptions of burden Social Science Computer Review 19 IssueID2 146–162

    Google Scholar 

  11. R. Curtin S. Presser E. Singer (2000) ArticleTitleThe effects of response rate changes on the index of consumer sentiment Public Opinion Quarterly 64 413–428

    Google Scholar 

  12. E. De Leeuw W. Heer (2002) Trends in household survey nonresponse: A longitudinal and international comparison R. M. Groves D. A. Dillman J. L. Eltinge R. J. A. Little (Eds) Survey Nonresponse John Wiley & Sons New York

    Google Scholar 

  13. E. Dey (1997) ArticleTitleWorking with low survey response rates: The efficacy of weighting adjustments Research in Higher Education 38 IssueID2 215–227

    Google Scholar 

  14. C. J. Dommeyer E. Moriarty (2000) ArticleTitleComparing two forms of an email survey: Embedded vs. attached International Journal of Market Research 42 IssueID1 39–50

    Google Scholar 

  15. R. Evans E. Donnerstein (1974) ArticleTitleSome implications for psychological research of early versus late term participation by college students Journal of Research in Personality 8 102–109

    Google Scholar 

  16. J. Goyder (1982) ArticleTitleFurther evidence on factors affecting response rates to mailed questionnaires American Sociological Review 47 IssueID4 550–553

    Google Scholar 

  17. J. Goyder (1986) ArticleTitleSurveys on surveys: Limitations and potentialities Public Opinion Quarterly 50 IssueID1 27–41

    Google Scholar 

  18. J. Goyder (1987) The Silent Minority: Nonrespondents in Sample Surveys Westview Press Boulder

    Google Scholar 

  19. J. Goyder K. Warriner S. Milller (2002) ArticleTitleEvaluating socio-economic status (SES) bias in survey nonresponse Journal of Official Statistics 18 IssueID1 1–11

    Google Scholar 

  20. W. H. Greene (1997) Econometric Analysis Prentice Hall Upper Saddle River, NJ

    Google Scholar 

  21. R. M. Groves R. B. Cialdini M. P. Couper (1992) ArticleTitleUnderstanding the decision to participate in a survey Public Opinion Quarterly 56 475–495

    Google Scholar 

  22. R. M. Groves M. P. Couper (1996) ArticleTitleContact-level influences on cooperation in face-to-face surveys Journal of Official Statistics 12 IssueID1 63–83

    Google Scholar 

  23. R. M. Groves E. Singer A. Corning (2000) ArticleTitleLeverage-saliency theory of survey participation Public Opinion Quarterly 64 299–308

    Google Scholar 

  24. T. A. Heberlein R. Baumgartner (1978) ArticleTitleFactors affecting response rates to mailed questionnaires: A quantitative analysis of the published literature American Sociological Review 43 IssueID4 447–462

    Google Scholar 

  25. J. L. Holland (1966) The Psychology of Vocational Choice Blaisdell Waltham, MA

    Google Scholar 

  26. J. L. Holland (1985) Making Vocational Choices Prentice-Hall Englewood Cliffs, NJ

    Google Scholar 

  27. J. Hutchinson N. Tollefson H. Wigington (1987) ArticleTitleResponse bias in college freshman’s response to mail surveys Research in Higher Education 26 IssueID1 99–106

    Google Scholar 

  28. J. S. Johnson R. R. Mowrer (2000) ArticleTitleMail surveys among college students:Do initial responders score differently from nonresponders on the 16PF? Psychological Reports 86 901–908

    Google Scholar 

  29. D. O. Kaldenberg H. F. Koenig B. W. Becker (1994) ArticleTitleMail survey response rate patterns in a population of the elderly: Does response deteriorate with age? Public Opinion Quarterly 58 IssueID1 68–76

    Google Scholar 

  30. D. Kandel J. L. Raveis (1983) ArticleTitleSex differences in the characteristics of members lost to a longitudinal panel: A speculative research note Public Opinion Quarterly 47 IssueID4 567–575

    Google Scholar 

  31. Kojetin, B. A., Borgida, E. , and Snyder, M. (1993). Survey Topic Involvement and Nonresponse Bias. Paper presented at the Proceeding of the Section on Survey Research Method, Volume II, American Statistical Association

  32. G. D. Kuh (2003) The National Survey of Student Engagement: Conceptual Framework and Overview of Psychometric Properties Indiana University Center for Postsecondary Research Bloomington, IN

    Google Scholar 

  33. N. Kwak B. Radler (2002) ArticleTitleA comparison between mail and web surveys: Response pattern, respondent profileand data quality Journal of Official Statistics 18 IssueID2 257–273

    Google Scholar 

  34. V. M. H. C. J. Lahaut H. A. M. Jansen D. van de Mheen H. F. L. Garretsen J. E. E. Verdurmen A. van Dijk (2003) ArticleTitleEstimating non-response bias in a survey on alcohol consumption: Comparison of response waves Alcohol and Alcoholism 38 128–134

    Google Scholar 

  35. L. M. Larson P. J. Rottingham F. Borgen (2002) ArticleTitleMeta-analysis of big six interests and big five personality factors Journal of Vocational Behavior 61 217–239

    Google Scholar 

  36. H. Laurie R. Smith L. Scott (1999) ArticleTitleStrategies for reducing nonresponse in a longitudinal panel survey Journal of Official Statistics 15 IssueID2 269–282

    Google Scholar 

  37. H. J. Liberty (1993) ArticleTitleThe relationship between extraversion and time of data collection Personality and Individual Differences 14 835–836

    Google Scholar 

  38. I.-F. Lin N. C. Schaeffer (1995) ArticleTitleUsing survey participants to estimate the impact of nonparticipation Public Opinion Quarterly 59 236–258

    Google Scholar 

  39. J. S. Long (1997) Regression Models for Categorical and Limited Dependent Variables Sage Publications Thousand Oaks, CA

    Google Scholar 

  40. F. Lowe T. C. McCormick (1955) ArticleTitleSome Survey Sampling Biases Public Opinion Quarterly 19 303–315

    Google Scholar 

  41. R. R. McCrae P. T. J. Costa (1996) Toward a new generation of personality theories: Theoretical contexts for the five-factor model J. S. Wiggins (Eds) The Five-Factor Model of Personality:Theoretical Perspectives Guilford New York 51–87

    Google Scholar 

  42. S. W. McDaniel C. S. Madden P. Verille (1987) ArticleTitleDo topic differences affect survey non-response? Journal of the Market Research Society 29 55–66

    Google Scholar 

  43. D. L. Moore J. Tarnai (2002) Evaluating nonresponse error in mail surveys R. M. Groves D. A. Dillman J. L. Eltinge R. J. A. Little (Eds) Survey Nonresponse John Wiley & Sons New York 197–211

    Google Scholar 

  44. MountM. K., Barrick, M. A., Scullen, S. M., and Rounds, J. (2004, April 2004). Higher Order Dimensions of Personality Traits and Interests. Paper presented at the Annual meeting of the Society for Industrial and Organizational Psychology, Chicago.

  45. National Center for Education Statistics (2002) National Postsecondary Student Aid Study 1999–2000 (NPSAS:2000), CATI Nonresponse Bias Analysis Report (No. 2002–03) Washington, DC

  46. National Survey of Student Engagement (2003). The College Student Report: 2003 Overview.

  47. L. N. Pealer R. M. Weiler R. M. Pigg SuffixJr D. Miller S. M. Dorman (2001) ArticleTitleThe feasibility of a web-based surveillance system to collect health risk data from college students Health Education & Behavior 28 IssueID5 547–559

    Google Scholar 

  48. Porter, S. R., Umbach, P. D. (2002) College Major Choice: An Analysis of Person-Environment Fit. Paper presented at the Association for Institutional Research. TorontoCanada

  49. S. R. Porter P. D. Umbach (2004) ArticleTitleWhat works best? Collecting alumni data with multiple technologies AIR Professional File 90 1–10

    Google Scholar 

  50. S. R. Porter M. E. Whitcomb (2003a) ArticleTitleThe impact of contact type on web survey response rates Public Opinion Quarterly 67 IssueID4 579–588

    Google Scholar 

  51. S. R. Porter M. E. Whitcomb (2003b) ArticleTitleThe impact of lottery incentives on student survey response rates Research in Higher Education 44 IssueID4 389–407

    Google Scholar 

  52. S. R. Porter M. E. Whitcomb W. H. Weitzer (2004) Multiple surveys of students and survey fatigue S. R. Porter (Eds) Overcoming Survey Research Problems, New Directions for Institutional Research (Vol. 121) Jossey-Bass San Francisco

    Google Scholar 

  53. R. J. Roman G. B. Moskowitz M. I. Stein R. F. Eisenberg (1995) ArticleTitleIndividual differences in experiment participation: Structureautonomy, and the time of semester Journal of Personality 63 113–138

    Google Scholar 

  54. L. J. Sax S. K. Gilmartin A. N. Bryant (2003) ArticleTitleAssessing response rates and nonresponse bias in web and paper surveys Research in Higher Education 44 IssueID4 409–432

    Google Scholar 

  55. L. J. Sax J. A. Lindholm A. W. Astin W. S. Korn K. M. Mahoney (2001) The American Freshman:National Norms for Fall 2001 Higher Education Research Institute UCLA, Los Angeles, CA

    Google Scholar 

  56. E. Singer R. M. Groves A. D. Corning (1999) ArticleTitleDifferential incentives:Beliefs about practices, perceptions of equity, and effects on survey participation Public Opinion Quarterly 63 251–260

    Google Scholar 

  57. E. Singer J. van Hoewyk M. P. Maher (2000) ArticleTitleExperiments with incentives in telephone surveys Public Opinion Quarterly 64 171–188

    Google Scholar 

  58. J. C. Smart K. A. Feldman C. A. Ethington (2000) Academic Disciplines:Holland’s Theory and the Study of College Students and Faculty Vanderbilt University Press Nashville

    Google Scholar 

  59. T. W. Smith (1995) ArticleTitleTrends in nonresponse rates International Journal of Public Opinion Research 7 157–171

    Google Scholar 

  60. K. Smith T. Bers (1987) ArticleTitleImproving alumni survey response rates: An experiment and cost-benefit analysis Research in Higher Education 27 IssueID3 218–226

    Google Scholar 

  61. C. G. Steeh (1981) ArticleTitleTrends in nonresponse rates, 1952–1979 Public Opinion Quarterly 59 66–77

    Google Scholar 

  62. C. D. Stevens R. A. Ash (2001) ArticleTitleThe conscientiousness of students in subject pools: Implications for “laboratory” research Journal of Research in Personality 35 91–97

    Google Scholar 

  63. A. L. Stinchcombe C. Jones P. Sheatsley (1981) ArticleTitleNonresponse bias for attitude questions Public Opinion Quarterly 45 IssueID3 359–375

    Google Scholar 

  64. Tuckel, P., and O’Neill, H. (2002). The vanishing respondent in telephone surveys. Journal of Advertising Research 26–48.

  65. P. D. Umbach S. R. Porter (2002) ArticleTitleHow do academic departments impact student satisfaction? Understanding the contextual effects of departments Research in Higher Education 23 IssueID2 209–233

    Google Scholar 

  66. P. Van Kenhove K. Wijnen K. De Wulf (2002) ArticleTitleThe influence of topic involvement on mail-survey response behavior Psychology & Marketing 19 293–301

    Google Scholar 

  67. L. F. Voigt T. D. Koepsell J. R. Daling (2003) ArticleTitleCharacteristics of telephone survey respondents according to willingness to participate American Journal of Epidemiology 157 66–73

    Google Scholar 

  68. WhitcombM. E., and PorterS. R. (2004). Email contacts: A test of complex graphical designs. Social Science Computer Review 22(3): 370–376.

  69. Zelenski, J. M., Rusting C. L., Larsen, R. J.(2000). Personality Differences and Consistency in the Time of Experiment Participation. Paper presented at the Poster presented at The Annual Meeting of the Society for Personality and Social Psychology. NashvilleTN

  70. B. J. Zusman P. Duby (1987) ArticleTitleAn evaluation of the use of monetary incentives in postsecondary survey research Journal of Research and Development in Education 20 IssueID4 73–78

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Wesleyan University, Middletown, CT

    Stephen R. Porter & Michael E. Whitcomb

  2. Office of Institutional Research, Wesleyan University, 237 High Street, 304 North College Bldg, Middletown, CT, 06459, USA

    Michael E. Whitcomb

Authors
  1. Stephen R. Porter
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael E. Whitcomb
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael E. Whitcomb.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Porter, S.R., Whitcomb, M.E. Non-response in student surveys: The Role of Demographics, Engagement and Personality. Res High Educ 46, 127–152 (2005). https://doi.org/10.1007/s11162-004-1597-2

Download citation

  • Received:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11162-004-1597-2

Keywords

Search

Navigation