Interpretation of significance levels by psychological researchers: the .05 cliff effect may be overstated

Psychon Bull Rev. 2001 Dec;8(4):847-50. doi: 10.3758/bf03196227.

Abstract

Comments about previous studies indicate that the interpretation of significance levels by psychological researchers is unequivocally dictated by a binary decision-making framework. In particular, confidence in a p level would drop abruptly just beyond the fateful .05 level (cliff effect). A replication of Rosenthal and Gaito's (1963) experiment on the degree of confidence in p levels shows that these claims should be moderated. Detailed analysis of individual curves reveals that the attitude of researchers toward p values is far from being as homogeneous as might be expected. However, most psychological researchers in our study rated graduated confidence judgments, as either exponential or linear. Only a minority of all-or-none respondents exhibited an abrupt drop in confidence.

MeSH terms

  • Decision Making*
  • Humans
  • Models, Theoretical
  • Psychology / methods
  • Random Allocation