A tutorial on a practical Bayesian alternative to null-hypothesis significance testing

Springer Science and Business Media LLC - Tập 43 - Trang 679-690 - 2011
Michael E. J. Masson1
1Department of Psychology, University of Victoria, Victoria, Canada

Tóm tắt

Null-hypothesis significance testing remains the standard inferential tool in cognitive science despite its serious disadvantages. Primary among these is the fact that the resulting probability value does not tell the researcher what he or she usually wants to know: How probable is a hypothesis, given the obtained data? Inspired by developments presented by Wagenmakers (Psychonomic Bulletin & Review, 14, 779–804, 2007), I provide a tutorial on a Bayesian model selection approach that requires only a simple transformation of sum-of-squares values generated by the standard analysis of variance. This approach generates a graded level of evidence regarding which model (e.g., effect absent [null hypothesis] vs. effect present [alternative hypothesis]) is more strongly supported by the data. This method also obviates admonitions never to speak of accepting the null hypothesis. An Excel worksheet for computing the Bayesian analysis is provided as supplemental material.

Tài liệu tham khảo

Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19, 716–723. doi:10.1109/TAC.1974.1100705 Berger, J. O., & Delampady, M. (1987). Testing precise hypotheses. Statistical Science, 2, 317–352. doi:10.1109/TAC.1974.1100705 Bernstein, D. M., Loftus, G. R., & Meltzoff, A. N. (2005). Object identification in preschool children and adults. Developmental Science, 8, 151–161. doi:10.1111/j.1467-7687.2005.00402.x Bortolussi, M., & Dixon, P. (2003). Psychonarratology: Foundations for the empirical study of literary response. Cambridge: Cambridge University Press. Breuer, A. T., Masson, M. E. J., Cohen, A.-L., & Lindsay, D. S. (2009). Long-term repetition priming of briefly identified objects. Journal of Experimental Psychology. Learning, Memory, and Cognition, 35, 487–498. doi:10.1037/a0014734 Bub, D. N., & Masson, M. E. J. (2010). Grasping beer mugs: On the dynamics of alignment effects induced by handled objects. Journal of Experimental Psychology: Human Perception and Performance, 36, 341–358. doi:10.1037/a0017606 Chow, S. L. (1998). Précis of statistical significance: Rationale, validity, and utility. The Behavioral and Brain Sciences, 21, 169–239. doi:10.1017/S0140525X98001162 Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale: Erlbaum. Cohen, J. (1994). The earth is round (p <.05). The American Psychologist, 49, 997–1003. doi:10.1037/0003-066X.49.12.997 Dixon, P. (2003). The p-value fallacy and how to avoid it. Canadian Journal of Experimental Psychology, 57(189-202), 133–149. doi:10.1037/h0087425 Dixon, P., & O'Reilly, T. (1999). Scientific versus statistical inference. Canadian Journal of Experimental Psychology, 53, 189–202. doi:10.1037/h0087305 Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175–191. Gallistel, C. R. (2009). The importance of proving the null. Psychological Review, 116, 439–453. doi:10.1037/a0015251 Glover, S., & Dixon, P. (2001). Dynamic illusion effects in a reaching task: Evidence for separate visual representations in the planning and control of reaching. Journal of Experimental Psychology: Human Perception and Performance, 27, 560–572. doi:10.1037/0096-1523.27.3.560 Glover, S., & Dixon, P. (2004). Likelihood ratios: A simple and flexible statistic for empirical psychologists. Psychonomic Bulletin & Review, 11, 791–806. Glover, S., Dixon, P., Castiello, U., & Rushworth, M. F. S. (2005). Effects of an orientation illusion on motor performance and motor imagery. Experimental Brain Research, 166, 17–22. Hagen, R. L. (1997). In praise of the null hypothesis statistical test. The American Psychologist, 52, 15–24. doi:10.1037/0003-066X.52.1.15 Hubbard, R., & Lindsay, R. M. (2008). Why p values are not a useful measure of evidence in statistical significance testing. Theory & Psychology, 18, 69–88. doi:10.1177/0959354307086923 Iverson, G. J., Wagenmakers, E.-J., & Lee, M. D. (2010). A model-averaging approach to replication: The case of p rep. Psychological Methods, 15, 172–181. Kantner, J., & Lindsay, D. S. (2010). Can corrective feedback improve recognition memory? Memory & Cognition, 38, 389–406. doi:10.3758/MC.38.4.389 Kass, R. E., & Raftery, A. E. (1995). Bayes factors. Journal of the American Statistical Association, 90, 773–795. Kass, R. E., & Wasserman, L. (1995). A reference Bayesian test for nested hypotheses and its relationship to the Schwarz criterion. Journal of the American Statistical Association, 90, 928–934. Krueger, J. (2001). Null hypothesis significance testing: On the survival of a flawed method. The American Psychologist, 56, 16–26. doi:10.1037/0003-066X.56.1.16 Lindley, D. V. (1957). A statistical paradox. Biometrika, 44, 187–192. doi:10.1093/biomet/44.1-2.187 Loftus, G. R., & Harley, E. M. (2005). Why is it easier to identify someone close than far away? Psychonomic Bulletin & Review, 12, 43–65. Loftus, G. R., & Irwin, D. E. (1998). On the relations among different measures of visible and informational persistence. Cognitive Psychology, 35, 135–199. doi:10.1006/cogp.1998.0678 Nickerson, R. S. (2000). Null hypothesis statistical testing: A review of an old and continuing controversy. Psychological Methods, 5, 241–301. doi:10.1037/1082-989X.5.2.241 R development core team. (2010). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from http://www.R-project.org Raftery, A. E. (1995). Bayesian model selection in social research. In P. V. Marsden (Ed.), Sociological methodology 1995 (pp. 111–196). Cambridge: Blackwell. Raftery, A. E. (1999). Bayes factors and BIC: Comment on "A critique of the Bayesian information criterion for model selection.". Sociological Methods & Research, 27, 411–427. doi:10.1177/0049124199027003005 Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009). Bayesian t tests for accepting and rejecting the null hypothesis. Psychonomic Bulletin & Review, 16, 225–237. doi:10.3758/PBR.16.2.225 Trafimow, D., & Rice, S. (2009). A test of the null hypothesis significance testing procedure correlation argument. The Journal of General Psychology, 136, 261–269. doi:10.3200/GENP.136.3.261-270 Wagenmakers, E.-J. (2007). A practical solution to the pervasive problems of p values. Psychonomic Bulletin & Review, 14, 779–804. Wilkinson, L., & the Task Force on Statistical Inference, (1999). Statistical methods in psychology journals: Guidelines and explanations. The American Psychologist, 54, 594–604. doi:10.1037/0003-066X.54.8.594 Winkel, J., Wijnen, J. G., Ridderinkof, K. R., Groen, I. I. A., Derrfuss, J., Danielmeier, C., et al. (2009). Your conflict matters to me! Behavioral and neural manifestations of control adjustment after self-experienced and observed decision-conflict. Frontiers in Human Neuroscience, 3, 57. doi:10.3389/neuro.09.057.2009