The Experiment is just as Important as the Likelihood in Understanding the Prior: a Cautionary Note on Robust Cognitive Modeling
Tóm tắt
Cognitive modeling shares many features with statistical modeling, making it seem trivial to borrow from the practices of robust Bayesian statistics to protect the practice of robust cognitive modeling. We take one aspect of statistical workflow—prior predictive checks—and explore how they might be applied to a cognitive modeling task. We find that it is not only the likelihood that is needed to interpret the priors, we also need to incorporate experiment information as well. This suggests that while cognitive modeling might borrow from statistical practices, especially workflow, care must be taken to make the necessary adaptions.
Tài liệu tham khảo
Betancourt, M. (2018). Towards a principled Bayesian workflow. https://betanalpha.github.io/assets/case_studies/principled_bayesian_workflow.html (visited on 05/23/2019).
Evans, N.J. (2019). What factors are most important in finding the best model of a psychological process? Comment on Navarro (2018). https://doi.org/10.31234/osf.io/rkyp3. https://psyarxiv.com/rkyp3.
Gabry, J., & et al. (2019). Visualization in Bayesian workflow. Journal of the Royal Statistical Society: Series A (Statistics in Society), 182(2), 389–402.
Gelman, A. (2008). The folk theorem of statistical computing. https://statmodeling.stat.columbia.edu/2008/05/13/the_folk_theore/ (visited on 05/23/2019).
Gelman, A., Simpson, D., Betancourt, M. (2017). The prior can often only be understood in the context of the likelihood, (Vol. 19 p. 555).
Gronau, Q.F., & Singmann, H. (2018). bridgesampling: Bridge sampling for marginal likelihoods and Bayes factors. R package version 0.6-0. https://CRAN.R-project.org/package=bridgesampling.
Gronau, Q.F., & Wagenmakers, E.-J. (2019). Rejoinder: more limitations of Bayesian leave-one-out cross-validation. Computational Brain and Behavior, 2(1), 35–47. https://doi.org/10.1007/s42113-018-0022-4.
Kennedy, L., & Gelman, A. (2019). Know your population and know your model: using model-based regression and poststratification to generalize findings beyond the observed sample. arXiv:1906.11323 [stat.AP].
Lee, M.D. (2018). Bayesian methods in cognitive modeling. Stevens’ Handbook of Experimental Psychology and Cognitive Neuroscience, 5, 1–48.
Lee, M.D., & Vanpaemel, W. (2018). Determining informative priors for cognitive models. Psychonomic Bulletin and Review, 25(1), 114–127.
Lee, M.D., & Wagenmakers, E.-J. (2014). Bayesian cognitive modeling: a practical course. Cambridge: Cambridge University Press.
Lee, M.D., & et al. (2019). Robust modeling in cognitive science. Computational Brain and Behavior. ISSN: 2522-087X. https://doi.org/10.1007/s42113-019-00029-y.
Lejuez, C.W., & et al. (2002). Evaluation of a behavioral measure of risk taking: the Balloon Analogue Risk Task (BART). Journal of Experimental Psychology: Applied, 8(2), 75.
Navarro, D.J. (2019). Between the devil and the deep blue sea: tensions between scientific judgement and statistical model selection. Computational Brain and Behavior, 2(1), 28–34.
Schad, D.J., Betancourt, M., Vasishth, S. (2019). Toward a principled Bayesian workflow in cognitive science. arXiv:1904.12765 [stat.ME].
Shavelson, R.J, Webb, N.M., Rowley, G.L. (1989). Generalizability theory. American Psychologist, 44 (6), 922.
Talts, S., & et al. (2018). Validating Bayesian inference algorithms with simulation-based calibration. arXiv:1804.06788.
Van Ravenzwaaij, D., Dutilh, G., Wagenmakers, E.-J. (2011). Cognitive model decomposition of the BART: assessment and application. Journal of Mathematical Psychology, 55(1), 94–105.
Vehtari, A., & et al. (2018). loo: Efficient leave-one-out cross-validation and WAIC for Bayesian models. R package version 2.0.0. https://CRAN.R-project.org/package=loo.
Veksler, V.D., Myers, C.W., Gluck, K.A. (2015). Model flexibility analysis. Psychological Review, 122 (4), 755.