A Bayesian approach to modeling topic-metadata relationships
AStA Advances in Statistical Analysis - Trang 1-17 - 2023
Tóm tắt
The objective of advanced topic modeling is not only to explore latent topical structures, but also to estimate relationships between the discovered topics and theoretically relevant metadata. Methods used to estimate such relationships must take into account that the topical structure is not directly observed, but instead being estimated itself in an unsupervised fashion, usually by common topic models. A frequently used procedure to achieve this is the method of composition, a Monte Carlo sampling technique performing multiple repeated linear regressions of sampled topic proportions on metadata covariates. In this paper, we propose two modifications of this approach: First, we substantially refine the existing implementation of the method of composition from the R package stm by replacing linear regression with the more appropriate Beta regression. Second, we provide a fundamental enhancement of the entire estimation framework by substituting the current blending of frequentist and Bayesian methods with a fully Bayesian approach. This allows for a more appropriate quantification of uncertainty. We illustrate our improved methodology by investigating relationships between Twitter posts by German parliamentarians and different metadata covariates related to their electoral districts, using the structural topic model to estimate topic proportions.
Tài liệu tham khảo
Atchison, J., Shen, S.M.: Logistic-normal distributions: some properties and uses. Biometrika 67(2), 261–272 (1980)
Benoit, K., Watanabe, K., Wang, H., Nulty, P., Obeng, A., Müller, S., Matsuo, A.: quanteda: an R package for the quantitative analysis of textual data. J. Open Source Softw. 3(30), 774 (2018). Retrieved from https://doi.org/10.21105/joss.00774https://quanteda.io
Blei, D.M., Lafferty, J.D.: A correlated topic model of science. Ann. Appl. Stat. 1(1), 17–35 (2007)
Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)
Bohr, J., Dunlap, R.E.: Key topics in environmental sociology, 1990–2014: results from a computational text analysis. Environ. Sociol. 4(2), 181–195 (2018)
Chandelier, M., Steuckardt, A., Mathevet, R., Diwersy, S., Gimenez, O.: Content analysis of newspaper coverage of wolf recolonization in France using structural topic modeling. Biol. Cons. 220, 254–261 (2018)
Cho, I., Wesslen, R., Karduni, A., Santhanam, S., Shaikh, S., Dou, W.: The anchoring effect in decision-making with visual analytics. In: 2017 IEEE Conference on Visual Analytics Science and Technology (vast), pp. 116–126 (2017)
Egami, N., Fong, C.J., Grimmer, J., Roberts, M.E., Stewart, B.M.: How to make causal inferences using texts. arXiv preprint arXiv:1802.02163 (2018)
Farrell, J.: Corporate funding and ideological polarization about climate change. Proc. Natl. Acad. Sci. 113(1), 92–97 (2016)
Ferrari, S., Cribari-Neto, F.: Beta regression for modelling rates and proportions. J. Appl. Stat. 31(7), 799–815 (2004)
Heberling, J.M., Prather, L.A., Tonsor, S.J.: The changing uses of herbarium data in an era of global change: an overview using automated content analysis. Bioscience 69(10), 812–822 (2019)
Kim, I.S.: Political cleavages within industry: firm-level lobbying for trade liberalization. Am. Political Sci. Rev. 111(1), 1 (2017)
Lucas, C., Nielsen, R.A., Roberts, M.E., Stewart, B.M., Storer, A., Tingley, D.: Computer-assisted text analysis for comparative politics. Polit. Anal. 23(2), 254–277 (2015)
Mimno, D., Wallach, H.M., Talley, E., Leenders, M., McCallum, A.: Optimizing semantic coherence in topic models. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 262–272 (2011)
Moschella, M., Pinto, L.: Central banks’ communication as reputation management: How the fed talks under uncertainty. Public Admin. 97(3), 513–529 (2019)
Richardson, L.: Beautiful soup documentation. April (2007)
Roberts, M.E., Stewart, B.M., Airoldi, E.M.: A model of text for experimentation in the social sciences. J. Am. Stat. Assoc. 111(515), 988–1003 (2016)
Roberts, M.E., Stewart, B.M., Tingley, D.: stm: An R package for structural topic models. J. Stat. Softw. 91(2), 1–40 (2019). https://doi.org/10.18637/jss.v091.i02
Roesslein, J.: Tweepy: Twitter for python! (2020). https://github.com/tweepy/tweepy
Taddy, M.: On estimation and selection for topic models. Artific. Intell Stat., pp. 1184–1193 (2012)
Tanner, M.A.: Tools for Statistical Inference. Springer, Berlin (2012)
Treier, S., Jackman, S.: Democracy as a latent variable. Am. J. Political Sci. 52(1), 201–217 (2008)
Walker, A.M.: On the asymptotic behaviour of posterior distributions. J. Roy. Stat. Soc.: Ser. B (Methodol.) 31(1), 80–88 (1969)
Wallach, H.M., Murray, I., Salakhutdinov, R., Mimno, D.: Evaluation methods for topic models. In: Proceedings of the 26th Annual International Conference On Machine Learning (pp. 1105–1112) (2009)
Wang, C., Blei, D.M.: Variational inference in nonconjugate models. J. Mach. Learn. Res. 14, 1005–1031 (2013)