CROWDSOURCING A WORD–EMOTION ASSOCIATION LEXICON

Computational Intelligence - Tập 29 Số 3 - Trang 436-465 - 2013
Saif M. Mohammad1, Peter D. Turney1
1National Research Council Canada, Ottawa, Ontario, Canada

Tóm tắt

Even though considerable attention has been given to the polarity of words (positive and negative) and the creation of large polarity lexicons, research in emotion analysis has had to rely on limited and small emotion lexicons. In this paper, we show how the combined strength and wisdom of the crowds can be used to generate a large, high‐quality, word–emotion and word–polarity association lexicon quickly and inexpensively. We enumerate the challenges in emotion annotation in a crowdsourcing scenario and propose solutions to address them. Most notably, in addition to questions about emotions associated with terms, we show how the inclusion of a word choice question can discourage malicious data entry, help to identify instances where the annotator may not be familiar with the target term (allowing us to reject such annotations), and help to obtain annotations at sense level (rather than at word level). We conducted experiments on how to formulate the emotion‐annotation questions, and show that asking if a term is associated with an emotion leads to markedly higher interannotator agreement than that obtained by asking if a term evokes an emotion.

Từ khóa


Tài liệu tham khảo

10.1145/1367497.1367587

10.3115/1220575.1220648

10.1007/978-3-540-74628-7_27

Ross W. D., 1913, The Works of Aristotle, 805

Bellegarda J., 2010, Proceedings of the NAACL‐HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text

J. Bernard, 1986, The Macquarie Thesaurus. Macquarie Library

Boucouvalas A. C., 2002, Emerging Communication: Studies on New Technologies and Practices in Communication, 305

Bougie J. R. G., 2003, Open access publications from Tilburg University

Bradley M., 1999, Technical Report, C‐1, The Center for Research in Psychophysiology

Brants T., 2006, Web 1T 5‐Gram Version 1, Linguistic Data Consortium

Breazeal C., 2004, Who Needs Emotions

10.1177/001316448104100307

10.3115/1699510.1699548

10.1177/001316446002000104

Cohen S. H, 2003, Maximum difference scaling: Improved measures of importance and preference for segmentation, Sawtooth Software Conference Proceedings, Sawtooth Software, Inc., 61

10.1016/S0022-4359(00)00029-4

10.1037/10001-000

Davenport T. H., 2001, How do they know their customers so well?, MIT Sloan Management Review, 42, 63

10.1108/09564230210425331

10.1016/S1067-5671(96)05054-8

10.1080/02699939208411068

Ekman P., 2005, Emotion in the Human Face

Ekman P., 2003, Malor Books

10.1037/0033-2909.115.1.102

Elliott C., 1992, Ph.D. Thesis, Institute for the Learning Sciences

10.1037/h0031619

10.1086/209086

10.1007/11846406_47

Genereux M., 2006, Distinguishing affective states in weblogs, AAAI-2006 Spring Symposium on Computational Approaches to Analysing Weblogs, Stanford, CA, 27

Goyal A., 2010, Proceedings of the NAACL‐HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text

10.1037/1089-2680.2.3.271

10.1007/s10071-008-0199-3

10.1016/j.jretai.2004.04.002

10.1109/IROS.2006.282327

Holsti O. R.1969.Content analysis for the social sciences and humanities.Addison‐Wesley: Reading MA .

Holzman L. E., 2003, Technical report

Howe J., 2006, Crowdsourcing: Tracking the Rise of the Amateur

10.1093/mind/os-IX.34.188

John D., 2006, Proceedings of the 24th IASTED International Conference on Internet and Multimedia Systems and Applications, 183

10.1007/11555261_81

10.1145/1835449.1835618

Kŏvecses Z, 2003, Culture, and Body in Human Feeling Studies in Emotion and Social Interaction

10.2307/2529310

10.1037/0003-066X.39.2.124

Lazarus R. S, 2000, Handbook of Cognition and Emotion, 1

Lehrer A.1974.Semantic Fields and Lexical Structure.North‐Holland: Amsterdam the Netherlands .

Litman D. J., 2004, Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics, ACL ’04, USA

10.1145/604045.604067

Louviere J. J., 1991, Technical report

Louviere J. J., 1992, Determining the appropriate response to evidence of public concern: The case of food safety, Journal of Public Policy and Marketing, 11, 12, 10.1177/074391569201100202

Ma C., 2005, First International Conference on Affective Computing and Intelligent Interaction ACII‐2005, 622

Masson J. M., 1996, When Elephants Weep: The Emotional Lives of Animals

Matykiewicz P., 2009, Proceedings of the Workshop on Current Trends in Biomedical Natural Language Processing, BioNLP ’09, USA, 179

10.1016/0167-4870(93)90001-2

Mihalcea R., 2006, AAAI‐2006 Spring Symposium on Computational Approaches to Analysing Weblogs, 139

Mohammad S. M., 2011, Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies

Mohammad S. M., 2011, Proceedings of the ACL 2011 Workshop on Language Technology for Cultural Heritage, Social Sciences, and Humanities LaTeCH

Mohammad S. M., 2010, Proceedings of the NAACL‐HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text

Mohammad S. M., 2011, Proceedings of the ACL 2011 Workshop on Computational Approaches to Subjectivity and Sentiment Analysis WASSA

10.3115/1699571.1699591

Neviarouskaya A., 2009, Proceedings of the Third International Conference on Weblogs and Social Media ICWSM‐09, 278

10.1017/S1351324910000239

10.1111/j.1749-6632.1956.tb49644.x

Oliver R. L.1997.Satisfaction a behavioral perspective on the consumer.McGraw‐Hill: New York .

10.1037/0033-295X.97.3.315

Ortony A. G. L.Clore andA.Collins.1988.The Cognitive Structure of Emotions.Cambridge University Press: Cambridge UK .

10.1037/h0047078

10.1561/1500000011

Parrot W., 2001, Emotions in Social Psychology

Pearl L., 2010, Proceedings of the NAACL‐HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text

10.2307/3172601

10.3115/1572306.1572327

Plutchik R., 1962, The Emotions

10.1016/B978-0-12-558701-3.50007-7

10.1007/BF00991576

Plutchik R., 1994, The Psychology and Biology of Emotion

10.1007/11788713_5

10.1162/pres.15.4.381

Read J., 2004, Ph.D. Thesis, Department of Informatics

Reichheld F. F., 2000, E‐loyalty: Your secret weapon on the web, Harvard Business Review, 78, 105

Rentoumi V., 2009, Proceedings of the International Conference RANLP‐2009, Borovets, Bulgaria, 370

10.1007/BF02722168

Richins M. L., 1984, Word of mouth communication as negative information, Advances in Consumer Research, 11, 697

10.1037/0033-2909.115.1.102

10.1073/pnas.0908239106

10.1177/053901882021004004

Scherer K. R., 1984, Emotion as a multicomponent process: A model and some cross‐cultural data, Review of Personality and Social Psychology, 5, 37

10.1086/266577

10.1016/S0963-8687(02)00022-7

10.1037/0022-3514.52.6.1061

10.2307/1251688

10.3115/1613715.1613751

Steunebrink B. R, 2010, Ph.D. Thesis

Stewart K. J., 2003, Trust transfer on the World Wide Web. Organization Science, 5

Stone P., 1966, The General Inquirer: A Computer Approach to Content Analysis

Strapparava C., 2004, Proceedings of the 4th International Conference on Language Resources and Evaluation LREC‐2004, 1083

10.1109/91.940962

10.1145/944012.944013

Velásquez J. D., 1997, Proceedings of the Fourteenth National Conference on Artificial Intelligence and Ninth Conference on Innovative Applications of Artificial Intelligence, AAAI’97/IAAI’97, 10

10.1037/0033-295X.92.4.548

Wiebe J. M., 1994, Tracking point of view in narrative, Computational Linguistics, 20, 233

10.1037/0003-066X.39.2.117

Zhe X., 2002, Text‐to‐emotion engine for real time Internet communication, International Symposium on CSNDSP, 164

Thông tin
Thông tin xuất bản

Computational Intelligence

Tập 29 Số 3

436-465

Thông tin tác giả