How do Individuals With and Without Traumatic Brain Injury Interpret Emoji? Similarities and Differences in Perceived Valence, Arousal, and Emotion Representation
Tóm tắt
Impaired facial affect recognition is common after traumatic brain injury (TBI) and linked to poor social outcomes. We explored whether perception of emotions depicted by emoji is also impaired after TBI. Fifty participants with TBI and 50 non-injured peers generated free-text labels to describe emotions depicted by emoji and rated their levels of valence and arousal on nine-point rating scales. We compared how the two groups’ valence and arousal ratings were clustered and examined agreement in the words participants used to describe emoji. Hierarchical clustering of affect ratings produced four emoji clusters in the non-injured group and three emoji clusters in the TBI group. Whereas the non-injured group had a strongly positive and a moderately positive cluster, the TBI group had a single positive valence cluster, undifferentiated by arousal. Despite differences in cluster numbers, hierarchical structures of the two groups’ emoji ratings were significantly correlated. Most emoji had high agreement in the words participants with and without TBI used to describe them. Participants with TBI perceived emoji similarly to non-injured peers, used similar words to describe emoji, and rated emoji similarly on the valence dimension. Individuals with TBI showed small differences in perceived arousal for a minority of emoji. Overall, results suggest that basic recognition processes do not explain challenges in computer-mediated communication reported by adults with TBI. Examining perception of emoji in context by people with TBI is an essential next step for advancing our understanding of functional communication in computer-mediated contexts after brain injury.
Tài liệu tham khảo
Ahmadi, R., Lim, H., Mutlu, B., Duff, M., Toma, C., & Turkstra, L. (2022). Facebook Experiences of Users With Traumatic Brain Injury: A Think-Aloud Study. JMIR Rehabilitation and Assistive Technologies, 9(4), e39984. https://doi.org/10.2196/39984
Anwyl-Irvine, A. L., Massonnié, J., Flitton, A., Kirkham, N., & Evershed, J. K. (2020). Gorilla in our midst: An online behavioral experiment builder. Behavior Research Methods, 52(1), 388–407. https://doi.org/10.3758/s13428-019-01237-x
Arnould, A., Rochat, L., Azouvi, P., & van der Linden, M. (2013). A multidimensional approach to apathy after traumatic brain injury. Neuropsychology Review, 23(3), 210–233. https://doi.org/10.1007/s11065-013-9236-3
Arnould, A., Rochat, L., Azouvi, P., & van der Linden, M. (2015). Apathetic symptom presentations in patients with severe traumatic brain injury: Assessment, heterogeneity and relationships with psychosocial functioning and caregivers’ burden. Brain Injury, 29(13–14), 1597–1603. https://doi.org/10.3109/02699052.2015.1075156
Babbage, D. R., Yim, J., Zupan, B., Neumann, D., Tomita, M. R., & Willer, B. (2011). Meta-analysis of facial affect recognition difficulties after traumatic brain injury. Neuropsychology, 25(3), 277–285. https://doi.org/10.1037/a0021908
Bai, Q., Dan, Q., Mu, Z., & Yang, M. (2019). A Systematic Review of Emoji: Current Research and Future Perspectives. Frontiers in Psychology. https://doi.org/10.3389/FPSYG.2019.02221
Bibby, H., & McDonald, S. (2005). Theory of mind after traumatic brain injury. Neuropsychologia, 43(1), 99–114. https://doi.org/10.1016/j.neuropsychologia.2004.04.027
Bothe, E., Palermo, R., Rhodes, G., Burton, N., & Jeffery, L. (2019). Expression recognition difficulty is associated with social but not attention-to-detail autistic traits and reflects both alexithymia and perceptual difficulty. Journal of Autism and Developmental Disorders, 49(11), 4559–4571. https://doi.org/10.1007/s10803-019-04158-y
Brunner, M., Hemsley, B., Palmer, S., Dann, S., & Togher, L. (2015). Review of the literature on the use of social media by people with traumatic brain injury (TBI). Disability and Rehabilitation, 37(17), 1511–1521. https://doi.org/10.3109/09638288.2015.1045992
Brunner, M., Palmer, S., Togher, L., & Hemsley, B. (2019). ‘I kind of figured it out’: The views and experiences of people with traumatic brain injury (TBI) in using social media—self-determination for participation and inclusion online. International Journal of Language and Communication Disorders, 54(2), 221–233. https://doi.org/10.1111/1460-6984.12405
Byom, L., Duff, M., Mutlu, B., & Turkstra, L. (2019). Facial emotion recognition of older adults with traumatic brain injury. Brain Injury, 33(3), 322–332. https://doi.org/10.1080/02699052.2018.1553066
Centers for Disease Control and Prevention. (2015). Report to Congress on Traumatic Brain Injury in the United States: Epidemiology and Rehabilitation.
Clough, S., Morrow, E., Mutlu, B., Turkstra, L., & Duff, M. C. (2023). Emotion recognition of faces and emoji in individuals with moderate-severe traumatic brain injury. Brain Injury. https://doi.org/10.1080/02699052.2023.2181401
Covington, N. V., & Duff, M. C. (2021). Heterogeneity Is a hallmark of traumatic brain injury, not a limitation: A new perspective on study design in rehabilitation research. American Journal of Speech-Language Pathology, 30(2S), 974–985. https://doi.org/10.1044/2020_AJSLP-20-00081
Cramer, H., de Juan, P., & Tetreault, J. (2016). Sender-intended functions of emojis in US messaging. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services, 504–509. https://doi.org/10.1145/2935334.2935370
Croker, V., & McDonald, S. (2005). Recognition of emotion from facial expression following traumatic brain injury. Brain Injury, 19(10), 787–799. https://doi.org/10.1080/02699050500110033
Dewan, M. C., Rattani, A., Gupta, S., Baticulon, R. E., Hung, Y.-C., Punchak, M., Agrawal, A., Adeleye, A. O., Shrime, M. G., Rubiano, A. M., Rosenfeld, J. V., & Park, K. B. (2019). Estimating the global incidence of traumatic brain injury. Journal of Neurosurgery, 130(4), 1080–1097. https://doi.org/10.3171/2017.10.JNS17352
Duff, M. C., Morrow, E. L., Edwards, M., McCurdy, R., Clough, S., Patel, N., Walsh, K., & Covington, N. V. (2022). The value of patient registries to advance basic and translational research in the area of traumatic brain injury. Frontiers in Behavioral Neuroscience, 16(April), 1–15. https://doi.org/10.3389/fnbeh.2022.846919
Everitt, B. S., Landau, S., Leese, M., & Stahl, D. (2010). Cluster Analysis. In S. Landau, M. Leese, & D. Stahl (Eds.), Book. Wiley.
Green, R. E. A., Turner, G. R., & Thompson, W. F. (2004). Deficits in facial emotion perception in adults with recent traumatic brain injury. Neuropsychologia, 42, 133–141. https://doi.org/10.1016/j.neuropsychologia.2003.07.005
Hand, C. J., Burd, K., Oliver, A., & Robus, C. M. (2022a). Interactions between text content and emoji types determine perceptions of both messages and senders. Computers in Human Behavior Reports, 8, 100242. https://doi.org/10.1016/j.chbr.2022.100242
Hand, C. J., Kennedy, A., Filik, R., Pitchford, M., & Robus, C. M. (2022b). Emoji identification and emoji effects on sentence emotionality in ASD-diagnosed adults and neurotypical controls. Journal of Autism and Developmental Disorders. https://doi.org/10.1007/s10803-022-05557-4
Jaeger, S. R., & Ares, G. (2017). Dominant meanings of facial emoji: Insights from Chinese consumers and comparison with meanings from internet resources. Food Quality and Preference, 62, 275–283.
Jaeger, S. R., Roigard, C. M., Jin, D., Vidal, L., & Ares, G. (2019). Valence, arousal and sentiment meanings of 33 facial emoji: Insights for the use of emoji in consumer research. Food Research International, 119(2018), 895–907. https://doi.org/10.1016/j.foodres.2018.10.074
Kersey, J., McCue, M., & Skidmore, E. (2020). Domains and dimensions of community participation following traumatic brain injury. Brain Injury, 34(6), 708–712. https://doi.org/10.1080/02699052.2020.1757153
Kutsuzawa, G., Umemura, H., Eto, K., & Kobayashi, Y. (2022a). Age differences in the interpretation of facial emojis: classification on the arousal-valence space. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2022.915550
Kutsuzawa, G., Umemura, H., Eto, K., & Kobayashi, Y. (2022b). Classification of 74 facial emoji’s emotional states on the valence-arousal axes. Scientific Reports, 12(1), 398. https://doi.org/10.1038/s41598-021-04357-7
Lane-Brown, A. T., & Tate, R. L. (2011). Apathy after traumatic brain injury: An overview of the current state of play. Brain Impairment, 12(1), 43–53. https://doi.org/10.1375/brim.12.1.43
Maas, A. I. R., Menon, D. K., Adelson, P. D., Andelic, N., Bell, M. J., Belli, A., Bragge, P., Brazinova, A., Büki, A., Chesnut, R. M., Citerio, G., Coburn, M., Cooper, D. J., Crowder, A. T., Czeiter, E., Czosnyka, M., Diaz-Arrastia, R., Dreier, J. P., Duhaime, A.-C., & Zumbo, F. (2017). Traumatic brain injury: integrated approaches to improve prevention, clinical care, and research. The Lancet Neurology, 16(12), 987–1048. https://doi.org/10.1016/S1474-4422(17)30371-X
Malec, J. F., Brown, A. W., Leibson, C. L., Flaada, J. T., Mandrekar, J. N., Diehl, N. N., & Perkins, P. K. (2007). The mayo classification system for traumatic brain injury severity. JOURNAL OF NEUROTRAUMA, 24, 1417–1424. https://doi.org/10.1089/neu.2006.0245
McDonald, S. (2013). Impairments in social cognition following severe traumatic brain injury. Journal of the International Neuropsychological Society, 19(3), 231–246. https://doi.org/10.1017/S1355617712001506
McDonald, S., & Flanagan, S. (2004). Social perception deficits after traumatic brain injury: Interaction between emotion recognition, mentalizing ability, and social communication. Neuropsychology, 18(3), 572–579. https://doi.org/10.1037/0894-4105.18.3.572
Miller, H., Thebault-Spieker, J., Chang, S., Johnson, I., Terveen, L., & Hecht, B. (2016). “blissfully happy” or “ready to fight”: Varying interpretations of emoji. In Proceedings of the 10th International Conference on Web and Social Media, ICWSM 2016, Icwsm, 259–268.
Morrow, E. L., Zhao, F., Turkstra, L., Toma, C., Mutlu, B., & Duff, M. C. (2021). Computer-mediated communication in adults with and without moderate-to-severe traumatic brain injury: survey of social media use. JMIR Rehabilitation and Assistive Technologies, 8(3), e26586. https://doi.org/10.2196/26586
Prada, M., Rodrigues, D. L., Garrido, M. V., Lopes, D., Cavalheiro, B., & Gaspar, R. (2018). Motives, frequency and attitudes toward emoji and emoticon use. Telematics and Informatics, 35(7), 1925–1934. https://doi.org/10.1016/j.tele.2018.06.005
Radice-Neumann, D., Zupan, B., Babbage, D. R., & Willer, B. (2007). Overview of impaired facial affect recognition in persons with traumatic brain injury. Brain Injury, 21(8), 807–816. https://doi.org/10.1080/02699050701504281
Rigon, A., Turkstra, L., Mutlu, B., & Duff, M. (2016). The female advantage: Sex as a possible protective factor against emotion recognition impairment following traumatic brain injury. Cognitive, Affective and Behavioral Neuroscience, 16(5), 866–875. https://doi.org/10.3758/s13415-016-0437-0
Rigon, A., Turkstra, L. S., Mutlu, B., & Duff, M. C. (2018a). Facial-affect recognition deficit as a predictor of different aspects of social-communication impairment in traumatic brain injury. Neuropsychology, 32(4), 476–483. https://doi.org/10.1037/neu0000368
Rigon, A., Voss, M. W., Turkstra, L. S., Mutlu, B., & Duff, M. C. (2017). Relationship between individual differences in functional connectivity and facial-emotion recognition abilities in adults with traumatic brain injury. NeuroImage: Clinical, 13, 370–377. https://doi.org/10.1016/j.nicl.2016.12.010
Rigon, A., Voss, M. W., Turkstra, L. S., Mutlu, B., & Duff, M. C. (2018b). Different aspects of facial affect recognition impairment following traumatic brain injury: The role of perceptual and interpretative abilities. Journal of Clinical and Experimental Neuropsychology, 00(00), 1–15. https://doi.org/10.1080/13803395.2018.1437120
Rosenberg, H., McDonald, S., Dethier, M., Kessels, R. P. C., & Westbrook, R. F. (2014). Facial emotion recognition deficits following moderate-severe traumatic brain injury (TBI): re-examining the valence effect and the role of emotion intensity. Journal of the International Neuropsychological Society, 20(10), 994–1003. https://doi.org/10.1017/S1355617714000940
Ruedin, D. (2021). agrmt: Calculate Concentration and Dispersion in Ordered Rating Scales (1.42.8). https://CRAN.R-project.org/package=agrmt.
Salas, C. E., Casassus, M., Rowlands, L., Pimm, S., & Flanagan, D. A. (2018). “Relating through sameness”: A qualitative study of friendship and social isolation in chronic traumatic brain injury. Neuropsychological Rehabilitation, 28(7), 1161–1178
Shorland, J., & Douglas, J. M. (2010). Understanding the role of communication in maintaining and forming friendships following traumatic brain injury. Brain Injury, 24(4), 569–580. https://doi.org/10.3109/02699051003610441
Subramanian, J., Sridharan, V., Shu, K., & Liu, H. (2019). Exploiting Emojis for Sarcasm Detection. In R. Thomson, H. Bisgin, C. Dancy, & A. Hyder (Eds.), Social, Cultural, and Behavioral Modeling (pp. 70–80). Springer.
Taylor, H., Hand, C. J., Howman, H., & Filik, R. (2022). Autism, attachment, and alexithymia: investigating emoji comprehension. International Journal of Human-Computer Interaction. https://doi.org/10.1080/10447318.2022.2154890
Teasdale, G., & Jennett, B. (1974). Assessment of coma and impaired consciousness: A practical scale. The Lancet, 304(7872), 81–84. https://doi.org/10.1016/S0140-6736(74)91639-0
Tigwell, G. W., & Flatla, D. R. (2016). Oh that’s what you meant! In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct, 859–866. Doi https://doi.org/10.1145/2957265.2961844
Turkstra, L. S., Kraning, S. G., Riedeman, S. K., Mutlu, B., Duff, M., & VanDenHeuvel, S. (2017). Labelling facial affect in context in adults with and without TBI. Brain Impairment, 18(1), 49–61. https://doi.org/10.1017/BrImp.2016.29
Turkstra, L. S., Norman, R. S., Mutlu, B., & Duff, M. C. (2018). Impaired theory of mind in adults with traumatic brain injury: A replication and extension of findings. Neuropsychologia, 111(January), 117–122. https://doi.org/10.1016/j.neuropsychologia.2018.01.016
van der Eijk, C. (2001). Measuring agreement in ordered rating scales. In Quality & Quantity. https://doi.org/10.1023/A:1010374114305
Wild, F. (2022). lsa: Latent Semantic Analysis (0.73.3). https://CRAN.R-project.org/package=lsa.
Yim, O., & Ramdeen, K. T. (2015). Hierarchical cluster analysis: comparison of three linkage measures and application to psychological data. The Quantitative Methods for Psychology, 11(1), 8–21. https://doi.org/10.20982/tqmp.11.1.p008
Ylvisaker, M., & Feeney, T. J. (1998). Collaborative brain injury intervention: Positive everyday routines. Singular Publishing Group
Zupan, B., Dempsey, L., & Hartwell, K. (2022). Categorising emotion words: the influence of response options. Language and Cognition. https://doi.org/10.1017/langcog.2022.24