Modelling the “transactive memory system” in multimodal multiparty interactions

Beatrice Biancardi1, Maurizio Mancini2, Brian Ravenet3, Giovanna Varni4
1CESI, LINEACT, 93 boulevard de la Seine, 92000, Nanterre, France
2Department of Computer Science, Sapienza University of Rome, Rome, Italy
3LISN-CNRS, Université Paris-Saclay, Rue du Belvédère, 91400, Orsay, France
4Department of Information Engineering and Computer Science, University of Trento, Trento, Italy

Tóm tắt

AbstractTransactive memory system (TMS) is a team emergent state representing the knowledge of each member about “who knows what” in a team performing a joint task. We present a study to show how the three TMS dimensions Credibility, Specialisation, Coordination, can be modelled as a linear combination of the nonverbal multimodal features displayed by the team performing the joint task. Results indicate that, to some extent, the three dimensions of TMS can be expressed as a linear combination of nonverbal multimodal features. Moreover, the higher the number of modalities (audio, movement, spatial), the better the modelling. Results could be used in future work to design human-centered computing applications able to automatically estimate TMS from teams’ behavioural patterns, to provide feedback and help teams’ interactions.

Từ khóa


Tài liệu tham khảo

Alborno P, Piana S, Mancini M, Niewiadomski R, Volpe G, Camurri A (2016) Analysis of intrapersonal synchronization in full-body movements displaying different expressive qualities. In: Proceedings of the international working conference on advanced visual interfaces, pp 136–143

Argote L, Aven BL, Kush J (2018) The effects of communication networks and turnover on transactive memory and group performance. Organ Sci 29(2):191–206

Austin JR (2003) Transactive memory in organizational groups: the effects of content, consensus, specialization, and accuracy on group performance. J Appl Psychol 88(5):866

Bachrach DG, Mullins R (2019) A dual-process contingency model of leadership, transactive memory systems and team performance. J Bus Res 96:297–308

Beattie GW (1981) Interruption in conversational interaction, and its relation to the sex and status of the interactants. Linguistics 19(1–2):15–36

Beyan C, Carissimi N, Capozzi F, Vascon S, Bustreo M, Pierro A, Becchio C, Murino V (2016) Detecting emergent leader in a meeting environment using nonverbal visual features only. In: Proceedings of the 18th ACM international conference on multimodal interaction, pp 317–324

Beyan C, Capozzi F, Becchio C, Murino V (2017) Prediction of the leadership style of an emergent leader using audio and visual nonverbal features. IEEE Trans Multimed 20(2):441–456

Beyan C, Katsageorgiou V-M, Murino V (2017) Moving as a leader: detecting emergent leadership in small groups using body pose. In: Proceedings of the 25th ACM international conference on multimedia, pp 1425–1433

Beyan C, Katsageorgiou V-M, Murino V (2019) A sequential data analysis approach to detect emergent leaders in small groups. IEEE Trans Multimed 21(8):2107–2116

Beyan C, Karumuri S, Volpe G, Camurri A, Niewiadomski R (2021) Modeling multiple temporal scales of full-body movements for emotion classification. IEEE Trans Affect Comput

Biancardi B, Maisonnave-Couterou L, Renault P, Ravenet B, Mancini M, Varni G (2020) The wonowa dataset: investigating the transactive memory system in small group interactions. In: Proceedings of the 2020 international conference on multimodal interaction, pp 528–537

Biancardi B, Giaccaglia I, Ravenet B, Varni G (2021) Virtual leaders supporting the development of transactive memory systems. In: 32e conférence francophone sur l’Interaction Humain-Machine (IHM’20.21). ACM, pp 4–1

Biancardi B, O’Toole P, Giaccaglia I, Ravenet B, Pitt I, Mancini M, Varni G (2021) How ECA vs human leaders affect the perception of transactive memory system (TMS) in a team. In: 2021 9th International conference on affective computing and intelligent interaction (ACII). IEEE, pp 1–8

Boyd Sr FD (2000) Non-verbal behaviors of effective teachers of at-risk African-American male middle school students. PhD thesis, Virginia Tech

Bradski G (2000) The OpenCV library. Dr. Dobb’s J Softw Tools

Bresin R, Mancini M, Elblaus L, Frid E (2020) Sonification of the self vs. sonification of the other: differences in the sonification of performed vs. observed simple hand movements. Int J Hum Comput Stud 144:102500

Campbell N, Sadanobu T, Imura M, Iwahashi N, Suzuki N, Douxchamps D (2006) Multimedia database of meetings and informal interactions for tracking participant involvement and discourse flow. In: LREC, pp 391–394

Camurri A, Mazzarino B, Volpe G (2003) Analysis of expressive gesture: the eyesweb expressive gesture processing library. In: International gesture workshop. Springer, Berlin, pp 460–467

Canny J (2001) Human-centered computing (technical report). Berkeley, CA: University of California, Berkeley

Cicchetti DV (1994) Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychol Assess 6(4):284

Clarkson E, Day JA, Foley JD (2006) An educational digital library for human-centered computing. In: CHI’06 Extended abstracts on human factors in computing systems, pp 646–651

Converse S, Cannon-Bowers JA, Salas E (1993) Shared mental models in expert team decision making. Individual and group decision making: Current Issues 221:221–46

Den Y (2018) F-formation and social context: how spatial orientation of participants’ bodies is organized in the vast field. In: Proceedings of LREC 2018 workshop: language and body in real life (LB-IRL2018) and multimodal corpora (MMC2018) joint workshop, pp 35–39

Feese S, Arnrich B, Troster G, Meyer B, Jonas K (2011) Detecting posture mirroring in social interactions with wearable sensors. In: 2011 15th Annual international symposium on wearable computers. IEEE, pp 119–120

Feese S, Muaremi A, Arnrich B, Troster G, Meyer B, Jonas K (2011) Discriminating individually considerate and authoritarian leaders by speech activity cues. In: 2011 IEEE third international conference on privacy, security, risk and trust and 2011 IEEE third international conference on social computing. IEEE, pp 1460–1465

Feese S, Arnrich B, Tröster G, Meyer B, Jonas K (2012) Quantifying behavioral mimicry by automatic detection of nonverbal cues from body motion. In: 2012 International conference on privacy, security, risk and trust and 2012 international conference on social computing. IEEE, pp 520–525

Forbes RJ, Jackson PR (1980) Non-verbal behaviour and the outcome of selection interviews. J Occup Psychol 53(1):65–72

Frid E, Bresin R, Alborno P, Elblaus L (2016) Interactive sonification of spontaneous movement of children-cross-modal mapping and the perception of body movement qualities through sound. Front Neurosci 10:521

Garrido-Jurado S, Muñoz-Salinas R, Madrid-Cuevas FJ, Marín-Jiménez MJ (2014) Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recognit 47(6):2280–2292

Grossman R, Friedman SB, Kalra S (2017) Teamwork processes and emergent states. In: The Wiley Blackwell handbook of the psychology of team working and collaborative processes, pp 243–269

Hall ET (1966) The hidden dimension, vol 609. Doubleday, Garden City

Heldner M, Edlund J (2010) Pauses, gaps and overlaps in conversations. J Phon 38(4):555–568

Hollingshead AB (1988) Distributed knowledge and transactive processes in decision-making groups

Hollingshead AB (1998) Groupand individual training: the impact of practice on performance. Small Group Res 29(2):254–280

Hollingshead AB (2000) Perceptions of expertise and transactive memory in work relationships. Group Process Intergroup Relations 3(3):257–267

Hollingshead AB, Brandon DP (2003) Potential benefits of communication in transactive memory systems. Hum Commun Res 29(4):607–615

Hung H, Gatica-Perez D (2010) Estimating cohesion in small groups using audio-visual nonverbal behavior. IEEE Trans Multimed 12(6):563–575

Janin A, Baron D, Edwards J, Ellis D, Gelbart D, Morgan N, Peskin B, Pfau T, Shriberg E, Stolcke A et al (2003) The ICSI meeting corpus. In: 2003 IEEE international conference on acoustics, speech, and signal processing, 2003. Proceedings (ICASSP’03), vol 1. IEEE, pp I–I

Jayagopi DB, Gatica-Perez D (2010) Mining group nonverbal conversational patterns using probabilistic topic models. IEEE Trans Multimed 12(8):790–802

Jayagopi DB, Ba S, Odobez J-M, Gatica-Perez D (2008) Predicting two facets of social verticality in meetings from five-minute time slices and nonverbal cues. In: Proceedings of the 10th international conference on Multimodal interfaces. ACM, pp 45–52

Jovanovic N, op den Akker R, Nijholt A (2006) A corpus for studying addressing behaviour in multi-party dialogues. Lang Resour Eval 40:5–23

Kantharaju RB, Langlet C, Barange M, Clavel C, Pelachaud C (2020) Multimodal analysis of cohesion in multi-party interactions. In: LREC, 2020

Kendon A (1990) Conducting interaction: patterns of behavior in focused encounters, vol 7. CUP Archive

Kleanthous S, Michael M, Samaras G, Christodoulou E (2016) Transactive memory in task-driven 3d virtual world teams. In: Proceedings of the 9th Nordic conference on human–computer interaction, pp 1–6

Koo TK, Li MY (2016) A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 15(2):155–163

Kozlowski SWJ, Ilgen DR (2006) Enhancing the effectiveness of work groups and teams. Psycho Sci Public Interest 7(3):77–124

Kubasova U, Murray G, Braley M (2019) Analyzing verbal and nonverbal features for predicting group performance. In: Proc. Interspeech 2019. ISCA, pp 1896–1900

Kwon K, Cho D (2016) How transactive memory systems relate to organizational innovation: the mediating role of developmental leadership. J Knowl Manag

Leo FM, González-Ponce I, García-Calvo T, Sánchez-Oliva D et al (2019) The relationship among cohesion, transactive memory systems, and collective efficacy in professional soccer teams: a multilevel structural equation analysis. Group Dyn Theory Res Pract 23(1):44

Lepri B, Mana N, Cappelletti A, Pianesi F (2009) Automatic prediction of individual performance from “thin slices” of social behavior. In: Proceedings of the 17th ACM international conference on multimedia, pp 733–736

Lewis K (2003) Measuring transactive memory systems in the field: scale development and validation. J Appl Psychol 88(4):587–604

Lewis K, Herndon B (2011) Transactive memory systems: current issues and future research directions. Organ Sci 22(5):1254–1265

Liang DW, Moreland R, Argote L (1995) Group versus individual training and group performance: the mediating role of transactive memory. Personal Soc Psychol Bull 21(4):384–393

Liao J, Jimmieson NL, O’Brien AT, Restubog SLD (2012) Developing transactive memory systems: theoretical contributions from a social identity perspective. Group Organ Manag 37(2):204–240

Marks MA, Mathieu JE, Zaccaro SJ (2001) A temporally based framework and taxonomy of team processes. Acad Manag Rev 26(3):356–376

Martínez-Cagigal V (2018) Sample entropy. https://it.mathworks.com/matlabcentral/fileexchange/69381-sample-entropy. Accessed 18 Mar 2021

Mawalim CO, Okada S, Nakano YI, Unoki M (2023) Personality trait estimation in group discussions using multimodal analysis and speaker embedding. J Multimodal User Interfaces 1–17

Michinov E (2007) Validation de l’échelle de mémoire transactive en langue française et adaptation au contexte académique. Revue Européenne de Psychologie Appliquée/Eur Rev Appl Psychol 57(1):59–68

Michinov E, Olivier-Chiron E, Rusch E, Chiron B (2008) Influence of transactive memory on perceived performance, job satisfaction and identification in anaesthesia teams. Br J Anaesth 100(3):327–332

Miller PW (1988) Nonverbal communication. what research says to the teacher. ERIC

Moreland RL (1999) Transactive memory: learning who knows what in work groups and organizations. In: Thompson L, Messick D, Levine J (eds) Shared cognition in organizations: the management of knowledge

Moreland RL (2010) Are dyads really groups? Small Group Res 41(2):251–267

Moreland RL, Myaskovsky L (2000) Exploring the performance benefits of group training: transactive memory or improved communication? Organ Behav Hum Decis Process 82(1):117–133

Moreland RL, Thompson L (2006) Transactive memory: learning who knows what in work groups and organizations. In: Small groups: key readings. Psychology Press, New York, pp 327–346

Murata K (1994) Intrusive or co-operative? A cross-cultural study of interruption. J Pragmat 21(4):385–400

Nanninga MC, Zhang Y, Lehmann-Willenbrock N, Szlávik Z, Hung H (2017) Estimating verbal expressions of task and social cohesion in meetings by quantifying paralinguistic mimicry. In: Proceedings of the 19th ACM international conference on multimodal interaction, pp 206–215

Oertel C, Funes MKA, Sheikhi S, Odobez J-M, Gustafson J (2014) Who will get the grant? A multimodal corpus for the analysis of conversational behaviours in group interviews. In: Proceedings of the 2014 workshop on understanding and modeling multiparty, multimodal interactions, pp 27–32

Pavitt C (2003) Colloquy: do interacting groups perform better than aggregates of individuals? Why we have to be reductionists about group memory. Hum Commun Res 29(4):592–599

Peltokorpi V, Hood AC (2019) Communication in theory and research on transactive memory systems: a literature review. Top Cognit Sci 11(4):644–667

Rahimpour M (2014) The nature of transactive memory systems in emergency medicine teams based on observations and communication analysis. PhD thesis, Carleton University

Rapp T, Maynard T, Domingo M, Klock E (2021) Team emergent states: what has emerged in the literature over 20 years. Small Group Res 52(1):68–102

Ren Y, Argote L (2011) Transactive memory systems 1985–2010: an integrative framework of key dimensions, antecedents, and consequences. Acad Manag Ann 5(1):189–229

Sanchez-Cortes D, Aran O, Mast MS, Gatica-Perez D (2010) Identifying emergent leadership in small groups using nonverbal communicative cues. In: International conference on multimodal interfaces and the workshop on machine learning for multimodal interaction, pp 1–4

Sanchez-Cortes D, Aran O, Mast MS, Gatica-Perez D (2011) A nonverbal behavior approach to identify emergent leaders in small groups. IEEE Trans Multimed 14(3):816–832

Sanchez-Cortes D, Aran O, Gatica-Perez D (2011) An audio visual corpus for emergent leader analysis. In: Workshop on multimodal corpora for machine learning: taking stock and road mapping the future, ICMI-MLMI. Citeseer

Sanchez-Cortes D, Aran O, Jayagopi DB, Mast MS, Gatica-Perez D (2013) Emergent leaders through looking and speaking: from audio-visual data to multimodal recognition. J Multimodal User Interfaces 7(1–2):39–53

Van Segbroeck M, Van hamme H (2009) Unsupervised learning of time-frequency patches as a noise-robust representation of speech. Speech Commun 51(11):1124–1138

Van hamme H (2008) Hac-models: a novel approach to continuous speech recognition. In: Ninth annual conference of the international speech communication association

Varni G, Mancini M (2020) Movement expressivity analysis: from theory to computation. In: Modelling human motion. Springer, Berlin, pp 213–233

Walocha F, Maman L, Chetouani M, Varni G (2020) Modeling dynamics of task and social cohesion from the group perspective using nonverbal motion capture-based features. In: Companion publication of the 2020 international conference on multimodal interaction, pp 182–190

Wegner DM (1987) Transactive memory: a contemporary analysis of the group mind. In: Theories of group behavior. Springer, Berlin, pp 185–208

Yoo Y, Kanawattanachai P (2001) Developments of transactive memory systems and collective mind in virtual teams. Int J Organ Anal 9(2):187–208

Zhong X, Huang Q, Davison RM, Yang X, Chen H (2012) Empowering teams through social network ties. Int J Inf Manag 32(3):209–220

Zhou Z, Pazos P (2020) Empirical perspectives of transactive memory systems: a meta-analysis. Team Perform Manag Int J