Gamification of an n-back working memory task – Is it worth the effort? An EEG and eye-tracking study

Alemdag, 2018, A systematic review of eye tracking research on multimedia learning, Computers & Education, 125, 413, 10.1016/j.compedu.2018.06.023 Antonenko, 2010, Using electroencephalography to measure cognitive load, Educational Psychology Review, 22, 425, 10.1007/s10648-010-9130-y Baddeley, 1996, Exploring the central executive, The Quarterly Journal of Experimental Psychology Section A, 49, 5, 10.1080/713755608 Baddeley, 2012, Working memory: Theories, models, and controversies, Annual Review of Psychology, 63, 1, 10.1146/annurev-psych-120710-100422 Bastos, 2015, Visual areas exert feedforward and feedback influences through distinct frequency channels, Neuron, 85, 390, 10.1016/j.neuron.2014.12.018 Bernecker, 2021, No pain, no gain? Investigating motivational mechanisms of game elements in cognitive tasks, Computers in Human Behavior, 114, 10.1016/j.chb.2020.106542 Bledowski, 2010, Basic operations in working memory: Contributions from functional imaging studies, Behavioural Brain Research, 214, 172, 10.1016/j.bbr.2010.05.041 Brandstätter, 2022, Persistence and disengagement in personal goal pursuit, Annual Review of Psychology, 73, 271, 10.1146/annurev-psych-020821-110710 Brehm, 1989, The intensity of motivation, Annual Review of Psychology, 40, 109, 10.1146/annurev.ps.40.020189.000545 Brom, 2018, How effective is emotional design? A meta-analysis on facial anthropomorphisms and pleasant colors during multimedia learning, Educational Research Review, 25, 100, 10.1016/j.edurev.2018.09.004 Cabestrero, 2009, Pupillary dilation as an index of task demands, Perceptual and Motor Skills, 109, 664, 10.2466/pms.109.3.664-678 Chen, 2008, Sub-processes of working memory in the n-back task: An investigation using ERPs, Clinical Neurophysiology, 119, 1546, 10.1016/j.clinph.2008.03.003 Cohen, 2014 Cowan, 2010, The magical mystery four: How is working memory capacity limited, and why?, Current Directions in Psychological Science, 19, 51, 10.1177/0963721409359277 Cowan, 2012, Models of verbal working memory capacity: What does it take to make them work?, Psychological Review, 119, 480, 10.1037/a0027791 Delorme, 2004, EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis, Journal of Neuroscience Methods, 134, 9, 10.1016/j.jneumeth.2003.10.009 Delorme, 2007, Enhanced detection of artifacts in EEG data using higher-order statistics and independent component analysis, NeuroImage, 34, 1443, 10.1016/j.neuroimage.2006.11.004 DeLosAngeles, 2016, Electroencephalographic correlates of states of concentrative meditation, International Journal of Psychophysiology, 110, 27, 10.1016/j.ijpsycho.2016.09.020 Diamond, 2013, Executive functions, Annual Review of Psychology, 64, 135, 10.1146/annurev-psych-113011-143750 Dimigen, 2011, Coregistration of eye movements and EEG in natural reading: Analyses and review, Journal of Experimental Psychology: General, 140, 552, 10.1037/a0023885 Duncan, 2009, Event-related potentials in clinical research: Guidelines for eliciting, recording, and quantifying mismatch negativity, P300, and N400, Clinical Neurophysiology, 120, 1883, 10.1016/j.clinph.2009.07.045 Van Gerven, 2004, Memory load and the cognitive pupillary response in aging, Psychophysiology, 41, 167, 10.1111/j.1469-8986.2003.00148.x Gevins, 2000, Neurophysiological measures of working memory and individual differences in cognitive ability and cognitive style, Cerebral Cortex, 10, 829, 10.1093/cercor/10.9.829 Gevins, 1997, High-resolution EEG mapping of cortical activation related to working memory: Effects of task difficulty, type of processing, and practice, Cerebral Cortex, 7, 374, 10.1093/cercor/7.4.374 van Gog, 2010, Eye tracking as a tool to study and enhance multimedia learning, Learning and Instruction, 20, 95, 10.1016/j.learninstruc.2009.02.009 Haciahmet, 2021, Target amplification and distractor inhibition: Theta oscillatory dynamics of selective attention in a flanker task, Cognitive, Affective & Behavioral Neuroscience, 21, 355, 10.3758/s13415-021-00876-y Hamari, J., Koivisto, J., & Sarsa, H. (2014). Does gamification work? A literature review of empirical studies on gamification. 2014 47th Hawaii International Conference on System Sciences. https://doi.org/10/gfj8gf. Hanslmayr, 2009, Brain oscillations dissociate between semantic and nonsemantic encoding of episodic memories, Cerebral Cortex, 19, 1631, 10.1093/cercor/bhn197 Harris, 2017, Distinct roles of theta and alpha oscillations in the involuntary capture of goal-directed attention, NeuroImage, 152, 171, 10.1016/j.neuroimage.2017.03.008 Hart, 1988, Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research, Advances in Psychology, 52, 139, 10.1016/S0166-4115(08)62386-9 Heidig, 2015, Emotional design in multimedia learning: Differentiation on relevant design features and their effects on emotions and learning, Computers in Human Behavior, 44, 81, 10.1016/j.chb.2014.11.009 Janke, 2014, Deutsche Version der Positive and Negative Affect Schedule (PANAS), Zusammenstellung sozialwissenschaftlicher Items und Skalen (ZIS) Jasper, 1958, The ten-twenty electrode system of the International Federation, Electroencephalogr Clin Neurophysiol, 10, 370 Jung, 2000, Removing electroencephalographic artifacts by blind source separation, Psychophysiology, 37, 163, 10.1111/1469-8986.3720163 Just, 1976, Eye fixations and cognitive processes, Cognitive Psychology, 8, 441, 10.1016/0010-0285(76)90015-3 Just, 1980, A theory of reading: From eye fixations to comprehension, Psychological Review, 87, 329, 10.1037/0033-295X.87.4.329 Kiili, 2012, The design principles for flow experience in educational games, Procedia Computer Science, 15, 78, 10.1016/j.procs.2012.10.060 Klimesch, 1999, EEG alpha and theta oscillations reflect cognitive and memory performance: A review and analysis, Brain Research Reviews, 29, 169, 10.1016/S0165-0173(98)00056-3 Lakens, 2022, Sample Size Justification, Collabra: Psychology, 8, 33267, 10.1525/collabra.33267 Landers, 2017, How to use game elements to enhance learning: Applications of the theory of gamified learning, Serious Games and Edutainment Applications, 457, 10.1007/978-3-319-51645-5_21 Laugwitz, 2008, Construction and evaluation of a user experience questionnaire, Lecture Notes in Computer Science, 63, 10.1007/978-3-540-89350-9_6 Lawrence, M.A. (2016). Package ‘ez’ (4.4–0). 〈https://CRAN.R-project.org/package=ez〉. Lenth, R.V. (2021). Package ‘emmeans’ (1.7.0). 〈https://CRAN.R-project.org/package=emmeans〉. Lu, 2023, Changes in pupil dilation and P300 amplitude indicate the possible involvement of the locus coeruleus-norepinephrine (LC-NE) system in psychological flow, Scientific Reports, 13, 1908, 10.1038/s41598-023-28781-z Van der Lubbe, 2023, Lateralized connectivity in the alpha band between parietal and occipital sources when spatial attention is externally and internally directed, Frontiers in Cognition, 2, 10.3389/fcogn.2023.1145854 Mayer, 2009 Mayer, 2014, Principles for Reducing Extraneous Processing in Multimedia Learning: Coherence, Signaling, Redundancy, Spatial Contiguity, and Temporal Contiguity Principles, 279 Miyake, 2012, The nature and organization of individual differences in executive functions: Four general conclusions, Current Directions in Psychological Science, 21, 8, 10.1177/0963721411429458 Moreno, 2010, Cognitive load theory: Historical development and relation to other theories, 9 Ninaus, 2020, To add or not to add game elements? Exploring the effects of different cognitive task designs using eye tracking, IEEE Transactions on Learning Technologies, 13, 847, 10.1109/TLT.2020.3031644 Ninaus, 2015, Game elements improve performance in a working memory training task, International Journal of Serious Games, 2, 10.17083/ijsg.v2i1.60 Oldfield, 1971, The assessment and analysis of handedness: The Edinburgh inventory, Neuropsychologia, 9, 97, 10.1016/0028-3932(71)90067-4 Owen, 2005, N-back working memory paradigm: A meta-analysis of normative functional neuroimaging studies, Human Brain Mapping, 25, 46, 10.1002/hbm.20131 Palomäki, 2012, Brain oscillatory 4–35Hz EEG responses during an n-back task with complex visual stimuli, Neuroscience Letters, 516, 141, 10.1016/j.neulet.2012.03.076 Pang, W.M., Wei Liew, T., & Leow, M.C. (2021). Emotional design for educational multimedia: A mini-review. 2021 14th International Conference on Human System Interaction (HSI) , 1–8. 〈https://doi.org/10.1109/HSI52170.2021.9538667〉. Parong, 2019, Cognitive consequences of playing brain‐training games in immersive virtual reality, Applied Cognitive Psychology, 34, 29, 10.1002/acp.3582 Peirce, 2019, PsychoPy2: Experiments in behavior made easy, Behavior Research Methods, 51, 195, 10.3758/s13428-018-01193-y Pesonen, 2007, Brain oscillatory 4–30 Hz responses during a visual n-back memory task with varying memory load, Brain Research, 1138, 171, 10.1016/j.brainres.2006.12.076 Pfister, 2016, schoRsch: An R package for analyzing and reporting factorial experiments, The Quantitative Methods for Psychology, 12, 147, 10.20982/tqmp.12.2.p147 Pfurtscheller, 1999, Event-related EEG/MEG synchronization and desynchronization: Basic principles, Clinical Neurophysiology, 110, 1842, 10.1016/S1388-2457(99)00141-8 Plass, 2015, Foundations of game-based learning, Educational Psychologist, 50, 258, 10.1080/00461520.2015.1122533 Plass, 2020, Emotional design for digital games for learning: The effect of expression, color, shape, and dimensionality on the affective quality of game characters, Learning and Instruction, 70, 10.1016/j.learninstruc.2019.01.005 Plass, 2016, Emotional design in digital media for learning, 131 R Core Team. (2020). R: a language and environment for statistical computing. R Foundation for Statistical Computing. 〈https://www.R-project.org/〉. Redick, 2013, Complex span and n-back measures of working memory: A meta-analysis, Psychonomic Bulletin & Review, 20, 1102, 10.3758/s13423-013-0453-9 Rey, 2012, A review of research and a meta-analysis of the seductive detail effect, Educational Research Review, 7, 216, 10.1016/j.edurev.2012.05.003 Rheinberg, F., Vollmeyer, R., & Engeser, S. (2003). Flow short scale. In PsycTESTS Dataset. American Psychological Association (APA). 〈https://doi.org/10.1037/t47787–000〉. Richter, 2016, Three Decades of Research on Motivational Intensity Theory, Vol. 3, 149 Robinson, D., R., & Bellotti, V. (2013). A preliminary taxonomy of gamification elements for varying anticipated commitment. Proc. ACM CHI 2013 Workshop on Designing Gamification: Creating Gameful and Playful Experiences. Sailer, 2017, How gamification motivates: An experimental study of the effects of specific game design elements on psychological need satisfaction, Computers in Human Behavior, 69, 371, 10.1016/j.chb.2016.12.033 Sanchez, D.R., & Lee, C.A. (2022). Understanding the challenges of game-based training: Recommendations for moving research forward in game-based learning. In O. Bernardes, V. Amorim, & A.C. Moreira (Eds.), Handbook of Research on the Influence and Effectiveness of Gamification in Education (pp. 541–578). IGI Global. 〈https://doi.org/10.4018/978–1-6684–4287-6.ch027〉. Sauseng, 2010, Control mechanisms in working memory: A possible function of EEG theta oscillations, Neuroscience & Biobehavioral Reviews, 34, 1015, 10.1016/j.neubiorev.2009.12.006 Sauseng, 2009, Brain oscillatory substrates of visual short-term memory capacity, Current Biology, 19, 1846, 10.1016/j.cub.2009.08.062 Sauseng, 2005, Fronto-parietal EEG coherence in theta and upper alpha reflect central executive functions of working memory, International Journal of Psychophysiology, 57, 97, 10.1016/j.ijpsycho.2005.03.018 Scharinger, 2015, Pupil dilation and EEG alpha frequency band power reveal load on executive functions for link-selection processes during text reading, PLOS ONE, 10, 10.1371/journal.pone.0130608 Scharinger, C., Prislan, L., Bernecker, K., & Ninaus, M. (2020). “N-back gamified” (AsPredicted #45524). 〈https://aspredicted.org/blind.php?x=QNN_GEE〉. Scharinger, 2020, Using eye-tracking and EEG to study the mental processing demands during learning of text-picture combinations, International Journal of Psychophysiology, 158, 201, 10.1016/j.ijpsycho.2020.09.014 Scharinger, 2015, When flanker meets the n‐back: What EEG and pupil dilation data reveal about the interplay between the two central‐executive working memory functions inhibition and updating, Psychophysiology, 52, 1293, 10.1111/psyp.12500 Scharinger, 2017, Comparison of the working memory load in n-back and working memory span tasks by means of eeg frequency band power and P300 amplitude, Frontiers in Human Neuroscience, 11, 10.3389/fnhum.2017.00006 Schrepp, 2017, Construction of a benchmark for the user experience questionnaire (UEQ), International Journal of Interactive Multimedia and Artificial Intelligence, 4, 40, 10.9781/ijimai.2017.445 Siegle, 2003, Use of concurrent pupil dilation assessment to inform interpretation and analysis of fMRI data, NeuroImage, 20, 114, 10.1016/S1053-8119(03)00298-2 Smit, 2004, Mental effort affects vigilance enduringly: After-effects in EEG and behavior, International Journal of Psychophysiology, 53, 239, 10.1016/j.ijpsycho.2004.04.005 St Clair-Thompson, 2006, Executive functions and achievements in school: Shifting, updating, inhibition, and working memory, Quarterly Journal of Experimental Psychology, 59, 745, 10.1080/17470210500162854 Sundararajan, 2020, Keep it coherent: A meta-analysis of the seductive details effect, Educational Psychology Review, 32, 707, 10.1007/s10648-020-09522-4 Sweller, 1998, Cognitive architecture and instructional design, Educational Psychology Review, 10, 251, 10.1023/A:1022193728205 Um, 2012, Emotional design in multimedia learning, Journal of Educational Psychology, 104, 485, 10.1037/a0026609 Vermeir, 2020, The effects of gamification on computerized cognitive training: Systematic review and meta-analysis, JMIR Serious Games, 8, 10.2196/18644 van der Wel, 2018, Pupil dilation as an index of effort in cognitive control tasks: A review, Psychonomic Bulletin & Review, 25, 2005, 10.3758/s13423-018-1432-y Woolley, 2015, The experience matters more than you think: People value intrinsic incentives more inside than outside an activity, Journal of Personality and Social Psychology, 109, 968, 10.1037/pspa0000035 Woolley, 2016, For the fun of it: Harnessing immediate rewards to increase persistence in long-term goals, Journal of Consumer Research, 42, 952, 10.1093/jcr/ucv098 Yang, 2017, Single neuron activity and theta modulation in the posterior parietal cortex in a visuospatial attention task, Hippocampus, 27, 263, 10.1002/hipo.22691