A comprehensive survey on emotion recognition based on electroencephalograph (EEG) signals

Abhang PA, Gawali B, Mehrotra SC (2016) Introduction to EEG-and speech-based emotion recognition. Academic Press Alhagry S, Fahmy AA, El-Khoribi RA (2017) Emotion recognition based on EEG using LSTM recurrent neural network. Emotion 8:355–358 Alickovic E, Babic Z (2015) The effect of denoising on classification of ECG signals. In: 2015 XXV International Conference on Information, Communication and Automation Technologies (ICAT), pp. 1–6 Bajaj V, Taran S, Sengur A (2018) Emotion classification using flexible analytic wavelet transform for electroencephalogram signals. Health Inf Sci Syst 6:12 Bălan O, Moise G, Petrescu L, Moldoveanu A, Leordeanu M, Moldoveanu F (2020) Emotion classification based on biophysical signals and machine learning techniques. Symmetry 12:21 Bhattacharyya A, Pachori RB (2017) A multivariate approach for patient-specific EEG seizure detection using empirical wavelet transform. IEEE Trans Biomed Eng 64:2003–2015 Bhattacharyya A, Singh L, Pachori RB (2018) Fourier–Bessel series expansion based empirical wavelet transform for analysis of non-stationary signals. Digit Signal Process 78:185–196 Bhattacharyya A, Sharma M, Pachori RB, Sircar P, Acharya UR (2018) A novel approach for automated detection of focal EEG signals using empirical wavelet transform. Neural Comput & Applic 29:47–57 Bhattacharyya A, Tripathy RK, Garg L, Pachori RB (2020) A novel multivariate-multiscale approach for computing EEG spectral and temporal complexity for human emotion recognition. IEEE Sensors J 21:3579–3591 Bigirimana AD, Siddique N, Coyle D (2016) A hybrid ICA-wavelet transform for automated artefact removal in EEG-based emotion recognition. In: 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 004429–004434 Chen T, He T, Benesty M, Khotilovich V, Tang Y, Cho H (2015) Xgboost: extreme gradient boosting. R package version 0.4–2, vol. 1, pp. 1–4 Chen J, Zhang P, Mao Z, Huang Y, Jiang D, Zhang Y (2019) Accurate EEG-based emotion recognition on combined features using deep convolutional neural networks. IEEE Access 7:44317–44328 Cheng Y, Liu G-Y, Zhang H-L (2010) The research of EMG signal in emotion recognition based on TS and SBS algorithm. In: The 3rd International Conference on Information Sciences and Interaction Sciences, pp. 363–366 Cheng J, Chen M, Li C, Liu Y, Song R, Liu A et al (2020) Emotion recognition from multi-channel eeg via deep forest. IEEE J Biomed Health Inform 25:453–464 Cui H, Liu A, Zhang X, Chen X, Wang K, Chen X (2020) EEG-based emotion recognition using an end-to-end regional-asymmetric convolutional neural network. Knowl-Based Syst 205:106243 Demir F, Sobahi N, Siuly S, Sengur A (2021) Exploring deep learning features for automatic classification of human emotion using EEG rhythms. IEEE Sensors J 21(13):14923–14930 Ekman P (1992) An argument for basic emotions. Cognit Emot 6:169–200 Ekman P, Friesen WV (2003) Unmasking the face: A guide to recognizing emotions from facial clues, vol 10. Ishk Frantzidis CA, Bratsas C, Papadelis CL, Konstantinidis E, Pappas C, Bamidis PD (2010) Toward emotion aware computing: an integrated approach using multichannel neurophysiological recordings and affective visual stimuli. IEEE Trans Inf Technol Biomed 14:589–597 Gao Z, Li Y, Yang Y, Wang X, Dong N, Chiang H-D (2020) A GPSO-optimized convolutional neural networks for EEG-based emotion recognition. Neurocomputing 380:225–235 García HF, Álvarez MA, Orozco ÁA (2016) Gaussian process dynamical models for multimodal affect recognition. In: 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 850–853 Garg A, Chaturvedi V, Kaur AB, Varshney V, Parashar A (2022) Machine learning model for mapping of music mood and human emotion based on physiological signals. Multimed Tools Appl:1–41 Gicić A, Subasi A (2019) Credit scoring for a microcredit data set using the synthetic minority oversampling technique and ensemble classifiers. Expert Syst 36:e12363 Grühn D, Scheibe S (2008) Age-related differences in valence and arousal ratings of pictures from the international affective picture system (IAPS): do ratings become more extreme with age? Behav Res Methods 40:512–521 Gupta V, Chopda MD, Pachori RB (2018) Cross-subject emotion recognition using flexible analytic wavelet transform from EEG signals. IEEE Sensors J 19:2266–2274 Hill NJ, Lal TN, Schröder M, Hinterberger T, Widman G, Elger CE et al (2006) Classifying event-related desynchronization in EEG, ECoG and MEG signals. In: Joint Pattern Recognition Symposium, pp. 404–413 Hjorth B (1970) EEG analysis based on time domain properties. Electroencephalogr Clin Neurophysiol 29:306–310 Huang NE, Shen Z, Long SR, Wu MC, Shih HH, Zheng Q, Yen NC, Tung CC, Liu HH (1998) The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis, Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences. 454:903–995 Huang J, Xu X, Zhang T (2017) Emotion classification using deep neural networks and emotional patches. In: 2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 958–962 Jenke R, Peer A, Buss M (2014) Feature extraction and selection for emotion recognition from EEG. IEEE Trans Affect Comput 5:327–339 Jerritta S, Murugappan M, Wan K, Yaacob S (2014) Emotion recognition from facial EMG signals using higher order statistics and principal component analysis. J Chin Inst Eng 37:385–394 Jirayucharoensak S, Pan-Ngum S, Israsena P (2014) EEG-based emotion recognition using deep learning network with principal component based covariate shift adaptation. Sci World J 2014:1–10 Kamble KS, Sengupta J (2021) Ensemble machine learning-based affective computing for emotion recognition using dual-decomposed EEG signals. IEEE Sensors J 22:2496–2507 Katsigiannis S, Ramzan N (2017) DREAMER: a database for emotion recognition through EEG and ECG signals from wireless low-cost off-the-shelf devices. IEEE J Biomed Health Inform 22:98–107 Khare SK, Bajaj V (2020) Constrained based tunable Q wavelet transform for efficient decomposition of EEG signals. Appl Acoust 163:107234 Khare SK, Bajaj V (2020) An evolutionary optimized variational mode decomposition for emotion recognition. IEEE Sensors J 21:2035–2042 Khare SK, Bajaj V (2020) Time-frequency representation and convolutional neural network-based emotion recognition. IEEE Trans Neural Netw Learn Syst 32:2901–2909 Khosrowabadi R, bin Abdul Rahman AW (2010) Classification of EEG correlates on emotion using features from Gaussian mixtures of EEG spectrogram. In: Proceeding of the 3rd International Conference on Information and Communication Technology for the Moslem World (ICT4M) 2010, pp. E102-E107 Khosrowabadi R, Quek HC, Wahab A, Ang KK (2010) EEG-based emotion recognition using self-organizing map for boundary detection. In: 2010 20th International Conference on Pattern Recognition, pp. 4242–4245 Kim J, André E (2008) Emotion recognition based on physiological changes in music listening. IEEE Trans Pattern Anal Mach Intell 30:2067–2083 Koelstra S, Patras I (2013) Fusion of facial expressions and EEG for implicit affective tagging. Image Vis Comput 31:164–174 Koelstra S, Muhl C, Soleymani M, Lee J-S, Yazdani A, Ebrahimi T et al (2011) Deap: a database for emotion analysis; using physiological signals. IEEE Trans Affect Comput 3:18–31 Kolodyazhniy V, Kreibig SD, Gross JJ, Roth WT, Wilhelm FH (2011) An affective computing approach to physiological emotion specificity: toward subject-independent and stimulus-independent classification of film-induced emotions. Psychophysiology 48:908–922 Krishna AH, Sri AB, Priyanka KYVS, Taran S, Bajaj V (2018) Emotion classification using EEG signals based on tunable-Q wavelet transform. IET Sci Meas Technol 13:375–380 Kroupi E, Yazdani A, Ebrahimi T (2011) EEG correlates of different emotional states elicited during watching music videos. In: International Conference on Affective Computing and Intelligent Interaction: Fourth International Conference, ACII, Memphis, TN, USA, October 9–12, 2011, Proceedings, Part II pp. 457–466. Springer Berlin, Heidelberg Lakhan P, Banluesombatkul N, Changniam V, Dhithijaiyratn R, Leelaarporn P, Boonchieng E, Hompoonsup S, Wilaiprasitporn T (2019) Consumer grade brain sensing for emotion recognition. IEEE Sensors J 19(21):9896–9907 Lan Z, Sourina O, Wang L, Liu Y (2016) Real-time EEG-based emotion monitoring using stable features. Vis Comput 32:347–358 Lemm S, Blankertz B, Curio G, Muller K-R (2005) Spatio-spectral filters for improving the classification of single trial EEG. IEEE Trans Biomed Eng 52:1541–1548 Li H, Qing C, Xu X, Zhang T (2017) A novel DE-PCCM feature for EEG-based emotion recognition. In: 2017 International Conference on Security, Pattern Analysis, and Cybernetics (SPAC), pp. 389–393 Li T-H, Liu W, Zheng W-L, Lu B-L (2019) Classification of five emotions from EEG and eye movement signals: Discrimination ability and stability over time. In: 2019 9th International IEEE/EMBS Conference on Neural Engineering (NER), pp. 607–610 Li R, Ren C, Zhang X, Hu B (2022) A novel ensemble learning method using multiple objective particle swarm optimization for subject-independent EEG-based emotion recognition. Comput Biol Med 140:105080 Liu Y, Sourina O (2013) Real-time fractal-based valence level recognition from EEG. In: Transactions on computational science XVIII. ed: Springer, pp 101–120 Liu Y, Sourina O (2013) EEG databases for emotion recognition. In: 2013 international conference on cyberworlds, pp. 302–309 Liu W, Zheng W-L, Lu B-L (2016) Emotion recognition using multimodal deep learning. In: International conference on neural information processing, pp. 521–529 Liu Y-J, Yu M, Zhao G, Song J, Ge Y, Shi Y (2017) Real-time movie-induced discrete emotion recognition from EEG signals. IEEE Trans Affect Comput 9:550–562 Liu J, Su Y, Liu Y (2017) Multi-modal emotion recognition with temporal-band attention based on LSTM-RNN. In: Pacific Rim Conference on Multimedia, pp. 194–204 Liu Y, Ding Y, Li C, Cheng J, Song R, Wan F, Chen X (2020) Multi-channel EEG-based emotion recognition via a multi-level features guided capsule network. Comput Biol Med 123:103927 Liu W, Qiu J-L, Zheng W-L, Lu B-L (2021) Comparing recognition performance and robustness of multimodal deep learning models for multimodal emotion recognition. IEEE Trans Cogn Develop Syst 14:715–729 Maheshwari D, Ghosh S, Tripathy R, Sharma M, Acharya UR (2021) Automated accurate emotion recognition system using rhythm-specific deep convolutional neural network technique with multi-channel EEG signals. Comput Biol Med 134:104428 Mahmoud S, Elamien M, Zaatar A, ElTinai M (2014) Low noise low power instrumentation amplifier for biomedical application Mehrabian A (1996) Pleasure-arousal-dominance: a general framework for describing and measuring individual differences in temperament. Curr Psychol 14:261–292 Murugappan M, Murugappan S (2013) Human emotion recognition through short time Electroencephalogram (EEG) signals using Fast Fourier Transform (FFT). In: 2013 IEEE 9th International Colloquium on Signal Processing and its Applications, pp. 289–294 Murugappan M, Ramachandran N, Sazali Y (2010) Classification of human emotion from EEG using discrete wavelet transform. J Biomed Sci Eng 3:390–396 Murugappan M, Nagarajan R, Yaacob S (2011) Combining spatial filtering and wavelet transform for classifying human emotions using EEG signals. J Med Biol Eng 31:45–51 Nakisa B, Rastgoo MN, Tjondronegoro D, Chandran V (2018) Evolutionary computation algorithms for feature selection of EEG-based emotion recognition using mobile sensors. Expert Syst Appl 93:143–155 Nie D, Wang X-W, Shi L-C, Lu B-L (2011) EEG-based emotion recognition during watching movies. In: 2011 5th International IEEE/EMBS Conference on Neural Engineering, pp. 667–670 Onton JA, Makeig S (2009) High-frequency broadband modulation of electroencephalographic spectra. Front Hum Neurosci 3:61 Paithane A, Bormane D (2014) Analysis of nonlinear and non-stationary signal to extract the features using Hilbert Huang transform. In: 2014 IEEE International Conference on Computational Intelligence and Computing Research, pp. 1–4 Pandeya YR, Lee J (2021) Deep learning-based late fusion of multimodal information for emotion classification of music video. Multimed Tools Appl 80:2887–2905 Patel R, Janawadkar MP, Sengottuvel S, Gireesan K, Radhakrishnan TS (2016) Suppression of eye-blink associated artifact using single channel EEG data by combining cross-correlation with empirical mode decomposition. IEEE Sensors J 16:6947–6954 Petrantonakis PC, Hadjileontiadis LJ (2010) Emotion recognition from brain signals using hybrid adaptive filtering and higher order crossings analysis. IEEE Trans Affect Comput 1:81–97 Plutchik R, Kellerman H (1980) Theories of emotion. Academic Press Rached TS, Perkusich A (2013) Emotion recognition based on brain-computer interface systems. Brain-computer interface systems-Recent progress and future prospects, pp. 253–270 Rázuri JG, Sundgren D, Rahmani R, Cardenas AM (2013) Automatic emotion recognition through facial expression analysis in merged images based on an artificial neural network. In: 2013 12th Mexican International Conference on Artificial Intelligence, pp. 85–96 Reuderink B, Mühl C, Poel M (2013) Valence, arousal and dominance in the EEG during game play. Int J Auton Adapt Commun Syst 6:45–62 Salama ES, El-Khoribi RA, Shoman ME, Shalaby MAW (2018) EEG-based emotion recognition using 3D convolutional neural networks. Int J Adv Comput Sci Appl 9:329–337 Salari S, Ansarian A, Atrianfar H (2018) Robust emotion classification using neural network models. In: 2018 6th Iranian Joint Congress on Fuzzy and Intelligent Systems (CFIS), pp. 190–194 Sangnark S, Autthasan P, Ponglertnapakorn P, Chalekarn P, Sudhawiyangkul T, Trakulruangroj M, Songsermsawad S, Assabumrungrat R, Amplod S, Ounjai K, Wilaiprasitporn T (2021) Revealing preference in popular music through familiarity and brain response. IEEE Sensors J 21:14931–14940 Sharma LD, Bhattacharyya A (2021) A computerized approach for automatic human emotion recognition using sliding mode singular Spectrum analysis. IEEE Sensors J 21:26931–26940 Shin D, Shin D, Shin D (2017) Development of emotion recognition interface using complex EEG/ECG bio-signal for interactive contents. Multimed Tools Appl 76:11449–11470 Shirke B, Wong J, Libut JC, George K, Oh SJ (2020) Brain-IoT based Emotion Recognition System. In: 2020 10th Annual Computing and Communication Workshop and Conference (CCWC), pp. 0991–0995 Siddharth S, Jung T-P, Sejnowski TJ (2019) Utilizing deep learning towards multi-modal bio-sensing and vision-based affective computing. IEEE Trans Affect Comput Siuly S, Khare SK, Bajaj V, Wang H, Zhang Y (2020) A computerized method for automatic detection of schizophrenia using EEG signals. IEEE Trans Neural Syst Rehabilitation Eng 28:2390–2400 Soleymani M, Lichtenauer J, Pun T, Pantic M (2011) A multimodal database for affect recognition and implicit tagging. IEEE Trans Affect Comput 3:42–55 Soleymani M, Pantic M, Pun T (2011) Multimodal emotion recognition in response to videos. IEEE Trans Affect Comput 3:211–223 Song T, Zheng W, Song P, Cui Z (2018) EEG emotion recognition using dynamical graph convolutional neural networks. IEEE Trans Affect Comput 11:532–541 Song T, Zheng W, Song P, Cui Z (2018) EEG emotion recognition using dynamical graph convolutional neural networks. IEEE Trans Affect Comput Song T, Liu S, Zheng W, Zong Y, Cui Z, Li Y, Zhou X (2021) Variational instance-adaptive graph for EEG emotion recognition. IEEE Trans Affect Comput Subasi A, Tuncer T, Dogan S, Tanko D, Sakoglu U (2021) EEG-based emotion recognition using tunable Q wavelet transform and rotation forest ensemble classifier. Biomed Signal Process Control 68:102648 Tao W, Li C, Song R, Cheng J, Liu Y, Wan F, Chen X (2020) EEG-based emotion recognition via channel-wise attention and self attention. IEEE Trans Affect Comput Taran S, Bajaj V (2019) Motor imagery tasks-based EEG signals classification using tunable-Q wavelet transform. Neural Comput & Applic 31:6925–6932 Taran S, Bajaj V, Sharma D, Siuly S, Sengur A (2018) Features based on analytic IMF for classifying motor imagery EEG signals in BCI applications. Measurement 116:68–76 Torres E, Hernández-Álvarez M, Yoo SG (2020) EEG-Based BCI Emotion Recognition Using the Stock-Emotion Dataset. In: The International Conference on Advances in Emerging Trends and Technologies, pp. 226–235 Tuncer T, Dogan S, Subasi A (2021) A new fractal pattern feature generation function based emotion recognition method using EEG. Chaos Solit Fractals 144:110671 Valenza G, Lanata A, Scilingo EP (2012) The role of nonlinear dynamics in affective valence and arousal recognition. IEEE Trans Affect Comput 3:237–249 Verma GK, Tiwary US (2014) Multimodal fusion framework: a multiresolution approach for emotion classification and recognition from physiological signals. NeuroImage 102:162–172 Wang Y, Mo J (2013) Emotion feature selection from physiological signals using tabu search. In: 2013 25th Chinese Control and Decision Conference (CCDC), pp. 3148–3150 Wang J, Wang M (2021) Review of the emotional feature extraction and classification using EEG signals. Cognitive Robotics 1:29–40 Wang X-W, Nie D, Lu B-L (2011) EEG-based emotion recognition using frequency domain features and support vector machines. In: International conference on neural information processing, pp. 734–743 Wang X-W, Nie D, Lu B-L (2014) Emotional state classification from EEG data using machine learning approach. Neurocomputing 129:94–106 Wilson GF, Russell CA (2003) Real-time assessment of mental workload using psychophysiological measures and artificial neural networks. Hum Factors 45:635–644 Wu M, Wan T, Wan X, Du Y, She J (2017) Fast, accurate localization of epileptic seizure onset zones based on detection of high-frequency oscillations using improved wavelet transform and matching pursuit methods. Neural Comput 29:194–219 Yang B, Han X, Tang J (2017) Three class emotions recognition based on deep learning using staked autoencoder. In: 2017 10th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), pp. 1–5 Yohanes RE, Ser W, Huang G-B (2012) Discrete Wavelet Transform coefficients for emotion recognition from EEG signals. In: 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 2251–2254 Zhang J, Chen M, Hu S, Cao Y, Kozma R (2016) PNN for EEG-based Emotion Recognition. In: 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 002319–002323 Zhao S, Li Z, Cui R, Kang Y, Sun F, Song R (2017) Brain–machine interfacing-based teleoperation of multiple coordinated Mobile robots. IEEE Trans Ind Electron 64:5161–5170 Zheng W (2016) Multichannel EEG-based emotion recognition via group sparse canonical correlation analysis. IEEE Trans Cogn Dev Syst 9:281–290 Zheng W-L, Lu B-L (2015) Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks. IEEE Trans Auton Ment Dev 7:162–175 Zheng W-L, Lu B-L (2017) A multimodal approach to estimating vigilance using EEG and forehead EOG. J Neural Eng 14:026017 Zheng W-L, Zhu J-Y, Peng Y, Lu B-L (2014) EEG-based emotion classification using deep belief networks. In: 2014 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6 Zheng W-L, Zhu J-Y, Lu B-L (2017) Identifying stable patterns over time for emotion recognition from EEG. IEEE Trans Affect Comput