Efficacy of novel attention-based gated recurrent units transformer for depression detection using electroencephalogram signals
Tóm tắt
Depression is a global challenge causing psychological and intellectual problems that require efficient diagnosis. Electroencephalogram (EEG) signals represent the functional state of the human brain and can help build an accurate and viable technique for the early prediction and treatment of depression. An attention-based gated recurrent units transformer (AttGRUT) time-series model is proposed to efficiently identify EEG perturbations in depressive patients. Statistical, spectral and wavelet features were first extracted from the 60-channel EEG signal data. Then, two feature selection techniques, recursive feature elimination and the Boruta algorithm, both with Shapley additive explanations, were utilised for selecting essential features. The proposed model outperformed the two baseline and two hybrid time-series models—long short-term memory (LSTM), gated recurrent units (GRU), convolutional neural network-LSTM (CNN-LSTM), and CNN-GRU—achieving an accuracy of up to 98.67%. Feature selection considerably increased the performance across all time-series models. Based on the obtained results, novel feature selection greatly affected the results of the baseline and hybrid time-series models. The proposed AttGRUT can be implemented and tested in other domains by using different modalities for prediction.
Tài liệu tham khảo
Benazzi F. Various forms of depression. Dialogues Clin Neurosci. 2022;8:151–61.
Paykel ES. Basic concepts of depression. Dialogues Clin Neurosci. 2022;10:279–89.
Kamenov K, Caballero FF, Miret M, Leonardi M, Sainio P, Tobiasz-Adamczyk B, Haro JM, Chatterji S, Ayuso-Mateos JL, Cabello M. Which are the most burdensome functioning areas in depression? A cross-national study. Front Psychol. 2016;7:1342.
Cacheda F, Fernandez D, Novoa FJ, Carneiro V. Early detection of depression: social network analysis and random forest techniques. J Med Internet Res. 2019;21(6): e12554.
Ayano G, Demelash S, Haile K, Tulu M, Assefa D, Tesfaye A, Haile K, Solomon M, Chaka A, Tsegay L. Misdiagnosis, detection rate, and associated factors of severe psychiatric disorders in specialized psychiatry centers in Ethiopia. Ann Gen Psychiatry. 2021;20(1):1.
Prabhakar SK, Rajaguru H, Lee SW. A framework for schizophrenia EEG signal classification with nature inspired optimization algorithms. IEEE Access. 2020;8:39875–97.
Usman SM, Khalid S, Bashir Z. Epileptic seizure prediction using scalp electroencephalogram signals. Biocybern Biomed Eng. 2021;41(1):211–20.
Sánchez-Reyes LM, Rodríguez-Reséndiz J, Avecilla-Ramírez GN, García-Gomar ML, Robles-Ocampo JB. Impact of EEG parameters detecting dementia diseases: a systematic review. IEEE Access. 2021;9:78060.
Cannon J, O’Brien AM, Bungert L, Sinha P. Prediction in autism spectrum disorder: a systematic review of empirical evidence. Autism Res. 2021;14(4):604–30.
Sharma G, Parashar A, Joshi AM. DepHNN: a novel hybrid neural network for electroencephalogram (EEG)-based screening of depression. Biomed Signal Process Control. 2021;66: 102393.
Liu GD, Li YC, Zhang W, Zhang L. A brief review of artificial intelligence applications and algorithms for psychiatric disorders. Engineering. 2020;6(4):462–7.
Chen X, Li C, Liu A, McKeown MJ, Qian R, Wang ZJ. Toward open-world electroencephalogram decoding via deep learning: a comprehensive survey. IEEE Signal Process Mag. 2022;39(2):117–34.
Safayari A, Bolhasani H. Depression diagnosis by deep learning using EEG signals: a systematic review. Med Novel Technol Devices. 2021;12: 100102.
Khosla A, Khandnor P, Chand T. Automated diagnosis of depression from EEG signals using traditional and deep learning approaches: a comparative analysis. Biocybern Biomed Eng. 2021;42:108–42.
Rivera MJ, Teruel MA, Maté A, Trujillo J. Diagnosis and prognosis of mental disorders by means of EEG and deep learning: a systematic mapping study. Artif Intell Rev. 2021;2021:1–43.
Roy Y, Banville H, Albuquerque I, Gramfort A, Falk TH, Faubert J. Deep learning-based electroencephalography analysis: a systematic review. J Neural Eng. 2019;16(5): 051001.
Hosseini MP, Hosseini A, Ahi K. A review on machine learning for EEG signal processing in bioengineering. IEEE Rev Biomed Eng. 2020;14:204–18.
Lei Y, Belkacem AN, Wang X, Sha S, Wang C, Chen C. A convolutional neural network-based diagnostic method using resting-state electroencephalograph signals for major depressive and bipolar disorders. Biomed Signal Process Control. 2022;72: 103370.
Song X, Yan D, Zhao L, Yang L. LSDD-EEGNet: an efficient end-to-end framework for EEG-based depression detection. Biomed Signal Process Control. 2022;75: 103612.
Aydemir E, Tuncer T, Dogan S, Gururajan R, Acharya UR. Automated major depressive disorder detection using melamine pattern with EEG signals. Appl Intell. 2021;51(9):6449–66.
Seal A, Bajpai R, Agnihotri J, Yazidi A, Herrera-Viedma E, Krejcar O. DeprNet: a deep convolution neural network framework for detecting depression using EEG. IEEE Trans Instrum Meas. 2021;70:1–3.
Čukić M, Stokić M, Simić S, Pokrajac D. The successful discrimination of depression from EEG could be attributed to proper feature extraction and not to a particular classification method. Cogn Neurodyn. 2020;14(4):443–55.
Zhao L, Yang L, Li B, Su Z, Liu C. Frontal alpha EEG asymmetry variation of depression patients assessed by entropy measures and Lemple-Ziv complexity. J Med Biol Eng. 2021;41(2):146–54.
Akbari H, Sadiq MT, Rehman AU, Ghazvini M, Naqvi RA, Payan M, Bagheri H, Bagheri H. Depression recognition based on the reconstruction of phase space of EEG signals and geometrical features. Appl Acoust. 2021;179: 108078.
Craik A, He Y, Contreras-Vidal JL. Deep learning for electroencephalogram (EEG) classification tasks: a review. J Neural Eng. 2019;16(3): 031001.
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I. Attention is all you need. Adv Neural Inf Process Syst. 2017;30:5998–6008.
Kenton JD, Toutanova LK. Bert: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of NAACL-HLT, pp. 4171–4186 (2019).
Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S, Uszkoreit J. An image is worth 16x16 words: transformers for image recognition at scale. https://arxiv.org/abs/2010.11929 (2020).
Dong L, Xu S, Xu B. Speech-transformer: a no-recurrence sequence-to-sequence model for speech recognition. In: 2018 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE. pp. 5884–5888 (2018).
Chen L, Lu K, Rajeswaran A, Lee K, Grover A, Laskin M, Abbeel P, Srinivas A, Mordatch I. Decision transformer: reinforcement learning via sequence modeling. Adv Neural Inf Process Syst. 2021;34:15084–97.
Wu N, Green B, Ben X, O'Banion S. Deep transformer models for time series forecasting: the influenza prevalence case. https://arxiv.org/abs/2001.08317 (2020).
Ahmed S, Nielsen IE, Tripathi A, Siddiqui S, Rasool G, Ramachandran RP. Transformers in time-series analysis: a tutorial. https://arxiv.org/abs/2205.01138 (2022).
Wen Q, Zhou T, Zhang C, Chen W, Ma Z, Yan J, Sun L. Transformers in time series: a survey. https://arxiv.org/abs/2202.07125 (2022).
Jha RR, Bhardwaj A, Garg D, Bhavsar A, Nigam A. MHATC: Autism Spectrum Disorder identification utilizing multi-head attention encoder along with temporal consolidation modules. https://arxiv.org/abs/2201.00404 (2021).
Yi P, Chen K, Ma Z, Zhao D, Pu X, Ren Y. EEGDnet: fusing non-local and local self-similarity for 1-D EEG signal denoising with 2-D transformer. https://arxiv.org/abs/2109.04235 (2021).
Bagchi S, Bathula DR. EEG-ConvTransformer for single-trial EEG-based visual stimulus classification. Pattern Recogn. 2022;129: 108757.
Wang YA, Chen YN. What do position embeddings learn? An empirical study of pre-trained language model positional encoding. https://arxiv.org/abs/2010.04903 (2020)
Cavanagh JF, EEG: Depression rest. OpenNeuro. (2021) https://openneuro.org/datasets/ds003478/versions/1.1.0. Accessed 9 June 2022.
Delorme A, Makeig S. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J Neurosci Methods. 2004;134(1):9–21.
Übeyli ED. Statistics over features: EEG signals analysis. Comput Biol Med. 2009;39(8):733–41.
Hjorth B. EEG analysis based on time domain properties. Electroencephalogr Clin Neurophysiol. 1970;29(3):306–10.
Li M, Chen W. FFT-based deep feature learning method for EEG classification. Biomed Signal Process Control. 2021;66: 102492.
Al-Fahoum AS, Al-Fraihat AA. Methods of EEG signal features extraction using linear analysis in frequency and time-frequency domains. Int Sch Res Not. 2014;2014: 730218.
Marcílio WE, Eler DM. From explanations to feature selection: assessing shap values as feature selection mechanism. In: 2020 33rd SIBGRAPI conference on graphics, patterns and images (SIBGRAPI). IEEE, pp. 340–347 (2020).
Gramegna A, Giudici P. Shapley feature selection. FinTech. 2022;1(1):72–80.
Erickson BJ, Kitamura F. Magician’s corner: 9. Performance metrics for machine learning models. Radiology. 2021;3(3): e200126.
Yu PN, Liu CY, Heck CN, Berger TW, Song D. A sparse multiscale nonlinear autoregressive model for seizure prediction. J Neural Eng. 2021;18(2): 026012.
Attia A, Moussaoui A, Chahir Y. Epileptic seizures identification with autoregressive model and firefly optimization based classification. Evol Syst. 2021;12(3):827–36.
Mohan P, Paramasivam I. Feature reduction using SVM-RFE technique to detect autism spectrum disorder. Evol Intell. 2021;14(2):989–97.
Zulfiker MS, Kabir N, Biswas AA, Nazneen T, Uddin MS. An in-depth analysis of machine learning approaches to predict depression. Curr Res Behav Sci. 2021;2: 100044.
Haque UM, Kabir E, Khanam R. Detection of child depression using machine learning methods. PLoS ONE. 2021;16(12): e0261131.
Byun S, Kim AY, Jang EH, Kim S, Choi KW, Yu HY, Jeon HJ. Detection of major depressive disorder from linear and nonlinear heart rate variability features during mental task protocol. Comput Biol Med. 2019;112:103381.l.
Alghowinem SM, Gedeon T, Goecke R, Cohn J, Parker G. Interpretation of depression detection models via feature selection methods. In: IEEE transactions on affective computing. (2020).
Zhou T, Ma Z, Wen Q, Wang X, Sun L, Jin R. FEDformer: frequency enhanced decomposed transformer for long-term series forecasting. https://arxiv.org/abs/2201.12740 (2022).
Yu C, Ma X, Ren J, Zhao H, Yi S. Spatio-temporal graph transformer networks for pedestrian trajectory prediction. In: European conference on computer vision. Springer, Cham; pp. 507–523 (2020).
Mei H, Yang C, Eisner J. Transformer embeddings of irregularly spaced events and their participants. In: International conference on learning representations (2021).
Xu J, Wu H, Wang J, Long M. Anomaly transformer: time series anomaly detection with association discrepancy. https://arxiv.org/abs/2110.02642 (2021).
Liu M, Ren S, Ma S, Jiao J, Chen Y, Wang Z, Song W. Gated transformer networks for multivariate time series classification. https://arxiv.org/abs/2103.14438 (2021).
Bachmann M, Päeske L, Kalev K, Aarma K, Lehtmets A, Ööpik P, Lass J, Hinrikus H. Methods for classifying depression in single channel EEG using linear and nonlinear signal analysis. Comput Methods Programs Biomed. 2018;155:11–7.
Mahato S, Paul S. Detection of major depressive disorder using linear and non-linear features from EEG signals. Microsyst Technol. 2019;25(3):1065–76.
Mahato S, Goyal N, Ram D, Paul S. Detection of depression and scaling of severity using six channel EEG data. J Med Syst. 2020;44(7):1–2.
Lin H, Jian C, Cao Y, Ma X, Wang H, Miao F, Fan X, Yang J, Zhao G, Zhou H. MDD-TSVM: a novel semisupervised-based method for major depressive disorder detection using electroencephalogram signals. Comput Biol Med. 2022;140: 105039.