SA-ASBA: Mô hình kết hợp cho phân tích cảm xúc dựa trên khía cạnh sử dụng sự chú ý tổng hợp trong mô hình ngôn ngữ BERT đã được tiền huấn luyện với phương pháp tăng cường độ dốc cực đại

Springer Science and Business Media LLC - Tập 79 - Trang 5516-5551 - 2022
Arvind Mewada1, Rupesh Kumar Dewang1
1Motilal Nehru National Institute of Technology Allahabad, Prayagraj, India

Tóm tắt

Phân tích cảm xúc dựa trên khía cạnh (ABSA) là một nhiệm vụ phân tích cảm xúc ở mức độ chi tiết nhằm phát hiện các thành phần cảm xúc đối với một khía cạnh cụ thể trong văn bản. Nghiên cứu này thể hiện sự tò mò thái quá trong việc mô hình hóa mục tiêu và ngữ cảnh thông qua các mạng chú ý nhằm đạt được các đại diện đặc trưng hiệu quả cho các công việc phát hiện cảm xúc. Chúng tôi đã đề xuất sự chú ý tổng hợp trong các biểu diễn bộ mã hóa hai chiều từ transformers (SA-BERT) với bộ phân loại tăng cường độ dốc cực đại (XGBoost) để phân loại cảm xúc trong tập dữ liệu đánh giá. Mô hình được đề xuất tạo ra mã hóa vector từ động cho khía cạnh và ngữ cảnh tương ứng của các đánh giá. Sau đó, khía cạnh và ngữ cảnh của các đánh giá được đại diện một cách có nghĩa bởi một transformer có thể nhập vector từ song song. Tiếp theo, mô hình sử dụng cơ chế chú ý tổng hợp để học các phần quan trọng của ngữ cảnh và các khía cạnh trong các đánh giá. Cuối cùng, mô hình đưa ra đại diện tổng thể trong lớp phân loại cảm xúc để dự đoán thành phần cảm xúc. Cả hai mô hình SA-BERT và SA-BERT-XGBoost được đề xuất đều đạt độ chính xác cao nhất (92,02 và 93,71%) trên tập dữ liệu restaurant16 và điểm F-1 cao nhất (81,19 và 81,64%) trên tập dữ liệu restaurant14. Độ chính xác trung bình và điểm số F1 cao hơn khoảng 2 và 3,04% so với các mô hình cơ sở (DLCF-DCA-CDM, R-GAT+BERT, ASGCN-DG, AEN-BERT và BERT-PT). Do đó, các mô hình được đề xuất vượt trội so với các mô hình cơ sở.

Từ khóa

#Phân tích cảm xúc #chú ý tổng hợp #BERT #tăng cường độ dốc cực đại #mạng đối kháng.

Tài liệu tham khảo

Pang B, Lee L (2008) Opinion mining and sentiment analysis. Found Trends Inf Retr 2:1–2 Guixian X, Zhang Z, Zhang T, Shaona Yu, Meng Y, Chen S (2022) Aspect-level sentiment classification based on attention-BiLSTM model and transfer learning. Know-Based Syst 245:108586 Mayur W, Sekhara RAC, Chaitanya K (2022) A survey on sentiment analysis methods, applications, and challenges. Artif Intell Rev 7:1–50 Schouten K, Frasincar F (2015) Survey on aspect-level sentiment analysis. IEEE Trans Know Data Eng 28(3):813–830 Dou Z-Y (2017) Capturing user and product information for document level sentiment analysis with deep memory network. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 521–526 Chakraborty K, Bhattacharyya S, Bag R (2020) A survey of sentiment analysis from social media data. IEEE Trans Comput Soc Syst 7(2):450–464 Sun J, Han P, Cheng Z, Enming W, Wang W (2020) Transformer based multi-grained attention network for aspect-based sentiment analysis. IEEE Access 8:211152–211163 Ayetiran EF, Eniafe Festus Ayetiran (2022) Attention-based aspect sentiment classification using enhanced learning through CNN-BiLSTM networks. Know-Based Syst 252:109409 Zhao P, Hou L, Ou W (2020) Modeling sentiment dependencies with graph convolutional networks for aspect-level sentiment classification. Know-Based Syst 193:105443 Xiao Z, Xin X, Xing H, Song F, Wang X, Zhao B (2021) A federated learning system with enhanced feature extraction for human activity recognition. Know-Based Syst 229:107338 Guangtao X, Liu P, Zhu Z, Liu J, Fuyong X (2021) Attention-enhanced graph convolutional networks for aspect-based sentiment classification with multi-head attention. Appl Sci 11(8):3640 Anand D, Naorem D (2016) Semi-supervised aspect based sentiment analysis for movies using review filtering. Procedia Comput Sci 84:86–93 Kiritchenko S, Zhu X, Cherry C, Mohammad SM (2014) Detecting aspects and sentiment in customer reviews. In: 8th International Workshop on Semantic Evaluation (SemEval), pp 437–442 Poria S, Ofek N, Gelbukh A, Hussain A, Rokach L(2014) Dependency tree-based rules for concept-level aspect-based sentiment analysis. In: Semantic Web Evaluation Challenge, 41–47. Springer: Cham Weichselbraun A, Gindl S, Scharl A (2013) Extracting and grounding contextualized sentiment lexicons. IEEE Intell Syst 28(2):39–46 Gao Z, Li Z, Luo J, Li X (2022) Short text aspect-based sentiment analysis based on CNN+ BiGRU. Appl Sci 12(5):2707 Wang X, Li F, Zhang Z, Guangluan X, Zhang J, Sun X (2021) A unified position-aware convolutional neural network for aspect based sentiment analysis. Neurocomputing 450:91–103 Bai Q, Zhou J, He L (2022) PG-RNN: using position-gated recurrent neural networks for aspect-based sentiment classification. J Supercomput 78(3):4073–4094 Zhang B, Li X, Xiaofei X, Leung K-C, Chen Z, Ye Y (2020) Knowledge guided capsule attention network for aspect-based sentiment analysis. IEEE/ACM Trans Audio, Speech, Lang Process 28:2538–2551 Lin Y, Wang C, Song H, Li Y (2021) Multi-head self-attention transformation networks for aspect-based sentiment analysis. IEEE Access 9:8762–8770 Liang B, Hang S, Gui L, Cambria E, Ruifeng X (2022) Aspect-based sentiment analysis via affective knowledge enhanced graph convolutional networks. Knowl-Based Syst 235:107643 Majumder N, Poria S, Gelbukh A, Akhtar MS, Cambria E, Ekbal A(2018) Iarm: Inter-aspect relation modeling with memory networks in aspect-based sentiment analysis. In: Proceedings of the 2018 conference on empirical methods in natural language processing, pp 3402–3411 Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. Adv Neural Inf Process Syst 30 Xing H, Xiao Z, Zhan D, Luo S, Dai P, Li K (2022) Selfmatch: Robust semisupervised time-series classification with self-distillation. Int J Intell Syst Zhang P, Huang X, Li M, Xue Yu (2021) Hybridization between neural computing and nature-inspired algorithms for a sentence similarity model based on the attention mechanism. ACM Trans Asian Low-Resour Lang Inf Process(TALLIP) 20(1):1–21 Yadav RK, Jiao L, Goodwin M, Granmo O-C (2021) Positionless aspect based sentiment analysis using attention mechanism. Know-Based Syst 226:107136 Huang B, Ou Y, Carley KM (2018) Aspect level sentiment classification with attention-over-attention neural networks. In: International Conference on Social Computing, Behavioral-Cultural Modeling and Prediction and Behavior Representation in Modeling and Simulation. pp 197–206. Springer: Cham Xiao Z, Xin X, Xing H, Luo S, Dai P, Zhan D (2021) RTFN: a robust temporal feature network for time series classification. Inf Sci 571:65–86 Wang Y, Huang M, Zhu X, Zhao L(2016) Attention-based LSTM for aspect-level sentiment classification. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pP 606–615 Tang D, Qin B, Liu T(2016) Aspect level sentiment classification with deep memory network. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp 214–224, Austin, Texas, November 2016a. Association for Computational Linguistics. https://doi.org/10.18653/v1/D16-1021 Ma D, Li S, Zhang X, Wang H(2017) Interactive attention networks for aspect-level sentiment classification. In: IJCAI’17: Proceedings of the 26th International Joint Conference on Artificial Intelligence, IJCAI’17, pp 4068-4074. AAAI Press,. ISBN 9780999241103 Bensoltane R, Zaki T (2022) Towards Arabic aspect-based sentiment analysis: a transfer learning-based approach. Soc Netw Anal Min 12(1):1–16 Cai X, Cao H, Ma J, Li M, Zhuang X(2021) Aspect level sentiment classification with semantic distance attention networks. In: 2021 2nd International Conference on Computing, Networks and Internet of Things, pp 1–5 Al-Smadi M, Talafha B, Al-Ayyoub M, Jararweh Y (2019) Using long short-term memory deep neural networks for aspect-based sentiment analysis of Arabic reviews. Int J Mach Learn Cybern 10(8):2163–2175 Peters ME, Neumann M, Iyyer M, Gardner M, Clark C, Lee K, Zettlemoyer L (2018) Deep contextualized word representations. CoRR, abs/1802.05365 Alec R, Karthik N, Tim S, Ilya S (2018) Improving language understanding by generative pre-training. The university of british columbia vancouver campus, Vancouver Ullah H, Ahmad B, Sana I, Sattar A, Khan A, Akbar S, Asghar MZ (2021) Comparative study for machine learning classifier recommendation to predict political affiliation based on online reviews. CAAI Trans Intell Technol 6(3):251–264 Karimi A, Rossi L, Prati A(2021) Adversarial training for aspect-based sentiment analysis with bert. In: 2020 25th International Conference on Pattern Recognition (ICPR), pp 8797–8803. IEEE Song Y, Wang J, Jiang T, Liu Z, Rao Y(2019) Attentional encoder network for targeted sentiment classification. arXiv preprint arXiv:1902.09314 Do HH, Prasad PWC, Maag A, Alsadoon A (2019) Deep learning for aspect-based sentiment analysis: a comparative review. Expert Syst Appl 118:272–299 Zhou J, Huang JX, Chen Q, Qinmin Vivian H, Wang T, He L (2019) Deep learning for aspect-level sentiment classification: survey, vision, and challenges. IEEE Access 7:78454–78483 Li N, Chow CY, Zhang JD (2020) SEML: a semi-supervised multi-task learning framework for aspect-based sentiment analysis. IEEE Access 8:189287–189297 Chen Z, Qian T(2019) Transfer capsule network for aspect level sentiment classification. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp 547–556 Jindian S, Shanshan Yu, Luo D (2020) Enhancing aspect-based sentiment analysis with capsule network. IEEE Access 8:100551–100561 Zhao F, Wu Z, Dai X(2020b) Attention transfer network for aspect-level sentiment classification. In: Proceedings of the 28th International Conference on Computational Linguistics, pp 811–821 Dong L, Wei F, Tan C, Tang D, Zhou M, Xu K(2014) Adaptive recursive neural network for target-dependent twitter sentiment classification. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (volume 2: Short papers), pp 49–54 Tang D, Qin B, Feng X, Liu T(2016b) Effective LSTMs for target-dependent sentiment classification. In: Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pp 3298–3307, Osaka, Japan, December. The COLING 2016 Organizing Committee. https://aclanthology.org/C16-1311 Tay Y, Tuan LA, Hui SC(2018) Learning to attend via word-aspect associative fusion for aspect-based sentiment analysis. In: Proceedings of the AAAI Conference on Artificial Intelligence Ma Y, Peng H, Cambria E(2018) Targeted aspect-based sentiment analysis via embedding commonsense knowledge into an attentive LSTM. In: Proceedings of the AAAI Conference on Artificial Intelligence Devlin J, Chang MW, Lee K, Toutanova K (2019) BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp 4171–4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. https://doi.org/10.18653/v1/N19-1423 Lan Z, Chen M, Goodman S, Gimpel K, Sharma P, Soricut R(2020) ALBERT: A lite BERT for self-supervised learning of language representations. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, https://openreview.net/forum?id=H1eA7AEtvS Li B, Pan F, Shou Z, Zhang H(2021) Aspect based sentiment analysis of catering field reviews via roberta-aoa model. In: J Phys: Conf Ser. pp 012064. IOP Publishing Tay Y, Bahri D, Metzler D, Juan DC, Zhao Z, Zheng C(2021) Synthesizer: Rethinking self-attention for transformer models. In: International Conference on Machine Learning, pp 10183–10192. PMLR Kitaev N, Kaiser Ł, Levskaya A(2020) Reformer: The efficient transformer. arXiv preprint arXiv:2001.04451 Aziz RHH, Dimililer N (2021) Sentixgboost: enhanced sentiment analysis in social media posts with ensemble XGBoost classifier. J Chin Inst Eng 44(6):562–572 Pontiki M, Galanis D, Pavlopoulos J, Papageorgiou H, Androutsopoulos I, Manandhar S (2014) SemEval-2014 task 4: Aspect based sentiment analysis. In: Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pp 27–35, Dublin, Ireland, August. Association for Computational Linguistics. https://doi.org/10.3115/v1/S14-2004 Pontiki M, Galanis D, Papageorgiou H, Manandhar S, Androutsopoulos I (2015) Semeval-2015 task 12: Aspect based sentiment analysis. In: Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pp 486–495 Pontiki M, Galanis D, Papageorgiou H, Androutsopoulos I, Manandhar S, Al-Smadi M, Al-Ayyoub M, Zhao Y, Qin B, De C, Orphé et al. (2016) Semeval-2016 task 5: Aspect based sentiment analysis. In: International Workshop on Semantic Evaluation, pp 19–30 Yang C, Zhang H, Jiang B, Li K (2019) Aspect-based sentiment analysis with alternating coattention networks. Inf Process Manag 56(3):463–478 Fan F, Feng Y, Zhao D(2018) Multi-grained attention network for aspect-level sentiment classification. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp 3433–3442 He R, Lee WS, Ng HT, Dahlmeier D (2018) Exploiting document knowledge for aspect-level sentiment classification. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp 579–585, Melbourne, Australia, July. Association for Computational Linguistics. https://doi.org/10.18653/v1/P18-2092 Zhang C, Li Q, Song D (2019) Aspect-based sentiment classification with aspect-specific graph convolutional networks. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp 4568–4578, Hong Kong, China, November. Association for Computational Linguistics. https://doi.org/10.18653/v1/D19-1464 Xu H, Liu B, Shu L, Yu P(2019) BERT post-training for review reading comprehension and aspect-based sentiment analysis. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp 2324–2335, Minneapolis, Minnesota, June. Association for Computational Linguistics. https://doi.org/10.18653/v1/N19-1242 Meng W, Wei Y, Liu P, Zhu Z, Yin H (2019) Aspect based sentiment analysis with feature enhanced attention CNN-BiLSTM. IEEE Access 7:167240–167249 Zeng B, Han X, Zeng F, Xu R, Yang H(2019) Multifeature interactive fusion model for aspect-based sentiment analysis. Math Probl Eng Wang K, Shen W, Yang Y, Quan X, Wang R(2020) Relational graph attention network for aspect-based sentiment analysis. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp 3229–3238, Online, Jul. Association for Computational Linguistics. https://doi.org/10.18653/v1/2020.acl-main.295 Qiannan X, Zhu L, Dai T, Yan C (2020) Aspect-based sentiment classification with multi-attention network. Neurocomputing 388:135–143 Tang H, Ji D, Li C, Zhou Q(2020) Dependency graph enhanced dual-transformer structure for aspect-based sentiment classification. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp 6578–6588 Shuang K, Mengyu G, Li R, Loo J, Sen S (2021) Interactive POS-aware network for aspect-level sentiment classification. Neurocomputing 420:181–196 Mayi X, Biqing Z, Heng Y, Junlong C, Jiatao C, Hongye L (2022) Combining dynamic local context focus and dependency cluster attention for aspect-level sentiment classification. Neurocomputing 478:49–69