Coarse-grained decomposition and fine-grained interaction for multi-hop question answering
Tóm tắt
In recent years, question answering (QA) and reading comprehension (RC) has attracted much attention, and most research on QA has focused on multi-hop QA task which requires connecting multiple pieces of evidence scattered in a long context to answer the question. The key to the multi-hop QA task is semantic feature interaction between documents and questions, which is widely processed by Bi-directional Attention Flow (Bi-DAF), but Bi-DAF generally captures only the surface semantics of words in complex questions, and fails to capture implied semantic feature of intermediate answers, as well as ignoring parts of contexts related to the question and failing to extract the most important parts of multiple documents. In this paper, we propose a new model architecture for multi-hop question answering by applying two completion strategies:(1) Coarse-Grained complex question Decomposition (CGDe) strategy is introduced to decompose complex questions into simple ones without any additional annotations; (2) Fine-Grained Interaction (FGIn) strategy is introduced to explicitly represent each word in documents and extract more comprehensive and accurate sentences related to the inference path. The above two strategies are combined and tested on the SQuAD and HotpotQA datasets, and the experimental results show that our method outperforms state-of-the-art baselines.
Tài liệu tham khảo
Bhargav, GPS., Glass, M., Garg, D., Shevade, S., Dana, S., Khandelwal, D., Subramaniam, L. V., & Gliozzo, A. (2020). In Proceedings of the AAAI conference on artificial intelligence, (Vol. 34 pp. 7700–7707).
Cao, Y., Fang, M., & Tao, D. (2019). BAG: Bi-directional attention entity graph convolutional network for multi-hop reasoning question answering. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: Human language technologies, volume 1 (long and short papers) (pp. 357–362).
Chen, D., Fisch, A., Weston, J., & Bordes, A. (2017a). Reading wikipedia to answer open-domain questions. In Proceedings of the 55th annual meeting of the association for computational linguistics (volume 1: long papers) (pp. 1870–1879).
Chen, H., Liu, X., Yin, D., & Tang, J. (2017b). Acm Sigkdd Explorations Newsletter, 19(2), 25–35.
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014). arXiv:1412.3555.
Cui, Y., Chen, Z., Wei, S., Wang, S., Liu, T., & Hu, G. (2017). Attention-over-attention neural networks for reading comprehension. In Proceedings of the 55th annual meeting of the association for computational linguistics (volume 1: long papers) (pp. 593–602).
Devlin, J., Chang, M-W, Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: Human language technologies, volume 1 (long and short papers) (pp. 4171–4186).
Dimitrakis, E., Sgontzos, K., & Tzitzikas, Y. (2020). J Intell Inf Syst, 55(2), 233–259.
Fang, Y., Sun, S., Gan, Z., Pillai, R., Wang, S., & Liu, J. (2020). Hierarchical graph network for multi-hop question answering. In Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP) (pp. 8823–8838).
Hajiramezanali, E., Hasanzadeh, A., Duffield, N., Narayanan, K.R., Zhou, M., & Qian, X. (2019). Variational graph recurrent neural networks. Adv. Neural Inf. Process. Syst., 32, 10701–10711.
Huang, H.-Y., Zhu, C., Shen, Y., & Chen, W. (2017). arXiv:1711.07341.
Huang, M., Zhu, X., & Gao, J. (2020). ACM Trans Inf Syst (TOIS), 38(3), 1–32.
Jiang, Y., & Bansal, M. (2019a). Avoiding reasoning shortcuts: Adversarial evaluation, training, and model development for multi-hop QA. In Proceedings of the 57th annual meeting of the association for computational linguistics (pp. 2726–2736).
Jiang, Y., & Bansal, M. (2019b). Self-assembling modular networks for interpretable multi-hop reasoning. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP) (pp. 4464–4474).
Jiang, Y., Joshi, N., Chen, Y.-C., & Bansal, M. (2019). Explore, propose, and assemble: an interpretable model for multi-hop reading comprehension. In Proceedings of the 57th annual meeting of the association for computational linguistics (pp. 2714–2725).
Khushhal, S., Majid, A., Abbas, S.A., Nadeem, M.S.A., & Shah, S.A. (2020). J Intell Inf Syst, 55(2), 307–327.
Kipf, T.N., & Welling, M. (2016). arXiv:1609.02907.
Kočiskỳ, T., Schwarz, J., Blunsom, P., Dyer, C., Hermann, K.M., Melis, G., & Grefenstette, E. (2018). Transactions of the Association for Computational Linguistics, 6, 317–328.
Kundu, S., Khot, T., Sabharwal, A., & Clark, P. (2019). Exploiting explicit paths for multi-hop reading comprehension. In Proceedings of the 57th annual meeting of the association for computational linguistics (pp. 2737–2747).
Lin, X. V., Socher, R., & Xiong, C. (2018). Multi-Hop knowledge graph reasoning with reward shaping. In Proceedings of the 2018 conference on empirical methods in natural language processing (pp. 3243–3253).
Liu, N., & Shen, B. (2020). Neurocomputing, 395, 66–77.
Min, S., Zhong, V., Zettlemoyer, L., & Hajishirzi, H. (2019). Multi-hop Reading comprehension through question decomposition and rescoring. In Proceedings of the 57th annual meeting of the association for computational linguistics (pp. 6097–6109).
Nishida, K., Nishida, K., Nagata, M., Otsuka, A., Saito, I., Asano, H., & Tomita, J. (2019). Answering while summarizing: Multi-task learning for multi-hop QA with evidence extraction. In Proceedings of the 57th annual meeting of the association for computational linguistics (pp. 2335–2345).
Pennington, J., Socher, R., & Manning, C. D. (2014). In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) (pp. 1532–1543).
Qi, P., Lin, X., Mehr, L., Wang, Z., & Manning, C.D. (2019). Answering complex open-domain questions through iterative query generation. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP) (pp. 2590–2602).
Qiu, L., Xiao, Y., Qu, Y., Zhou, H., Li, L., Zhang, W., & Yu, Y. (2019). In Proceedings of the 57th annual meeting of the association for computational linguistics (pp. 6140–6150).
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016). SQuAD: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 conference on empirical methods in natural language processing (pp. 2383–2392).
Rajpurkar, P., Jia, R., & Liang, P. (2018). Know what you don’t know: Unanswerable questions for SQuAD. In Proceedings of the 56th annual meeting of the association for computational linguistics (volume 2: short papers) (pp. 784–789).
Reddy, S., Chen, D., & Manning, C.D. (2019). Transactions of the Association for Computational Linguistics, 7, 249–266.
Seo, M., Kembhavi, A., Farhadi, A., & Hajishirzi, H. (2016). arXiv:1611.01603.
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). The Journal of Machine Learning Research, 15(1), 1929–1958.
Srivastava, R.K., Greff, K., & Schmidhuber, J. (2015). arXiv:1505.00387.
Tang, Y., Ng, H.T., & Tung, A. KH. (2020). arXiv:2002.09919.
Tu, M., Wang, G., Huang, J., Tang, Y., He, X., & Zhou, B. (2019). Multi-hop reading comprehension across multiple documents by reasoning over heterogeneous graphs. In Proceedings of the 57th annual meeting of the association for computational linguistics (pp. 2704–2713).
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., & Bengio, Y. (2017). arXiv:1710.10903.
Weissenborn, D., Wiese, G., & Seiffe, L. (2017). arXiv:1703.04816.
Welbl, J., Stenetorp, P., & Riedel, S. (2018). Transactions of the Association for Computational Linguistics, 6, 287–302.
Xiong, C., Zhong, V., & Socher, R. (2016). arXiv:1611.01604.
Xu, K., Hu, W., Leskovec, J., & Jegelka, S. (2018). arXiv:1810.00826.
Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W.W., Salakhutdinov, R., & Manning, C.D. (2018). HotpotQA: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 conference on empirical methods in natural language processing (pp. 2369–2380).
Ye, D., Lin, Y., Liu, Z., Liu, Z., & Sun, M. (2019). arXiv:1911.02170.
Zhong, V., Xiong, C., Keskar, N. S., & Socher, R. (2019). arXiv:1901.00603.