ReBoost: a retrieval-boosted sequence-to-sequence model for neural response generation
Tóm tắt
Human–computer conversation is an active research topic in natural language processing. One of the representative methods to build conversation systems uses the sequence-to-sequence (Seq2seq) model through neural networks. However, with limited input information, the Seq2seq model tends to generate meaningless and trivial responses. It can be greatly enhanced if more supplementary information is provided in the generation process. In this work, we propose to utilize retrieved responses to boost the Seq2seq model for generating more informative replies. Our method, called ReBoost, incorporates retrieved results in the Seq2seq model by a hierarchical structure. The input message and retrieved results can influence the generation process jointly. Experiments on two benchmark datasets demonstrate that our model is able to generate more informative responses in both automatic and human evaluations and outperforms the state-of-the-art response generation models.
Tài liệu tham khảo
Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. CoRR abs/1409.0473.
Bartl, A., & Spanakis, G. (2017). A retrieval-based dialogue system utilizing utterance and context embeddings. In 16th IEEE international conference on machine learning and applications, ICMLA 2017 (pp. 1120–1125). Cancun, Mexico, December 18–21, 2017.
Fleiss, J. L., & Cohen, J. (1973). The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educational and Psychological Measurement, 33(3), 613–619.
Ghazvininejad, M., Brockett, C., Chang, M., Dolan, B., Gao, J., Yih, W., & Galley, M. (2018). A knowledge-grounded neural conversation model. In Proceedings of the thirty-second AAAI conference on artificial intelligence. New Orleans, Louisiana, USA, February 2–7, 2018.
Ji, Z., Lu, Z., & Li, H. (2014). An information retrieval approach to short text conversation. CoRR abs/1408.6988.
Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. CoRR abs/1412.6980.
Li, J., Galley, M., Brockett, C., Gao, J., & Dolan, B. (2016a). A diversity-promoting objective function for neural conversation models. In NAACL HLT 2016, the 2016 conference of the North American chapter of the association for computational linguistics: Human language technologies (pp. 110–119). San Diego California, USA, June 12–17, 2016.
Li, X., Mou, L., Yan, R., & Zhang, M. (2016b). Stalematebreaker: A proactive content-introducing approach to automatic human–computer conversation. In Proceedings of the twenty-fifth international joint conference on artificial intelligence, IJCAI 2016 (pp. 2845–2851). New York, NY, USA, 9-15 July 2016.
Luong, T., Pham, H., & Manning, C.D. (2015). Effective approaches to attention-based neural machine translation. In Proceedings of the 2015 conference on empirical methods in natural language processing, EMNLP 2015 (pp. 1412–1421). Lisbon, Portugal, September 17–21, 2015.
Mou, L., Song, Y., Yan, R., Li, G., Zhang, L., & Jin, Z. (2016). Sequence to backward and forward sequences: A content-introducing approach to generative short-text conversation. In COLING 2016, 26th international conference on computational linguistics, proceedings of the conference: Technical papers (pp. 3349–3358). Osaka, Japan, December 11–16, 2016.
Papineni, K., Roukos, S., Ward, T., & Zhu, W. (2002). Bleu: A method for automatic evaluation of machine translation. In ACL 2002 (pp. 311–318). Philadelphia, PA, USA, July 6–12, 2002.
Robertson, S. E., & Zaragoza, H. (2009). The probabilistic relevance framework: BM25 and beyond. Foundations and Trends in Information Retrieval, 3(4), 333–389.
Shang, L., Lu, Z., & Li, H. (2015). Neural responding machine for short-text conversation. In Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing of the Asian federation of natural language processing, ACL 2015 (pp. 1577–1586). Beijing, China, Volume 1: Long Papers, July 26–31, 2015.
Shang, L., Sakai, T., Li, H., Higashinaka, R., Miyao, Y., & Nomoto, M. (2017). Overview of the NTCIR-13 short text conversation task. In Proceedings of the 13th NTCIR conference on evaluation of information access technologies. Tokyo Japan, December 5–8, 2017.
Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to sequence learning with neural networks. In Advances in neural information processing systems 27: Annual conference on neural information processing systems 2014 (pp. 3104–3112). Montreal, Quebec, Canada, December 8–13 2014.
Tian, Z., Yan, R., Mou, L., Song, Y., Feng, Y., & Zhao, D. (2017). How to make context more useful? An empirical study on context-aware neural conversational models. In Proceedings of the 55th annual meeting of the Association for Computational Linguistics, ACL 2017 (pp. 231–236). Vancouver, Canada, July 30–August 4, Volume 2: Short Papers.
Wu, Y., Li, Z., Wu, W., & Zhou, M. (2018a). Response selection with topic clues for retrieval-based chatbots. Neurocomputing, 316, 251–261.
Wu, Y., Wu, W., Yang, D., Xu, C., & Li, Z. (2018b). Neural response generation with dynamic vocabularies. In Proceedings of the thirty-second AAAI conference on artificial intelligence. New Orleans, Louisiana, USA, February 2–7, 2018.
Xing, C., Wu, W., Wu, Y., Liu, J., Huang, Y., Zhou, M., & Ma, W. (2017). Topic aware neural response generation. In Proceedings of the thirty-first AAAI conference on artificial intelligence. San Francisco, California, USA, February 4–9, 2017.
Yan, R., Song, Y., & Wu, H. (2016). Learning to respond with deep neural networks for retrieval-based human–computer conversation system. In Proceedings of the 39th international ACM SIGIR conference on research and development in information retrieval, SIGIR 2016 (pp. 55–64). Pisa, Italy, July 17–21, 2016.
Yan, X., Guo, J., Lan, Y., & Cheng, X. (2013). A biterm topic model for short texts. In 22nd international world wide web conference, WWW ’13 (pp. 1445–1456). Rio de Janeiro, Brazil, May 13–17, 2013.
Zhou, H., Huang, M., Zhang, T., Zhu, X., & Liu, B. (2018a). Emotional chatting machine: Emotional conversation generation with internal and external memory. In Proceedings of the thirty-second AAAI conference on artificial intelligence. New Orleans, Louisiana, USA, February 2–7, 2018.
Zhou, H., Young, T., Huang, M., Zhao, H., Xu, J., & Zhu, X. (2018b). Commonsense knowledge aware conversation generation with graph attention. In Proceedings of the twenty-seventh international joint conference on artificial intelligence, IJCAI 2018 (pp. 4623–4629). Stockholm, Sweden, July 13–19, 2018.