Learning to decode to future success for multi-modal neural machine translation

Lin, H., Meng, F., Su, J., et al., 2020. Dynamic context-guided capsule network for multimodal machine translation, in: Proceedings of the Twenty Eighth ACM International Conference on Multimedia, Seattle, USA, 1320–1329. Yang, P., Chen, B., Zhang, P., et al., 2020. Visual agreement regularized training for multi-modal machine translation, in: Proceedings of the AAAI Conference on Artificial Intelligence, 34, 9418–9425. Yao, S., Wan, X. 2020. Multimodal transformer for multimodal machine translation, in: Proceedings of the Fifty Eighth Annual Meeting of the Association for Computational Linguistics, Online, 4346–4350. O. Caglayan, P. Madhyastha, L. Specia, et al., Probing the need for visual context in multimodal machine translation, in: Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, .Minneapolis, USA, 2019, 1, 4159-4170. Wang, D., Xiong, D. , 2021. Efficient object-level visual context modeling for multimodal machine translation: masking irrelevant objects helps grounding, in: Proceedings of the AAAI Conference on Artificial Intelligence, 35, 2720–2728. Vaswani, 2017, Attention is all you need, Adv. Neural Inf. Process. Syst., 30 Duan, 2021, Modeling future cost for neural machine translation. actions on, IEEE/ACM Trans. Audio Speech Lang. Process., 29, 770, 10.1109/TASLP.2020.3042006 Zheng, Z., Huang, S., Tu, Z., et al., 2019. Dynamic past and future for neural machine translation, in: Proceedings of the Conference on Empirical Methods in Natural Language Processing and the Ninth International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China, 2019, 931–941. Li, X., Liu, L., Tu, Z., et al., 2018. Target foresight based attention for neural machine translation, in: Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, USA, 2018, 1, 1380–1390. D. Elliott, S. Frank, K. Sima'an, et al., Multi30K: multilingual English-German image descriptions, in: Proceedings of the Fifth Workshop on Vision and Language, Berlin, Germany, 2016, 70-74. Zheng, 2018, Modeling past and future for neural machine translation, Trans. Assoc. Comput. Linguist., 6, 145, 10.1162/tacl_a_00011 Weng, R., Huang, S., Zheng, Z., et al., 2017. Neural machine translation with word predictions, in: Proceedings of the Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark, 2017, 136–145. Zhou, M., Cheng, R., Lee, Y.J., et al., 2018. A visual attention grounding neural model for multimodal machine translation, in: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). Brussels, Belgium. Yin, Y., Meng, F., Su, J., et al., 2020. A Novel Graph-based Multi-modal Fusion Encoder for Neural Machine Translation, in: Proceedings of the Fifty Eighth Annual Meeting of the Association for Computational Linguistics, Online, 3025–3035. Li, J., Monroe, W., Jurafsky, D. 2017. Learning to decode for future success. arXiv Preprint arXiv:1701.06549. Lin, T.Y., Maire, M., Belongie, S., et al., 2014. Microsoft coco: Common objects in context, in: Proceedings of the European Conference on Computer Vision, Zurich, Switzerland, 740–755. P. Koehn, H. Hoang, A. Birch et al., Moses: open source toolkit for statistical machine translation, in: Proceedings of the Forty Fifth Annual Meeting of the Association for Computational Linguistics Companion Volume Proceedings of the Demo and Poster Sessions, 2007, 177-180. Papineni, K., Roukos, S., Ward, T., et al., 2002. Bleu: a method for automatic evaluation of machine translation, in: Proceedings of the Fortieth Annual Meeting of the Association for Computational Linguistics, Philadelphia, USA, 311–318. M. Denkowski, A. Lavie, 2014. Meteor universal: language specific translation evaluation for any target language, in: Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, USA, 376–380. Calixto, I., Rios, M., Aziz, W. 2019. Latent variable model for multi-modal translation, in: Proceedings of the Fifty Seventh Annual Meeting of the Association for Computational Linguistics, Firenze, Italy, 6392–6405. Caglayan, O., Aransa, W., & Bardet, A., et al. 2017. LIUM-CVC submissions for WMT17 multimodal translation task, in: Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark, 2, 432–439. Xia, 2017, Deliberation networks: sequence generation beyond one-pass decoding, Adv. Neural Inf. Process. Syst., 30 D. Elliott, Á. Kádár, 2017. Imagination improves multimodal translation, in: Proceedings of the Eighth International Joint Conference on Natural Language Processing, Taipei, Taiwan, 1, 130-141. J. Ive, P.S. Madhyastha, L. Specia, Distilling translations with visual awareness, in: Proceedings of the Fifty Seventh Annual Meeting of the Association for Computational Linguistics, Firenze, Italy, 2019, 6525-6538. Nishihara, T., Tamura, A., Ninomiya, T., et al., 2020. Supervised visual attention for multimodal neural machine translation, in: Proceedings of the Twenty Eighth International Conference on Computational Linguistics, 4304–4314. Online. Wang, X., Wu, J., Chen, J., et al., 2019. Vatex: a large-scale, high-quality multilingual dataset for video-and-language research, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 4581–4591.