Applications of Language Modeling in Speech-To-Speech Translation
Tóm tắt
This paper describes various language modeling issues in a speech-to-speech translation system. These issues are addressed in the IBM speech-to-speech system we developed for the DARPA Babylon program in the context of two-way translation between English and Mandarin Chinese. First, the language models for the speech recognizer had to be adapted to the specific domain to improve the recognition performance for in-domain utterances, while keeping the domain coverage as broad as possible. This involved considerations of disfluencies and lack of punctuation, as well as domain-specific utterances. Second, we used a hybrid semantic/syntactic representation to minimize the data sparseness problem in a statistical natural language generation framework. Serious inflection and synonym issues arise when words in the target language are to be determined in the translation output. Instead of relying on tedious handcrafted grammar rules, we used N-gram models as a post-processing step to enhance the generation performance. When an interpolated language model was applied to a Chinese-to-English translation task, the translation performance, measured by an objective metric of BLEU, improved substantially to 0.514 from 0.318 when we used the correct transcription as input. Similarly, the BLEU score improved to 0.300 from 0.194 for the same task when the input was speech data.
Tài liệu tham khảo
Berger, A., Della Pietra, S., and Della Pietra, V. (1996). A maximum entropy approach to natural language processing. Computer Linguistics, 22(1):39–71.
Brown, P.F., Della Pietra, S.A., Della Pietra, V.J., and Mercer, R.L. (1993). The mathematics of statistical machine translation: Parameter estimation. Computational Linguistics, 19:263–311.
Davies, K., Donovan, R., Epstein, M., Franz, M., Ittycheriah, A., Jan, E., LeRoux, J., Lubensky, D., Neti, C., Padmanabhan, M., Papineni, K., Roukos, S., Sakrajda, A., Sorensen, J., Tydlitat, B., and Ward, T. (1999). The IBM conversational telephony system for financial applications. EuroSpeech-1999 Proceedings. Budapest, Hungary: EuroSpeech, pp. 275–278.
Donovan, R.E., Franz, M., Sorensen, J.S., and Roukos, S. (1999). Phrase splicing and variable substitution using the IBM trainable speech synthesis system. ICASSP-1999 Proceedings. Phoenix, AZ: ICASSP, pp. 373–376.
Lavie, A., Mztze, F., Cattoni, R., and Costantini, E. (2002). A multi-perspective evaluation of the NESPOLE! speech-tospeech translation system. Speech-to-Speech Translation Workshop. Philadelphia, PA: ACL, pp. 121–128.
Lavie, A., Waibel, A., Levin, L., Finke, M., Gates, D., Gavalda, M., Zeppenfeld, T., and Zhan, P. (1997). JANUS-III: Speech-to-speech translation in multiple languages. ICASSP-1997 Proceedings. Munich, Germany: ICASSP, pp. 99–102.
Levin, L., Gates, D., Wallace, D., Peterson, K., Lavie, A., Pianesi, F., Pianta, E., Cattoni, R., and Mana, N. (2002). Balancing expressiveness and simplicity in an interlingua for task based dialogue. Speech-to-Speech Translation Workshop, Philadelphia, PA: ACL, pp. 53–60.
Magerman, D. (1994). Natural language parsing as statistical pattern recognition. Ph. D. thesis, Stanford University, Palo Alto, CA.
McCord, M.C. (1989). A new version of the machine translation system LMT. Literary and Linguistic Computing, 4:218–229.
Ney, H., NieBen, S., Och, F.J., Sawaf, H., Tillmann, C., and Vogel, S. (2000). Algorithms for statistical translation of spoken language. IEEE Trans. on Speech And Audio Processing, 8(1):24–36.
Papineni, K., Roukos, S., Ward, T., and Zhu, W. (2001). Bleu: A method for automatic evaluation of machine translation. Research Report RC22176, IBM. Yorktown Heights, NY.
Ratnaparkhi, A. (2000). Trainable methods for surface natural language generation.NAACL-2000 Proceedings. Seattle, WA: NAACL, pp. 194–201.
Rayner, M., Carter, D., Bouillon, P., Digalakis, V., and Wiren, M. (2000). The Spoken Language Translator. Cambridge, UK: Cambridge University Press.
Takezawa, T., Morimoto, T., Sagisaka, Y., Campbell, N., Iida, H., Sugaya, F., Yokoo, A., and Yamamoto, S. (1998). A Japanese-to-English speech translation system: ART-MATRIX. ICSLP-1998 Proceedings. Sydney, Australia: ICSLP, pp. 2779–2782.
Wahlster, W. (2000). Verbmobil: Foundations of Speech-to-Speech Translation. Berlin: Springer-Verlag.
Zhou, B., Gao, Y., Sorensen, J., Diao, Z., and Picheny, M. (2002). Statistical natural language generation for speech-to-speech machine translation systems. ICSLP-2002 Proceedings. Denver, CO: ICSLP, pp. 1897–1900.