Enriching Word Vectors with Subword Information

Transactions of the Association for Computational Linguistics - Tập 5 - Trang 135-146 - 2017
Piotr Bojanowski1, Édouard Grave1, Jean‐Philippe Vert1, Tomáš Mikolov1
1Facebook AI Research

Tóm tắt

Continuous word representations, trained on large unlabeled corpora are useful for many natural language processing tasks. Popular models that learn such representations ignore the morphology of words, by assigning a distinct vector to each word. This is a limitation, especially for languages with large vocabularies and many rare words. In this paper, we propose a new approach based on the skipgram model, where each word is represented as a bag of character n-grams. A vector representation is associated to each character n-gram; words being represented as the sum of these representations. Our method is fast, allowing to train models on large corpora quickly and allows us to compute word representations for words that did not appear in the training data. We evaluate our word representations on nine different languages, both on word similarity and analogy tasks. By comparing to recently proposed morphological word representations, we show that our vectors achieve state-of-the-art performance on these tasks.

Từ khóa


Tài liệu tham khảo

10.1162/coli_a_00016

Cui Qing, 2015, ACM Transactions on Information Systems, 34, 1

Deerwester Scott, 1990, Journal of the American Society for Information Science, 41, 391, 10.1002/(SICI)1097-4571(199009)41:6<391::AID-ASI1>3.0.CO;2-9

Harris Zellig S, 1954, Word, 10, 146, 10.1080/00437956.1954.11659520

Lund Kevin, 1996, Behavior Research Methods, Instruments, & Computers, 28, 203, 10.3758/BF03204766

Spearman Charles, 1904, The American Journal of Psychology, 15, 72, 10.2307/1412159

Turney Peter D., 2010, Journal of Artificial Intelligence Research, 37, 141, 10.1613/jair.2934

Thông tin
Thông tin xuất bản

Transactions of the Association for Computational Linguistics

Tập 5

135-146

Thông tin tác giả