Learning similarity measures from data

Progress in Artificial Intelligence - 2020
Bjørn Magnus Mathisen1, Agnar Aamodt1, Kerstin Bach1, Helge Langseth2
1Department of Computer Science, Norwegian University of Science and Technology, Trondheim, Norway
2Department of Computer Science, EXPOSED Aquaculture Research Centre, Trondheim, Norway

Tóm tắt

AbstractDefining similarity measures is a requirement for some machine learning methods. One such method is case-based reasoning (CBR) where the similarity measure is used to retrieve the stored case or a set of cases most similar to the query case. Describing a similarity measure analytically is challenging, even for domain experts working with CBR experts. However, datasets are typically gathered as part of constructing a CBR or machine learning system. These datasets are assumed to contain the features that correctly identify the solution from the problem features; thus, they may also contain the knowledge to construct or learn such a similarity measure. The main motivation for this work is to automate the construction of similarity measures using machine learning. Additionally, we would like to do this while keeping training time as low as possible. Working toward this, our objective is to investigate how to apply machine learning to effectively learn a similarity measure. Such a learned similarity measure could be used for CBR systems, but also for clustering data in semi-supervised learning, or one-shot learning tasks. Recent work has advanced toward this goal which relies on either very long training times or manually modeling parts of the similarity measure. We created a framework to help us analyze the current methods for learning similarity measures. This analysis resulted in two novel similarity measure designs: The first design uses a pre-trained classifier as basis for a similarity measure, and the second design uses as little modeling as possible while learning the similarity measure from data and keeping training time low. Both similarity measures were evaluated on 14 different datasets. The evaluation shows that using a classifier as basis for a similarity measure gives state-of-the-art performance. Finally, the evaluation shows that our fully data-driven similarity measure design outperforms state-of-the-art methods while keeping training time low.

Từ khóa


Tài liệu tham khảo

Abdel-Aziz, A., Strickert, M., Hüllermeier, E.: Learning solution similarity in preference-based CBR. In: International Conference on Case-Based Reasoning, pp. 17–31. Springer, Berlin (2014)

Arandjelovic, R., Zisserman, A.: Look, listen and learn. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 609–617. IEEE (2017)

Bergmann, R.: Experience Management: Foundations, Development Methodology, and Internet-Based Applications. Springer, Berlin (2002)

Berlemont, S., Lefebvre, G., Duffner, S., Garcia, C.: Siamese neural network based similarity metric for inertial gesture classification and rejection. In: 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), vol. 1, pp. 1–6. IEEE (2015)

Bromley, J., Guyon, I., LeCun, Y., Säckinger, E., Shah, R.: Signature verification using a” siamese” time delay neural network. In: Advances in neural information processing systems, pp. 737–744 (1994)

Chopra, S., Hadsell, R., LeCun, Y.: Learning a similarity metric discriminatively, with application to face verification. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005. CVPR 2005, vol. 1, pp. 539–546. IEEE (2005)

Cunningham, P.: A taxonomy of similarity mechanisms for case-based reasoning. IEEE Trans. Knowl. Data Eng. 21(11), 1532–1543 (2009)

Dheeru, D., Karra Taniskidou, E.: UCI machine learning repository. http://archive.ics.uci.edu/ml (2017). Accessed 1 June 2019

Florescu, C., Igel, C.: Resilient backpropagation (RPROP) for batch-learning in tensorflow. In: ICLR 2018 Workshop Permission Proceedings (to appear) (2018)

Gabel, T., Godehardt, E.: Top-down induction of similarity measures using similarity clouds. In: Hüllermeier, E., Minor, M. (eds.) Case-Based Reasoning Research and Development, pp. 149–164. Springer International Publishing, Cham (2015)

Hadsell, R., Chopra, S., LeCun, Y.: Dimensionality reduction by learning an invariant mapping. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06) IEEE, pp. 1735–1742 (2006)

He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

Hoffer, E., Ailon, N.: Deep metric learning using triplet network. In: International Workshop on Similarity-Based Pattern Recognition, pp. 84–92. Springer, Berlin (2015)

Hüllermeier, E., Cheng, W.: Preference-based CBR: general ideas and basic principles. In: IJCAI, pp. 3012–3016 (2013)

Hüllermeier, E., Schlegel, P.: Preference-based CBR: first steps toward a methodological framework. In: International Conference on Case-Based Reasoning, pp. 77–91. Springer, Berin (2011)

Koch, G., Zemel, R., Salakhutdinov, R.: Siamese neural networks for one-shot image recognition. In: ICML Deep Learning Workshop, vol. 2 (2015)

Lake, B.M., Salakhutdinov, R., Tenenbaum, J.B.: Human-level concept learning through probabilistic program induction. Science 350(6266), 1332–1338 (2015)

Langseth, H., Aamodt, A., Winnem, O.M.: Learning retrieval knowledge from data. In: Sixteenth International Joint Conference on Artificial Intelligence, Workshop ML-5: Automating the Construction of Case-Based Reasoners, pp. 77–82. Stockholm, Citeseer (1999)

Leake, D.B.: Case-Based Reasoning: Experiences, Lessons and Future Directions. MIT Press, Cambridge (1996)

Lefebvre, G., Garcia, C.: Learning a bag of features based nonlinear metric for facial similarity. In: 2013 10th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 238–243. IEEE (2013)

Maggini, M., Melacci, S., Sarti, L.: Learning from pairwise constraints by similarity neural networks. Neural Netw. 26, 141–158 (2012)

Martin, K., Wiratunga, N., Sani, S., Massie, S., Clos, J.: A convolutional siamese network for developing similarity knowledge in the selfback dataset. In: Proceedings of the International Conference on Case-Based Reasoning Workshops, CEUR Workshop Proceedings, ICCBR (Organisers), pp. 85–94 (2017)

Nikpour, H., Aamodt, A., Bach, K.: Bayesian-supported retrieval in BNCreek: A knowledge-intensive case-based reasoning system. In: International Conference on Case-Based Reasoning, pp. 323–338. Springer, Berlin (2018)

Reategui, E.B., Campbell, J.A., Leao, B.F.: Combining a neural network with case-based reasoning in a diagnostic system. Artif. Intell. Med. 9(1), 5–27 (1997)

Riedmiller, M., Braun, H.: A direct adaptive method for faster backpropagation learning: The RPROP algorithm. In: IEEE International Conference on Neural Networks, pp. 586–591. IEEE (1993)

Shawe-Taylor, J.: Symmetries and discriminability in feedforward network architectures. IEEE Trans. Neural Netw. 4(5), 816–826 (1993)

Stahl, A.: Learning feature weights from case order feedback. In: International Conference on Case-Based Reasoning, pp. 502–516. Springer, Berlin (2001)

Stahl, A., Gabel, T.: Using evolution programs to learn local similarity measures. In: International Conference on Case-Based Reasoning, pp. 537–551 (2003)

Stahl, A., Gabel, T.: Optimizing similarity assessment in case-based reasoning. In: Proceedings of the National Conference on Artificial Intelligence, Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 1999, vol. 21, p. 1667 (2006)

Tversky, A.: Features of similarity. Psychol. Rev. 84(4), 327 (1977)

Vinyals, O., Blundell, C., Lillicrap, T., Wierstra, D., et al.: Matching networks for one shot learning. In: Advances in Neural Information Processing Systems, pp. 3630–3638 (2016)

Wienhofen, L.W.M., Mathisen, B.M.: Defining the Initial Case-Base for a CBR Operator Support System in Digital Finishing, pp. 430–444. Springer International Publishing, Cham (2016). https://doi.org/10.1007/978-3-319-47096-2_29

Zagoruyko, S., Komodakis, N.: Learning to compare image patches via convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4353–4361 (2015)

Thông tin
Thông tin xuất bản

Progress in Artificial Intelligence

Thông tin tác giả