An integrated classification model for incremental learning
Tóm tắt
Incremental Learning is a particular form of machine learning that enables a model to be modified incrementally, when new data becomes available. In this way, the model can adapt to the new data without the lengthy and time-consuming process required for complete model re-training. However, existing incremental learning methods face two significant problems: 1) noise in the classification sample data, 2) poor accuracy of modern classification algorithms when applied to modern classification problems. In order to deal with these issues, this paper proposes an integrated classification model, known as a Pre-trained Truncated Gradient Confidence-weighted (Pt-TGCW) model. Since the pre-trained model can extract and transform image information into a feature vector, the integrated model also shows its advantages in the field of image classification. Experimental results on ten datasets demonstrate that the proposed method outperform the original counterparts.
Tài liệu tham khảo
Ben-David S, Kushilevitz E, Mansour Y (1997) Online learning versus offline learning. Mach Learn 29(1):45–63
Bottou L (2012) Stochastic gradient descent tricks. Neural networks: tricks of the trade. https://doi.org/10.1007/978-3-642-35289-8_25
Crammer K, Dekel et al (2006) Online passive-aggressive algorithms. J Mach Learn Res 7(3):551–585
Crammer K, Dredze M, Kulesza (2009) Multi-class confidence weighted algorithms. Conference on Empirical Methods in Natural Language Processing, pp. 496–504
Crammer K, Kulesza A, Dredze (2013) Adaptive regularization of weight vectors. Machine Learning, pp. 345–352
Dredze M, Crammer K, Pereira F (2008) Confidence-weighted linear classification. International Conference DBLP, pp. 264–271
Dredze M, Crammer K, Pereira F (2008) Confidence-weighted linear classification. Proceedings of the 25th international conference on Machine learning, pp. 264–271
Duchi J, Singer Y (2009) Efficient online and batch learning using forward backward splitting. J Mach Learn Res 10:2899–2934
Freund Y, Schapire RE (1999) Large margin classification using the perceptron algorithm. Mach Learn 37(3):277–296
Giraud-Carrier C (2000) A note on the utility of incremental learning. AI Commun 13(4):215–223
Hu J, Yan C, Liu X, et al (2019) Truncated gradient confidence-weighted based online learning for imbalance streaming data. IEEE International Conference on Multimedia and Expo (ICME) pp. 133-138
Langford J, Li L, Zhang T (2009) Sparse on learning via truncated gradient. J Mach Learn Res 10:777–801
Orabona F, Crammer K (2010) New adaptive algorithms for online classification. Advances in Neural Information Processing Systems, pp 1840–1848
Rossenblatt F (1958) The perceptron: a probabilistic model for information storage and organization in the brain. Psychol Rev 65:386–408
Schmidhuber J (2015) Deep Learning in Neural Networks: An Overview. Neural Netw 61:85–117
Shalev-Shwartz S (2012) Online learning and online convex optimization. Found Trends® Mach Learn 4(2):107–194
Simonyan K, Zisserman A (2014) Very deep convolution networks for large-scale image recognition. In ICLR. https://arxiv.org/abs/1409.1556
Syed NA, Huan S, Kah L, Sung K (1999) Incremental learning with support vector machines. IEEE International Conference on Data Mining, IEEE Computer Society, 1999
Wang J, Zhao P, Hoi SCH (2014) Cost-sensitive online classification. IEEE Trans Knowl Data Eng 26(10):2425–2438
Wang J, Zhao P, Hoi SCH (2016) Soft confidence-weighted learning. ACM. 8:1–32. https://doi.org/10.1145/2932193
Xie H, Yang D, Sun N, Chen Z, Zhang Y (2019) Automated pulmonary nodule detection in CT images using deep convolution neural networks. Pattern Recogn 85:109–119
Xie H, Fang S, Zha Z, Yang Y et al (2019) Convolution attention networks for scene text recognition. ACM Trans Multimed Comput Commun Appl 15:1–17
Kivinen, J. , Smola, A. J. , & Williamson, R. C.(2004) Learning with Kernels. IEEE Transactions on Signal Processing 52(8):2165–2176
Yang L, Jin R, Lansing E, et al. (2009) Online learning by ellipsoid method. International Conference on Machine Learning. ACM
Yosinski J, Clune J, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks? International Conference on Neural Information Processing Systems, MIT Press, 2014
Beichen Zhang, Liang Li, et al. (2020) State-relabeling adversarial active learning, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8753–8762