Supervised Training Using an Unsupervised Approach to Active Learning

Springer Science and Business Media LLC - 2002
A. P. Engelbrecht1, R. Brits1
1Department of Computer Science, University of Pretoria, Pretoria, South Africa

Tóm tắt

Active learning algorithms allow neural networks to dynamically take part in the selection of the most informative training patterns. This paper introduces a new approach to active learning, which combines an unsupervised clustering of training data with a pattern selection approach based on sensitivity analysis. Training data is clustered into groups of similar patterns based on Euclidean distance, and the most informative pattern from each cluster is selected for training using the sensitivity analysis incremental learning algorithm in (Engelbrecht and Cloete, 1999). Experimental results show that the clustering approach improves on standard active learning as presented in (Engelbrecht and Cloete, 1999).

Từ khóa


Tài liệu tham khảo

Cohn, D. A.: Neural network exploration using optimal experiment design. AI Memo No 1491, Artificial Intelligence Laboratory, Massachusetts Institute of Technology, 1994.

Cohn, D. A., Atlas, L. and Ladner, R.: Improving Generalization with Active Learning. Machine Learning, 15 (1994), 201–221.

Cohn, D. A., Ghahramani, Z. and Jordan, M. I.: Active learning with statistical models. Journal of Artificial Intelligence Research, 4 (1996), 129–145.

Engelbrecht, A. P. and Cloete, I.: Selective learning using sensitivity analysis. In: Proceedings of IEEE World Congress on Computational Intelligence, Anchorage, Alaska, (1988), pp. 1150–1155.

Engelbrecht, A. P. and Cloete, I.: Incremental learning using sensitivity analysis. In IEEE International Joint Conference on Neural Networks, Washington DC, USA, paper 380, 1999.

Fukumizu, K.: Active learning in multilayer perceptrons. In: D. S. Touretzky, M. C. Mozer and M. E. Hasselmo, (eds.), Advances in Neural Information Processing Systems, 8 (1996), 295–301.

Hampshire, J. B. and Waibel, A. H.: A Novel objective function for improved phoneme recognition using time-delay neural networks. IEEE Transactions on Neural Networks, 1(2) (1990), 216–228.

Hunt, S. D. and Deller, Jr. J. R.: Selective training of feedforward artificial neural networks using matrix perturbation theory. Neural Networks, 8(6) (1995), 931–944.

Hwang, J-N., Choi, J. J., Oh, S. and Marks II, R. J.: Query-based learning applied to partially trained multilayer perceptrons. IEEE Transactions on Neural Networks, 2(1) (1991), 131–136.

MacKay, D. J. C.: Bayesian Methods for Adaptive Models. PhD Thesis, California Institute of Technology, 1992a.

MacKay, D. J. C.: Information-based objective functions for active data selection. Neural Computation, 4 (1992b), 590–604.

Plutowski, H. and White, H. Selecting concise training sets from clean data. IEEE Transactions on Neural Networks, 4(2) (1993), 305–318.

Röbel, A.: Dynamic pattern selection: effectively training backpropagation neural networks. International Conference on Artificial Neural Networks, 1 (1994a), 643–646.

Röbel, A.: Dynamic pattern selection for faster learning and controlled generalization of neural networks. European Symposium on Artificial Neural Networks, 1994b.

Seung, H. S., Opper, M. and Sompolinsky, H.: Query by Committee. In: Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory (1992), pp. 287–299.

Sung, K. K. and Niyogi, P.: A Formulation for active learning with applications to object detection. AI Memo No 1438, Artificial Intelligence Laboratory, Massachusetts Institute of Technology, 1996.

Zhang, B.-T.: Accelerated learning by active example Selection. International Journal of Neural Systems, 5(1) (1994), 67–75.

Thông tin
Thông tin xuất bản

Springer Science and Business Media LLC

Thông tin tác giả