A novel multistage deep belief network based extreme learning machine ensemble learning paradigm for credit risk assessment
Tóm tắt
To achieve high assessment accuracy for credit risk, a novel multistage deep belief network (DBN) based extreme learning machine (ELM) ensemble learning methodology is proposed. In the proposed methodology, three main stages, i.e., training subsets generation, individual classifiers training and final ensemble output, are involved. In the first stage, bagging sampling algorithm is applied to generate different training subsets for guaranteeing enough training data. Second, the ELM, an effective AI forecasting tool with the unique merits of time-saving and high accuracy, is utilized as the individual classifier, and diverse ensemble members can be accordingly formulated with different subsets and different initial conditions. In the final stage, the individual results are fused into final classification output via the DBN model with sufficient hidden layers, which can effectively capture the valuable information hidden in ensemble members. For illustration and verification, the experimental study on one publicly available credit risk dataset is conducted, and the results show the superiority of the proposed multistage DBN-based ELM ensemble learning paradigm in terms of high classification accuracy.
Tài liệu tham khảo
Breiman L (1996) Bagging predictors. Mach Learn 24(2):123–140
Chi BW, Hsu CC (2012) A hybrid approach to integrate genetic algorithm into dual scoring model in enhancing the performance of credit scoring model. Expert Syst Appl 39(3):2650–2661
Grablowsky BJ, Talley WK (1981) Probit and discriminant functions for classifying credit applicants—a comparison. J Econ Bus 33(3):254–261
Hansen LK, Salamon P (1990) Neural network ensembles. IEEE Trans Pattern Anal 12(10):993–1001
Henley WE, Hand DJ (1996) A k-nearest-neighbour classifier for assessing consumer credit risk. Statistician 45:77–95
Hinton GE (2012) A practical guide to training restricted boltzmann machines. In: Montavon G, Orr GB, Müller K-R (eds) Neural networks: tricks of the trade. Lecture notes in computer science, 2nd edn. Springer, Berlin, pp 599–619
Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507
Hinton GE, Osindero S, Teh YW (2006) A fast learning algorithm for deep belief nets. Neural Comput 18(7):1527–1554
Hopfield JJ (1982) Neural networks and physical systems with emergent collective computational abilities. Proc Natl Acad Sci USA 79(8):2554–2558
Huang GB, Zhu QY, Siew CK (2004) Extreme learning machine: a new learning scheme of feedforward neural networks. In: Neural networks, proceedings. 2004 IEEE International Joint Conference on, IEEE, vol 2. pp 985–990
Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1):489–501
Kim Y, Lee H, Provost EM (2013) Deep learning for robust feature generation in audiovisual emotion recognition. In: Acoustics, speech and signal processing (ICASSP), 2013 IEEE International Conference on, IEEE. pp 3687–3691
Kou G, Peng Y, Shi Y, Wise M, Xu W (2005) Discovering credit cardholders’ behavior by multiple criteria linear programming. Ann Oper Res 135(1):261–274
Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ (eds) Advances in neural information processing systems, pp 1097–1105
Lai KK, Yu L, Zhou L, Wang S (2006a) Credit risk evaluation with least square support vector machine. Lect Notes Comput Sci 4062:490–495
Lai KK, Yu L, Wang S, Zhou L (2006b) Credit risk analysis using a reliability-based neural network ensemble model. Lect Notes Comput Sci 4132:682–690
Lai KK, Yu L, Wang S, Zhou L (2006c) Neural network metalearning for credit scoring. Lect Notes Comput Sci 4113:403–408
Lin WY, Hu YH, Tsai CF (2012) Machine learning in financial crisis prediction: a survey. IEEE Trans Syst Man Cybern C 42(4):421–436
Mangasarian OL (1965) Linear and nonlinear separation of patterns by linear programming. Oper Res 13(3):444–452
Mohamed A, Dahl GE, Hinton GE (2012) Acoustic modeling using deep belief networks. IEEE Trans Audio Speech 20(1):14–22
Oreski S, Oreski D, Oreski G (2012) Hybrid system with genetic algorithm and artificial neural networks and its application to retail credit risk assessment. Expert Syst Appl 39(16):12605–12617
Rao CR, Mitra SK (1971) Generalized inverse of matrices and its applications. Wiley, New York
Serre D (2002) Matrices: theory and applications. Springer, New York
Szymanski L, McCane B (2014) Deep networks are effective encoders of periodicity. IEEE Trans Neural Netw 25(10):1816–1827
Tang TC, Chi LC (2005) Predicting multilateral trade credit risks: comparisons of logit and fuzzy logic models using ROC curve analysis. Expert Syst Appl 28(3):547–556
Tang L, Dai W, Yu L, Wang S (2015) A novel CEEMD-based EELM ensemble learning paradigm for crude oil price forecasting. Int J Inf Technol Decis 14(01):141–169
Wang Y, Wang S, Lai KK (2005) A new fuzzy support vector machine to evaluate credit risk. IEEE Trans Fuzzy Syst 13(6):820–831
West D (2000) Neural network credit scoring models. Comput Oper Res 27(11):1131–1152
Yang S, Browne A (2004) Neural network ensembles: combining multiple models for enhanced performance using a multistage approach. Expert Syst 21(5):279–288
Yu L, Yao X (2013) A total least squares proximal support vector classifier for credit risk evaluation. Soft Comput 17(4):643–650
Yu L, Lai KK, Wang S, Huang W (2006) A bias-variance-complexity trade-off framework for complex system modeling. Lect Notes Comput Sci 3980:518–527
Yu L, Wang S, Lai KK (2008) Credit risk assessment with a multistage neural network ensemble learning approach. Expert Syst Appl 34(2):1434–1444
Yu L, Yue W, Wang S, Lai KK (2010) Support vector machine based multiagent ensemble learning for credit risk evaluation. Expert Syst Appl 37(2):1351–1360
Yu L, Yao X, Wang S, Lai KK (2011) Credit risk evaluation using a weighted least squares SVM classifier with design of experiment for parameter selection. Expert Syst Appl 38(12):15392–15399
Yu L, Dai W, Tang L (2015) A novel decomposition ensemble model with extended extreme learning machine for crude oil price forecasting. Eng Appl Artif Intell. doi:10.1016/j.engappai.2015.04.016
Zhang D, Zhou X, Leung SC, Zheng J (2010) Vertical bagging decision trees model for credit scoring. Expert Syst Appl 37(12):7838–7843
Zhou H, Niu WJ, Wang Y (2005) Discriminant analysis of clients’ credit management in electricity market. In: Transmission and distribution conference and exhibition: Asia and Pacific, 2005 IEEE/PES. pp 1–6