Appropriateness of performance indices for imbalanced data classification: An analysis

Bishop, 2006 Rumelhart, 1986, Learning representations by back-propagating errors, Nature, 323, 533, 10.1038/323533a0 Sokolova, 2006, Beyond accuracy, f-score and ROC: a family of discriminant measures for performance evaluation, 1015 Das, 2018, Handling data irregularities in classification: foundations, trends, and future challenges, Pattern Recognit., 81, 674, 10.1016/j.patcog.2018.03.008 He, 2013 Haixiang, 2017, Learning from class-imbalanced data: Review of methods and applications, Expert Syst. Appl., 73, 220, 10.1016/j.eswa.2016.12.035 Branco, 2016, A survey of predictive modeling on imbalanced domains, ACM Comput. Surv., 49, 31, 10.1145/2907070 Japkowicz, 2006, Why question machine learning evaluation methods (an illustrative review of the shortcomings of current methods), 6 Buckland, 1994, The relationship between recall and precision, J. Am. Soc. Inf. Sci., 45, 12, 10.1002/(SICI)1097-4571(199401)45:1<12::AID-ASI2>3.0.CO;2-L Kubat, 1997, Addressing the curse of imbalanced training sets: one-sided selection, 97, 179 Hand, 2001, A simple generalisation of the area under the ROC curve for multiple class classification problems, Mach. Learn., 45, 171, 10.1023/A:1010920819831 Davis, 2006, The relationship between Precision-Recall and ROC curves, 233 Huang, 2016, Learning deep representation for imbalanced classification, 5375 N. Japkowicz, Assessment Metrics for Imbalanced Learning, John Wiley & Sons, Ltd, pp. 187–206. Daskalaki, 2006, Evaluation of classifiers for an uneven class distribution problem, Appl. Artif. Intell., 20, 381, 10.1080/08839510500313653 Ferri, 2009, An experimental comparison of performance measures for classification, Pattern Recognit. Lett., 30, 27, 10.1016/j.patrec.2008.08.010 Ballabio, 2018, Multivariate comparison of classification performance measures, Chemom. Intell. Lab. Syst., 174, 33, 10.1016/j.chemolab.2017.12.004 Joshi, 2002, On evaluating performance of classifiers for rare classes, 641 Liu, 2007, A framework for analyzing skew in evaluation metrics, 1 Sokolova, 2007, Performance measures in classification of human communications, 159 Sokolova, 2009, A systematic analysis of performance measures for classification tasks, Inf. Process. Manag., 45, 427, 10.1016/j.ipm.2009.03.002 Brzezinski, 2018, Visual-based analysis of classification measures and their properties for class imbalanced problems, Inf. Sci., 462, 242, 10.1016/j.ins.2018.06.020 Luque, 2019, The impact of class imbalance in classification performance metrics based on the binary confusion matrix, Pattern Recognit., 91, 216, 10.1016/j.patcog.2019.02.023 Núñez, 2017, Improving SVM classification on imbalanced datasets by introducing a new bias, J. Classif., 34, 427, 10.1007/s00357-017-9242-x Brzezinski, 2019, On the dynamics of classification measures for imbalanced and streaming data, IEEE Trans. Neural Netw. Learn. Syst., 1, 10.1109/TNNLS.2019.2899061 Yamazaki, 2007, Asymptotic bayesian generalization error when training and test distributions are different, 1079 Alaiz-Rodríguez, 2008, Assessing the impact of changing environments on classifier performance, 13 Rudd, 2018, The extreme value machine, IEEE Trans. Pattern Anal. Mach. Intell., 40, 762, 10.1109/TPAMI.2017.2707495 Bradley, 2006, Precision-recall operating characteristic (p-roc) curves in imprecise environments, 4, 123 F. Yu, Y. Zhang, S. Song, A. Seff, J. Xiao, LSUN: Construction of a large-scale image dataset using deep learning with humans in the loop, arXiv preprint arXiv:1506.03365 (2015). Deng, 2009, ImageNet: a large-scale hierarchical image database, 248 López, 2014, On the importance of the validation technique for classification with imbalanced datasets: Addressing covariate shift when data is skewed, Inf. Sci., 257, 1, 10.1016/j.ins.2013.09.038 Datta, 2018, Multiobjective support vector machines: Handling class imbalance with pareto optimality, IEEE Trans. Neural Netw. Learn. Syst., 30, 1602, 10.1109/TNNLS.2018.2869298 Szegedy, 2016, Rethinking the inception architecture for computer vision, 2818 Datta, 2019, Boosting with lexicographic programming: Addressing class imbalance without cost tuning, IEEE Trans. Knowl. Data Eng., 10.1109/TKDE.2019.2894148 Datta, 2015, Near-bayesian support vector machines for imbalanced data classification with equal or unequal misclassification costs, Neural Netw., 70, 39, 10.1016/j.neunet.2015.06.005 Seiffert, 2010, RUSBoost: a hybrid approach to alleviating class imbalance, IEEE Trans. Syst. Man Cybern. Part A: Syst. Hum., 40, 185, 10.1109/TSMCA.2009.2029559 Japkowicz, 2000, The class imbalance problem: Significance and strategies, 111 Breiman, 1984 Chawla, 2002, SMOTE: synthetic minority over-sampling technique, J. Artif. Intell. Res., 16, 321, 10.1613/jair.953 Madjarov, 2012, An extensive experimental comparison of methods for multi-label learning, Pattern Recognit., 45, 3084, 10.1016/j.patcog.2012.03.004 Carbonneau, 2018, Multiple instance learning: a survey of problem characteristics and applications, Pattern Recognit., 77, 329, 10.1016/j.patcog.2017.10.009 Lin, 2017, Focal loss for dense object detection, 2980 Zimbra, 2018, The state-of-the-art in twitter sentiment analysis: a review and benchmark evaluation, ACM Trans. Manag. Inf. Syst., 9, 5:1, 10.1145/3185045 Rudd, 2017, The extreme value machine, IEEE Trans. Pattern Anal. Mach. Intell., 40, 762, 10.1109/TPAMI.2017.2707495