A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms

Higgins, 2003

García, 2009, A study of statistical techniques and performance measures for genetics-based machine learning: Accuracy and interpretability, Soft Computing, 13, 959, 10.1007/s00500-008-0392-y

García, 2009, A study on the use of nonparametric tests for analyzing the evolutionary algorithms’ behaviour: A case study on the CEC’2005 special session on real parameter optimization, Journal of Heuristics, 15, 617, 10.1007/s10732-008-9080-4

Demšar, 2006, Statistical comparisons of classifiers over multiple data sets, Journal of Machine Learning Research, 7, 1

García, 2008, An extension on Statistical Comparisons of Classifiers over Multiple Data Sets for all pairwise comparisons, Journal of Machine Learning Research, 9, 2677

P. Suganthan, N. Hansen, J. Liang, K. Deb, Y. Chen, A. Auger, S. Tiwari, Problem definitions and evaluation criteria for the CEC’2005 special session on real parameter optimization, Nanyang Technological University, Tech. Rep., 2005, Available in http://www.ntu.edu.sg/home/epnsugan/index_files/cec-05/Tech-Report-May-30-05.pdf.

J. Kennedy, R. Eberhart, Particle swarm optimization, in: Proceedings of IV IEEE International Conference on Neural Networks, Piscataway, New Jersey, 1995, pp. 1942–1948.

A. Auger, N. Hansen, A restart CMA evolution strategy with increasing population size, in: Proceedings of the 2005 IEEE Congress on Evolutionary Computation, 2005, pp. 1769–1776.

Eshelman, 1991, The CHC adaptative search algorithm: How to have safe search when engaging in nontraditional genetic recombination, 265, 10.1016/B978-0-08-050684-5.50020-3

Eshelman, 1993, Real-coded genetic algorithms and interval-schemata, 187, 10.1016/B978-0-08-094832-4.50018-0

C. Fernandes, A. Rosa, A study of non-random matching and varying population size in genetic algorithm using a royal road function, in: Proceedings of the 2001 Congress on Evolutionary Computation, Piscataway, New Jersey, 2001, pp. 60–66.

Mülenbein, 1993, Predictive models for the breeding genetic algorithm in continuous parameter optimization, Evolutionary Computation, 1, 25, 10.1162/evco.1993.1.1.25

Laguna, 2003

Herrera, 2006, Continuous scatter search: An analysis of the integration of some combination methods and improvement strategies, European Journal of Operational Research, 169, 450, 10.1016/j.ejor.2004.08.009

Price, 2005

A.K. Qin, P.N. Suganthan, Self-adaptive differential evolution algorithm for numerical optimization, in: Proceedings of the 2005 IEEE Congress on Evolutionary Computation, vol. 2, 2005, pp. 1785–1791.

Bartz-Beielstein, 2006

Ortiz-Boyer, 2007, Improving crossover operators for real-coded genetic algorithms using virtual parents, Journal of Heuristics, 13, 265, 10.1007/s10732-007-9018-2

Conover, 1998

Hollander, 1999

Fisher, 1959

Sheskin, 2006

García-Martínez, 2010, Evaluating a local genetic algorithm as context-independent local search operator for metaheuristics, Soft Computing, 14, 1117, 10.1007/s00500-009-0506-1

Gibbons, 2010

Zar, 2009

Rhyne, 1965, Tables for a treatments versus control multiple comparisons sign test, Technometrics, 7, 293, 10.2307/1266590

Steel, 1959, A multiple comparison sign test: treatments versus control, Journal of American Statistical Association, 54, 767, 10.2307/2282500

Friedman, 1937, The use of ranks to avoid the assumption of normality implicit in the analysis of variance, Journal of the American Statistical Association, 32, 674, 10.2307/2279372

Friedman, 1940, A comparison of alternative tests of significance for the problem of m rankings, Annals of Mathematical Statistics, 11, 86, 10.1214/aoms/1177731944

Iman, 1980, Approximations of the critical region of the friedman statistic, Communications in Statistics, 9, 571, 10.1080/03610928008827904

Hodges, 1962, Ranks methods for combination of independent experiments in analysis of variance, Annals of Mathematical Statistics, 33, 482, 10.1214/aoms/1177704575

Kruskal, 1952, Use of ranks in one-criterion variance analysis, Journal of the American Statistical Association, 47, 583, 10.2307/2280779

Quade, 1979, Using weighted rankings in the analysis of complete blocks with additive block effects, Journal of the American Statistical Association, 74, 680, 10.2307/2286991

Abramowitz, 1974

Daniel, 2000

Westfall, 2004

Dunn, 1961, Multiple comparisons among means, Journal of the American Statistical Association, 56, 52, 10.2307/2282330

Holm, 1979, A simple sequentially rejective multiple test procedure, Scandinavian Journal of Statistics, 6, 65

Holland, 1987, An improved sequentially rejective Bonferroni test procedure, Biometrics, 43, 417, 10.2307/2531823

Finner, 1993, On a monotonicity problem in step-down multiple test procedures, Journal of the American Statistical Association, 88, 920, 10.2307/2290782

Hochberg, 1988, A sharper Bonferroni procedure for multiple tests of significance, Biometrika, 75, 800, 10.1093/biomet/75.4.800

Hommel, 1988, A stagewise rejective multiple test procedure based on a modified Bonferroni test, Biometrika, 75, 383, 10.1093/biomet/75.2.383

Rom, 1990, A sequentially rejective test procedure based on a modified Bonferroni inequality, Biometrika, 77, 663, 10.1093/biomet/77.3.663

Li, 2008, A two-step rejection procedure for testing multiple hypotheses, Journal of Statistical Planning and Inference, 138, 1521, 10.1016/j.jspi.2007.04.032

Doksum, 1967, Robust procedures for some linear models with one observation per cell, Annals of Mathematical Statistics, 38, 878, 10.1214/aoms/1177698881

García, 2010, Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power, Information Sciences, 180, 2044, 10.1016/j.ins.2009.12.010

P.B. Nemenyi, Distribution-free Multiple comparisons, Master’s thesis, Princeton University, 1963.

Shaffer, 1986, Modified sequentially rejective multiple test procedures, Journal of American Statistical Association, 81, 826, 10.2307/2289016

Bergmann, 1988, Improvements of general multiple test procedures for redundant systems of hypotheses, 100

Hommel, 1994, A rapid algorithm and a computer program for multiple test procedures using procedures using logical structures of hypotheses, Computer Methods and Programs in Biomedicine, 43, 213, 10.1016/0169-2607(94)90072-8

Wright, 1992, Adjusted p-values for simultaneous inference, Biometrics, 48, 1005, 10.2307/2532694

Alcalá-Fdez, 2008, KEEL: a software tool to assess evolutionary algorithms for data mining problems, Soft Computing, 13, 307, 10.1007/s00500-008-0323-y

Alcalá-Fdez, 2011, KEEL data-mining software tool: Data set repository, integration of algorithms and experimental analysis framework, Journal of Multiple-Valued Logic and Soft Computing, 17, 255