Evolutionary architecture search via adaptive parameter control and gene potential contribution

LeCun, 2015, Deep learning, Nature, 521, 436, 10.1038/nature14539 A. Plebe, J.F. Kooij, G.P.R. Papini, M. Da Lio, Occupancy grid mapping with cognitive plausibility for autonomous driving applications, in: Proceedings of the IEEE International Conference on Computer Vision, 2021, pp. 2934–2941. Shang, 2022, Hyperspectral image classification based on pyramid coordinate attention and weighted self-distillation, IEEE Trans. Geosci. Remote Sens., 1 Darwish, 2020, An optimized model based on convolutional neural networks and orthogonal learning particle swarm optimization algorithm for plant diseases diagnosis, Swarm Evol. Comput., 52, 10.1016/j.swevo.2019.100616 Shang, 2022, Spatial feature-based convolutional neural network for PolSAR image classification, Appl. Soft Comput., 123, 10.1016/j.asoc.2022.108922 Galassi, 2021, Attention in natural language processing, IEEE Trans. Neural Networks Learn. Syst., 32, 4291, 10.1109/TNNLS.2020.3019893 Krizhevsky, 2012, Imagenet classification with deep convolutional neural networks, 1097 K. Simonyan, A. Zisserman, Very Deep Convolutional Networks for Large-Scale Image Recognition, in: Proceedings of the 3rd International Conference on Learning Representations, 2015, pp. 1–14. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich, Going deeper with convolutions, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 1–9. K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778. X. Zhang, X. Zhou, M. Lin, J. Sun, Shufflenet: An extremely efficient convolutional neural network for mobile devices, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 6848–6856. M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, L.-C. Chen, Mobilenetv2: Inverted residuals and linear bottlenecks, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 4510–4520. He, 2021, Automl: A survey of the state-of-the-art, Knowl.-Based Syst., 212, 10.1016/j.knosys.2020.106622 B. Zoph, Q.V. Le, Neural architecture search with reinforcement learning, in: Proceedings of the 5th International Conference on Learning Representations, 2017, pp. 1–16. H. Liu, K. Simonyan, Y. Yang, DARTS: Differentiable Architecture Search, in: 7th International Conference on Learning Representations, 2019, pp. 1–13. Shang, 2022, Evolutionary neural architecture search based on evaluation correction and functional units, Knowl.-Based Syst., 251, 10.1016/j.knosys.2022.109206 Fernandes, 2019, Particle swarm optimization of deep neural networks architectures for image classification, Swarm Evol. Comput., 49, 62, 10.1016/j.swevo.2019.05.010 B. Zoph, V. Vasudevan, J. Shlens, Q.V. Le, Learning transferable architectures for scalable image recognition, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 8697–8710. S. Xie, H. Zheng, C. Liu, L. Lin, SNAS: stochastic neural architecture search, in: Proceedings of the 7th International Conference on Learning Representations, 2019, pp. 1–17. X. Dong, Y. Yang, Searching for a robust neural architecture in four gpu hours, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 1761–1770. B. Wu, X. Dai, P. Zhang, Y. Wang, F. Sun, Y. Wu, Y. Tian, P. Vajda, Y. Jia, K. Keutzer, Fbnet: Hardware-aware efficient convnet design via differentiable neural architecture search, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 10734–10742. Li, 2020, Evolution strategies for continuous optimization: A survey of the state-of-the-art, Swarm Evol. Comput., 56, 10.1016/j.swevo.2020.100694 E. Real, S. Moore, A. Selle, S. Saxena, Y.L. Suematsu, J. Tan, Q.V. Le, A. Kurakin, Large-scale evolution of image classifiers, in: Proceedings of the 34th International Conference on Machine Learning, 2017, pp. 2902–2911. M. Suganuma, S. Shirakawa, T. Nagao, A Genetic Programming Approach to Designing Convolutional Neural Network Architectures, in: Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, 2018, pp. 5369–5373. Shi, 2022, Genetic-GNN: Evolutionary architecture search for graph neural networks, Knowl.-Based Syst., 247, 10.1016/j.knosys.2022.108752 Sun, 2020, Completely automated CNN architecture design based on blocks, IEEE Trans. Neural Networks Learn. Syst., 31, 1242, 10.1109/TNNLS.2019.2919608 Sun, 2020, Evolving deep convolutional neural networks for image classification, IEEE Trans. Evol. Comput., 24, 394, 10.1109/TEVC.2019.2916183 He, 2021, Simple-encoded evolving convolutional neural network and its application to skin disease image classification, Swarm Evol. Comput., 67, 10.1016/j.swevo.2021.100955 Sun, 2020, Surrogate-assisted evolutionary deep learning using an end-to-end random forest-based performance predictor, IEEE Trans. Evol. Comput., 24, 350, 10.1109/TEVC.2019.2924461 B. Wang, Y. Sun, B. Xue, M. Zhang, A hybrid differential evolution approach to designing deep convolutional neural networks for image classification, in: Proceedings of Australasian Conference on Artificial Intelligence, 2018, pp. 237–250. Sun, 2020, Automatically designing CNN architectures using the genetic algorithm for image classification, IEEE Trans. Cybern., 50, 3840, 10.1109/TCYB.2020.2983860 Eiben, 2003, Multimodal problems and spatial distribution, 153 D. Song, C. Xu, X. Jia, Y. Chen, C. Xu, Y. Wang, Efficient residual dense block search for image super-resolution, in: Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence, Vol. 34, 2020, pp. 12007–12014. P.R. Lorenzo, J. Nalepa, Memetic evolution of deep neural networks, in: Proceedings of the Genetic and Evolutionary Computation Conference, 2018, pp. 505–512. K. Maziarz, A. Khorlin, Q.d. Laroussilhe, A. Gesmundo, Evolutionary-Neural Hybrid Agents for Architecture Search, in: International Conference on Learning Representations, 2019. Lu, 2021, Multiobjective evolutionary design of deep convolutional neural networks for image classification, IEEE Trans. Evol. Comput., 25, 277, 10.1109/TEVC.2020.3024708 E. Real, A. Aggarwal, Y. Huang, Q.V. Le, Regularized Evolution for Image Classifier Architecture Search, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33, No. 01, 2019, pp. 4780–4789. McGinley, 2011, Maintaining healthy population diversity using adaptive crossover, mutation, and selection, IEEE Trans. Evol. Comput., 15, 692, 10.1109/TEVC.2010.2046173 G. Chaslot, S. Bakkes, I. Szita, P. Spronck, Monte-Carlo Tree Search: A New Framework for Game AI, in: Proceedings of the Fourth Artificial Intelligence and Interactive Digital Entertainment Conference, Vol. 8, 2008, pp. 216–217. Y. Tian, J. Ma, Q. Gong, S. Sengupta, Z. Chen, J. Pinkerton, L. Zitnick, Elf opengo: An analysis and open reimplementation of alphazero, in: Proceedings of the 36th International Conference on Machine Learning, 2019, pp. 6244–6253. Xiao, 2017 H. Larochelle, D. Erhan, A. Courville, J. Bergstra, Y. Bengio, An empirical evaluation of deep architectures on problems with many factors of variation, in: Proceedings of the 24th International Conference on Machine Learning, 2007, pp. 473–480. Krizhevsky, 2009 Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, A.Y. Ng, Reading Digits in Natural Images with Unsupervised Feature Learning, in: NIPS Workshop on Deep Learning and Unsupervised Feature Learning 2011, 2011. J.T. Springenberg, A. Dosovitskiy, T. Brox, M. Riedmiller, Striving for Simplicity: The All Convolutional Net, in: Proceedings of the 2015 International Conference on Learning Representations, San Diego, CA, 2015. G. Huang, Z. Liu, L. Van Der Maaten, K.Q. Weinberger, Densely connected convolutional networks, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 4700–4708. G. Larsson, M. Maire, G. Shakhnarovich, FractalNet: Ultra-Deep Neural Networks without Residuals, in: ICLR, 2017. M. Lin, Q. Chen, S. Yan, Network In Network, in: International Conference on Learning Representations, Banff, Canada, 2014. P. Sermanet, S. Chintala, Y. LeCun, Convolutional neural networks applied to house numbers digit classification, in: Proceedings of the 21st International Conference on Pattern Recognition, ICPR2012, 2012, pp. 3288–3291. Iandola, 2016 S. Rifai, P. Vincent, X. Muller, X. Glorot, Y. Bengio, Contractive auto-encoders: Explicit invariance during feature extraction, in: Proceedings of the 28th International Conference on Machine Learning, 2011, pp. 833–840. K. Sohn, H. Lee, Learning invariant representations with local transformations, in: Proceedings of the 29th International Conference on Machine Learning, 2012, pp. 1–8. K. Sohn, G. Zhou, C. Lee, H. Lee, Learning and selecting features jointly with point-wise gated Boltzmann machines, in: International Conference on Machine Learning, 2013, pp. 217–225. Bruna, 2013, Invariant scattering convolution networks, IEEE Trans. Pattern Anal. Mach. Intell., 35, 1872, 10.1109/TPAMI.2012.230 Chan, 2015, PCANet: A simple deep learning baseline for image classification?, IEEE Trans. Image Process., 24, 5017, 10.1109/TIP.2015.2475625 G. Yuan, B. Xue, M. Zhang, A Graph-Based Approach to Automatic Convolutional Neural Network Construction for Image Classification, in: 35th International Conference on Image and Vision Computing New Zealand, 2020, pp. 1–6. B. Wang, Y. Sun, B. Xue, M. Zhang, Evolving deep convolutional neural networks by variable-length particle swarm optimization for image classification, in: 2018 IEEE Congress on Evolutionary Computation, CEC, 2018, pp. 1–8. Bi, 2020, Genetic programming with image-related operators and a flexible program structure for feature learning in image classification, IEEE Trans. Evol. Comput., 25, 87, 10.1109/TEVC.2020.3002229 Assunção, 2019, DENSER: deep evolutionary network structured representation, Genet. Program. Evolvable Mach., 20, 5, 10.1007/s10710-018-9339-y E. Dufourq, B.A. Bassett, Eden: Evolutionary deep networks for efficient machine learning, in: 2017 Pattern Recognition Association of South Africa and Robotics and Mechatronics, PRASA-RobMech, 2017, pp. 110–115. Paszke, 2019, Pytorch: An imperative style, high-performance deep learning library, 8026