Pruning and quantization algorithm with applications in memristor-based convolutional neural network

Abdel-Hamid O, Ar Mohamed, Jiang H et al (2014) Convolutional neural networks for speech recognition. IEEE/ACM Trans Audio, Speech, Lang Process 22(10):1533–1545 Ajit A, Acharya K, Samanta A (2020) A review of convolutional neural networks. In: 2020 international conference on emerging trends in information technology and engineering (ic-ETITE), IEEE, pp 1–5 Albawi S, Mohammed TA, Al-Zawi S (2017) Understanding of a convolutional neural network. In: 2017 international conference on engineering and technology (ICET), IEEE, pp 1–6 Alzubaidi L, Zhang J, Humaidi AJ et al (2021) Review of deep learning: concepts, CNN architectures, challenges, applications, future directions. J big Data 8(1):1–74 Bisen D (2021) Deep convolutional neural network based plant species recognition through features of leaf. Multimed Tools Appl 80(4):6443–6456 Burr GW, Shelby RM, Sebastian A et al (2017) Neuromorphic computing using non-volatile memory. Adv Phys: X 2(1):89–124 Chen B, Cai F, Zhou J, et al (2015) Efficient in-memory computing architecture based on crossbar arrays. In: 2015 IEEE International Electron Devices Meeting (IEDM), IEEE, pp 17–5 Chen J, Wu Y, Yang Y et al (2021) An efficient memristor-based circuit implementation of squeeze-and-excitation fully convolutional neural networks. IEEE Trans Neural Netw Learn Syst 33(4):1779–1790 Dou G, Yu Y, Guo M et al (2017) Memristive behavior based on Ba-doped SrTIO3 films. Chin Phys Lett 34(3):038502 Drakopoulos F, Baby D, Verhulst S (2021) A convolutional neural-network framework for modelling auditory sensory cells and synapses. Commun Biol 4(1):1–17 Gandhi UD, Malarvizhi Kumar P, Chandra Babu G, et al (2021) Sentiment analysis on twitter data by using convolutional neural network (CNN) and long short term memory (LSTM). Wirel Pers Commun. https://doi.org/10.1007/s11277-021-08580-3 Gao F, Huang T, Sun J et al (2019) A new algorithm for SAR image target recognition based on an improved deep convolutional neural network. Cogn Comput 11(6):809–824 Ghimire D, Kil D, Kim Sh (2022) A survey on efficient convolutional neural networks and hardware acceleration. Electronics 11(6):945 Goldani MH, Safabakhsh R, Momtazi S (2021) Convolutional neural network with margin loss for fake news detection. Inf Process Manag 58(1):102418 Gu J, Wang Z, Kuen J et al (2018) Recent advances in convolutional neural networks. Pattern Recogn 77:354–377 Guo M, Zhu Y, Liu R et al (2022) An associative memory circuit based on physical memristors. Neurocomputing 472:12–23 Han S, Mao H, Dally WJ (2015) Deep compression: compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149 Hu X, Shi W, Zhou Y, et al (2022) Quantized and adaptive memristor based CNN (QA-mCNN) for image processing. Sci chin-inf Sci 65(1):119104. https://doi.org/10.1007/s11432-020-3031-9 Huang HM, Wang Z, Wang T et al (2020) Artificial neural networks based on memristive devices: from device to system. Adv Intell Syst 2(12):2000149 Jeong H, Shi L (2018) Memristor devices for neural networks. J Phys D Appl Phys 52(2):023003 Ji X, Hu X, Zhou Y et al (2019) Adaptive sparse coding based on memristive neural network with applications. Cogn Neurodyn 13(5):475–488 LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444 Lehtonen E, Laiho M (2010) CNN using memristors for neighborhood connections. In: 2010 12th International Workshop on Cellular Nanoscale Networks and their Applications (CNNA 2010), IEEE, pp 1–4 Li C, Belkin D, Li Y et al (2018) Efficient and self-adaptive in-situ learning in multilayer memristor neural networks. Nat Commun 9(1):1–8 Li H, Wang S, Zhang X et al (2021a) Memristive crossbar arrays for storage and computing applications. Adv Intell Syst 3(9):2100017 Li Z, Liu F, Yang W et al (2021) A survey of convolutional neural networks: analysis, applications, and prospects. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/TNNLS.2021.3084827 Liu J, Zhou Y, Duan S, et al (2022) Memristive neural network circuit implementation of associative learning with overshadowing and blocking. Cogn Neurodyn. https://doi.org/10.1007/s11571-022-09882-3 Mao WL, Chen WC, Wang CT et al (2021) Recycling waste classification using optimized convolutional neural network. Resour Conserv Recycl 164(105):132 Shafiee A, Nag A, Muralimanohar N et al (2016) ISAAC: a convolutional neural network accelerator with in-situ analog arithmetic in crossbars. ACM SIGARCH Comput Archit News 44(3):14–26 Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 Wang J, Zhuge F (2019) Memristive synapses for brain-inspired computing. Adv Mater Technol 4(3):1800544 Wang Z, Li C, Lin P et al (2019) In situ training of feed-forward and recurrent convolutional memristor networks. Nat Mach Intell 1(9):434–442 Wen S, Wei H, Yan Z et al (2019) Memristor-based design of sparse compact convolutional neural network. IEEE Trans Netw Sci Eng 7(3):1431–1440 Xia Z, Chen J, Huang Q et al (2020) Neural synaptic plasticity-inspired computing: a high computing efficient deep convolutional neural network accelerator. IEEE Trans Circuits Syst I Regul Pap 68(2):728–740 Xu X, Zhu Z, Wang Y et al (2022) Odor pattern recognition of a novel bio-inspired olfactory neural network based on kernel clustering. Commun Nonlinear Sci Numer Simul 109(106):274 Yakopcic C, Alom MZ, Taha TM (2016) Memristor crossbar deep network implementation based on a convolutional neural network. In: 2016 International joint conference on neural networks (IJCNN), IEEE, pp 963–970 Yakopcic C, Alom MZ, Taha TM (2017) Extremely parallel memristor crossbar architecture for convolutional neural network implementation. In: 2017 International Joint Conference on Neural Networks (IJCNN), IEEE, pp 1696–1703 Yang CHH, Qi J, Chen SYC et al (2021) Decentralizing feature extraction with quantum convolutional neural network for automatic speech recognition. In: ICASSP 2021–2021 IEEE International Conference on Acoustics. Speech and Signal Processing (ICASSP), IEEE, pp 6523–6527 Yao P, Wu H, Gao B et al (2020) Fully hardware-implemented memristor convolutional neural network. Nature 577(7792):641–646 Zhang Q, Zhang M, Chen T et al (2019) Recent advances in convolutional neural network acceleration. Neurocomputing 323:37–51 Zhang Y, Wang X, Friedman EG (2017) Memristor-based circuit design for multilayer neural networks. IEEE Trans Circuits Syst I Regul Pap 65(2):677–686 Zhang Y, Wang Z, Zhu J et al (2020) Brain-inspired computing with memristors: challenges in devices, circuits, and systems. Appl Phys Rev 7(1):011308 Zhang Y, Wu Z, Liu S et al (2021) A quantized convolutional neural network implemented with memristor for image denoising and recognition. Front Neurosci. https://doi.org/10.3389/fnins.2021.717222 Zhou Y, Hu X, Wang L et al (2021) Quantbayes: weight optimization for memristive neural networks via quantization-aware bayesian inference. IEEE Trans Circuits Syst I Regul Pap 68(12):4851–4861 Zhu M, Gupta S (2017) To prune, or not to prune: exploring the efficacy of pruning for model compression. arXiv preprint arXiv:1710.01878