1D convolutional neural networks and applications: A survey

A. Cichoki, R. Unbehauen, Neural Networks for Optimization and Signal Processing, thirrd ed., 1994.

S.O. Haykin, Neural Networks and Learning Machines, 2008. doi:978-0131471399.

Warren, 1943, A logical calculus of the ideas immanent in nervous activity, Bull. Math. Biophys., 5, 115, 10.1007/BF02478259

Rosenblatt, 1958, The perceptron: a probabilistic model for information storage and organization in the brain, Psychol. Rev., 65, 386, 10.1037/h0042519

J.P. Resop, A Comparison of Artificial Neural Networks and Statistical Regression with Biological Resources Applications, 2006.

Ince, 2010, Evaluation of global and local training techniques over feed-forward neural network architecture spaces for computer-aided medical diagnosis, Expert Syst. Appl., 37, 8450, 10.1016/j.eswa.2010.05.033

Kiranyaz, 2016, Progressive operational perceptrons, Neurocomputing

S. Kiranyaz, T. Ince, A. Iosifidis, M. Gabbouj, Generalized model of biological neural networks: progressive operational perceptrons, in: Proc. Int. Jt. Conf. Neural Networks. 2017–May (2017) 2477–2485. https://doi.org/10.1109/IJCNN.2017.7966157.

T.W. Rauber, K. Berns, Kernel multilayer perceptron, in: Proc. - 24th SIBGRAPI Conf. Graph. Patterns Images, 2011 pp. 337–343. https://doi.org/10.1109/SIBGRAPI.2011.21.

H. Ogai, B. Bhattacharya, Pipe Inspection Robots for Structural Health and Condition Monitoring, 2018. https://doi.org/10.1007/978-81-322-3751-8.

Mashor, 2000, Hybrid multilayered perceptron networks, Int. J. Syst. Sci., 31, 771, 10.1080/00207720050030815

Kiranyaz, 2009, Evolutionary artificial neural networks by multi-dimensional particle swarm optimization, Neural Networks, 22, 1448, 10.1016/j.neunet.2009.05.013

Ince, 2009, A generic and robust system for automated patient-specific classification of ECG signals, IEEE Trans. Biomed. Eng., 56, 1415, 10.1109/TBME.2009.2013934

Kiranyaz, 2014

Mohseni, 2012, Optimization of neural networks using variable structure systems, IEEE Trans. Syst. Man, Cybern. Part B Cybern., 42, 1645, 10.1109/TSMCB.2012.2197610

Hubel, 1959, Receptive fields of single neurones in the cat’s striate cortex, J. Physiol., 148, 574, 10.1113/jphysiol.1959.sp006308

Hubel, 1960, Single unit activity in lateral geniculate body and optic tract of unrestrained cats, J. Physiol., 150, 91, 10.1113/jphysiol.1960.sp006375

Hubel, 1962, Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex, J. Physiol., 160, 106, 10.1113/jphysiol.1962.sp006837

Hubel, 1963, Receptive fields of cells in striate cortex of very young, visually inexperienced kittens, J. Neurophysiol., 26, 994, 10.1152/jn.1963.26.6.994

Hubel, 1968, Receptive fields and functional architecture of monkey striate cortex, J. Physiol., 195, 215, 10.1113/jphysiol.1968.sp008455

Fukushima, 1982, Neocognitron: a new algorithm for pattern recognition tolerant of deformations and shifts in position, Pattern Recognit., 15, 455, 10.1016/0031-3203(82)90024-3

Rumelhart, 1986, Learning representations by back-propagation errors, Nature, 323, 533, 10.1038/323533a0

LeCun, 1990, Handwritten digit recognition with a back-propagation network, Adv. Neural Inf. Process. Syst., 396

Y. LeCun, Gradient Based Learning Applied To Document Recognition, 1998, pp. 1–46.

Yann, 2018

Krizhevsky, 2012, ImageNet classification with deep convolutional neural networks, Adv. Neural Inf. Process. Syst., 25, 1097

J. Deng, W. Dong, R. Socher, L.-J. Li, Kai Li, Li Fei-Fei, ImageNet: a large-scale hierarchical image database, in: 2009 IEEE Conf. Comput. Vis. Pattern Recognit., 2009, pp. 248–255. https://doi.org/10.1109/CVPR.2009.5206848.

Juan, 2010, A comparison of SIFT, PCA-SIFT and SURF, Int. J. Image Process, 3, 143

Ojala, 2005, Multiresolution gray-scale and rotation invariant texture classification with local binary patterns, IEEE Trans. Pattern Anal. Mach. Intell., 24, 971, 10.1109/TPAMI.2002.1017623

Srivastava, 2004, Dropout: a simple way to prevent neural networks from overfitting, J. Mach. Learn. Res., 1, 11

M.D. Zeiler, R. Fergus, Visualizing and understanding convolutional networks, Lect. Notes Comput. Sci. (Including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics). 8689 LNCS (2014) 818–833. https://doi.org/10.1007/978-3-319-10590-1_53.

O. Abdel-Hamid, A.R. Mohamed, H. Jiang, G. Penn, Applying convolutional neural networks concepts to hybrid NN-HMM model for speech recognition, in: ICASSP, IEEE Int. Conf. Acoust. Speech Signal Process. - Proc., 2012, pp. 4277–4280. https://doi.org/10.1109/ICASSP.2012.6288864.

Y. Kim, Convolutional neural networks for sentence classification, in: Proc. 2014 Conf. Empir. Methods Nat. Lang. Process., 2014, pp. 1746–1751.

Lu, 2017, Intelligent fault diagnosis of rolling bearing using hierarchical convolutional network based health state classification, Adv. Eng. Inform., 32, 139, 10.1016/j.aei.2017.02.005

Ding, 2017, Energy-fluctuated multiscale feature learning with deep ConvNet for Intelligent spindle bearing fault diagnosis, IEEE Trans. Instrum. Meas., 66, 1926, 10.1109/TIM.2017.2674738

Guo, 2016, Hierarchical adaptive deep convolution neural network and its application to bearing fault diagnosis, Meas. J. Int. Meas. Confed., 93, 490, 10.1016/j.measurement.2016.07.054

Janssens, 2016, Convolutional neural network based fault detection for rotating machinery, J. Sound Vib., 377, 331, 10.1016/j.jsv.2016.05.027

D. Hoang, H. Kang, Convolutional Neural Network Based Bearing Fault Diagnosis, 10362 (2017) 105–111. https://doi.org/10.1007/978-3-319-63312-1.

Z. Wei, P. Gaoliang, L. Chuanhao, Bearings Fault Diagnosis Based on Convolutional Neural Networks with 2- D Representation of Vibration Signals as Input, 13001 (2017) 1–5.

Appana, 2017, Speed invariant bearing fault characterization using convolutional, Neural Networks, 189

Li, 2017, An ensemble deep convolutional neural network model with improved D-S evidence fusion for bearing fault diagnosis, Sensors, 17, 1729, 10.3390/s17081729

Lee, 2017, An adaptive deep convolutional neural network for rolling bearing fault diagnosis, Hindawi Shock Vib., 30, 1

J. Ruiz, J. Pérez, J. Blázquez, Arrhythmia detection using convolutional neural models, in: Int. Symp. Distrib. Comput. Artif. Intell., 2018.

Zihlmann, 2017, Convolutional recurrent neural networks for electrocardiogram classification, Computing

S. Kiranyaz, T. Ince, R. Hamila, M. Gabbouj, Convolutional Neural Networks for patient-specific ECG classification, in: Proc. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. EMBS, 2015. https://doi.org/10.1109/EMBC.2015.7318926.

Kiranyaz, 2016, Real-time patient-specific ECG classification by 1-D convolutional neural networks, IEEE Trans. Biomed. Eng., 63, 664, 10.1109/TBME.2015.2468589

Kiranyaz, 2017, Personalized monitoring and advance warning system for cardiac arrhythmias, Sci. Rep., 7, 10.1038/s41598-017-09544-z

Avci, 2018, Wireless and real-time structural damage detection: a novel decentralized method for wireless sensor networks, J. Sound Vib., 10.1016/j.jsv.2018.03.008

O. Avci, O. Abdeljaber, S. Kiranyaz, D. Inman, Structural damage detection in real time: implementation of 1D convolutional neural networks for SHM applications, in: C. Niezrecki (Ed.), Struct. Heal. Monit. Damage Detect. Vol. 7 Proc. 35th IMAC, A Conf. Expo. Struct. Dyn. 2017, Springer International Publishing, Cham, 2017, pp. 49–54. https://doi.org/10.1007/978-3-319-54109-9_6.

Abdeljaber, 2017, Real-time vibration-based structural damage detection using one-dimensional convolutional neural networks, J. Sound Vib., 388, 10.1016/j.jsv.2016.10.043

O. Avci, O. Abdeljaber, S. Kiranyaz, B. Boashash, H. Sodano, D.J. Inman, Efficiency Validation of One Dimensional Convolutional Neural Networks for Structural Damage Detection Using a SHM Benchmark Data, 2018.

Abdeljaber, 2017, 1-D CNNs for structural damage detection: verification on a structural health monitoring benchmark data, Neurocomputing

Ince, 2016, Real-time motor fault detection by 1-D convolutional neural networks, IEEE Trans. Ind. Electron., 63, 7067, 10.1109/TIE.2016.2582729

Kiranyaz, 2018, Real-time fault detection and identification for MMC using 1D convolutional neural networks, IEEE Trans. Ind. Electron.

Abdeljaber, 2018, Fault detection and severity identification of ball bearings by online condition monitoring, IEEE Trans. Ind. Electron.

Eren, 2019, A generic intelligent bearing fault diagnosis system using compact adaptive 1D CNN classifier, J. Signal Process. Syst., 91, 179, 10.1007/s11265-018-1378-3

Eren, 2017, Bearing fault detection by one-dimensional convolutional neural networks, Math. Probl. Eng., 2017, 10.1155/2017/8617315

Zhang, 2018, A deep convolutional neural network with new training methods for bearing fault diagnosis under noisy environment and different working load, Mech. Syst. Signal Process., 100, 439, 10.1016/j.ymssp.2017.06.022

Hinton, 2006, Reducing the dimensionality of data with neural networks, Science (80-), 10.1126/science.1127647

Acharya, 2017, Automated detection of arrhythmias using different intervals of tachycardia ECG segments with convolutional neural network, Inf. Sci. (Ny)

Z. Xiong, M. Stiles, J. Zhao, Robust ECG signal classification for detection of atrial fibrillation using a novel neural network, in: 2017 Comput. Cardiol., 2017.

Acharya, 2017, A deep convolutional neural network model to classify heartbeats, Comput. Biol. Med., 10.1016/j.compbiomed.2017.08.022

O. Avci, S. Kiranyaz, O. Abdeljaber, Structural Damage Detection (Public Website), 2019. http://www.structuraldamagedetection.com/.

Qian, 1999, On the momentum term in gradient descent learning algorithms, Neural Networks, 12, 145, 10.1016/S0893-6080(98)00116-6

J. Duchi, E. Hazan, Y. Singer, Adaptive subgradient methods for online learning and stochastic optimization, COLT 2010 - 23rd Conf. Learn. Theory (2010) 257–269.

T. Tieleman, G. Hinton, Lecture 6.5 - RMSProp, Neural Networks for Machine Learning | Coursera, (n.d.).

Diederik, 2014, ADAM: a method for stochastic optimization, AIP Conf. Proc., 1631, 58

S. Ruder, An overview of gradient descent optimization algorithms, 2016.

Jiang, 2010, Discriminative training of HMMs for automatic speech recognition: a survey, Comput. Speech Lang., 24, 589, 10.1016/j.csl.2009.08.002

He, 2008, Discriminative learning in sequential pattern recognition: a unifying review for optimization-oriented speech recognition, IEEE Signal Process. Mag., 25, 14, 10.1109/MSP.2008.926652

Deng, 2013, Machine learning paradigms for speech recognition: an overview, IEEE Trans. Audio, Speech Lang. Process., 21, 1060, 10.1109/TASL.2013.2244083

Hinton, 2012, Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups, IEEE Signal Process. Mag., 29, 82, 10.1109/MSP.2012.2205597

Abdel-Hamid, 2014, Convolutional neural networks for speech recognition, IEEE Trans. Audio, Speech Lang. Process., 22, 1533, 10.1109/TASLP.2014.2339736

H. Lee, L. Yan, P. Pham, A.Y. Ng, Unsupervised feature learning for audio classification using convolutional deep belief networks, in: Adv. Neural Inf. Process. Syst. 22 - Proc. 2009 Conf., 2009, pp. 1096–1104.

D. Hau, K. Chen, Exploring hierarchical speech representations with a deep convolutional neural network, in: Proc. UKCI’11, 2011.

M. Bi, Y. Qian, K. Yu, Very deep convolutional neural networks for LVCSR, in: Proc. Annu. Conf. Int. Speech Commun. Assoc. INTERSPEECH. 2015–Janua, 2015, pp. 3259–3263.

T. Sercu, C. Puhrsch, B. Kingsbury, Y. Lecun, Very deep multilingual convolutional neural networks for LVCSR, ICASSP, in: IEEE Int. Conf. Acoust. Speech Signal Process. - Proc. 2016–May, 2016, pp. 4955–4959. https://doi.org/10.1109/ICASSP.2016.7472620.

D. Yu, W. Xiong, J. Droppo, A. Stolcke, G. Ye, J. Li, G. Zweig, Deep convolutional neural networks with layer-wise context expansion and attention, in: Proc. Annu. Conf. Int. Speech Commun. Assoc. INTERSPEECH. 08–12–Sept, 2016, pp. 17–21. https://doi.org/10.21437/Interspeech.2016-251.

T. Zhao, Y. Zhao, X. Chen, Time-frequency kernel-based CNN for speech recognition, in: Proc. Annu. Conf. Int. Speech Commun. Assoc. INTERSPEECH. 2015–Janua, 2015, pp. 1888–1892.

Yu, 2019, A novel deep learning-based method for damage identification of smart building structures, Struct. Heal. Monit., 18, 143, 10.1177/1475921718804132

Khodabandehlou, 2018, Vibration-based structural condition assessment using convolution neural networks, Struct. Control Heal. Monit., 10.1002/stc.2308

Cha, 2017, Deep learning-based crack damage detection using convolutional neural networks, Comput. Civ. Infrastruct. Eng., 10.1111/mice.12263

N.S. Gulgec, M. Takáč, S.N. Pakzad, Structural damage detection using convolutional neural networks, in: Conf. Proc. Soc. Exp. Mech. Ser., 2017, https://doi.org/10.1007/978-3-319-54858-6_33.

O. Abdeljaber, A. Younis, O. Avci, N. Catbas, M. Gul, O. Celik, H. Zhang, Dynamic testing of a laboratory stadium structure, in: Geotech. Struct. Eng. Congr. 2016, 2016, pp. 1719–1728. https://doi.org/10.1061/9780784479742.147.

S. Kiranyaz, O. Avci, O. Abdeljaber, Real-time structural damage detection by convolutional neural networks, US 20190017911A1, 2019.

O. Avci, O. Abdeljaber, S. Kiranyaz, D.J. Inman, Convolutional neural networks for real-time and wireless damage detection, in: IMAC XXXVII, Int. Modal Anal. Conf., Springer International Publishing, Orlando, FL, USA, 2019.

S. Dyke, D. Bernal, J. Beck, C. Ventura, Experimental phase II of the structural health monitoring benchmark problem, in: Proc. 16th ASCE Eng. Mech. Conf., 2003, pp. 1–7.

Sassi, 2008, Tracking surface degradation of ball bearings by means of new time domain scalar indicators, Int. J. COMADEM, 11, 36

Mohamed, 2018, Model-based analysis of gears’ dynamic behavior in the presence of multiple cracks, J. Shock Vib., 10.1155/2018/1913289

Serpen, 2014, Complexity analysis of multilayer perceptron neural network embedded into a wireless sensor network, Proc. Comput. Sci., 36, 192, 10.1016/j.procs.2014.09.078

Blodt, 2008, Models for bearing damage detection in induction motors using stator current monitoring, IEEE Trans. Ind. Electron., 55, 1813, 10.1109/TIE.2008.917108

Eren, 2004, Bearing damage detection via wavelet packet decomposition of the stator current, IEEE Trans. Instrum. Meas., 53, 431, 10.1109/TIM.2004.823323

L. Eren, A. Karahoca, M.J. Devaney, Neural network based motor bearing fault detection, in: Conf. Rec. - IEEE Instrum. Meas. Technol. Conf., 2004. https://doi.org/10.1109/IMTC.2004.1351399.

Bin, 2012, Early fault diagnosis of rotating machinery based on wavelet packets - empirical mode decomposition feature extraction and neural network, Mech. Syst. Signal Process., 27, 696, 10.1016/j.ymssp.2011.08.002

Fan

Kiranyaz, 2020, Operational Neural Networks”, Neural Computing and Applications (Springer-Nature), 1

Avci, 2021, A Review of Vibration-Based Damage Detection in Civil Structures: From Traditional Methods to Machine Learning and Deep Learning Applications, Mechanical Systems and Signal Processing, 147, 10.1016/j.ymssp.2020.107077

Tran, 2019, Heterogeneous Multilayer Generalized Operational Perceptron, IEEE Transactions on Neural Networks and Learning Systems, 1

Tran, 2019, Progressive Operational Perceptron with Memory, Neurocomputing

Tran, 2019, PyGOP: A Python Library for Generalized Operational Perceptron, Knowledge-Based Systems, 182, 10.1016/j.knosys.2019.06.009

Kiranyaz, 2020, Exploiting Heterogeneity in Operational Neural Networks by Synaptic Plasticity, arXiv:2004.11778

Kiranyaz, 2020, Self-Organized Operational Neural Networks with Generative Neurons, arXiv:2004.11778

Malik, 2020, FastONN--Python based open-source GPU implementation for Operational Neural Networks, arXiv:2006.02267