A survey on deep learning and its applications

Freedman, 2009

Mood, 2010, Logistic regression: Why we cannot do what we think we can do, and what we can do about it, Eur. Sociol. Rev., 26, 67, 10.1093/esr/jcp006

Kleinbaum, 2002, Analysis of matched data using logistic regression, 227

Hosmer Jr, 2013

Soentpiet, 1999

Hearst, 1998, Support vector machines, IEEE Intell. Syst. Appl., 13, 18, 10.1109/5254.708428

Steinwart, 2008

Schraudolph, 2002, Fast curvature matrix-vector products for second-order gradient descent, Neural Comput., 14, 1723, 10.1162/08997660260028683

Li, 2009

Verbeek, 2003, Efficient greedy learning of Gaussian mixture models, Neural Comput., 15, 469, 10.1162/089976603762553004

Hinton, 2006, A fast learning algorithm for deep belief nets, Neural Comput., 18, 1527, 10.1162/neco.2006.18.7.1527

Hebb, 1949, The organization of behavior; a neuropsycholocigal theory, A Wiley Book in Clinical Psychology, 62, 78

Crevier, 1993

McCarthy, 2006, A proposal for the dartmouth summer research project on artificial intelligence, august 31, 1955, AI Mag., 27, 12

LeCun, 1998, Gradient-based learning applied to document recognition, Proc. IEEE, 86, 2278, 10.1109/5.726791

Wang, 2018, Visualizing deep neural network by alternately image blurring and deblurring, Neural Netw., 97, 162, 10.1016/j.neunet.2017.09.007

Nouiehed, 2018

Diakonikolas, 2016

Yun, 2018

Haeffele, 2015

B.D. Haeffele, R. Vidal, Global optimality in neural network training, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 7331–7339.

Srivastava, 2014, Dropout: a simple way to prevent neural networks from overfitting, J. Mach. Learn. Res., 15, 1929

P. Mianjy, R. Arora, R. Vidal, On the implicit bias of dropout, in: ICML, 2018.

H. Salehinejad, S. Valaee, Ising-dropout: A regularization method for training and compression of deep neural networks, in: ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019, pp. 3602–3606.

Sengupta, 2018

S. Zheng, Y. Song, T. Leung, I.J. Goodfellow, Improving the robustness of deep neural networks via stability training, in: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 4480–4488.

Giryes, 2015

Haber, 2017, Stable architectures for deep neural networks, Inverse Problems, 34, 10.1088/1361-6420/aa9a90

Malladi, 2018

Chang, 2018, Reversible architectures for arbitrarily deep residual neural networks

Bengio, 2009

Vincent, 2010, Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion, J. Mach. Learn. Res., 11

Fiore, 2013, Network anomaly detection with the restricted Boltzmann machine, Neurocomputing, 122, 13, 10.1016/j.neucom.2012.11.050

Ackley, 1985, A learning algorithm for boltzmann machines, Cognitive Science, 9, 147, 10.1207/s15516709cog0901_7

Ranzato, 2011, On deep generative models with applications to recognition, 2857

Rifai, 2012, Disentangling factors of variation for facial expression recognition, 808

Salakhutdinov, 2009, Deep boltzmann machines, 448

Gu, 2018, Recent advances in convolutional neural networks, Pattern Recognit., 77, 354, 10.1016/j.patcog.2017.10.013

Hubel, 1962, Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex, J. Physiol., 160, 106, 10.1113/jphysiol.1962.sp006837

A. Krizhevsky, I. Sutskever, G.E. Hinton, ImageNet classification with deep convolutional neural networks, in: NIPS, 2012.

Zhang, 2020, Deep learning on graphs: A survey, IEEE Trans. Knowl. Data Eng.

Shuman, 2013, The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains, IEEE Signal Process. Mag., 30, 83, 10.1109/MSP.2012.2235192

Zhou, 2018

Kipf, 2016

Hamilton, 2017, Inductive representation learning on large graphs, 1024

Defferrard, 2016, Convolutional neural networks on graphs with fast localized spectral filtering, 3844

Mohebali, 2020, Probabilistic neural networks: a brief overview of theory, implementation, and application, 347

J. Gast, S. Roth, Lightweight probabilistic deep networks, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 3369–3378.

Zhang, 2016

Fan, 2017

Deng, 2016, A hierarchical fused fuzzy deep neural network for data classification, IEEE Trans. Fuzzy Syst., 25, 1006, 10.1109/TFUZZ.2016.2574915

Zhou, 2014, Fuzzy deep belief networks for semi-supervised sentiment classification, Neurocomputing, 131, 312, 10.1016/j.neucom.2013.10.011

Goodfellow, 2014, Generative adversarial nets, 2672

Salimans, 2016, Improved techniques for training gans, 2234

H. Schwenk, Continuous space translation models for phrase-based statistical machine translation, in: Proceedings of COLING 2012: Posters, 2012, pp. 1071–1080.

L. Dong, F. Wei, M. Zhou, K. Xu, Adaptive multi-compositionality for recursive neural models with applications to sentiment analysis, in: Proceedings of the National Conference on Artificial Intelligence, vol. 2, 2014, pp. 1537–1543.

D. Tang, F. Wei, B. Qin, T. Liu, M. Zhou, Coooolll: A deep learning system for twitter sentiment classification, in: Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), 2014, pp. 208–212.

You, 2015, An investigation on DNN-derived bottleneck features for GMM-HMM based robust speech recognition, 30

Maas, 2017, Building DNN acoustic models for large vocabulary speech recognition, Comput. Speech Lang., 41, 195, 10.1016/j.csl.2016.06.007

Li, 2014, Medical image classification with convolutional neural network, 844

Li, 2015, A robust deep model for improved classification of AD/MCI patients, IEEE J. Biomed. Health Inf., 19, 1610, 10.1109/JBHI.2015.2429556

Sirinukunwattana, 2016, Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images, IEEE Trans. Med. Imaging, 35, 1196, 10.1109/TMI.2016.2525803

Dou, 2016, Automatic detection of cerebral microbleeds from MR images via 3D convolutional neural networks, IEEE Trans. Med. Imaging, 35, 1182, 10.1109/TMI.2016.2528129

Mallik, 2012, Acquisition of multimedia ontology: an application in preservation of cultural heritage, Int. J. Multimedia Inf. Retr., 1, 249, 10.1007/s13735-012-0021-5

Höft, 2014, Fast semantic segmentation of RGB-D scenes with GPU-accelerated deep neural networks, 80

Y. Sun, X. Wang, X. Tang, Deep learning face representation from predicting 10,000 classes, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 1891–1898.

P. Pinheiro, R. Collobert, Recurrent convolutional neural networks for scene labeling, in: International Conference on Machine Learning, 2014, pp. 82–90.

Y. Taigman, M. Yang, M. Ranzato, L. Wolf, Deepface: Closing the gap to human-level performance in face verification, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 1701–1708.

J. Long, E. Shelhamer, T. Darrell, Fully convolutional networks for semantic segmentation, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 3431–3440.

F. Schroff, D. Kalenichenko, J. Philbin, Facenet: A unified embedding for face recognition and clustering, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 815–823.

Wang, 2018, Understanding convolution for semantic segmentation, 1451

S. Zheng, S. Jayasumana, B. Romera-Paredes, V. Vineet, Z. Su, D. Du, C. Huang, P.H. Torr, Conditional random fields as recurrent neural networks, in: Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 1529–1537.

Ronneberger, 2015, U-net: Convolutional networks for biomedical image segmentation, 234

Badrinarayanan, 2015

Z. Liu, X. Li, P. Luo, C.-C. Loy, X. Tang, Semantic image segmentation via deep parsing network, in: Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 1377–1385.

W. Byeon, T.M. Breuel, F. Raue, M. Liwicki, Scene labeling with lstm recurrent neural networks, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 3547–3555.

G. Lin, C. Shen, A. Van Den Hengel, I. Reid, Efficient piecewise training of deep structured models for semantic segmentation, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 3194–3203.

Shen, 2016

Chandra, 2016, Fast, exact and multi-scale inference for semantic image segmentation with deep gaussian crfs, 402

Luc, 2016

Hoffman, 2016

B. Shuai, Z. Zuo, B. Wang, G. Wang, Dag-recurrent neural networks for scene labeling, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 3620–3629.

K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778.

Chen, 2017

Koziński, 2017

Chen, 2017, Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs, IEEE Trans. Pattern Anal. Mach. Intell., 40, 834, 10.1109/TPAMI.2017.2699184

Souly, 2017

Yu, 2015

Teichmann, 2018

Simonyan, 2015, Very deep convolutional networks for large-scale image recognition

Sermanet, 2014, Overfeat: Integrated recognition, localization and detection using convolutional networks

Russakovsky, 2015, Imagenet large scale visual recognition challenge, Int. J. Comput. Vis., 115, 211, 10.1007/s11263-015-0816-y

Chatfield, 2015, On-the-fly learning for visual search of large-scale image and video datasets, Int. J. Multimedia Inf. Retr., 4, 75, 10.1007/s13735-015-0077-0

Pi, 2020, Convolutional neural networks for object detection in aerial imagery for disaster response and recovery, Adv. Eng. Inform., 43, 10.1016/j.aei.2019.101009

Gu, 2020, Automatic and robust object detection in x-ray baggage inspection using deep convolutional neural networks, IEEE Transactions on Industrial Electronics

S. Caelles, K.-K. Maninis, J. Pont-Tuset, L. Leal-Taixé, D. Cremers, L. Van Gool, One-shot video object segmentation, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 221–230.

J. Shin Yoon, F. Rameau, J. Kim, S. Lee, S. Shin, I. So Kweon, Pixel-level matching for video object segmentation using convolutional neural networks, in: Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 2167–2176.

W.-D. Jang, C.-S. Kim, Online video object segmentation via convolutional trident network, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 5849–5858.

Hu, 2017, Maskrnn: Instance level video object segmentation, 325

Sasikumar, 2018

Li, 2018, Deep video foreground target extraction with complex scenes, 440

H. Xiao, J. Feng, G. Lin, Y. Liu, M. Zhang, Monet: Deep motion exploitation for video object segmentation, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 1140–1148.

Goel, 2018, Unsupervised video object segmentation for deep reinforcement learning, 5683

Schofield, 1996, A system for counting people in video images using neural networks to identify the background scene, Pattern Recognit., 29, 1421, 10.1016/0031-3203(95)00163-8

Tavakkoli, 2005, Foreground-background segmentation in video sequences using neural networks

D. Culibrk, O. Marques, D. Socek, H. Kalva, B. Furht, A neural network approach to bayesian background modeling for video object segmentation, in: VISAPP (1), 2006, pp. 474–479.

Maddalena, 2007, A self-organizing approach to detection of moving patterns for real-time applications, 181

Ramírez-Quintana, 2013, Self-organizing retinotopic maps applied to background modeling for dynamic object segmentation in video sequences, 1

Guo, 2013, Partially-sparse restricted boltzmann machine for background modeling and subtraction, 209

P. Xu, M. Ye, X. Li, Q. Liu, Y. Yang, J. Ding, Dynamic background learning through deep auto-encoder networks, in: Proceedings of the 22nd ACM International Conference on Multimedia, 2014, pp. 107–116.

Xu, 2014, Motion detection via a couple of auto-encoder networks, 1

Ramirez-Quintana, 2015, Self-adaptive SOM-CNN neural system for dynamic object detection in normal and complex scenarios, Pattern Recognit., 48, 1137, 10.1016/j.patcog.2014.09.009

Qu, 2016, Motion background modeling based on context-encoder, 1

Minematsu, 2018, Analytics of deep neural network-based background subtraction, J. Imaging, 4, 78, 10.3390/jimaging4060078

Ammar, 2019, Moving objects segmentation based on deepsphere in video surveillance, 307

Sultana, 2020, Unsupervised adversarial learning for dynamic background modeling, 248

Duvenaud, 2015, Convolutional networks on graphs for learning molecular fingerprints, 2224

Kearnes, 2016, Molecular graph convolutions: moving beyond fingerprints, J. Comput. Aided Mol. Des., 30, 595, 10.1007/s10822-016-9938-8

Berg, 2017

Monti, 2017, Geometric matrix completion with recurrent multi-graph neural networks, 3697

Gilmer, 2017

Coley, 2017, Convolutional embedding of attributed molecular graphs for physical property prediction, J. Chem. Inf. Model., 57, 1757, 10.1021/acs.jcim.6b00601

Ktena, 2017, Distance metric learning using graph convolutional networks: Application to functional brain networks, 469

Parisot, 2017, Spectral graph convolutions for population-based disease prediction, 177

Parisot, 2018, Disease prediction using graph convolutional networks: Application to autism spectrum disorder and Alzheimer’s disease, Med. Image Anal., 48, 117, 10.1016/j.media.2018.06.001

J. Qiu, J. Tang, H. Ma, Y. Dong, K. Wang, J. Tang, Deepinf: Social influence prediction with deep learning, in: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2018, pp. 2110–2119.

R. Ying, R. He, K. Chen, P. Eksombatchai, W.L. Hamilton, J. Leskovec, Graph convolutional neural networks for web-scale recommender systems, in: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2018, pp. 974–983.

You, 2018, Graph convolutional policy network for goal-directed molecular graph generation, 6410

De Cao, 2018

Zitnik, 2018, Modeling polypharmacy side effects with graph convolutional networks, Bioinformatics, 34, i457, 10.1093/bioinformatics/bty294

Xie, 2018, Crystal graph convolutional neural networks for an accurate and interpretable prediction of material properties, Phys. Rev. Lett., 120, 10.1103/PhysRevLett.120.145301

Moreira-Matias, 2013, Predicting taxi–passenger demand using streaming data, IEEE Trans. Intell. Transp. Syst., 14, 1393, 10.1109/TITS.2013.2262376

De Brébisson, 2015

Vinyals, 2015, Pointer networks, 2692

Li, 2015

Bello, 2016

Zhang, 2016

Q. Chen, X. Song, H. Yamada, R. Shibasaki, Learning deep representation from big and heterogeneous data for traffic accident inference, in: Thirtieth AAAI Conference on Artificial Intelligence, 2016.

Endo, 2017, Predicting destinations from partial trajectories using recurrent neural network, 160

Ke, 2017, Short-term forecasting of passenger demand under on-demand ride services: A spatio-temporal deep learning approach, Transp. Res. C, 85, 591, 10.1016/j.trc.2017.10.016

Yao, 2018

Khalil, 2017, Learning combinatorial optimization algorithms over graphs, 6348

Ma, 2017, Learning traffic as images: a deep convolutional neural network for large-scale transportation network speed prediction, Sensors, 17, 818, 10.3390/s17040818

Jiang, 2017

Yao, 2017, Trajectory clustering via deep representation learning, 3880

Yang, 2018, Learning urban navigation via value iteration network, 800

Jindal, 2018, Optimizing taxi carpool policies via reinforcement learning and spatio-temporal mining, 1417

Y. Li, K. Fu, Z. Wang, C. Shahabi, J. Ye, Y. Liu, Multi-task representation learning for travel time estimation, in: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2018, pp. 1695–1704.

Kool, 2018

Lv, 2018, T-CONV: A convolutional neural network for multi-scale taxi trajectory prediction, 82

Y. Yuan, Z. Xiong, Q. Wang, Acm: Adaptive cross-modal graph convolutional neural networks for rgb-d scene recognition, in: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, 2019, pp. 9176–9184.

Li, 2019, A traffic prediction enabled double rewarded value iteration network for route planning, IEEE Trans. Veh. Technol., 68, 4170, 10.1109/TVT.2019.2893173

Tu, 2005, Image parsing: Unifying segmentation, detection, and recognition, Int. J. Comput. Vis., 63, 113, 10.1007/s11263-005-6642-x

Pavlidis, 1977, Fundamentals of picture segmentation, 65

Coleman, 1979, Image segmentation by clustering, Proc. IEEE, 67, 773, 10.1109/PROC.1979.11327

Geman, 1984, Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images, IEEE Trans. Pattern Anal. Mach. Intell., 721, 10.1109/TPAMI.1984.4767596

Zhang, 1996, A survey on evaluation methods for image segmentation, Pattern Recogn., 29, 1335, 10.1016/0031-3203(95)00169-7

Narkhede, 2013, Review of image segmentation techniques, Int. J. Sci. Modern Eng., 1, 54

Kaur, 2014, Various image segmentation techniques: a review, Int. J. Comput. Sci. Mobile Comput., 3, 809

Kuruvilla, 2016, A review on image processing and image segmentation, 198

Huang, 2012, Learning hierarchical representations for face verification with convolutional deep belief networks, 2518

Fischler, 1973, The representation and matching of pictorial structures, IEEE Trans. Comput., 100, 67, 10.1109/T-C.1973.223602

Liu, 2020, Deep learning for generic object detection: A survey, Int. J. Comput. Vis., 128, 261, 10.1007/s11263-019-01247-4

Sultana, 2020, A review of object detection models based on convolutional neural network, 1

Ren, 2015, Faster r-cnn: Towards real-time object detection with region proposal networks, 91

Dai, 2016, R-fcn: Object detection via region-based fully convolutional networks, 379

Hinton, 2006, Reducing the dimensionality of data with neural networks, Science, 313, 504, 10.1126/science.1127647

LeCun, 2015, Deep learning, Nature, 521, 436, 10.1038/nature14539

Dollar, 2011, Pedestrian detection: An evaluation of the state of the art, IEEE Trans. Pattern Anal. Mach. Intell., 34, 743, 10.1109/TPAMI.2011.155

Enzweiler, 2008, Monocular pedestrian detection: Survey and experiments, IEEE Trans. Pattern Anal. Mach. Intell., 31, 2179, 10.1109/TPAMI.2008.260

Geronimo, 2009, Survey of pedestrian detection for advanced driver assistance systems, IEEE Trans. Pattern Anal. Mach. Intell., 32, 1239, 10.1109/TPAMI.2009.122

Sun, 2006, On-road vehicle detection: A review, IEEE Trans. Pattern Anal. Mach. Intell., 28, 694, 10.1109/TPAMI.2006.104

Sakhare, 2020, Review of vehicle detection systems in advanced driver assistant systems, Arch. Comput. Methods Eng., 27, 591, 10.1007/s11831-019-09321-3

Yuan, 2020, Vehicle detection based on area and proportion prior with faster-RCNN, 435

Zafeiriou, 2015, A survey on face detection in the wild: past, present and future, Comput. Vis. Image Underst., 138, 1, 10.1016/j.cviu.2015.03.015

Masi, 2018, Deep face recognition: A survey, 471

Zeng, 2020

Zhao, 2019, Object detection with deep learning: A review, IEEE Trans. Neural Netw. Learn. Syst., 30, 3212, 10.1109/TNNLS.2018.2876865

Cane, 2018, Evaluating deep semantic segmentation networks for object detection in maritime surveillance, 1

R. Girshick, J. Donahue, T. Darrell, J. Malik, Rich feature hierarchies for accurate object detection and semantic segmentation, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 580–587.

R. Girshick, Fast r-cnn, in: Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 1440–1448.

Jian-Wei, 2014, Learning technique of probabilistic graphical models: a review, Acta Automat. Sinica, 40, 1025

Bouwmans, 2014, Traditional and recent approaches in background modeling for foreground detection: An overview, Comput. Sci. Rev., 11, 31, 10.1016/j.cosrev.2014.04.001

Bouwmans, 2017, Decomposition into low-rank plus additive matrices for background/foreground separation: A review for a comparative evaluation with a large-scale dataset, Comp. Sci. Rev., 23, 1, 10.1016/j.cosrev.2016.11.001

Garcia-Garcia, 2020, Background subtraction in real applications: Challenges, current models and future directions, Comp. Sci. Rev., 35

Bouwmans, 2014, Robust PCA via principal component pursuit: A review for a comparative evaluation in video surveillance, Comput. Vis. Image Underst., 122, 22, 10.1016/j.cviu.2013.11.009

Javed, 2014, OR-PCA with MRF for robust foreground detection in highly dynamic backgrounds, 284

L. Xu, Y. Li, Y. Wang, E. Chen, Temporally adaptive restricted Boltzmann machine for background modeling, in: Twenty-Ninth AAAI Conference on Artificial Intelligence, 2015.

Bouwmans, 2019, Deep neural network concepts for background subtraction: A systematic review and comparative evaluation, Neural Netw., 117, 8, 10.1016/j.neunet.2019.04.024

Assouel, 2018

Zhang, 2011, Data-driven intelligent transportation systems: A survey, IEEE Trans. Intell. Transp. Syst., 12, 1624, 10.1109/TITS.2011.2158001

Veres, 2019, Deep learning for intelligent transportation systems: a survey of emerging trends, IEEE Transactions on Intelligent Transportation Systems, 21, 3152, 10.1109/TITS.2019.2929020

Siripanpornchana, 2016, Travel-time prediction with deep learning, 1859

Zhang, 2018

Vlachos, 2002, Discovering similar multidimensional trajectories, 673

Tamar, 2016, Value iteration networks, 2154

Yau, 2017, A survey on reinforcement learning models and algorithms for traffic signal control, ACM Comput. Surv., 50, 1, 10.1145/3068287

Salakhutdinov, 2015, Learning deep generative models, Annu. Rev. Stat. Appl., 2, 361, 10.1146/annurev-statistics-010814-020120

J. Masci, E. Rodolà, D. Boscaini, M. Bronstein, H. Li, Geometric deep learning, in: SIGGRAPH ASIA 2016 Courses, 2016, pp. 1–50.

O.-E. Ganea, G. Bécigneul, T. Hofmann, Hyperbolic neural networks, arXiv preprint arXiv:1805.09112.

Wang, 2020, Deep learning for spatio-temporal data mining: a survey, IEEE Transactions on Knowledge and Data Engineering

You, 2020, Graph structure of neural networks, 119, 10881