A survey on deep learning for big data

Kuang, 2015, An integration framework on cloud for cyber physical social systems big data, IEEE Trans. Cloud Comput.

Li, 2013, Communication energy modeling and optimization through joint packet size analysis of BSN and wifi networks, IEEE Trans. Parallel Distrib. Syst., 24, 1741, 10.1109/TPDS.2012.264

Chen, 2014, Data-intensive applications, challenges, techniques and technologies: a survey on big data, Inf. Sci., 275, 314, 10.1016/j.ins.2014.01.015

Wu, 2014, Data mining with big data, IEEE Trans. Knowl. Data Eng., 26, 97, 10.1109/TKDE.2013.109

Orgaz, 2016, Social big data: recent achievements and new challenges, Inf. Fusion, 28, 45, 10.1016/j.inffus.2015.08.005

Samuel, 2015, A framework for composition and enforcement of privacy-aware and context-driven authorization mechanism for multimedia big data, IEEE Trans. Multimedia, 17, 1484, 10.1109/TMM.2015.2458299

Saha, 2014, Data quality: the other face of big data, 1294

Chen, 2014, Big data deep learning: challenges and perspectives, IEEE Access, 2, 514, 10.1109/ACCESS.2014.2325029

Bengio, 2013, Representation learning: a review and new perspectives, IEEE Trans. Pattern Anal. Mach. Intell., 35, 1798, 10.1109/TPAMI.2013.50

Y. LeCun, Y. Bengio, G. Hinton, Deep learning, Nature 521(7553) 436–444.

Schmidhuber, 2015, Deep learning in neural networks: an overview, Neural Netw., 61, 85, 10.1016/j.neunet.2014.09.003

Hinton, 2006, Reducing the dimensionality of data with neural networks, Science, 313, 504, 10.1126/science.1127647

Raina, 2009, Large-scale deep unsupervised learning using graphics processors, 873 C880

V. Sze, Y. Chen, T. Yang, J. Emer, Efficient processing of deep neual networks: a tutoria and survey, 2017, arXiv:1703.09039.

Islam, 2016, Application of deep learning to compuer vision: a comprehensive study, 592

Kruger, 2013, Deep hierarchies in the primate visual cortex: what can we learn for computer vision, IEEE Trans. Pattern Anal. Mach. Intell., 35, 1847, 10.1109/TPAMI.2012.272

Deng, 2013, New types of deep neual network learning for speech recognition and related applications: an overview, 26

Chen, 2015, Multitask learning of deep neural networks for low-resource speech recognition, IEEE/ACM Trans. Audio Speech Lang. Process., 23, 1172

Majumder, 2017, Deep learning-based document modeling for personality detection from text, IEEE Intell. Syst., 32, 74, 10.1109/MIS.2017.23

Jiang, 2015, Training word embeddings for deep learning in biomedical text mining tasks, 625

Khumoyun, 2016, Spark based distributed deep learning framework for big data applications, 1

Alsheikh, 2016, Mobile big data analytics using deep learning and apache spark, IEEE Netw., 30, 22, 10.1109/MNET.2016.7474340

Wilamowski, 2016, Big data and deep learning, 11

Gehring, 2013, Extracting deep bottleneck features using stacked auto-encoders, 26

Weng, 2016, Learning cascaded deep auto-encoder networks for face alignment, IEEE Trans. Multimedia, 18, 2066, 10.1109/TMM.2016.2591508

Sun, 2016, Unseen noise estimation using separable deep auto encoder for speech enhancement, IEEE/ACM Trans. Audio Speech Lang. Process., 24, 93, 10.1109/TASLP.2015.2498101

Goodfellow, 2009, Measuring invariances in deep networks, 646

A. Ng, Sparse autoencoder, CS294A Lecture notes 72(2011) 1–19.

G.E. Hinton, S. Osindero, Y.-W. Teh, A fast learning algorithm for deep belief nets, Neural Comput. 18(7) 1527–1554.

Larochelle, 2012, Learning algorithms for the classification restricted boltzmann machine, J. Mach. Learn. Res., 13, 643

Salakhutdinov, 2007, Restricted boltzmann machines for collaborative filtering, 791

Cho, 2011, Improved learning of gaussian-bernoulli restricted boltzmann machines, 10C17

Li, 2014, Classification of hyperspectral image based on deep belief networks, 5132

Niu, 2014, An improved bilinear deep belief network algorithm for image classification, 189

Mohamed, 2012, Acoustic modeling using deep belief networks, IEEE Trans. Audio Speech Lang. Process., 20, 14, 10.1109/TASL.2011.2109382

Zhang, 2013, Deep belief networks based voice activity detection, IEEE Trans. Audio Speech Lang. Process., 21, 697, 10.1109/TASL.2012.2229986

Huang, 2014, Deep architecture for traffic flow prediction: deep belief networks with multitask learning, IEEE Trans. Intell. Transp. Syst., 15, 2191, 10.1109/TITS.2014.2311123

Sarikaya, 2014, Application of deep belief networks for natural language understanding, IEEE/ACM Trans. Audio Speech Lang. Process., 22, 778, 10.1109/TASLP.2014.2303296

Karpathy, 2014, Large-scale video classification with convolutional neural networks, 1725

Maggiori, 2017, Convolutional neural networks for large-scale remote-sensing image classification, IEEE Trans. Geosci. Remote Sens., 55, 645, 10.1109/TGRS.2016.2612821

Han, 2017, Deep convolutional neural networks for predominant instrument recognition in polyphonic music, IEEE/ACM Trans. Audio Speech Lang. Process., 25, 208, 10.1109/TASLP.2016.2632307

Krizhevsky, 2012, Imagenet classification with deep convolutional neural networks, 1097

K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition, 2014, arXiv:1409.1556.

R. Zhao, R. Yan, Z. Chen, K. Mao, P. Wang, R.X. Gao, Deep learning and its applications to machine health monitoring: a survey, 2016, arXiv:1612.07640.

N. Kalchbrenner, E. Grefenstette, P. Blunsom, A convolutional neural network for modeling sentences, 2014, arXiv:1404.2188.

Shi, 2016, A multichannel convolutional neural network for cross-language dialog state tracking, 559

Abdel-Hamid, 2012, Applying convolutional neural networks concepts to gybrid NN-HMM model for speech recognition, 4277

Qian, 2016, Very deep convolutional neural networks for noise robust speech recognition, IEEE/ACM Trans. Audio Speech Lang. Process., 24, 2263, 10.1109/TASLP.2016.2602884

Abdel-Hamid, 2014, Convolutional neural networks for speech recognition, IEEE/ACM Trans. Audio Speech Lang. Process., 22, 1533, 10.1109/TASLP.2014.2339736

Swietojanski, 2014, Convolutional neural networks for distant speech recognition, IEEE Signal Process. Lett., 21, 1120, 10.1109/LSP.2014.2325781

Hochreiter, 1997, Long short-term memory, Neural Comput., 9, 1725, 10.1162/neco.1997.9.8.1735

Gers, 2000, Learning to torget: continual prediction with istm, Neural Comput., 12, 2451, 10.1162/089976600300015015

Gers, 2002, Learning precise timing with istm recurrent networks, J. Mach. Learn. Res., 3, 115

J. Chung, C. Gulcehre, K. Cho, Y. Bengio, Empirical evaluation of gated recurrent neural networks on sequence modeling, 2014, 1412.3555.

Chen, 2016, CUED-RNNLM-an open-source toolkit for efficient training and evaluation of recurrent neural network language models, 6000

Fu, 2015, Integrating prosodic information into recurrent neual network language model for speech recognition, 1194

Hermanto, 2015, Recurrent neural network language model for english-indonesian machine translation: Experimental study, 132

Zhang, 2017, End-to-end online writer identification with recurrent neural network, IEEE Trans. Hum. Mach. Syst., 47, 285, 10.1109/THMS.2016.2634921

Chien, 2016, Bayesian recurrent neural network for language modeling, IEEE Trans. Neural Netw. Learn. Syst., 27, 361, 10.1109/TNNLS.2015.2499302

Deng, 2012, Scalable stacking and learning for building deep architectures

Hutchinson, 2013, Tensor deep stacking networks, IEEE Trans. Pattern Anal. Mach. Intell., 35, 1944, 10.1109/TPAMI.2012.268

Dean, 2012, Large scale distributed deep networks, 1223

Heigold, 2013, Multilingual acoustic models using distributed deep neural networks, 8619

Heigold, 2014, Asynchronous stochastic optimization for sequence training of deep neural networks, 5587

Vanhoucke, 2011, Improving the speed of neural networks on CPUs, 1

Geman, 1984, Stochastic relaxation, gibbs distributions, and the bayesian restoration of images, IEEE Trans. Pattern Anal. Mach. Intell., 6, 721, 10.1109/TPAMI.1984.4767596

Casella, 1992, Explaining the gibbs sampler, Am. Stat., 46, 167, 10.1080/00031305.1992.10475878

Coats, 2013, Deep learning with COTS HPC systems, J. Mach. Learn. Res., 28, 1337

Zhang, 2015, Optimizing FPGA-based accelerator design for deep convolutional neural networks, 161

Wang, 2017, DLAU: a scalable deep learning accelerator unit on FPGA, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst., 36, 513

Denil, 2013, Predicting parameters in deep learning, 2148

Sainath, 2013, Low-rank matrix factorization for deep neural network training with high-dimensional output targets

Xue, 2013, Restructuring of deep neural network acoustic models with singular value decomposition

Chen, 2015, Compressing neural networks with the hashing trick

Novikov, 2015, Tensorizing neural netwroks, 442

V. Lebedev, Y. Ganin, M. Rakhuba, I. Oseledets, V. Lempitsky, Speeding-up convolutional neural networks ssing fine-tuned CP-decomposition, 2014, 1412.6553.

Ngiam, 2011, Multimodal deep learning, 689

Srivastava, 2012, Multimodal learning with deep boltzmann machines, 2222

Ouyang, 2014, Multi-source deep learning for human pose estimation, 2337

Zhou, 2015, Combining heterogeneous deep neural networks with conditional random fields for chinese dialogue act recognition, Neurocomputing, 168, 408, 10.1016/j.neucom.2015.05.086

Zhao, 2015, Heterogeneous feature selection with multi-modal deep neural networks and sparse group LASSO, IEEE Trans. Multimedia, 17, 1936, 10.1109/TMM.2015.2477058

Wang, 2015, Large-margin multi-modal deep learning for RGB-d object recognition, IEEE Trans. Multimedia, 17, 1887, 10.1109/TMM.2015.2476655

Neverova, 2016, Moddrop: adaptive multi-modal gesture recognition, IEEE Trans. Pattern Anal. Mach. Intell., 38, 1692, 10.1109/TPAMI.2015.2461544

Rastegar, 2016, MDL-CW: a multimodal deep learning framework with crossweights, 2601

Zhang, 2016, Deep computation model for unsupervised feature learning on big data, IEEE Trans. Serv. Comput., 9, 161, 10.1109/TSC.2015.2497705

Zhang, 2016, Privacy preserving deep computation model on cloud for big data feature learning, IEEE Trans. Comput., 65, 1351, 10.1109/TC.2015.2470255

Fu, 1996, Incremental backpropagation learning networks, IEEE Trans. Neural Netw., 7, 757, 10.1109/72.501732

Wan, 2006, Parameter incremental learning algorithm for neural networks, IEEE Trans. Neural Netw., 17, 1424, 10.1109/TNN.2006.880581

Shalev-Shwartz, 2012, Online learning and online convex optimization, Found. Trends Mach. Learn., 4, 107, 10.1561/2200000018

Zhao, 2015, Simnest: social media nested epidemic simulation via online semi-supervised deep learning, 639

Yu, 2017, Integrating online and offline three-dimensional deep learning for automated polyp detection in colonoscopy videos, IEEE J. Biomed. Health Inf., 21, 65, 10.1109/JBHI.2016.2637004

Bu, 2015, Incremental updating method for big data feature learning, Comput. Eng. Appl., 51, 21

Elwell, 2009, Incremental learning in nonstationary environments with controlled forgetting, 771

Chen, 2016, Scalable training of deep learning machines by incremental block training with intra-block parallel optimization and blockwise model-update filtering, 5880

Bizer, 2012, The meaningful use of big data: four perspectives – four challenges, ACM SIGMOD Rec., 40, 56, 10.1145/2094114.2094129

Kwon, 2014, Data quality management, data usage experience and acquisition intention of big data analytics, Int. J. Inf. Manage., 34, 387, 10.1016/j.ijinfomgt.2014.02.002

Becker, 2015, Big data, big data quality problem, 2644

Ciancarini, 2016, Big data quality: a roadmap for open data, 210

Vincent, 2008, Extracting and composing robust features with denoising autoencoders, 1096

Vincent, 2010, Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion, J. Mach. Learn. Res., 11, 3371

Bu, 2014, Incomplete big data impputation algorithm based on deep learning, Microelectr. Comput., 31, 173

R, 2016, Non-local auto-encoder with collaborative stabilization for image restoration, IEEE Trans. Image Process., 25, 2117, 10.1109/TIP.2016.2541318