Shallow and deep learning for image classification

Ososkov, G., Robust tracking by cellular automata and neural network with non-local weights, Rogers, S.K. and Ruck, D.W., Ed., Appl. Sci. Artificial Neural Networks, Proc. SPIE 2492, 1995, p. 1180A1192.

Lebedev, S., Hoehne, C., Lebedev, A., and Ososkov, G., Electron reconstruction and identication capabilities of the CBM experiment at FAIR, J. Phys.: Conf. Ser., 2012, vol. 396, p. 022029.

Baginyan, S. et al., Tracking by modified rotor model of neural network, Comput. Phys. Commun., 1994, vol. 79, p. 95.

Galkin, I. et al., Feedback neural networks for ARTIST ionogram processing, Radio Sci., 1996, vol. 31, no. 5, pp. 1119–1128.

TMVA Users Guide. http://tmva.sf.net.

Kisel, I., Neskoromnyi, V., and Ososkov, G., Applications of neural networks in experimental physics, Phys. Part. Nucl., 1993, vol. 24, no. 6, pp. 657–676.

Peterson, C., Track finding with neural networks, Nucl. Instr. Meth. A, 1986, vol. 279, p. 537.

Denby, B., Neural networks and cellular automata in experimental high energy physics, Comput. Phys. Commun., 1988, vol. 49, p. 429.

Peterson, C. and Hartman, E., Explorations of the mean field theory learning algorithm, Neural Networks, 1989, vol. 2, pp. 475–494.

Kirkpatrick, S., Gelatt, C.D., and Vecchi, M.P., Optimization by simulated annealing, Science, 1983, vol. 22, p. 671.

Ososkov, G.A., Polanski, A., and Puzynin, I.V., Current methods of processing experimental data in high energy physics, Phys. Part. Nucl., 2002, vol. 33, pp. 347–382.

Rumelhart, D., Hinton, G., and Williams, R., Learning representations by back-propagating errors, Nature, 1986, vol. 323, pp. 533–536.

Gyulassy, M. and Harlander, M., Elastic tracking and neural network algorithms for complex pattern recognition, Comput. Phys. Commun., 1991, vol. 66, pp. 31–46.

Ohlsson, M., Petereson, C., and Yuille, A.L., Track finding with deformable template–The elastic arm approach, Comput. Phys. Commun., 1992, vol. 71, p. 77.

Hinton, G.E. and Salakhutdinov, R., Reducing the dimensionality of data with neural networks, Science, 2006, vol. 313, pp. 504–507.

Bengio, Y., Learning Deep Architectures for AI, Now Publishers Inc., 2009, p. 144.

LeCun, Y., Bottou, L., Orr, G., and Müller, K.-R., Efficient backprop, Orr, G. and Müller, K.-R., Eds., Neural Networks: Tricks Trade, 1998a, pp. 9–50.

Krizhevsky, A., Learning Multiple Layers of Features from Tiny Images. https://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf.

Felleman, D.J. and van Essen, D.C., Distributed hierarchical processing in the primate cerebral cortex, Cereb. Cortex, 1991, vol. 1, pp. 1–47. doi 10.1093/cercor/1.1.1-a

Simon Thorpe, Denis Fize, Catherine Marlot, et al., Speed of processing in the human visual system, Nature, 1996, vol. 381, no. 6582, pp. 520–522.

Lennie, P., The cost of cortical computation, Curr. Biol., 2003, vol. 13, no. 6, pp. 493–497.

Glorot, X. and Bengio, Y., Understanding the difficulty of training deep feedforward neural networks, Aistats, 2010, vol. 9, pp. 249–256.

Hinton, G., Osindero, S., and Yee-Whye Teh, A fast learning algorithm for deep belief nets, Neural Comput., 2006, vol. 18, pp. 1527–1554.

Smolensky, P., Information processing in dynamical systems: Foundations of harmony theory, in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol. 1: Foundations, Rumelhart, D.E., McClelland, J.L., Eds., MIT Press, 1986, pp. 194–281.

Hinton, G.E., A Practical Guide to training restricted Boltzmann machines, Tech. Rep. 2010-000, Toronto: Machine Learning Group, University of Toronto, 2010.

Diaconis, P., The Marcov chain Monte Carlo revolution, Bull. Am. Math. Soc., 2009, vol. 46, no. 2, pp. 179–205.

Carreira-Perpinan, M.A. and Hinton, G., On contrastive divergence learning, Aistats, Citeseer, 2005, vol. 10, pp. 33–40.

Kramer, M., Nonlinear principal component analysis using autoassociative neural networks, AIChE J., 1991, vol. 37, no. 2, pp. 161–310.

Baldi, P. and Hornik, K., Neural networks and principal components analysis: Learning from examples without local minima, Neural Networks, 1989, vol. 2, pp. 53–58.

Ranzato, M., Poultney, C., Chopra, S., and LeCun, Y., Efficient Learning of Sparse Representations with an Energy-Based Model. http://yann.lecun.com/exdb/publis/pdf/ranzato-06.pdf.

Bengio, Y. and LeCun, Y., Scaling learning algorithms towards AI, in Large Scale Kernel Machines, Bottou, L., Chapelle, O., DeCoste, D., and Weston, J., Eds., MIT Press, 2007.

Vincent, P., Larochelle, H., Bengio, Y., and Manzagol, P., Extracting and Composing Robust Features with Denoising Autoencoders, 2008. http://machinelearning.org/archive/icml2008/papers/592.pdf.

Vincent, P., Larochelle, H., Bengio, Y., and Manzagol, P., Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion, J. Mach. Learning Res., 2010, vol. 11, pp. 3371–3408.

Masci, J., Meier, U., Ciresan, D., and Schmidhuber, J., Stacked convolutional auto-encoders for hierarchical feature extraction, ICANN 2011, Honkela, T. et al., Ed., Springer-Verlag: Berlin Heidelberg, 2011, Part I, LNCS 6791, pp. 52–59. https://pdfs.semanticscholar.org/1c6d/990c80e60aa0b0059415444cdf94b3574f0f.pdf.

LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P., Gradient-based learning applied to document recognition, Proc. IEEE, 1998, vol. 86, no. 11, pp. 2278–2324.

Simonyan, K. and Zisserman, A., Very Deep Convolutional Networks for Large-Scale Image Recognition. https://arxiv.org/pdf/1409.1556.pdf.

Sepp, H. and Schmidhuber, J., Long short-term memory, Neural Comput., 1997, vol. 9.8, pp. 1735–1780.

Schmidhuber, J., Deep learning in neural networks: An overview, Neural Networks, 2015, vol. 61, pp. 85–117.

Chung, J., Gulcehre, C., Cho, K., and Bengio, Y., Gated feedback recurrent neural networks, arXiv preprint, 2015. https://arxiv.org/pdf/1502.02367.pdf.

Sutton, R. and Barto, A., Reinforcement Learning: An Introduction, MIT Press, 1998.

Mnih, V. et al., Human-level control through deep reinforcement learning, Nature, 2015, vol. 518, pp. 529–533.

He, K., Zhang, X., Ren, Sh., and Sun, J., Identity mappings in deep residual networks. https://arxiv.org/pdf/1603.05027.pdf.

Mao, X.-J., Shen, C., and Yang, Y.-B., Image restoration using convolutional auto-encoders with symmetric skip connections. https://arxiv.org/pdf/1606.08921.pdf.

Phillips, A., Moon, H., Rauss, P., and Rizvi, S., The FERET evaluation methodology for face recognition algorithms, IEEE Trans. Pattern Analysis Mach. Intelligence, 2000, vol. 22, no. 10.

MNIST. https://www.nist.gov/sites/default/files/documents/srd/nistsd19.pdf.

Arlot, S., A survey of cross-validation procedures for model selection, Statistics Surv., 2010, vol. 4, pp. 40–79.

Xu, B., Wang, N., Chen, T., and Li, M., Empirical evaluation of rectified activations in convolutional network. https://arxiv.org/abs/1505.00853.

Hua, K.L., Hsu, C.H., Hidayati, S.C., Cheng, W.H., and Chen, Y.J., Computer-aided classification of lung nodules on computed tomography images via deep learning technique, OncoTargets Therapy, 2014, vol. 8, pp. 2015–2022.

Bengio, Y., Courville, A., and Vincent, P., Representation learning: A review and new perspectives. https://arxiv.org/pdf/1206.5538.pdf.

Sutskever, I., Training Recurrent Neural Networks, PhD Thesis. http://www.cs.utoronto.ca/~ilya/pubs/ilya_- sutskever_phd_thesis.pdf.

Nair, V. and Hinton, G., Rectified linear units improve restricted Boltzmann machines, Proc. 27th Int. Conference on Machine Learning, Furnkranz, J. and Joachims, Th., Eds., Haifa, Israel, 2010, pp. 807–814C.

Dugas, Y., Bengio, F., Bélisle, C., and Nadeau, R., Garcia, Incorporating second-order functional knowledge for better option pricing, NIPS'2000, 2001. http://papers.nips.cc/paper/1920-incorporating-second-orderfunctional- knowledge-for-better-option-pricing.pdf.

Zuriani, M. and Yuhanis, Y., A comparision of normalization techiques in predicting dengue outbreak, 2010 International Conference on Business and Economics Research, 2011, vol. 1, Kuala Lumpur, Malaysia: IACSIT Press, pp. 345–349.

Glorot, X., Bordes, A., and Bengio, Y., Deep sparse rectifier neural networks, Aistats, 2011, vol. 15, no. 106, pp. 315–323.

Kingma, D. and Ba, J.L., ADAM: A method for stochastic optimization, arXiv preprint arXiv:1412.6980, 2014. https://arxiv.org/abs/1412.6980.

Bengio, Y., Practical recommendations for gradient-based training of deep architectures, Neural Networks: Tricks of the Trade, Springer, 2012, pp. 437–478.

Duchi, J., Hazan, E., and Singer, Y., Adaptive subgradient methods for online learning and stochastic optimization, J. Mach. Learning Res., 2011, vol. 12, no. Jul, pp. 2121–2159.

Srivastava, N. et al., Dropout: A simple way to prevent neural networks from overfitting, J. Mach. Learning Res., 2014, vol. 15, pp. 1929–1958.

Stone, M., Cross-validatory choice and assessment of statistical predictions, J. Royal Statistical Soc., Ser. B: Methodological, 1974, pp. 111–147.

Cawley, G.C. and Talbot, N.L.C., On over-fitting in model selection and subsequent selection bias in performance evaluation, J. Mach. Learning Res., 2010, vol. 11, pp. 2079–2107.

Tuning the hyper-parameters of an estimator. http://scikit-learn.org/stable/modules/grid_search.html#gridsearch.

Deep Learning Frameworks. https://www.microway.com/hpc-tech-tips/deep-learning-frameworks-surveytensorflow-torch-theano-caffe-neon-ibm-machine-learning-stack/.

Tensorflow. https://www.tensorflow.org/.

Torch. http://torch.ch/.

Theano at a Glance. http://deeplearning.net/software/theano/introduction.html. Cited March 5. 2017.

Jia, Y. et al., Caffe: Convolutional architecture for fast feature embedding, Proceedings of the 22nd ACM International Conference on Multimedia, ACM, 2014, pp. 675–678.

Easy benchmarking of all publicly accessible implementations of convents. https://github.com/soumith/convnet- benchmarks.

Cuda-convnet2 project. https://github.com/akrizhevsky/cuda-convnet2.

Sermanet, P. et al., Overfeat: Integrated recognition, localization and detection using convolutional networks, arXiv preprint arXiv:1312.6229, 2013.

Simonyan, K. and Zisserman, A., Very deep convolutional networks for large-scale image recognition, arXiv preprint arXiv:1409.1556, 2014.

Szegedy, C. et al., Going deeper with convolutions, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 1–9.

Torch vs. TensorFlow vs. Theano. http://www.ccri.com/2016/12/09/torch-vs-tensorflow-vs-theano/.

Porting a model to TensorFlow. https://medium.com/@sentimentron/faceoff-theano-vs-tensorflowe25648c31800.

Speed up training with GPU-accelerated TensorFlow. http://www.nvidia.com/object/gpu-accelerated-applications- tensorflow-benchmarks.html.

TensorBoard: Visualizing Learning. https://www.tensorflow.org/get_started/summaries_and_tensorboard.

Web-portal of HybriLIT JINR computation facility. http://hybrilit.jinr.ru.