High performance inference of gait recognition models on embedded systems

West, 2003, Nonlinear dynamical model of human gait, Phys. Rev. E, 67, 10.1103/PhysRevE.67.051917 Scafetta, 2009, Understanding the complexity of human gait dynamics, Chaos, 19, 10.1063/1.3143035 Gupta, 2018, Re-thinking EEG-based non-invasive brain interfaces: Modeling and analysis, 275 Xue, 2016, A spatio-temporal fractal model for a CPS approach to brain-machine-body interfaces, 642 Di Stasi, 2013, Gait patterns differ between ACL-reconstructed athletes who pass return-to-sport criteria and those who fail, Am. J. Sports Med., 41, 1310, 10.1177/0363546513482718 Sepas-Moghaddam, 2022, Deep gait recognition: A survey, IEEE Trans. Pattern Anal. Mach. Intell. Zeng, 2014, Silhouette-based gait recognition via deterministic learning, Pattern Recognit., 47, 3568, 10.1016/j.patcog.2014.04.014 Han, 2005, Individual recognition using gait energy image, IEEE PAMI, 28, 316, 10.1109/TPAMI.2006.38 Wan, 2019, A survey on gait recognition, ACM Comput. Surv., 51, 89, 10.1145/3230633 Zhang, 2019, Gait recognition via disentangled representation learning, 4710 H. Chao, Y. He, J. Zhang, J. Feng, Gaitset: Regarding gait as a set for cross-view gait recognition, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2019. K. Shiraga, Y. Makihara, D. Muramatsu, T. Echigo, Y. Yagi, Geinet: View-invariant gait recognition using a convolutional neural network, in: Intl. Conference on Biometrics, ICB, 2016, pp. 1–8. Castro, 2017, Automatic learning of gait signatures for people identification, Vol. 10306, 257 Delgado-Escaño, 2018, An end-to-end multi-task and fusion CNN for inertial-based gait recognition, IEEE Access, 7, 1897, 10.1109/ACCESS.2018.2886899 He, 2016, Deep residual learning for image recognition, 770 X. Zhai, A. Kolesnikov, N. Houlsby, L. Beyer, Scaling vision transformers, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 12104–12113. Castro, 2019, Energy-based tuning of convolutional neural networks on multi-GPUs, Concurr. Comput.: Pract. Exper., 31, 10.1002/cpe.4786 Zhao, 2019, Object detection with deep learning: A review, IEEE Trans. Neural Netw. Learn. Syst., 30, 3212, 10.1109/TNNLS.2018.2876865 Saddik, 2021, Real-time evaluation of different indexes in precision agriculture using a heterogeneous embedded system, Sustain. Comput. Inform. Syst., 30 Qasaimeh, 2021, Benchmarking vision kernels and neural network inference accelerators on embedded platforms, J. Syst. Archit., 113, 10.1016/j.sysarc.2020.101896 Jaiswal, 2017, Deep neural network compression via knowledge distillation for embedded applications, 1 Liang, 2021, Pruning and quantization for deep neural network acceleration: A survey, Neurocomputing, 461, 370, 10.1016/j.neucom.2021.07.045 M. Mathew, K. Desappan, P. Kumar Swami, S. Nagori, Sparse, Quantized, Full Frame CNN for Low Power Embedded Devices, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2017. Kochura, 2019 Kim, 2021, Guaranteeing that multi-level prioritized DNN models on an embedded gpu have inference performance proportional to respective priorities, IEEE Embedded Syst. Lett., 1 Xu, 2019, Minimizing energy consumption with reliability goal on heterogeneous embedded systems, J. Parallel Distrib. Comput., 127, 44, 10.1016/j.jpdc.2019.01.006 HajiRassouliha, 2018, Suitability of recent hardware accelerators (DSPs, FPGAs, and GPUs) for computer vision and image processing algorithms, Signal Process., Image Commun., 68, 101, 10.1016/j.image.2018.07.007 NVIDIA, 2018 2014 2022 2022 Makihara, 2006, Gait recognition using a view transformation model in the frequency domain, 151 Bashir, 2009, Gait recognition using gait entropy image Lam, 2011, Gait flow image: A silhouette-based gait representation for human identification, Pattern Recognit., 44, 973, 10.1016/j.patcog.2010.10.011 Zhao, 2017, Wearable device-based gait recognition using angle embedded gait dynamic images and a convolutional neural network, Sensors, 17, 478, 10.3390/s17030478 Nakajima, 2000, Footprint-based personal recognition, IEEE Trans. Biomed. Eng., 47, 1534, 10.1109/10.880106 Delgado-Escaño, 2020, A cross-dataset deep learning-based classifier for people fall detection and identification, Comput. Methods Progr. Biomed., 184, 10.1016/j.cmpb.2019.105265 Z. Meng, S. Fu, J. Yan, H. Liang, A. Zhou, S. Zhu, H. Ma, J. Liu, N. Yang, Gait Recognition for Co-existing Multiple People Using Millimeter Wave Sensing, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2019. V.U. Prabhu, J. Whaley, Vulnerability of deep learning-based gait biometric recognition to adversarial perturbations, in: CVPR Workshop on the Bright and Dark Sides of Computer Vision: Challenges and Opportunities for Privacy and Security (CV-COPS 2017), 2017. He, 2020 Sheng, 2020, Siamese denoising autoencoders for joints trajectories reconstruction and robust gait recognition, Neurocomputing, 10.1016/j.neucom.2020.01.098 Narayanan, 2020 Feng, 2016, Learning effective gait features using LSTM, 325 Liao, 2020, A model-based gait recognition method with body pose and human prior knowledge, Pattern Recognit. Hasan, 2020, Multi-level feature fusion for robust pose-based gait recognition using RNN, Int. J. Comput. Sci. Inf. Secur. (IJCSIS) Castro, 2017, Evaluation of CNN architectures for gait recognition based on optical flow maps, 251 M.J. Marín-Jiménez, F.M. Castro, N. Guil, F. de la Torre, R. Medina-Carnicer, Deep multi-task learning for gait-based biometrics, in: IEEE Intl. Conference on Image Processing, ICIP, 2017, pp. 106–110. Xu, 2020, Cross-view gait recognition using pairwise spatial transformer networks, IEEE Trans. Circuits Syst. Video Technol. Takemura, 2018, Multi-view large population gait dataset and its performance evaluation for cross-view gait recognition, IPSJ Trans. Comput. Vis. Appl., 10, 4, 10.1186/s41074-018-0039-6 Zhang, 2019, Cross-view gait recognition by discriminative feature learning, IEEE Trans. Image Process. Jaderberg, 2015, Spatial transformer networks, 2017 A. Sokolova, A. Konushin, View Resistant Gait Recognition, in: Proceedings of the 3rd International Conference on Video and Image Processing, 2019, pp. 7–12. Luo, 2019, A sequence-based multi-scale network for cross-view gait recognition, 1179 Khan, 2020, A non-linear view transformations model for cross-view gait recognition, Neurocomputing, 10.1016/j.neucom.2020.03.101 Wu, 2019, Spatial-temporal graph attention network for video-based gait recognition, 274 Chao, 2021, GaitSet: Cross-view gait recognition through utilizing gait as a deep set, IEEE PAMI, 1, 10.1109/TPAMI.2021.3057879 Fan, 2020, Gaitpart: Temporal part-based model for gait recognition, 14225 Hou, 2020, Gait lateral network: Learning discriminative and compact representations for gait recognition, 382 Lin, 2021, Gait recognition via effective global-local feature representation and local temporal aggregation, 14648 Sakata, 2018, Gait-based age estimation using a DenseNet, 55 Bouças, 2019, Generating individual gait kinetic patterns using machine learning, 53 Castro, 2020, Multimodal feature fusion for CNN-based gait recognition: an empirical comparison, Neural Comput. Appl., 1 Hofmann, 2014, The TUM Gait from Audio, Image and Depth (GAID) database: Multimodal recognition of subjects and traits, J. Vis. Commun. Image Represent., 25, 195, 10.1016/j.jvcir.2013.02.006 Castro, 2016, Multimodal features fusion for gait, gender and shoes recognition, Mach. Vis. Appl., 27, 1213, 10.1007/s00138-016-0767-5 Marín-Jiménez, 2021, UGaitNet: Multimodal gait recognition with missing input modalities, IEEE Trans. Inf. Forensics Secur., 16, 5452, 10.1109/TIFS.2021.3132579 Delgado-Escaño, 2021, GaitCopy: Disentangling appearance for gait recognition by signature copy, IEEE Access, 9, 164339, 10.1109/ACCESS.2021.3134705 M. Jia, H. Yang, D. Huang, Y. Wang, Attacking Gait Recognition Systems via Silhouette Guided GANs, in: Proceedings of the 27th ACM International Conference on Multimedia, 2019. Choi, 2019 Chattopadhyay, 2014, Pose depth volume extraction from RGB-D streams for frontal gait recognition, J. Vis. Commun. Image Represent., 25, 53, 10.1016/j.jvcir.2013.02.010 Mazzia, 2020, Real-time apple detection system using embedded systems with hardware accelerators: An edge AI application, IEEE Access, 8, 9102, 10.1109/ACCESS.2020.2964608 Jeong, 2021, Deep learning inference parallelization on heterogeneous processors with tensorrt, IEEE Embedded Syst. Lett. Nguyen, 2020, Towards real-time vehicle detection on edge devices with nvidia jetson tx2, 1 McMahan, 2017, Communication-efficient learning of deep networks from decentralized data, 1273 J. Zhao, R. Mortier, J. Crowcroft, L. Wang, Privacy-preserving machine learning based data analytics on edge devices, in: Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, 2018, pp. 341–346. Z. Tao, Q. Li, {eSGD}: Communication Efficient Distributed Deep Learning on the Edge, in: USENIX Workshop on Hot Topics in Edge Computing (HotEdge 18), 2018. Deng, 2020, Model compression and hardware acceleration for neural networks: A comprehensive survey, Proc. IEEE, 108, 485, 10.1109/JPROC.2020.2976475 Guo, 2020, Model compression using progressive channel pruning, IEEE Trans. Circuits Syst. Video Technol., 31, 1114, 10.1109/TCSVT.2020.2996231 Z. Wang, C. Li, X. Wang, Convolutional neural network pruning with structural redundancy reduction, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 14913–14922. Luo, 2020, Autopruner: An end-to-end trainable filter pruning method for efficient deep model inference, Pattern Recognit., 107, 10.1016/j.patcog.2020.107461 Singh, 2019 A. Goncharenko, S. Alyamkin, A. Denisov, E. Terentev, Winning Solution on LPIRC-ll Competition, in: CVPR Workshops, 2019, pp. 10–16. Sheng, 2018, A quantization-friendly separable convolution for mobilenets, 14 Kang, 2018, Joint optimization of speed, accuracy, and energy for embedded image recognition systems, 715 Wu, 2019, Compressing deep neural networks with sparse matrix factorization, IEEE Trans. Neural Netw. Learn. Syst., 31, 3828, 10.1109/TNNLS.2019.2946636 M. Masana, J. van de Weijer, L. Herranz, A.D. Bagdanov, J.M. Alvarez, Domain-adaptive deep network compression, in: Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 4289–4297. Kim, 2015 Paupamah, 2020, Quantisation and pruning for neural network compression and regularisation, 1 Ma, 2021, Light-YOLOv4: An edge-device oriented target detection method for remote sensing images, IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., 14, 10808, 10.1109/JSTARS.2021.3120009 Y. Kim, J. Kim, D. Chae, D. Kim, J. Kim, MLayer: Low Latency On-Device Inference Using Cooperative Single-Layer Acceleration and Processor-Friendly Quantization, in: Proceedings of the Fourteenth EuroSys Conference 2019, EuroSys ’19, 2019. Jia, 2022, Codl: Efficient CPU-gpu co-execution for deep learning inference on mobile devices, 209 2022 2022 Seichter, 2021, Efficient rgb-d semantic segmentation for indoor scene analysis, 13525 Qasaimeh, 2021, Benchmarking vision kernels and neural network inference accelerators on embedded platforms, J. Syst. Archit., 113, 10.1016/j.sysarc.2020.101896 M. Antonini, T.H. Vu, C. Min, A. Montanari, A. Mathur, F. Kawsar, Resource characterisation of personal-scale sensing models on edge accelerators, in: Proceedings of the First International Workshop on Challenges in Artificial Intelligence and Machine Learning for Internet of Things, 2019, pp. 49–55. David, 2020 Dokic, 2020, Inference speed and quantisation of neural networks with TensorFlow lite for microcontrollers framework, 1 Zeroual, 2019, Using a fine-tuning method for a deep authentication in mobile cloud computing based on tensorflow lite framework, 1 Horn, 1981, Determining optical flow, Artificial Intelligence, 17, 185, 10.1016/0004-3702(81)90024-2 Zhu, 2017 Li, 2018, Optimization based layer-wise magnitude-based pruning for dnn compression, 2383 Lee, 2020 Hofmann, 2014, The tum gait from audio, image and depth (gaid) database: Multimodal recognition of subjects and traits, J. Vis. Commun. Image Represent., 25, 195, 10.1016/j.jvcir.2013.02.006 Yu, 2006, A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition, Vol. 4, 441 Abadi, 2015