Real-time video denoising on multicores and GPUs with Kalman-based and Bilateral filters fusion
Tóm tắt
In the context of video processing, image noise caused by acquisition, transfer and image compression can be attenuated by video denoising algorithms. However, their computational cost must be as low as possible to allow them to be applied to real-time applications. In this paper, we propose stmkf, a real-time video denoising algorithm based on Kalman and Bilateral filters. We evaluate the effectiveness of stmkf using several common videos used in the literature and we compare it to other denoising algorithms using both the PSNR and SSIM metrics. Our experimental results show that stmkf is competitive with other filters, especially for videos that feature stationary backgrounds such as in videoconferencing, video lectures and video surveillance. We also evaluate the performance of our parallel implementations of stmkf for CPUs and GPUs. stmkf achieved a performance improvement of up to
$$2.9\times $$
on a Intel i7 multicore processor with 4 cores compared to the sequential solution. The results obtained with the GPU version of stmkf on a NVIDIA Tesla K40 showed a performance improvement of up to
$$7.6\times $$
compared to the Intel i7 multicore processor.
Tài liệu tham khảo
Bardu, T.: Variational image denoising approach with diffusion porous media flow. Abstr. Appl. Anal. 2013, 8 (2013)
Buades, A., Coll, B., Morel, J.-M.: Nonlocal image and movie denoising. Int. J. Comput. Vision 76(2), 123–139 (2008)
Buades, A., Coll, B., Morel, J.-M.: A non-local algorithm for image denoising. In: Conference on Computer Vision and Pattern Recognition, CVPR ’05, pp. 60–65. Washington, DC, USA, IEEE Computer Society (2005)
Chan, T.-W., Au, O.C., Chong, T.-S., Chau, W.-S.: A novel content-adaptive video denoising filter. In: IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pp. 649–652, Philadelphia, USA (2005)
Chaudhury, K.N.: Acceleration of the shiftable algorithm for bilateral filtering and nonlocal means. IEEE Trans. Image Process. 22(4), 1291–1300 (2013)
Chen, T.-Y., Chen, T.-H., Su, C.-P., Chen, Y.-J.: The study on video enhancement in the low-light environment by spatio-temporal filtering. In: International Conference on Intelligent Systems Design and Applications (ISDA), vol. 3, pp. 561–564, Kaohsiung, Taiwan (2008)
Chenglin Z., Yu, L., Xin, T., Wei, W., Maojun, Z. (2013) Video denoising based on a spatiotemporal Kalman-bilateral mixture model. Sci. World J. 2013 (2013)
Dabov, K., Foi, A., Egiazarian, K.: Video denoising by sparse 3D transform-domain collaborative filtering. In: European Signal Processing Conference, pp. 145–149, Poznan, Poland. IEEE (2007)
Davis, L., Rosenfeld, A.: Noise cleaning by iterated cleaning. IEEE Trans. Syst. Man Cybern. SMC 8(9), 705–710 (1978)
Dufaux, F., Callet, P.L., Mantiuk, R., Mrak, M.: High Dynamic Range Video: From Acquisition, to Display and Applications. Elsevier (2016). ISBN 9780128030394
Farooque, M.A., Sohankar, J.S.: Survey on various noises and techniques for denoising the color image. Int. J. Appl. Innov. Eng. Manage. (IJAIEM) 2, 217 (2013)
Garg, R., Kumar, A.: Comparision of various noise removals using bayesian framework. Int. J. Mod. Eng. Res. (IJMER) 2, 265 (2012)
Han, Y., Chen, R.: Efficient video denoising based on dynamic nonlocal means. Image Vision Comput. 30, 78–85 (2012)
Hong-Zhi, W., Ling, C., Shu-Liang, X.: Improved video denoising algorithm based on spatial-temporal combination. In: International Conference on Image and Graphics (ICIG), pp. 64–67, Qingdao, China. IEEE (2013)
Jojy, C., Nair, M.S., Subrahmanyam, G.R.K.S., Raji, R.: Discontinuity adaptive non-local means with importance sampling unscented Kalman filter for de-speckling SAR images. IEEE J. Sel. Top. Appl. Earth Obser. Remote Sens. 6(4), 1964–1970 (2013)
Jung, B., Sukhatme, G.S.: Detecting moving objects using a single camera on a mobile robot in an outdoor environment. In: International Conference on Intelligent Autonomous Systems, pp. 980–987 (2004)
Kalman, R.E.: A new approach to linear filtering and prediction problems. Trans. ASME J. Basic Eng. 82(D), 35–45 (1960)
Karnati, V., Uliyar, M., Dey, S.: Fast non-local algorithm for image denoising. In: International Conference on Image Processing (ICIP), pp. 3873–3876. IEEE (Nov 2009)
Kirk, D.B., Wen-mei W.H.: Programming Massively Parallel Processors: A Hands-on Approach. Morgan Kaufmann Publishers Inc., San Francisco, 1st edn. (2010). ISBN 0123814723
Kokkonis, G., Psannis, K.E., Roumeliotis, M., Ishibashi, Y.: Efficient Algorithm for transferring a real-time HEVC stream with haptic data through the internet. J. Real-Time Image Process. pp. 1–13, (2015). ISSN 1861-8219. doi:10.1007/s11554-015-0505-7
Kokkonis, G., Psannis, K.E., Roumeliotis, M., Schonfeld, D.: Real-time wireless multisensory smart surveillance with 3D-HEVC streams for internet-of-things (iot). J. Supercomput. pp 1–19, (2016). ISSN 1573-0484. doi:10.1007/s11227-016-1769-9
Kostadin D., Alessandro F., Vladimir K., Karen E.: Image denoising with block-matching and 3D filtering. In: SPIE-IS&T Electronic Imaging, p. 6064 (2006)
Li, W., Zhang, J., Dai, Q.: Video denoising using shape-adaptive sparse representation over similar spatio-temporal patches. Signal Proc.: Image. Communication 26(4–5), 250–265 (2011)
Li, X., Zheng, Y.: Patch-based video proc.: a variational bayesian approach. IEEE Trans. Circuits Syst Video Technol 19(1), 27–40 (2009)
Mahmoud, R.O., Faheem, M.T., Sarhan, A.: Intelligent denoising technique for spatial video denoising for real-time applications. In: International Conference on Computer Engineering Systems (ICCES), pp. 407–412, Cairo, Egypt. IEEE (2008)
Mahmoudi, M., Sapiro, G.: Fast image and video denoising via nonlocal means of similar neighborhoods. IEEE Signal Process. Lett. 12(12), 839–842 (2005)
Memos, V.A., Psannis, K.E.: Encryption algorithm for efficient transmission of hevc media. J. Real-Time Image Process. pp. 1–10, (2015). ISSN 1861-8219. doi:10.1007/s11554-015-0509-3
Mitchell, H.B., Mashkit, N.: Noise smoothing by a fast k-nearest neighbour algorithm. Signal Process. Image Commun. 4(3), 227–232 (1992)
OpenMP Architecture Review Board. OpenMP application program interface version 4.0, July 2013. URL http://www.openmp.org/mp-documents/OpenMP4.0.0.pdf
Pauwels, K., Tomasi, M., Alonso, J. Diaz., Ros, E., Van Hulle, M. M.: A comparison of fpga and GPU for real-time phase-based optical flow, stereo, and local image features. IEEE Trans. Comput. 61(7): 999–1012, (2012). ISSN 0018-9340
Pizurica, A., Zlokolica, V., Philips, W.: Noise reduction in video sequences using wavelet-domain and temporal filtering. In: Photonics Technologies for Robotics, Automation, and Manufacturing, Int. Soc. for Optics and Photonics, pp. 48–59 (2004)
Psannis, K.E.: Hevc in wireless environments. J. Real-Time Image Process. pp. 1–8, (2015). ISSN 1861-8219. doi:10.1007/s11554-015-0514-6
Pulli, K., Baksheev, A., Kornyakov, K., Eruhimov, V.: Real-time computer vision with OpenCV. Commun. ACM 55(6): 61–69, (2012). ISSN 0001-0782
Rahman, S.M.M., Ahmad, M.O., Swamy, M.N.S.: Video denoising based on inter-frame statistical modeling of wavelet coefficients. IEEE Trans. Circuits Syst. Video Technol. 17(2), 187–198 (2007)
Ryu, J., Nishimura, T. H.: Fast image blurring using lookup table for real time feature extraction. In: 2009 IEEE International Symposium on Industrial Electronics, pp. 1864–1869 (2009)
Seiller, N., Singhal, N., Park, I.K.: Object oriented framework for real-time image processing on GPU. In: International Conference on Image Processing (ICIP), pp. 4477–4480, Hong Kong, China. IEEE (2010)
Selesnick, I.W, Li, K.Y.: Video denoising using 2D and 3D dual-tree complex wavelet transforms. In: Annual Meeting on Optical Science and Technology (SPIE), Int. Soc. for Optics and Photonics, pp. 607–618. (2003)
Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: International Conference on Computer Vision, Bombay, India, pp. 839–846, IEEE (1998)
Van De Ville, D., Kocher, M.: SURE-based non-local means. IEEE Signal Process. Lett. 16(11), 973–976 (2009)
Wang, Z., Bovik, A.C., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Proc. 13(4), 600 (2004)
Wolf, W., Ozer, B., Lv, T.: Smart cameras as embedded systems. Computer 35(9), 48–53 (2002)
Zlokolica, V., Pizurica, A., Philips, W.: Wavelet-domain video denoising based on reliability measures. IEEE Trans. Circuits Syst. Video Technol. 16(8), 993–1007 (2006)
Zlokolica, V., Philips, W., Van De Ville, D.: A new non-linear filter for video processing. In: IEEE Benelux Signal Processing Symposium, pp. 221–224 (2002)