Referenceless image quality assessment by saliency, color-texture energy, and gradient boosting machines
Tóm tắt
In most practical multimedia applications, processes are used to manipulate the image content. These processes include compression, transmission, or restoration techniques, which often create distortions that may be visible to human subjects. The design of algorithms that can estimate the visual similarity between a distorted image and its non-distorted version, as perceived by a human viewer, can lead to significant improvements in these processes. Therefore, over the last decades, researchers have been developing quality metrics (i.e., algorithms) that estimate the quality of images in multimedia applications. These metrics can make use of either the full pristine content (full-reference metrics) or only of the distorted image (referenceless metric). This paper introduces a novel referenceless image quality assessment (RIQA) metric, which provides significant improvements when compared to other state-of-the-art methods. The proposed method combines statistics of the opposite color local variance pattern (OC-LVP) descriptor with statistics of the opposite color local salient pattern (OC-LSP) descriptor. Both OC-LVP and OC-LSP descriptors, which are proposed in this paper, are extensions of the opposite color local binary pattern (OC-LBP) operator. Statistics of these operators generate features that are mapped into subjective quality scores using a machine-learning approach. Specifically, to fit a predictive model, features are used as input to a gradient boosting machine (GBM). Results show that the proposed method is robust and accurate, outperforming other state-of-the-art RIQA methods.
Tài liệu tham khảo
Seshadrinathan K, Bovik AC (2011) Automatic prediction of perceptual quality of multimedia signals—a survey. Multimed Tools Appl 51(1):163–186.
Chandler DM (2013) Seven challenges in image quality assessment: past, present, and future research. ISRN Signal Process, vol. 2013. https://doi.org/10.1155/2013/905685. https://www.hindawi.com/journals/isrn/2013/905685/.
Telecom I (2000) Recommendation 500-10: Methodology for the subjective assessment of the quality of television pictures. ITU-R Rec. BT.500.
Wang Z, Bovik AC (2009) Mean squared error: love it or leave it? A new look at signal fidelity measures. IEEE Signal Proc Mag 26(1):98–117.
Moghadam A, Mohammadi P, Shirani S (2015) Subjective and objective quality assessment of image: a survey. Majlesi J Electr Eng 9(1):55–83. https://profdoc.um.ac.ir/paper-abstract-1048833.html.
Fang Y, Ma K, Wang Z, Lin W, Fang Z, Zhai G (2015) No-reference quality assessment of contrast-distorted images based on natural scene statistics. Signal Process Lett IEEE 22(7):838–842.
Bahrami K, Kot AC (2014) A fast approach for no-reference image sharpness assessment based on maximum local variation. Signal Process Lett IEEE 21(6):751–755.
Golestaneh SA, Chandler DM (2014) No-reference quality assessment of JPEG images via a quality relevance map. Signal Process Lett IEEE 21(2):155–158.
Li L, Lin W, Zhu H (2014) Learning structural regularity for evaluating blocking artifacts in jpeg images. Signal Process Lett IEEE 21(8):918–922.
Li L, Zhou Y, Lin W, Wu J, Zhang X, Chen B (2016) No-reference quality assessment of deblocked images. Neurocomputing 177:572–584.
Li L, Zhu H, Yang G, Qian J (2014) Referenceless measure of blocking artifacts by Tchebichef kernel analysis. Signal Process Lett IEEE 21(1):122–125.
Liu L, Hua Y, Zhao Q, Huang H, Bovik AC (2016) Blind image quality assessment by relative gradient statistics and adaboosting neural network. Signal Process Image Commun 40:1–15.
Li Q, Lin W, Fang Y (2016) No-reference quality assessment for multiply-distorted images in gradient domain. IEEE Signal Process Lett 23(4):541–545. https://doi.org/10.1109/LSP.2016.2537321.
Saad MA, Bovik AC, Charrier C (2012) Blind image quality assessment: a natural scene statistics approach in the DCT domain. Image Process IEEE Trans 21(8):3339–3352.
Moorthy AK, Bovik AC (2011) Blind image quality assessment: from natural scene statistics to perceptual quality. Image Process IEEE Trans 20(12):3350–3364.
Mittal A, Moorthy AK, Bovik AC (2012) No-reference image quality assessment in the spatial domain. Image Process IEEE Trans 21(12):4695–4708.
Saad MA, Bovik AC, Charrier C (2010) A DCT statistics-based blind image quality index. IEEE Signal Process Lett 17(6):583–586.
Liu L, Liu B, Huang H, Bovik AC (2014) No-reference image quality assessment based on spatial and spectral entropies. Signal Process Image Commun 29(8):856–863.
Freitas PG, Akamine WY, Farias MC (2016) Blind image quality assessment using multiscale local binary patterns. J Imaging Sci Technol 60(6):60405–1.
Freitas PG, Akamine WY, Farias MC (2016) No-reference image quality assessment based on statistics of local ternary pattern In: 2016 Eighth International Conference on Quality of Multimedia Experience (QoMEX), 1–6.. IEEE. https://ieeexplore.ieee.org/abstract/document/7498959/.
Freitas PG, Akamine WY, Farias MC (2016) No-reference image quality assessment using texture information banks In: Intelligent Systems (BRACIS), 2016 5th Brazilian Conference On, 127–132.. IEEE. https://ieeexplore.ieee.org/abstract/document/7839574/.
Ye P, Doermann D (2012) No-reference image quality assessment using visual codebooks. Image Process IEEE Trans 21(7):3129–3138.
Ye P, Kumar J, Kang L, Doermann D (2012) Unsupervised feature learning framework for no-reference image quality assessment In: Computer vision and pattern recognition (CVPR), 2012 IEEE Conference On, 1098–1105. IEEE. https://ieeexplore.ieee.org/abstract/document/6247789/.
Zhang M, Muramatsu C, Zhou X, Hara T, Fujita H (2015) Blind image quality assessment using the joint statistics of generalized local binary pattern. Signal Process Lett IEEE 22(2):207–210.
Zhang Y, Wu J, Xie X, Shi G (2016) Blind image quality assessment based on local quantized pattern In: Pacific Rim Conference on Multimedia, 241–251.. Springer. https://link.springer.com/chapter/10.1007/978-3-319-48896-7_24.
Wu Q, Wang Z, Li H (2015) A highly efficient method for blind image quality assessment In: Image processing (ICIP), 2015 IEEE International Conference On, 339–343.. IEEE. https://ieeexplore.ieee.org/abstract/document/7350816/.
Wu J, Lin W, Shi G (2014) Image quality assessment with degradation on spatial structure. Signal Process Lett IEEE 21(4):437–440.
Larson EC, Chandler DM (2010) Most apparent distortion: full-reference image quality assessment and the role of strategy. J Electron Imaging 19(1):011006–011006.
Charrier C, Saadane A, Fernandez-Maloigne C (2017) No-reference learning-based and human visual-based image quality assessment metric In: 19th International Conference on Image Analysis and Processing, Catania. https://link.springer.com/chapter/10.1007/978-3-319-68548-9_23.
Sheikh HR, Bovik AC (2006) Image information and visual quality. IEEE Trans Image Process 15(2):430–444.
Chandler DM, Hemami SS (2007) VSNR: a wavelet-based visual signal-to-noise ratio for natural images. IEEE Trans Image Process 16(9):2284–2298.
Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. Image Process IEEE Trans 13(4):600–612.
Gu K, Zhai G, Yang X, Zhang W (2015) Using free energy principle for blind image quality assessment. IEEE Trans Multimed 17(1):50–63.
Zhang L, Zhang L, Mou X (2010) RFSIM: a feature based image quality assessment metric using riesz transforms In: Image Processing (ICIP), 2010 17th IEEE International Conference On, 321–324.. IEEE. https://ieeexplore.ieee.org/abstract/document/5649275/.
Kang L, Ye P, Li Y, Doermann D (2014) Convolutional neural networks for no-reference image quality assessment In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1733–1740. https://ieeexplore.ieee.org/abstract/document/6909620/.
Li J, Zou L, Yan J, Deng D, Qu T, Xie G (2016) No-reference image quality assessment using Prewitt magnitude based on convolutional neural networks. SIViP 10(4):609–616.
Bosse S, Maniry D, Wiegand T, Samek W (2016) A deep neural network for image quality assessment In: Image Processing (ICIP), 2016 IEEE International Conference On, 3773–3777.. IEEE. https://ieeexplore.ieee.org/abstract/document/7533065/.
Kuzovkin I, Vicente R, Petton M, Lachaux JP, Baciu M, Kahane P, Rheims S, Vidal JR, Aru J (2017) Frequency-resolved correlates of visual object recognition in human brain revealed by deep convolutional neural networks. bioRxiv. https://doi.org/10.1101/133694. https://www.biorxiv.org/content/early/2017/05/03/133694.full.pdf.
Yamins DL, DiCarlo JJ (2016) Using goal-driven deep learning models to understand sensory cortex. Nat Neurosci 19(3):356.
Zhang L, Shen Y, Li H (2014) VSI: a visual saliency-induced index for perceptual image quality assessment. IEEE Trans Image Process 23(10):4270–4281.
Farias MC, Akamine WY (2012) On performance of image quality metrics enhanced with visual attention computational models. Electron Lett 48(11):631–633.
Engelke U, Kaprykowsky H, Zepernick HJ, Ndjiki-Nya P (2011) Visual attention in quality assessment. IEEE Signal Proc Mag 28(6):50–59.
Gu K, Wang S, Yang H, Lin W, Zhai G, Yang X, Zhang W (2016) Saliency-guided quality assessment of screen content images. IEEE Trans Multimed 18(6):1098–1110.
You J, Perkis A, Hannuksela MM, Gabbouj M (2009) Perceptual quality assessment based on visual attention analysis In: Proceedings of the 17th ACM International Conference on Multimedia, 561–564.. ACM. https://doi.org/10.1145/1631272.1631356.
Le Meur O, Ninassi A, Le Callet P, Barba D (2010) Overt visual attention for free-viewing and quality assessment tasks: impact of the regions of interest on a video quality metric. Signal Process Image Commun 25(7):547–558.
Le Meur O, Ninassi A, Le Callet P, Barba D (2010) Do video coding impairments disturb the visual attention deployment?. Signal Process Image Commun 25(8):597–609.
Akamine WY, Farias MC (2014) Video quality assessment using visual attention computational models. J Electron Imaging 23(6):061107.
Ortiz-Jaramillo B, Kumcu A, Philips W (2016) Evaluating color difference measures in images In: Quality of Multimedia Experience (QoMEX), 2016 Eighth International Conference On, 1–6.. IEEE. https://ieeexplore.ieee.org/abstract/document/7498922/.
Gu K, Zhai G, Lin W, Liu M (2016) The analysis of image contrast: from quality assessment to automatic enhancement. IEEE Trans Cybern 46(1):284–297.
Liu L, Dong H, Huang H, Bovik AC (2014) No-reference image quality assessment in curvelet domain. Signal Process Image Commun 29(4):494–505.
Maenpaa T, Pietikainen M, Viertola J (2002) Separating color and pattern information for color texture discrimination In: Pattern Recognition, 2002. Proceedings. 16th International Conference On, 668–671. IEEE. https://ieeexplore.ieee.org/abstract/document/1044840/.
Freitas PG, Akamine WYL, de Farias MCQ (2017) Blind image quality assessment using local variant patterns In: 2017 Brazilian Conference on Intelligent Systems (BRACIS), 252–257.. IEEE. https://ieeexplore.ieee.org/abstract/document/8247062/.
Friedman JH (2001) Greedy function approximation: a gradient boosting machine. Ann Stat:1189–1232. https://www.jstor.org/stable/2699986?seq=1#page_scan_tab_contents.
Natekin A, Knoll A (2013) Gradient boosting machines, a tutorial. Front Neurorobotics 7:21.
Ojala T, Pietikäinen M, Mäenpää T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. Pattern Anal Mach Intell IEEE Trans 24(7):971–987.
He DC, Wang L (1990) Texture unit, texture spectrum, and texture analysis. Geosci Remote Sens IEEE Trans 28(4):509–512.
Ojala T, Pietikäinen M, Mäenpää T (2000) Gray scale and rotation invariant texture classification with local binary patterns In: Computer Vision-ECCV 2000, 404–420.. Springer, Berlin.
Pietikäinen M, Ojala T, Xu Z (2000) Rotation-invariant texture classification using feature distributions. Pattern Recog 33(1):43–52.
Jain A, Healey G (1998) A multiscale representation including opponent color features for texture recognition. IEEE Trans Image Process 7(1):124–128.
Sheikh HR, Sabir MF, Bovik AC (2006) A statistical evaluation of recent full reference image quality assessment algorithms. Image Process IEEE Trans 15(11):3440–3451.
Ponomarenko N, Jin L, Ieremeiev O, Lukin V, Egiazarian K, Astola J, Vozel B, Chehdi K, Carli M, Battisti F, et al. (2015) Image database TID2013: peculiarities, results and perspectives. Signal Process Image Commun 30:57–77.
Zhang J, Sclaroff S (2016) Exploiting surroundedness for saliency detection: a boolean map approach. IEEE Trans Pattern Anal Mach Intell 38(5):889–902.
Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825–2830.
Chen T, Guestrin C (2016) Xgboost: a scalable tree boosting system In: Proceedings of the 22nd Acm Sigkdd International Conference on Knowledge Discovery and Data Mining, 785–794.. ACM. https://doi.org/10.1145/2939672.2939785.