Supporting visual quality assessment with machine learning
Tóm tắt
Objective metrics for visual quality assessment often base their reliability on the explicit modeling of the highly non-linear behavior of human perception; as a result, they may be complex and computationally expensive. Conversely, machine learning (ML) paradigms allow to tackle the quality assessment task from a different perspective, as the eventual goal is to mimic quality perception instead of designing an explicit model the human visual system. Several studies already proved the ability of ML-based approaches to address visual quality assessment; nevertheless, these paradigms are highly prone to overfitting, and their overall reliability may be questionable. In fact, a prerequisite for successfully using ML in modeling perceptual mechanisms is a profound understanding of the advantages and limitations that characterize learning machines. This paper illustrates and exemplifies the good practices to be followed.
Tài liệu tham khảo
Hemami SS, Reibman AR: No-reference image and video quality estimation: applications and human-motivated design. Signal Processing: Image Commun 2010, 25(7):469-481. 10.1016/j.image.2010.05.009
Lin W, Jay Kuo C-C: Perceptual visual quality metrics: a survey. J. Visual Commun. Image Representation 2011, 22(4):297-312. 10.1016/j.jvcir.2011.01.005
VQEG, Final report from the video quality experts group on the validation of objective models of video quality assessment. 2003. . Accessed 3 September 2013 http://www.vqeg.org/
Gastaldo P, Redi JA: Machine learning solutions for objective visual quality assessment. Paper presented at the 6th international workshop on video processing and quality metrics for consumer electronics, VPQM-12. Scottsdale; 2012. http://enpub.fulton.asu.edu/resp/vpqm/vpqm12/
Lin F-H, Mersereau RM: Rate–quality tradeoff MPEG video encoder. Signal Processing: Image Commun 1999, 14: 297-309. 10.1016/S0923-5965(98)00014-9
Gastaldo P, Rovetta S, Zunino R: Objective quality assessment of MPEG-2 video streams by using CBP neural networks. IEEE Trans. on Neural Networks 2002, 13(4):939-947. 10.1109/TNN.2002.1021894
Yao S, Lin W, Lu Z, Ong E, Yang X: Video quality assessment using neural network based on multi-feature extraction. Proceedings of the SPIE, Lugano, vol. 5150. In Visual Communications and Image Processing. Edited by: Ebrahimi T, Sikora T. SPIE, Bellingham; 2003:604.
Le Callet P, Viard-Gaudin C, Barba D: A convolutional neural network approach for objective video quality assessment. IEEE Trans Neural Networks 2006, 17(5):1316-1327. 10.1109/TNN.2006.879766
Kanumuri S, Cosman P, Reibman A, Vaishampayan V: Modeling packet-loss visibility in MPEG-2 video. IEEE Transactions on Multimedia 2006, 8(2):341-355.
El Khattabi H, Tamtaoui A, Aboutajdine D: Video quality assessment measure with a neural network. Int. J. Comput. Inf. Eng 2010, 4(3):167-171.
Narwaria M, Lin W, Anmin L: Low-complexity video quality assessment using temporal quality variations. IEEE Trans Multimedia 2012, 14(3):525-535.
Staelens N, Deschrijver D, Vladislavleva E, Vermeulen B, Dhaene T, Demeester P: Constructing a no-reference H.264/AVC bitstream-based video quality metric using genetic programming-based symbolic regression. IEEE Trans on Circuits Systems Video Technol 2013, 23(8):1322-1333.
Gastaldo P, Zunino R: Neural networks for the no-reference assessment of perceived quality. SPIE J Electron Imaging 2005, 14: 033004. 10.1117/1.1988313
Suresh S, Babu RV, Kim HJ: No-reference image quality assessment using modified extreme learning machine classifier. Applied Soft Computing 2009, 9: 541-552. 10.1016/j.asoc.2008.07.005
Narwaria M, Lin W: Objective image quality assessment based on support vector regression. IEEE Trans. Neural Networks 2010, 21(3):515-519.
Redi J, Gastaldo P, Heynderickx I, Zunino R: Color distribution information for the reduced-reference assessment of perceived image quality. IEEE Trans. Circ. Syst. Video Technol 2010, 20(12):1757-1769.
Liu H, Redi J, Alers H, Zunino R, Heynderickx I: An efficient neural-network based no-reference approach to an overall quality metric for JPEG and JPEG2000 compressed images. SPIE J. of Elect. Imaging 2011, 20(4):1-15.
Li C, Bovik AC, Wu X: Blind image quality assessment using a general regression neural network. IEEE Trans Neural Networks 2011, 22(5):793-799.
Moorthy AK, Bovik AC: Blind image quality assessment: from natural scene statistics to perceptual quality. IEEE Trans. Image Proc 2011, 20(12):3350-3364.
Tang H, Joshi N, Kapoor A IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs. In Learning a blind measure of perceptual image quality. IEEE, Piscataway; 2011:305.
Decherchi S, Gastaldo P, Redi J, Zunino R, Cambria E: Circular-ELM for the reduced-reference assessment of perceived image quality. Neurocomputing 2013, 102: 78-89.
Charrier C, Lézoray O, Lebrun G: Machine learning to design full-reference image quality assessment algorithm. Signal Processing: Image Communication 2012, 27: 209-219. 10.1016/j.image.2012.01.002
Narwaria M, Lin W, Enis A: Cetin, Scalable image quality assessment with 2D mel-cepstrum and machine learning approach. Pattern Recognition 2012, 45: 299-313. 10.1016/j.patcog.2011.06.023
Narwaria M, Lin W: SVD-based quality metric for image and video using machine learning. IEEE Trans. on Systems, Man and Cybernetics, Part B. Cybernetics 2012, 42(2):347-364.
Thiede T, Treurniet W, Bitto R, Schmidmer C, Sporer T, Beerends J, Colomes C, Keyhl M, Stoll G, Brandenburg K, Feiten B: PEAQ—the ITU standard for objective measurement of perceived audio quality. J. Audio Eng. Soc 2000, 48(1/2):3-29.
Sebe N, Cohen I, Garg A, Huang TS: Machine Learning in Computer Vision. Springer, Dordrecht; 2005.
Da San Martino G, Sperduti A: Mining structured data. IEEE Comput. Intell. Mag 2010, 5: 42-49.
Rajapakse JC, Zhang Y-Q, Fogel GB IEEE/ACM Transactions on Computational Biology and Bionformatics. Computational intelligence approaches in computational biology and bioinformatics, vol 4 2007.
Bishop CM: Pattern Recognition and Machine Learning. Springer, Dordrecht; 2006.
Guyon I, Elisseeff A: An introduction to variable and feature selection. J. Mach. Learn. Res 2003, 3: 1157-1182.
van der Maaten L, Postma E, van den Herik J: Dimensionality reduction: a comparative review. J Mach. Learn. Res 2009, 10: 1-41.
Vapnik V: Statistical Learning Theory. Wiley, New York; 1998.
Misra J, Saha I: Artificial neural networks in hardware: a survey of two decades of progress. Neurocomputing 2010, 74: 239-255. 10.1016/j.neucom.2010.03.021
Rumelhart DE, McClelland JL: Parallel Distributed Processing. MIT Press, Cambridge; 1986.
Hornik K, Stinchcombe M, White H: Multilayer feedforward networks are universal approximators. Neural Networks 1989, 2(5):359-356. 10.1016/0893-6080(89)90020-8
Baum EB, David H: What size net gives valid generalization? Neural Comput 1989, 1(1):151-160. 10.1162/neco.1989.1.1.151
Widrow B, Lehr MA: 30 years of adaptive neural networks: perceptron Madaline and back propagation. Proc. IEEE 1990, 78(9):1415-1442. 10.1109/5.58323
Koza JR: Genetic programming as a means for programming computers by natural selection. Stat Comput 1994, 4(2):87-112.
Filho JLR, Treleaven PC, Alippi C: Genetic-algorithm programming environments. IEEE Computer 1994, 27(6):28-43.
Espejo PG, Ventura S, Herrera FG: A survey on the application of genetic programming to classification. IEEE Trans. Systems, Man, and Cybernetics, Part C: Applications and Reviews 2010, 40(2):121-144.
Moody JE: The effective number of parameters: an analysis of generalization and regularization in nonlinear learning systems. In Advances in Neural Information Processing Systems 4. Edited by: Moody JE, Hanson SJ, Lippmann RP. Morgan Kaufmann, San Mateo; 1992:847.
Fliegel K, Timmerer C: WG4 Databases Whitepaper v1.5: QUALINET multimedia database enabling QoE evaluations and benchmarking (QUALINET, COST Action IC1003, 2013). . Accessed 3 September 2013 http://dbq-wiki.multimediatech.cz/_media/qi0306.pdf
Seshadrinathan K, Soundararajan R, Bovik A, Cormack L: Study of subjective and objective quality assessment of video. IEEE Trans. Image Process 2010, 19(6):1427-1441.
Simone F, Naccari M, Tagliasacchi M, Dufaux F, Tubaro S, Ebrahimi T Proceedings of the 1st International Workshop on Quality of Multimedia Experience, QoMEX 2009, San Diego. In Subjective assessment of H.264/AVC video sequences transmitted over a noisy channel. IEEE, Piscataway; 2009:204.
LIVE Image Quality Assessment Database (Laboratory for Image & Video Engineering, The University of Texas at Austin). . Accessed 3 September 2013 http://live.ece.utexas.edu/research/quality/
Ponomarenko N, Carli M, Lukin V, Egiazarian K, Astola J, Battisti F Proceedings of IEEE 10th Workshop on Multimedia Signal Processing, Cairns. In Color image database for evaluation of image quality metrics. IEEE, Piscataway; 2008:403.
Larson E, Chandler D: Most apparent distortion: full-reference image quality assessment and the role of strategy. SPIE J. on Electron. Imag 2010, 19(1):011006. 10.1117/1.3267105
Subjective Quality Assessment IRCCyN/IVC Database (IVC, Institut de Recherche en Communications et Cybernétique de Nantes). . Accessed 3 September 2013 http://www2.irccyn.ec-nantes.fr/ivcdb/
Image Quality Evaluation Database (MICT, University of Toyama). . Accessed 3 September 2013 http://160.26.142.130/toyama_database.zip
Pinson MH, Wolf S: Comparing subjective video quality testing methodologies. Proceedings of the SPIE, Lugano. In Visual Communications and Image Processing, vol. 5150. Edited by: Ebrahimi T, Sikora T. SPIE, Bellingham; 2003:573.
Specht DF: A general regression neural network. IEEE Trans. Neural Networks 1991, 2(6):568-576. 10.1109/72.97934
Huang G-B, Wang D, Lan Y: Extreme learning machines: a survey. Int. J. Mach. Learn. Cybern 2011, 2(2):107-122. 10.1007/s13042-011-0019-y
Argyropoulos S, Raake A, Garcia M-N, List P: No-reference video quality assessment of SD and HD H.264/AVC sequences based on continuous estimates of packet loss visibility. In Proceedings of the 3rd International Workshop Quality of Multimedia Experience, QoMEX 2011, Mechelen. IEEE, Piscataway; 2011:31.
Ridella S, Rovetta S, Zunino R: Circular back-propagation networks for classification. IEEE Trans. Neural Networks 1997, 8(1):84-97. 10.1109/72.554194
Strohmeier D, Kunzem K, Göbel K, Liebetrau J: Evaluation of differences in quality of experience features for test stimuli of good-only and bad-only overall audio visual quality. Proceedings of the SPIE, Burlingame, 3. In Image Quality and System Performance X, vol. 8653. Edited by: Burns PD, Triantaphillidou S. SPIE, Bellingham; 2013. doi:10.1117/12.2001363
Chetouani A, Beghdadi A: Image quality assessment based on distortion identification. Proceedings of the SPIE, San Francisco. In Image Quality and System Performance VIII, vol. 7867. Edited by: Farnand SP, Gaykema F. SPIE, Bellingham; 2011. doi:10.1117/12.876308
Simoncelli EP, Freeman WT, Adelson EH, Heeger DJ: Shiftable multiscale transforms. IEEE Trans. Inf. Theory 1992, 38(2):587-607. 10.1109/18.119725
Wang Z, Bovik AC: A universal quality index. IEEE Transactions on Image Processing 2002, 9(3):81-84.
Dempster A: Upper and lower probabilities induced by multi-valued mapping. Annals of Math. Stat 1967, 38: 325-339. 10.1214/aoms/1177698950
Wang Z, Bovik A, Sheikh H, Simoncelli E: Image quality assessment: from error measurement to structural similarity. IEEE Trans. Image Process 2004, 13(1):1-14.
Moorthy AK, Bovik AC: A two-step framework for constructing blind image quality indices. IEEE Signal Processing Letters 2010, 17(5):513-516.
Saad MA, Bovik AC: Blind image quality assessment: a natural scene statistics approach in the DCT domain. IEEE Transactions on Image Processing 2012, 21(8):3339-3352.
International Telecommunication Union (ITU): Objective perceptual video quality measurement techniques for digital cable television in the presence of a full reference. Rec J 2004., 144:
Chang C-C, Lin C-J: LIBSVM: a library for support vector machines. ACM Trans. on Intelligent Systems and Technology 2011, 2(3):1-39. doi:10.1145/1961189.1961199
International Telecommunication Union (ITU): BT.500–11: Methodology for the Subjective Assessment of the Quality of Television Pictures. ITU, Geneva; 2002.
Redi J, Liu H, Zunino R, Heynderickx I: Comparing subjective image quality measurement methods for the creation of public databases. Proceedings of the SPIE, San Jose. In Image Quality and System Performance VII, vol. 7529. Edited by: Farnand SP, Gaykema F. SPIE, Bellingham; 2010. doi:10.1117/12.839195
Chapelle O, Schòlkopf B, Zien A: Semi-Supervised Learning. MIT Press, Cambridge; 2006.