Image auto-annotation via concept interdependency network

Multimedia Tools and Applications - Tập 75 - Trang 6237-6261 - 2015
HaiJiao Xu1, Peng Pan1, ChunYan Xu2, YanSheng Lu1, Deng Chen1
1School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, People’s Republic of China
2Electrical and Computer Engineering, National University of Singapore, Singapore, Singapore

Tóm tắt

With the explosive growth of multimedia data such as unlabeled images on the Web, image auto-annotation has been receiving increasing research interest. By automatically assigning a set of concepts to unlabeled images, image retrieval can be performed over labeled concepts. Most existing studies focus on the relations between images and concepts, and ignore the interdependencies between labeled concepts. In this paper, we propose a novel image auto-annotation model which utilizes the concept interdependency network to achieve better image auto-annotation. When a concept and its interdependent concepts have a high co-occurrence frequency in the training set, we consider boosting the chance of predicting this concept if there is strong visual evidence for the interdependent concepts in an unlabeled image. Additionally, we combine the global concept interdependency and the local concept interdependency to enhance the auto-annotation performance. Extensive experiments on Corel and IAPR datasets show that the proposed approach almost outperforms all existing methods.

Tài liệu tham khảo

Bishop CM, et al. (2006) Pattern recognition and machine learning, vol 1. Springer, New York Carneiro G, Chan AB, Moreno PJ, Vasconcelos N (2007) Supervised learning of semantic classes for image annotation and retrieval. IEEE Trans Pattern Anal Mach Int 29(3):394–410 Chang CC, Lin CJ (2011) Libsvm: a library for support vector machines. ACM Trans Int Syst Technol (TIST) 2(3):27 Chen M, Zheng A, Weinberger K (2013) Fast image tagging. In: Proceedings of the 30th international conference on Machine Learning, pp 1274–1282 Chen PI, Lin SJ, Chu YC (2011) Using google latent semantic distance to extract the most relevant information. Expert Syst Appl 38(6):7349–7358 Choi J, Cho M, Park SH, Kim P (2003) Concept-based image retrieval using the new semantic similarity measurement. In: Computational Science and Its ApplicationsłCCSA 2003. Springer, pp 79–88 Cilibrasi RL, Vitanyi PM (2007) The google similarity distance. IEEE Trans Knowl Data Eng 19(3): 370–383 Cui C, Ma J, Lian T, Wang X, Ren Z (2013) Ranking-oriented nearest-neighbor based method for automatic image annotation. In: Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval. ACM, pp 957–960 Das SR, Panigrahi PK, Das K, Mishra D (2012) Improving rbf kernel function of support vector machine using particle swarm optimization. International Journal Duygulu P, Barnard K, de Freitas J F, Forsyth DA (2002) Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary. In: Computer VisionłECCV 2002. Springer, pp 97–112 Fellbaum C (1998) WordNet. Wiley Online Library Feng S, Manmatha R, Lavrenko V (2004) Multiple bernoulli relevance models for image and video annotation. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004, vol 2. IEEE, pp II–1002 Feng Z, Jin R, Jain A (2013) Large-scale image annotation by efficient and robust kernel metric learning. In: 2013 IEEE International Conference on Computer Vision (ICCV). IEEE, pp 1609–1616 Fu H, Zhang Q, Qiu G (2012) Random forest for image annotation. In: Computer Vision–ECCV 2012. Springer, pp 86–99 Grubinger M, Clough P, Müller H, Deselaers T (2006) The iapr tc-12 benchmark: A new evaluation resource for visual information systems. In: International Workshop OntoImage, pp 13–23 Guillaumin M, Küttel D, Ferrari V (2014) Imagenet auto-annotation with segmentation propagation. International Journal of Computer Vision: pp 1–21 Guillaumin M, Mensink T, Verbeek J, Schmid C (2009) Tagprop: Discriminative metric learning in nearest neighbor models for image auto-annotation. In: IEEE 12th International Conference on Computer Vision, 2009. IEEE, pp 309–316 Gruber T (1993) A translation approach to portable ontology specifications. Knowl Acquis 5(2):199–220 Gruber T (2009) Ontology. Encyclopedia of database systems, pp 1963–1965 Hu J, Lam KM (2013) An efficient two-stage framework for image annotation. Pattern Recog 46(3):936–947 Huang Z, Qiu Y (2010) A multiple-perspective approach to constructing and aggregating citation semantic link network. Futur Gener Comput Syst 26(3):400–407 Jeon J, Lavrenko V, Manmatha R (2003) Automatic image annotation and retrieval using cross-media relevance models. In: Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval. ACM, pp 119–126 Jin R, Chai JY, Si L (2004) Effective automatic image annotation via a coherent language model and active learning. In: Proceedings of the 12th annual ACM international conference on Multimedia. ACM, pp 892–899 Jin Y, Khan L, Wang L, Awad M (2005) Image annotations by combining multiple evidence & wordnet. In: Proceedings of the 13th annual ACM international conference on Multimedia. ACM, pp 706–715 Lavrenko V, Manmatha R, Jeon J (2003) A model for learning the semantics of pictures. In: Advances in neural information processing systems, p. None Liu J, Li M, Liu Q, Lu H, Ma S (2009) Image annotation via graph learning. Pattern Recog 42(2):218–228 Maji S, Berg AC, Malik J (2008) Classification using intersection kernel support vector machines is efficient. In: IEEE Conference on Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE, pp 1–8 Makadia A, Pavlovic V, Kumar S (2010) Baselines for image annotation. Int J Comput Vis 90(1):88–105 Manevitz LM, Yousef M (2002) One-class svms for document classification. J Mach Learn Res 2:139–154 Metzler D, Manmatha R (2004) An inference network approach to image retrieval. In: Image and video retrieval. Springer, pp 42–50 Moran S, Lavrenko V (2014) Sparse kernel learning for image annotation. In: Proceedings of international conference on Multimedia Retrieval. ACM, p 113 Nguyen CT, Kaothanthong N, Tokuyama T, Phan XH (2013) A feature-word-topic model for image annotation and retrieval. ACM Trans Web (TWEB) 7(3):12 Srikanth M, Varner J, Bowden M, Moldovan D (2005) Exploiting ontologies for automatic image annotation. In: Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval. ACM, pp 552–558 Verma Y, Jawahar C (2013) Exploring svm for image annotation in presence of confusing labels. In: Proceedings of the 24th British Machine Vision Conference Wang C, Zhang L, Zhang HJ (2008) Scalable markov model-based image annotation. In: Proceedings of the 2008 international conference on Content-based image and video retrieval. ACM, pp 113–118 Wang M, Xia X, Le J, Zhou X (2014) Effective automatic image annotation via integrated discriminative and generative models. Inf Sci 262:159–171 Wang Z, Guan G, Qiu Y, Zhuo L, Feng D (2013) Semantic context based refinement for news video annotation. Multimedia Tools Appl 67(3):607–627 Wei XY, Jiang YG, Ngo CW (2011) Concept-driven multi-modality fusion for video search. IEEE Trans Circ Syst Video Technol 21(1):62–73 Xiang Y, Zhou X, Chua TS, Ngo CW (2009) A revisit of generative model for automatic image annotation using markov random fields. In: IEEE Conference on Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE, pp 1153–1160 Xie L, Pan P, Lu Y, Wang S, Zhu T, Xu H, Chen D (2013) A two-phase generation model for automatic image annotation. In: 2013 IEEE International Symposium on Multimedia (ISM), pp 155–162 Xu H, Pan P, Lu Y, Xu C, Chen D (2014) Improving automatic image annotation with google semantic link. In: 2014 10th international conference on Semantics, Knowledge and Grids (SKG), pp 177–184 Yavlinsky A, Schofield E, Rüger S (2005) Automated image annotation using global features and robust nonparametric density estimation. In: Image and video retrieval. Springer, pp 507–517 Yu Y, Pedrycz W, Miao D (2013) Neighborhood rough sets based multi-label classification for automatic image annotation. Int J Approx Reason 54(9):1373–1387 Zhang D, Islam MM, Lu G (2012) A review on automatic image annotation techniques. Pattern Recognit 45(1):346–362 Zhang S, Huang J, Li H, Metaxas DN (2012) Automatic image annotation and retrieval using group sparsity. IEEE Trans on Systems, Man, and Cybernetics, Part B: Cybernetics 42(3):838–849 Zhuge H (2004) The knowledge grid, vol 2012. World Scientific Zhuge H (2010) Interactive semantics. Artif Intell 174(2):190–204 Zhuge H (2011) Semantic linking through spaces for cyber-physical-socio intelligence: A methodology. Artif Intell 175(5):988–1019 Zhuge H, Sun Y (2010) The schema theory for semantic link network. Futur Gener Comput Syst 26(3):408–420