Đề xuất đối tượng cho phân đoạn đối tượng nổi bật trong video

Multimedia Tools and Applications - Tập 79 - Trang 8677-8693 - 2019
Rahma Kalboussi1, Aymen Azaza2, Joost van de Weijer3, Mehrez Abdellaoui4, Ali Douik4
1National Engineering School of Sousse, ISITCOM Hammam Sousse, University of Sousse, Sahloul, Tunisia
2National Engineering School of Monastir, University of Monastir, Monastir, Tunisia
3Computer Vision Center, Bellaterra (Cerdanyola), Spain
4National Engineering School of Sousse, University of Sousse, Sousse, Tunisia

Tóm tắt

Phân đoạn đối tượng nổi bật trong video thường được tách thành hai phần: phân đoạn video và phân bổ độ nổi bật. Gần đây, các đề xuất đối tượng, được sử dụng để phân đoạn hình ảnh, đã có tác động đáng kể đến nhiều ứng dụng của thị giác máy tính, bao gồm phân đoạn hình ảnh, phát hiện đối tượng và gần đây là phát hiện độ nổi bật trong hình ảnh tĩnh. Tuy nhiên, việc sử dụng chúng vẫn chưa được đánh giá cho phân đoạn đối tượng nổi bật trong video. Do đó, trong bài báo này, chúng tôi điều tra ứng dụng của các đề xuất đối tượng vào phân đoạn đối tượng nổi bật trong video. Ngoài ra, chúng tôi đề xuất một đặc tính chuyển động mới được suy diễn từ tensor cấu trúc dòng quang học để phát hiện độ nổi bật trong video. Các thí nghiệm trên hai tập dữ liệu chuẩn cho độ nổi bật video cho thấy đặc tính chuyển động đề xuất cải thiện kết quả ước lượng độ nổi bật, và các đề xuất đối tượng là một phương pháp hiệu quả cho phân đoạn đối tượng nổi bật. Kết quả trên các tập dữ liệu thách thức SegTrack v2 và Fukuchi cho thấy chúng tôi vượt trội hơn đáng kể so với công nghệ tiên tiến nhất hiện nay.

Từ khóa

#phân đoạn đối tượng nổi bật #đề xuất đối tượng #phát hiện độ nổi bật video #đặc tính chuyển động #thị giác máy tính

Tài liệu tham khảo

Achanta R, Hemami S, Estrada F, Susstrunk S (2009) Frequency-tuned salient region detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 1597–1604 Azaza A, Van de Weijer J, Douik A, Masana M (2018) Context proposals for saliency detection. Comput Vis Image Underst 174:1–11 Bigun J, Granlund GH, Wiklund J (1991) Multidimensional orientation estimation with applications to texture analysis and optical flow. IEEE Trans Pattern Anal Mach Intell 13(8):775–790 Borji A, Cheng M-M, Jiang H, Li J (2014) Salient object detection:, A survey, arXiv preprint arXiv., pp 1411–5878 Borji A, Cheng M-M, Jiang H, Li J (2015) Salient object detection: A benchmark. IEEE Trans Image Process, pp 5706–5722 Breiman L (2001) Random forests. Machine learning, pp 5–32 Brox T, Malik J (2010) Object segmentation by long term analysis of point trajectories. European Conference on Computer Vision, pp 282–295 Bruce N, Tsotsos J (2010) Saliency based on information maximization. In: Advances in Neural Information Processing Systems, pp 282–295 Carreira J, Sminchisescu C (2010) Constrained parametric min-cuts for automatic object segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 3241–3248 Fragkiadaki K, Zhang G, Shi J (2012) Video segmentation by tracing discontinuities in a trajectory embedding. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 1846–1853 Frintrop S, Werner T, García GM (2015) Traditional saliency reloaded: a good old model in new shape. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 82–90 Fukuchi K, Miyazato K, Kimura A, Takagi S, Yamato J (2009) Saliency-based video segmentation with graph cuts and sequentially updated priors. In: IEEE International Conference on Multimedia and Expo, pp 638–641 Goferman S, Zelnik-Manor L, Tal A (2012) Context-aware saliency detection. IEEE Trans Pattern Anal Mach Intell 34(10):1915–1926 Itti L, Koch C, Niebur E, et al. (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259 Kim H, Kim Y, Sim J-Y, Kim C-S (2015) Spatiotemporal saliency detection for video sequences based on random walk with restart. IEEE Trans Image Process 24 (8):2552–2564 Harel J, Koch C, Perona PO (2006) Graph-based visual saliency. In: Advances in Neural Information Processing Systems, pp 545–5524 Han B, Li X, Gao X, Tao D (2012) A biological inspired features based saliency map. In: International Conference on Computing, Networking and Communications, pp 371–375 Itti L, Baldi P (2005) A Principled approach to detecting surprising events in video. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 631–637 Jiang H, Wang J, Yuan Z, Wu Y, Zheng N, Li S (2013) Salient object detection: A discriminative regional feature integration approach. in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2083–2090 Jiang H, Wang J, Yuan Z, Liu T, Zheng N, Li S (2011) Automatic salient object segmentation based on context and shape prior. In: British Machine Vision Conference, pp 2083–2090 Judd T, Ehinger K, Durand F, Torralba A (2009) Learning to predict where humans look. in: IEEE 12th International Conference on Computer Vision, pp 2106–2113 Kim J, Han D, Tai Y-W, Kim J (2014) Salient region detection via high-dimensional color transform. In: Proc IEEE Conf Comput Vis Pattern Recognit, pp 883–890 Krähenbühl P, Koltun V (2014) Geodesic object proposals. European Conference on Computer Vision, pp 725–739 Li F, Kim T, Humayun A, Tsai D, Rehg JM (2013) Video segmentation by tracking many figure-ground segments. In: Proceedings of the IEEE International Conference on Computer Vision, pp 2192–2199 Liu Z, Zhang X, Luo S, Le Meur O (2014) Superpixel-based spatiotemporal saliency detections. IEEE Trans Circuits Syst Video Technol 24(9):1522–1540 Lin X, Casas JR, Pardas M (2017) 3D point cloud segmentation using a fully connected conditional random field. In: 25th European Signal Processing Conference (EUSIPCO), pp 66–70 Lee YJ, Kim J, Grauman K (2011) Key-segments for video object segmentation. In: International Conference on Computer Vision, pp 1995–2002 Lezama Jo, Alahari K, Sivic J, Laptev I (2011) Track to the future: Spatio-temporal video segmentation with long-range motion cues. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 3369–3376 Li Y, Hou X, Koch C, Rehg JM, Yuille Al (2014) The secrets of salient object segmentation. IEEE Conference on Computer Vision and Pattern Recognition, pp 280–287 Lucas BD, Kanade T, et al. (1981) An iterative image registration technique with an application to stereo vision. In: International Joint Conference on Artificial Intelligence, pp 674–679 Mancas M, Riche N, Leroy J, Gosselin B (2011) Abnormal motion selection in crowds using bottom-up saliency. In: IEEE International Conference on Image Processing, pp 229–232 Ma T, Latecki LJ (2012) Maximum weight cliques with mutex constraints for video object segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 670–677 Marat S, Phuoc TH, Granjon L, Guyader N, Pellerin D, Guérin-Dugué A (2009) Modelling spatio-temporal saliency to predict gaze direction for short videos. In: International journal of computer vision, vol 82,3, Springer, p 213 Mauthner T, Possegger HD, Waltner G, Bischof H (2015) Encoding based saliency detection for videos and images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2494–2502 Nah S, Lee KM (2015) Random forest with data ensemble for saliency detection. In: Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, pp 604–607 Oneata D, Revaud J, Verbeek J, Schmid C (2014) Spatio-temporal object detection proposals. In: European Conference on Computer Vision, pp 737–752 Papazoglou A, Ferrari V (2013) Fast object segmentation in unconstrained video. In: Proceedings of the IEEE International Conference on Computer Vision, pp 1777–1784 Rahman A, Houzet D, Pellerin D, Marat S, Guyader N (2011) Parallel implementation of a spatio-temporal visual saliency model. J Real-Time Image Proc 6 (1):3–14 Rahtu E, Kannala J, Salo M, Heikkilä JE (2010) Segmenting salient objects from images and videos. In: European Conference on Computer Vision, pp 366–379 Simoncelli EP, Adelson EH, Heeger DJ (1991) Probability distributions of optical flow. In: Computer Society Conference on Computer Vision and Pattern Recognition, pp 310–315 Singh A, Chu C-HH, Pratt M (2015) Learning to predict video saliency using temporal superpixels. In: International Conference on Pattern Recognition Applications and Methods, pp 201–209 Uijlings JRR, van de Sande KEA, Gevers T, Smeulders AWM (2013) Selective search for object recognition. Int J Comput Vis 104(2):154–171 Van De Weijer J, Gevers T, Smeulders AWM (2006) Robust Photometric invariant features from the color tensor. IEEE Trans Image Process 15(1):118–127 Wang J, Tavakoli H, Laaksonen J (2017) Fixation prediction in videos using unsupervised hierarchical features. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp 50–57 Wang X, Ma H, Chen X (2016) Geodesic weighted Bayesian model for saliency optimization. Pattern Recogn Lett 75:1–8 Wang W, Shen J, Porikli F (2015) Saliency-aware geodesic video object segmentation. In: Proc IEEE Conf Comput Vis Pattern Recognit, pp 3395–3402 Wang L, Lu H, Ruan Xg, Yang M (2015) Deep networks for saliency detection via local estimation and global search. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 3183–3192 Wang P, Wang J, Zeng G, Feng J, Zha H, Li S (2012) Salient object detection for searched web images via global saliency. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 3194–3201 Yang C, Zhang L, Lu H, Ruan X, Yang M-H (2013) Saliency detection via graph-based manifold ranking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3166–3173 Zhang D, Javed O, Shah M (2013) Video object segmentation through spatially accurate and temporally dense extraction of primary object regions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 628–635 Zhong S-h, Liu Y, Ren F, Zhang J, Ren T (2013) Video Saliency detection via dynamic consistent Spatio-Temporal attention modelling. In: National Conference of the American Association for Artificial Intelligence, pp 1063–1069