Nội dung được dịch bởi AI, chỉ mang tính chất tham khảo
Phương pháp phân đoạn ngữ nghĩa được giám sát yếu dựa trên chuyển đổi siêu điểm cục bộ
Tóm tắt
Phân đoạn ngữ nghĩa được giám sát yếu (WSSS) có thể thu được các mặt nạ ngữ nghĩa giả thông qua việc sử dụng nhãn giám sát ở mức yếu hơn, giảm thiểu nhu cầu về các chú thích ở mức pixel đắt đỏ. Tuy nhiên, phương pháp thu thập mặt nạ giả dựa trên bản đồ kích hoạt lớp chung (CAM) gặp phải vấn đề phủ sóng thưa thớt, dẫn đến các vùng dương tính giả và âm tính giả làm giảm độ chính xác. Chúng tôi đề xuất một phương pháp WSSS dựa trên chuyển đổi siêu điểm cục bộ kết hợp lý thuyết siêu điểm và thông tin cục bộ của hình ảnh. Phương pháp của chúng tôi sử dụng hàm mất mát chéo phân phối trọng số theo nhất quán cục bộ của siêu điểm để sửa chữa các vùng sai và một phương pháp xử lý hậu kỳ dựa trên ma trận liên kết siêu điểm kề nhau (ASAM) để mở rộng các âm tính giả, triệt tiêu các dương tính giả và tối ưu hóa các ranh giới ngữ nghĩa. Phương pháp của chúng tôi đạt được 73,5% mIoU trên tập xác thực PASCAL VOC 2012, cao hơn 2,5% so với chuẩn EPS của chúng tôi và đạt 73,9% trên tập kiểm tra, đồng thời phương pháp xử lý hậu kỳ ASAM được xác thực trên nhiều phương pháp hiện đại nhất.
Từ khóa
#Phân đoạn ngữ nghĩa #giám sát yếu #siêu điểm #bản đồ kích hoạt lớp #xử lý hậu kỳ #ma trận liên kết.Tài liệu tham khảo
Chen L-C, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2016) DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. Cornell University - arXiv
Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440
Zhang J, Yang J, Yu J, Fan J (2022) Semisupervised image classification by mutual learning of multiple self-supervised models. Int J Intell Syst 37(5):3117–3141
Dai J, He K, Sun J (2015) Boxsup: exploiting bounding boxes to supervise convolutional networks for semantic segmentation. In: Proceedings of the IEEE international conference on computer vision, pp 1635–1643
Lin D, Dai J, Jia J, He K, Sun J (2016) Scribblesup: scribble-supervised convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3159–3167
Papandreou G, Chen L-C, Murphy KP, Yuille AL (2015) Weakly-and semi-supervised learning of a deep convolutional network for semantic image segmentation. In: Proceedings of the IEEE international conference on computer vision, pp 1742–1750
Bearman A, Russakovsky O, Ferrari V, Fei-Fei L (2016) What’s the point: semantic segmentation with point supervision. In: Computer vision—ECCV 2016: 14th European conference, Amsterdam, The Netherlands, October 11–14, 2016, proceedings, part VII 14. Springer, pp 549–565
Hou Q, Jiang P, Wei Y, Cheng M-M (2018) Self-erasing network for integral object attention. In: Advances in neural information processing systems, p 31
Wei Y, Feng J, Liang X, Cheng M-M, Zhao Y, Yan S (2017) Object region mining with adversarial erasing: a simple classification to semantic segmentation approach. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1568–1576
Krähenbühl P, Koltun V (2011) Efficient inference in fully connected CRFS with Gaussian edge potentials. In: Advances in neural information processing systems, vol 24
Zhang X, Peng Z, Zhu P, Zhang T, Li C, Zhou H, Jiao L (2021) Adaptive weakly supervised semantic segmentation with a unified framework. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 12293–12302
Ahn J, Cho S, Kwak S (2019) Weakly supervised learning of instance segmentation with inter-pixel relations. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 2209–2218
Wu T, Huang J, Gao G, Wei X, Wei X, Luo X, Liu CH (2021) Embedded discriminative attention mechanism for weakly supervised semantic segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 16765–16774
Everingham M, Eslami SMA, Van Gool L, Williams CKI, Winn J, Zisserman A (2015) The Pascal visual object classes challenge: a retrospective. Int J Comput Vis 111:98–136
Tu W-C, Liu M-Y, Jampani V, Sun D, Chien S-Y, Yang M-H, Kautz J (2018) Learning superpixels with segmentation-aware affinity loss. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 568–576
Yang F, Sun Q, Jin H, Zhou Z (2020) Superpixel segmentation with fully convolutional networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 13964–13973
Jampani V, Sun D, Liu M-Y, Yang M-H, Kautz J (2018) Superpixel sampling networks. In: Proceedings of the European conference on computer vision (ECCV), pp 352–368
Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S (2012) SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34(11):2274–2282
Suzuki T (2020) Superpixel segmentation via convolutional neural networks with regularized information maximization. In: ICASSP 2020-2020 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 2573–2577
Yu J, Tao D, Wang M (2012) Adaptive hypergraph learning and its application in image classification. IEEE Trans Image Process 21(7):3262–3272
Zhu L, She Q, Zhang B, Lu Y, Lu Z, Li D, Hu J (2021) Learning the superpixel in a non-iterative and lifelong manner. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 1225–1234
Zhang J, Cao Y, Wu Q (2021) Vector of locally and adaptively aggregated descriptors for image feature representation. Pattern Recognit 116:107952
Yu J, Tao D, Wang M, Rui Y (2014) Learning to rank using user clicks and visual features for image retrieval. IEEE Trans Cybern 45(4):767–779
Pinheiro PO, Collobert R (2015) From image-level to pixel-level labeling with convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1713–1721
Araslanov N, Roth S (2020) Single-stage semantic segmentation from image labels. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 4253–4262
Zhang B, Xiao J, Wei Y, Sun M, Huang K (2020) Reliability does matter: an end-to-end weakly supervised semantic segmentation approach. Proc AAAI Conf Artif Intell 34(07):12765–12772
Tang M, Perazzi F, Djelouah A, Ben AI, Schroers C, Boykov Y (2018) On regularized losses for weakly-supervised CNN segmentation. In: Proceedings of the European conference on computer vision (ECCV), pp. 507–522
Ke T-W, Hwang J-J, Liu Z, Yu SX (2018) Adaptive affinity fields for semantic segmentation. In: Proceedings of the European conference on computer vision (ECCV), pp 587–602
Gao W, Wan F, Pan X, Peng Z, Tian Q, Han Z, Zhou B, Ye Q (2021) Ts-cam: token semantic coupled attention map for weakly supervised object localization. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 2886–2895
Ru L, Zhan Y, Yu B, Du B (2022) Learning affinity from attention: end-to-end weakly-supervised semantic segmentation with transformers. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 16846-16855
Li K, Wu Z, Peng K-C, Ernst J, Fu Y (2019) Guided attention inference network. IEEE Trans Pattern Anal Mach Intell 42(12):2996–3010
Zhang F, Gu C, Zhang C, Dai Y (2021) Complementary patch for weakly supervised semantic segmentation. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 7242–7251
Sun K, Shi H, Zhang Z, Huang Y (2021) Ecs-net: improving weakly supervised semantic segmentation by using connections between class activation maps. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 7283–7292
Kolesnikov A, Lampert CH (2016) Seed, expand and constrain: three principles for weakly-supervised image segmentation. In: Computer vision-ECCV, (2016) 14th European conference, Amsterdam, The Netherlands, October 11–14, 2016. Proceedings, part IV, vol 14, pp 695–711
Jiang P-T, Hou Q, Cao Y, Cheng M-M, Wei Y, Xiong H-K (2019) Integral object mining via online attention accumulation. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 2070–2079
Chang Y-T (2020) Weakly-supervised semantic segmentation via self-regularization. University of California, Merced
Ahn J, Kwak S (2018) Learning pixel-level semantic affinity with image-level supervision for weakly supervised semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4981–4990
Chen L, Wu W, Fu C, Han X, Zhang Y (2020) Weakly supervised semantic segmentation with boundary exploration. In: Computer vision-ECCV, (2020) 16th European conference, Glasgow, UK, August 23–28, 2020. Proceedings, part XXVI, vol 16, pp 347–362
Sun G, Wang W, Dai J, Van Gool L (2020) Mining cross-image semantics for weakly supervised semantic segmentation. In: Computer vision—ECCV 2020: 16th European conference, Glasgow, UK, August 23–28, 2020, proceedings, part II, vol 16, pp 347–365
Lee S, Lee M, Lee J, Shim H (2021) Railroad is not a train: saliency as pseudo-pixel supervision for weakly supervised semantic segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5495–5505
Li X, Zhou T, Li J, Zhou Y, Zhang Z (2021) Group-wise semantic mining for weakly supervised semantic segmentation. Proc AAAI Conf Artif Intell 35(3):1984–1992
Zhang D, Zhang H, Tang J, Hua X-S, Sun Q (2020) Causal intervention for weakly-supervised semantic segmentation. In: Advances in neural information processing systems, vol 33, pp 655–666
Lee J, Choi J, Mok J, Yoon S (2021) Reducing information bottleneck for weakly supervised semantic segmentation. In: Advances in neural information processing systems, vol 34, pp 27408–27421
Lee J, Kim E, Yoon S (2021) Anti-adversarially manipulated attributions for weakly and semi-supervised semantic segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 4071–4080
Wang Y, Zhang J, Kan M, Shan S, Chen X (2020) Self-supervised equivariant attention mechanism for weakly supervised semantic segmentation. In: Computer vision and pattern recognition
Wei Y, Xiao H, Shi H, Jie Z, Feng J, Huang TS (2018) Revisiting dilated convolution: a simple approach for weakly-and semi-supervised semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7268–7277
Jiang P-T, Yang Y, Hou Q, Wei Y (2022) L2G: a simple local-to-global knowledge transfer framework for weakly supervised semantic segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition
Wu Z, Shen C, van den Hengel A (2019) Wider or deeper: revisiting the ResNet model for visual recognition. In: Pattern recognition
Liu J-J, Hou Q, Cheng M-M, Feng J, Jiang J (2019) A simple pooling-based design for real-time salient object detection. In: IEEE CVPR
Chen L-C, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2014) Semantic image segmentation with deep convolutional nets and fully connected CRFS. arXiv preprint arXiv:1412.7062
Chen L-C, Zhu Y, Papandreou G, Schroff F, Adam H (2018) Encoder–decoder with atrous separable convolution for semantic image segmentation. Cornell University - arXiv
Du Y, Fu Z, Liu Q, Wang Y (2022) Weakly supervised semantic segmentation by pixel-to-prototype contrast. In: Proceedings of the IEEE conference on computer vision and pattern recognition
Chen Z, Wang T, Wu X, Hua X-S, Zhang H, Sun Q (2022) Class re-activation maps for weakly-supervised semantic segmentation. In: The IEEE conference on computer vision and pattern recognition (CVPR)
Jo S, Yu I-J (2021) Puzzle-CAM: improved localization via matching partial and full features. In: 2021 IEEE international conference on image processing (ICIP). IEEE, pp 639–643
Fan J, Zhang Z, Song C, Tan T (2020) Learning integral objects with intra-class discriminator for weakly-supervised semantic segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 4283–4292
Kim B, Han S, Kim J (2021) Discriminative region suppression for weakly-supervised semantic segmentation. Proc AAAI Conf Artif Intell 35(2):1754–1761
Wang X, You S, Li X, Ma H (2018) Weakly-supervised semantic segmentation by iteratively mining common object features. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1354–1362
Huang Z, Wang X, Wang J, Liu W, Wang J (2018) Weakly-supervised semantic segmentation network with deep seeded region growing. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7014–7023
Chang Y-T, Wang Q, Hung W-C, Piramuthu R, Tsai Y-H, Yang M-H (2020) Weakly-supervised semantic segmentation via sub-category exploration. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 8991–9000
Fan J, Zhang Z, Tan T (2020) Employing multi-estimations for weakly-supervised semantic segmentation. In: Computer vision—ECCV 2020: 16th European conference, Glasgow, UK, August 23–28, 2020, proceedings, part XVII 16. Springer, pp 332–348
Liu Y, Wu Y-H, Wen P, Shi Y, Qiu Y, Cheng M-M (2020) Leveraging instance-, image-and dataset-level information for weakly supervised instance segmentation. IEEE Trans Pattern Anal Mach Intell 44(3):1415–1428
Lee J, Kim E, Lee S, Lee J, Yoon S (2019) Ficklenet: weakly and semi-supervised semantic image segmentation using stochastic inference. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5267–5276
Yao Y, Chen T, Xie G-S, Zhang C, Shen F, Wu Q, Tang Z, Zhang J (2021) Non-salient region object mining for weakly supervised semantic segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 2623–2632
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
Shimoda W, Yanai K (2019) Self-supervised difference detection for weakly-supervised semantic segmentation. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 5208–5217
Jiang P-T, Han L-H, Hou Q, Cheng M-M, Wei Y (2021) Online attention accumulation for weakly supervised semantic segmentation. IEEE Trans Pattern Anal Mach Intell 44(10):7062–7077
