Phương pháp phân đoạn ngữ nghĩa được giám sát yếu dựa trên chuyển đổi siêu điểm cục bộ

Springer Science and Business Media LLC - Tập 55 - Trang 12039-12060 - 2023
Zhiming Ma1, Dali Chen1, Yilin Mo1, Yue Chen2, Yumin Zhang1
1College of Information Science and Engineering, Northeastern University, Shenyang, China
2College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, China

Tóm tắt

Phân đoạn ngữ nghĩa được giám sát yếu (WSSS) có thể thu được các mặt nạ ngữ nghĩa giả thông qua việc sử dụng nhãn giám sát ở mức yếu hơn, giảm thiểu nhu cầu về các chú thích ở mức pixel đắt đỏ. Tuy nhiên, phương pháp thu thập mặt nạ giả dựa trên bản đồ kích hoạt lớp chung (CAM) gặp phải vấn đề phủ sóng thưa thớt, dẫn đến các vùng dương tính giả và âm tính giả làm giảm độ chính xác. Chúng tôi đề xuất một phương pháp WSSS dựa trên chuyển đổi siêu điểm cục bộ kết hợp lý thuyết siêu điểm và thông tin cục bộ của hình ảnh. Phương pháp của chúng tôi sử dụng hàm mất mát chéo phân phối trọng số theo nhất quán cục bộ của siêu điểm để sửa chữa các vùng sai và một phương pháp xử lý hậu kỳ dựa trên ma trận liên kết siêu điểm kề nhau (ASAM) để mở rộng các âm tính giả, triệt tiêu các dương tính giả và tối ưu hóa các ranh giới ngữ nghĩa. Phương pháp của chúng tôi đạt được 73,5% mIoU trên tập xác thực PASCAL VOC 2012, cao hơn 2,5% so với chuẩn EPS của chúng tôi và đạt 73,9% trên tập kiểm tra, đồng thời phương pháp xử lý hậu kỳ ASAM được xác thực trên nhiều phương pháp hiện đại nhất.

Từ khóa

#Phân đoạn ngữ nghĩa #giám sát yếu #siêu điểm #bản đồ kích hoạt lớp #xử lý hậu kỳ #ma trận liên kết.

Tài liệu tham khảo

Chen L-C, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2016) DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. Cornell University - arXiv Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440 Zhang J, Yang J, Yu J, Fan J (2022) Semisupervised image classification by mutual learning of multiple self-supervised models. Int J Intell Syst 37(5):3117–3141 Dai J, He K, Sun J (2015) Boxsup: exploiting bounding boxes to supervise convolutional networks for semantic segmentation. In: Proceedings of the IEEE international conference on computer vision, pp 1635–1643 Lin D, Dai J, Jia J, He K, Sun J (2016) Scribblesup: scribble-supervised convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3159–3167 Papandreou G, Chen L-C, Murphy KP, Yuille AL (2015) Weakly-and semi-supervised learning of a deep convolutional network for semantic image segmentation. In: Proceedings of the IEEE international conference on computer vision, pp 1742–1750 Bearman A, Russakovsky O, Ferrari V, Fei-Fei L (2016) What’s the point: semantic segmentation with point supervision. In: Computer vision—ECCV 2016: 14th European conference, Amsterdam, The Netherlands, October 11–14, 2016, proceedings, part VII 14. Springer, pp 549–565 Hou Q, Jiang P, Wei Y, Cheng M-M (2018) Self-erasing network for integral object attention. In: Advances in neural information processing systems, p 31 Wei Y, Feng J, Liang X, Cheng M-M, Zhao Y, Yan S (2017) Object region mining with adversarial erasing: a simple classification to semantic segmentation approach. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1568–1576 Krähenbühl P, Koltun V (2011) Efficient inference in fully connected CRFS with Gaussian edge potentials. In: Advances in neural information processing systems, vol 24 Zhang X, Peng Z, Zhu P, Zhang T, Li C, Zhou H, Jiao L (2021) Adaptive weakly supervised semantic segmentation with a unified framework. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 12293–12302 Ahn J, Cho S, Kwak S (2019) Weakly supervised learning of instance segmentation with inter-pixel relations. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 2209–2218 Wu T, Huang J, Gao G, Wei X, Wei X, Luo X, Liu CH (2021) Embedded discriminative attention mechanism for weakly supervised semantic segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 16765–16774 Everingham M, Eslami SMA, Van Gool L, Williams CKI, Winn J, Zisserman A (2015) The Pascal visual object classes challenge: a retrospective. Int J Comput Vis 111:98–136 Tu W-C, Liu M-Y, Jampani V, Sun D, Chien S-Y, Yang M-H, Kautz J (2018) Learning superpixels with segmentation-aware affinity loss. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 568–576 Yang F, Sun Q, Jin H, Zhou Z (2020) Superpixel segmentation with fully convolutional networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 13964–13973 Jampani V, Sun D, Liu M-Y, Yang M-H, Kautz J (2018) Superpixel sampling networks. In: Proceedings of the European conference on computer vision (ECCV), pp 352–368 Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S (2012) SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34(11):2274–2282 Suzuki T (2020) Superpixel segmentation via convolutional neural networks with regularized information maximization. In: ICASSP 2020-2020 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 2573–2577 Yu J, Tao D, Wang M (2012) Adaptive hypergraph learning and its application in image classification. IEEE Trans Image Process 21(7):3262–3272 Zhu L, She Q, Zhang B, Lu Y, Lu Z, Li D, Hu J (2021) Learning the superpixel in a non-iterative and lifelong manner. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 1225–1234 Zhang J, Cao Y, Wu Q (2021) Vector of locally and adaptively aggregated descriptors for image feature representation. Pattern Recognit 116:107952 Yu J, Tao D, Wang M, Rui Y (2014) Learning to rank using user clicks and visual features for image retrieval. IEEE Trans Cybern 45(4):767–779 Pinheiro PO, Collobert R (2015) From image-level to pixel-level labeling with convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1713–1721 Araslanov N, Roth S (2020) Single-stage semantic segmentation from image labels. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 4253–4262 Zhang B, Xiao J, Wei Y, Sun M, Huang K (2020) Reliability does matter: an end-to-end weakly supervised semantic segmentation approach. Proc AAAI Conf Artif Intell 34(07):12765–12772 Tang M, Perazzi F, Djelouah A, Ben AI, Schroers C, Boykov Y (2018) On regularized losses for weakly-supervised CNN segmentation. In: Proceedings of the European conference on computer vision (ECCV), pp. 507–522 Ke T-W, Hwang J-J, Liu Z, Yu SX (2018) Adaptive affinity fields for semantic segmentation. In: Proceedings of the European conference on computer vision (ECCV), pp 587–602 Gao W, Wan F, Pan X, Peng Z, Tian Q, Han Z, Zhou B, Ye Q (2021) Ts-cam: token semantic coupled attention map for weakly supervised object localization. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 2886–2895 Ru L, Zhan Y, Yu B, Du B (2022) Learning affinity from attention: end-to-end weakly-supervised semantic segmentation with transformers. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 16846-16855 Li K, Wu Z, Peng K-C, Ernst J, Fu Y (2019) Guided attention inference network. IEEE Trans Pattern Anal Mach Intell 42(12):2996–3010 Zhang F, Gu C, Zhang C, Dai Y (2021) Complementary patch for weakly supervised semantic segmentation. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 7242–7251 Sun K, Shi H, Zhang Z, Huang Y (2021) Ecs-net: improving weakly supervised semantic segmentation by using connections between class activation maps. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 7283–7292 Kolesnikov A, Lampert CH (2016) Seed, expand and constrain: three principles for weakly-supervised image segmentation. In: Computer vision-ECCV, (2016) 14th European conference, Amsterdam, The Netherlands, October 11–14, 2016. Proceedings, part IV, vol 14, pp 695–711 Jiang P-T, Hou Q, Cao Y, Cheng M-M, Wei Y, Xiong H-K (2019) Integral object mining via online attention accumulation. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 2070–2079 Chang Y-T (2020) Weakly-supervised semantic segmentation via self-regularization. University of California, Merced Ahn J, Kwak S (2018) Learning pixel-level semantic affinity with image-level supervision for weakly supervised semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4981–4990 Chen L, Wu W, Fu C, Han X, Zhang Y (2020) Weakly supervised semantic segmentation with boundary exploration. In: Computer vision-ECCV, (2020) 16th European conference, Glasgow, UK, August 23–28, 2020. Proceedings, part XXVI, vol 16, pp 347–362 Sun G, Wang W, Dai J, Van Gool L (2020) Mining cross-image semantics for weakly supervised semantic segmentation. In: Computer vision—ECCV 2020: 16th European conference, Glasgow, UK, August 23–28, 2020, proceedings, part II, vol 16, pp 347–365 Lee S, Lee M, Lee J, Shim H (2021) Railroad is not a train: saliency as pseudo-pixel supervision for weakly supervised semantic segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5495–5505 Li X, Zhou T, Li J, Zhou Y, Zhang Z (2021) Group-wise semantic mining for weakly supervised semantic segmentation. Proc AAAI Conf Artif Intell 35(3):1984–1992 Zhang D, Zhang H, Tang J, Hua X-S, Sun Q (2020) Causal intervention for weakly-supervised semantic segmentation. In: Advances in neural information processing systems, vol 33, pp 655–666 Lee J, Choi J, Mok J, Yoon S (2021) Reducing information bottleneck for weakly supervised semantic segmentation. In: Advances in neural information processing systems, vol 34, pp 27408–27421 Lee J, Kim E, Yoon S (2021) Anti-adversarially manipulated attributions for weakly and semi-supervised semantic segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 4071–4080 Wang Y, Zhang J, Kan M, Shan S, Chen X (2020) Self-supervised equivariant attention mechanism for weakly supervised semantic segmentation. In: Computer vision and pattern recognition Wei Y, Xiao H, Shi H, Jie Z, Feng J, Huang TS (2018) Revisiting dilated convolution: a simple approach for weakly-and semi-supervised semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7268–7277 Jiang P-T, Yang Y, Hou Q, Wei Y (2022) L2G: a simple local-to-global knowledge transfer framework for weakly supervised semantic segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition Wu Z, Shen C, van den Hengel A (2019) Wider or deeper: revisiting the ResNet model for visual recognition. In: Pattern recognition Liu J-J, Hou Q, Cheng M-M, Feng J, Jiang J (2019) A simple pooling-based design for real-time salient object detection. In: IEEE CVPR Chen L-C, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2014) Semantic image segmentation with deep convolutional nets and fully connected CRFS. arXiv preprint arXiv:1412.7062 Chen L-C, Zhu Y, Papandreou G, Schroff F, Adam H (2018) Encoder–decoder with atrous separable convolution for semantic image segmentation. Cornell University - arXiv Du Y, Fu Z, Liu Q, Wang Y (2022) Weakly supervised semantic segmentation by pixel-to-prototype contrast. In: Proceedings of the IEEE conference on computer vision and pattern recognition Chen Z, Wang T, Wu X, Hua X-S, Zhang H, Sun Q (2022) Class re-activation maps for weakly-supervised semantic segmentation. In: The IEEE conference on computer vision and pattern recognition (CVPR) Jo S, Yu I-J (2021) Puzzle-CAM: improved localization via matching partial and full features. In: 2021 IEEE international conference on image processing (ICIP). IEEE, pp 639–643 Fan J, Zhang Z, Song C, Tan T (2020) Learning integral objects with intra-class discriminator for weakly-supervised semantic segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 4283–4292 Kim B, Han S, Kim J (2021) Discriminative region suppression for weakly-supervised semantic segmentation. Proc AAAI Conf Artif Intell 35(2):1754–1761 Wang X, You S, Li X, Ma H (2018) Weakly-supervised semantic segmentation by iteratively mining common object features. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1354–1362 Huang Z, Wang X, Wang J, Liu W, Wang J (2018) Weakly-supervised semantic segmentation network with deep seeded region growing. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7014–7023 Chang Y-T, Wang Q, Hung W-C, Piramuthu R, Tsai Y-H, Yang M-H (2020) Weakly-supervised semantic segmentation via sub-category exploration. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 8991–9000 Fan J, Zhang Z, Tan T (2020) Employing multi-estimations for weakly-supervised semantic segmentation. In: Computer vision—ECCV 2020: 16th European conference, Glasgow, UK, August 23–28, 2020, proceedings, part XVII 16. Springer, pp 332–348 Liu Y, Wu Y-H, Wen P, Shi Y, Qiu Y, Cheng M-M (2020) Leveraging instance-, image-and dataset-level information for weakly supervised instance segmentation. IEEE Trans Pattern Anal Mach Intell 44(3):1415–1428 Lee J, Kim E, Lee S, Lee J, Yoon S (2019) Ficklenet: weakly and semi-supervised semantic image segmentation using stochastic inference. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5267–5276 Yao Y, Chen T, Xie G-S, Zhang C, Shen F, Wu Q, Tang Z, Zhang J (2021) Non-salient region object mining for weakly supervised semantic segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 2623–2632 He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778 Shimoda W, Yanai K (2019) Self-supervised difference detection for weakly-supervised semantic segmentation. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 5208–5217 Jiang P-T, Han L-H, Hou Q, Cheng M-M, Wei Y (2021) Online attention accumulation for weakly supervised semantic segmentation. IEEE Trans Pattern Anal Mach Intell 44(10):7062–7077