Efficient point cloud segmentation approach using energy optimization with geometric features for 3D scene understanding

Journal of the Optical Society of America A: Optics and Image Science, and Vision - Tập 38 Số 1 - Trang 60 - 2021
Xurui Li1, Guangshuai Liu1, Si Sun1
1Southwest Jiaotong University, School of Mechanical Engineering, Chengdu, Sichuan 610031, China

Tóm tắt

Efficient and quick extraction of unknown objects in cluttered 3D scenes plays a significant role in robotics tasks such as object search, grasping, and manipulation. This paper describes a geometric-based unsupervised approach for the segmentation of cluttered scenes into objects. The proposed method first over-segments the raw point clouds into supervoxels to provide a more natural representation of 3D point clouds and reduce the computational cost with a minimal loss of geometric information. Then the fully connected local area linkage graph is used to distinguish between planar and nonplanar adjacent patches. Then the initial segmentation is completed utilizing the geometric features and local surface convexities. After the initial segmentation, many subgraphs are generated, each of which represents an individual object or part of it. Finally, we use the plane extracted from the scene to refine the initial segmentation result under the framework of global energy optimization. Experiments on the Object Cluttered Indoor Dataset dataset indicate that the proposed method can outperform the representative segmentation algorithms in terms of weighted overlap and accuracy, while our method has good robustness and real-time performance.

Từ khóa


Tài liệu tham khảo

Danielczuk, 2019, Segmenting unknown 3D objects from real depth images using mask R-CNN trained on synthetic data, International Conference on Robotics and Automation (ICRA), 7283, 10.1109/ICRA.2019.8793744

Nguyen, 2013, 3D point cloud segmentation: a survey, 6th IEEE Conference on Robotics, Automation and Mechatronics (RAM), 225

Stein, 2014, Convexity based object partitioning for robot applications, IEEE International Conference on Robotics and Automation (ICRA), 3213, 10.1109/ICRA.2014.6907321

Ioannou, 2012, Difference of normals as a multi-scale operator in unorganized point clouds, 2nd International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission, 501

Rabbani, 2006, Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., 36, 248

Wang, 2011, Photogramm. Rec., 26, 32, 10.1111/j.1477-9730.2011.00624.x

Xie, 2020, The best of both modes: separately leveraging RGB and depth for unseen object instance segmentation, Conference on robot learning (PMLR), 1369

Long, 2015, Fully convolutional networks for semantic segmentation, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 3431

Lai, 2011, A large-scale hierarchical multi-view RGB-D object dataset, International Conference on Robotics and Automation (ICRA), 1817

Suchi, 2019, EasyLabel: a semi-automatic pixel-wise object annotation tool for creating robotic RGB-D datasets, International Conference on Robotics and Automation (ICRA), 6678, 10.1109/ICRA.2019.8793917

Aldoma, 2015, Segmentation of depth data in piece-wise smooth parametric surfaces, Computer Vision Winter Workshop (CVWW)

Boyer, 1999, Guest editors’ introduction: perceptual organization in computer vision: status, challenges, and potential, Computer Vision and Image Understanding, 76, 1

Felzenszwalb, 2004, Int. J. Comput. Vis., 59, 167, 10.1023/B:VISI.0000022288.19776.77

Rother, 2004, ACM Trans. Graph., 23, 309, 10.1145/1015706.1015720

Vicente, 2009, Joint optimization of segmentation and appearance models, 12th International Conference on Computer Vision (ICCV), 755

Werlberger, 2009, A variational model for interactive shape prior segmentation and real-time tracking, International Conference on Scale Space and Variational Methods in Computer Vision, 200, 10.1007/978-3-642-02256-2_17

Strekalovskiy, 2014, Real-time minimization of the piecewise smooth Mumford-Shah functional, European Conference on Computer Vision (ECCV), 127

Kootstra, 2010, Fast and automatic detection and segmentation of unknown objects, 10th IEEE-RAS International Conference on Humanoid Robots, 442

Ückermann, 2012, Real-time 3D segmentation of cluttered scenes for robot grasping, 12th IEEE-RAS International Conference on Humanoid Robots, 198

Richtsfeld, 2012, Segmentation of unknown objects in indoor environments, IEEE/RSJ International Conference on Intelligent Robots and Systems, 4791

Mörwald, 2013, Geometric data abstraction using B-splines for range image segmentation, International Conference on Robotics and Automation (ICRA), 148

Xie, 2020, Linking points with labels in 3D: a review of point cloud semantic segmentation, IEEE Geoscience and Remote Sensing Magazine, 10.1109/MGRS.2019.2937630

Golovinskiy, 2009, Min-cut based segmentation of point clouds, 12th International Conference on Computer Vision (ICCV), 39

Boykov, 2006, Int. J. Comput. Vis., 70, 109, 10.1007/s11263-006-7934-5

Ural, 2012, Int. Arch. Photogramm. Remote Sens. Spatial. Inf. Sci., XXXIXB3, 167, 10.5194/isprsarchives-XXXIX-B3-167-2012

Dutta, 2014, Int. Arch. Photogramm. Remote Sens. Spatial. Inf. Sci., XL3, 81, 10.5194/isprsarchives-XL-3-81-2014

Johnson-Roberson, 2010, Attention-based active 3D point cloud segmentation, IEEE/RSJ International Conference on Intelligent Robots and Systems, 1165

Rusu, 2009, Model-based and learned semantic object labeling in 3D point cloud maps of kitchen environments, IEEE/RSJ International Conference on Intelligent Robots and Systems, 3601

Papon, 2013, Voxel cloud connectivity segmentation-supervoxels for point clouds, IEEE Conference on Computer Vision and Pattern Recognition, 2027

Ao, 2019, Int. Arch. Photogramm. Remote. Sens. Spatial Inf. Sci., XLII2/W13, 919, 10.5194/isprs-archives-XLII-2-W13-919-2019

Verdoja, 2017, Fast 3D point cloud segmentation using supervoxels with geometry and color for 3D scene understanding, International Conference on Multimedia and Expo (ICME), 1285

Ben-Shabat, 2018, Comput. Vis. Image Underst., 174, 12, 10.1016/j.cviu.2018.06.004

Christoph Stein, 2014, Object partitioning using local convexity, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 304

Xu, 2017, ISPRS Ann. Photogram. Remote Sens. Spatial Inf. Sci., IV1/W1, 43, 10.5194/isprs-annals-IV-1-W1-43-2017

Saglam, 2020, Expert. Syst. Appl., 157, 113439, 10.1016/j.eswa.2020.113439

Czerniawski, 2018, Automat. Constr., 88, 44, 10.1016/j.autcon.2017.12.029

Vosselman, 2017, ISPRS. J. Photogramm., 128, 354, 10.1016/j.isprsjprs.2017.03.010

Kim, 2016, Sensors, 16, 140, 10.3390/s16020140

Pham, 2015, Hierarchical higher-order regression forest fields: an application to 3D indoor scene labelling, IEEE International Conference on Computer Vision (ICCV), 2246, 10.1109/ICCV.2015.259

Landrieu, 2018, Large-scale point cloud semantic segmentation with superpoint graphs, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 4558

Kumra, 2017, Robotic grasp detection using deep convolutional neural networks, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 769

Asif, 2017, IEEE Trans. Robot., 33, 547, 10.1109/TRO.2016.2638453

Levine, 2018, Int. J. Robot. Res., 37, 421, 10.1177/0278364917710318

Huang, 2017, Densely connected convolutional networks, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 4700

Milan, 2018, Semantic segmentation from limited training data, International Conference on Robotics and Automation (ICRA), 1908

Neven, 2019, Instance segmentation by jointly optimizing spatial embeddings and clustering bandwidth, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 8837

Novotny, 2018, Semi-convolutional operators for instance segmentation, European Conference on Computer Vision (ECCV), 86

Pinheiro, 2015, Learning to segment object candidates, Advances in Neural Information Processing Systems, 1990

Pinheiro, 2016, Learning to refine object segments, European Conference on Computer Vision (ECCV), 75

Yang, 2015, ISPRS. J. Photogramm., 99, 45, 10.1016/j.isprsjprs.2014.10.005

Neubeck, 2006, Efficient non-maximum suppression, 18th International Conference on Pattern Recognition (ICPR), 850

Isack, 2012, Int. J. Comput. Vis., 97, 123, 10.1007/s11263-011-0474-7

Delong, 2012, Int. J. Comput. Vis., 96, 1, 10.1007/s11263-011-0437-z

Boykov, 2001, IEEE. Trans. Pattern. Anal., 23, 1222, 10.1109/34.969114

Loghmani, 2018, Recognizing objects in-the-wild: where do we stand?, International Conference on Robotics and Automation (ICRA), 2170

Calli, 2017, Int. J. Robot. Res., 36, 261, 10.1177/0278364917700714

Schnabel, 2007, Comput. Graph. Forum, 26, 214, 10.1111/j.1467-8659.2007.01016.x

Verdoja, 2017, Fast 3D point cloud segmentation using supervoxels with geometry and color for 3D scene understanding, International Conference on Multimedia and Expo (ICME), 1285

Pham, 2018, SceneCut: joint geometric and object segmentation for indoor scenes, IEEE International Conference on Robotics and Automation (ICRA), 3213, 10.1109/ICRA.2018.8461108

Potapova, 2014, Incremental attention-driven object segmentation, IEEE-RAS International Conference on Humanoid Robots, 252, 10.1109/HUMANOIDS.2014.7041368

Vo, 2015, ISPRS. J. Photogramm., 104, 88, 10.1016/j.isprsjprs.2015.01.011

Awrangjeb, 2014, Remote Sens., 6, 3716, 10.3390/rs6053716

Silberman, 2012, Indoor segmentation and support inference from rgbd images, European Conference on Computer Vision (ECCV), 746

Choi, 2015, Robust reconstruction of indoor scenes, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 5556

Thông tin
Thông tin xuất bản

Journal of the Optical Society of America A: Optics and Image Science, and Vision

Tập 38 Số 1

60

Thông tin tác giả