Three-stage generative network for single-view point cloud completion

The Visual Computer - Tập 38 - Trang 4373-4382 - 2021
Bingling Xiao1,2, Feipeng Da1,2,3
1School of Automation, Southeast University, Nanjing, China
2Key Laboratory of Measurement and Control of Complex Systems of Engineering, Ministry of Education, Nanjing, China
3Shenzhen Research Institute, Southeast University, Shenzhen, China

Tóm tắt

3D shape completion from single-view scan is an important task for follow-up applications such as recognition and segmentation, but it is challenging due to the critical sparsity and structural incompleteness of single-view point clouds. In this paper, a three-stage generative network (TSGN) is proposed for single-view point cloud completion, which generates fine-grained dense point clouds step by step and effectively overcomes the ubiquitous problem—the imbalance between general and individual characteristics. In the first stage, an encoder–decoder network consumes a partial point cloud and generates a rough sparse point cloud inferring the complete geometric shape. Then, a bi-channel residual network is designed to refine the preliminary result with assistance of the original partial input. A local-based folding network is introduced in the last stage to extract local context information from the revised result and build a dense point cloud with finer-grained details. Experiments on ShapeNet dataset and KITTI dataset validate the effectiveness of TSGN. The results on ShapeNet demonstrate the competitive performance on both CD and EMD.

Tài liệu tham khảo

Marini, S., Patané, G., Spagnuolo, M., et al.: Spectral feature selection for shape characterization and classification. Vis Comput 27, 1005 (2011)

Luciano, L., Ben Hamza, A.: Deep similarity network fusion for 3D shape classification. Vis Comput 35, 1171–1180 (2019)

Sun, Y., Miao, Y., Chen, J., et al.: PGCNet: patch graph convolutional network for point cloud segmentation of indoor scenes. Vis Comput 36, 2407–2418 (2020)

Fan, Y., Wang, M., Geng, N., et al.: A self-adaptive segmentation method for a point cloud. Vis Comput 34, 659–673 (2018)

Dai, A., Qi, C. R., Nießner, M.: Shape completion using 3D-encoder-predictor CNNs and shape synthesis. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, pp. 6545–6554 (2017). doi: https://doi.org/10.1109/CVPR.2017.693

Han, X, Li, Z., Huang, H., Kalogerakis, E., Yu, Y.: High-resolution shape completion using deep neural networks for global structure and local geometry inference. In: 2017 IEEE International Conference on Computer Vision (ICCV), Venice, 2017, pp. 85–93, doi: https://doi.org/10.1109/ICCV.2017.19.

Litany, O., Bronstein, A., Bronstein, M., Makadia, A.: Deformable shape completion with graph convolutional autoencoders. In: 2018 IEEE/CVF conference on computer vision and pattern recognition, Salt Lake City, UT, pp. 1886–1895 (2018). doi: https://doi.org/10.1109/CVPR.2018.00202

Pan, J., Han, X., Chen, W., Tang, J., Jia, K.: Deep mesh reconstruction from single RGB images via topology modification networks. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), pp. 9963–9972 (2019). doi: https://doi.org/10.1109/ICCV.2019.01006

Yuan, W., Khot, T., Held, D., Mertz, C., Hebert, M.: PCN: point completion network. In: 2018 International Conference on 3D Vision (3DV), Verona, pp. 728–737 (2018). doi:https://doi.org/10.1109/3DV.2018.00088

Tchapmi, L. P., Kosaraju, V., Rezatofighi, H., Reid, I., Savarese, S.: TopNet: structural point cloud decoder. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, pp. 383–392 2019. doi: https://doi.org/10.1109/CVPR.2019.00047

Huang, Z., Yu, Y., Xu, J., Ni, F., Le, X.: PF-net: point fractal network for 3D point cloud completion. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, pp. 7659–7667 (2020). doi: https://doi.org/10.1109/CVPR42600.2020.00768

Liu, M., Sheng, L., Yang, S., Shao, J., Hu, S.: Morphing and sampling network for dense point cloud completion. arXiv e-prints (2019)

Wang, X., Ang, M. H., Lee, G. H.: Cascaded refinement network for point cloud completion. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, pp. 787–796 (2020). doi: https://doi.org/10.1109/CVPR42600.2020.00087

Sung, M., Kim, V., Angst, R., Guibas, L.: Data-driven structural priors for shape completion. ACM Trans. Graphics 34, 1–11 (2015). https://doi.org/10.1145/2816795.2818094

Sipiran, V., Gregor, R., Schreck, T.: Approximate symmetry detection in partial 3D meshes. Comput. Graphics Forum (2014). doi: https://doi.org/10.1111/cgf.12481

Thrun, S., Wegbreit, B.: Shape from symmetry. In: Tenth IEEE International Conference on Computer Vision (ICCV'05), vol. 1, Beijing, 2005, pp. 1824–1831 vol. 2. doi: https://doi.org/10.1109/ICCV.2005.221

Rock, J., Gupta, T., Thorsen, J., Gwak, J., Shin, D., Hoiem, D.: Completing 3D object shape from one depth image. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, pp. 2484–2493 (2015). doi: https://doi.org/10.1109/CVPR.2015.7298863

Li, D., Shao, T., Wu, H., Zhou, K.: Shape completion from a single RGBD image. IEEE Trans. Visual Comput. Graphics 23(7), 1809–1822 (2017). https://doi.org/10.1109/TVCG.2016.2553102

Pauly, M., Mitra, N., Giesen, J., Gross, M., Guibas, L.: Example-based 3D scan completion. SGP.

Sharma, A., Grau, O., Fritz, M.: VConv-DAE: deep volumetric shape learning without object labels. arXiv e-prints (2016)

Smith, E., Meger, D.: Improved adversarial systems for 3D object generation and reconstruction, arXiv e-prints (2017)

Charles, R. Q., Su, H., Kaichun, M., Guibas, L. J.: PointNet: deep learning on point sets for 3D classification and segmentation. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, 2017, pp. 77–85, doi: https://doi.org/10.1109/CVPR.2017.16

Charles, R. Q., Yi, L., Su, H., Guibas, L. J.: "PointNet++: deep hierarchical feature learning on point sets in a metric space. arXiv e-prints (2017)

Achlioptas, P., Diamanti, O., Mitliagkas, I., Guibas, L.: Learning representations and generative models for 3D point clouds," arXiv e-prints (2017)

Fan, H., Su, H., Guibas, L.: A point set generation network for 3D object reconstruction from a single image. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, pp. 2463–2471 (2017). doi: https://doi.org/10.1109/CVPR.2017.264

Yang, Y., Feng, C., Shen, Y., Tian, D.: FoldingNet: point cloud auto-encoder via deep grid deformation. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, pp. 206–215 (2018). doi: https://doi.org/10.1109/CVPR.2018.00029

Groueix, T., Fisher, M., Kim, V. G., Russell, B. C., Aubry, M.: A Papier-Mache approach to learning 3D surface generation. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, pp. 216–224 (2018). doi: https://doi.org/10.1109/CVPR.2018.00030

Wu, Z. et al.: 3D ShapeNets: a deep representation for volumetric shapes. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, pp. 1912–1920 (2015). doi: https://doi.org/10.1109/CVPR.2015.7298801

Geiger, A., et al.: Vision meets robotics: the KITTI dataset. Int. J. Robot. Res. 32(11), 1231–1237 (2013)

Thông tin
Thông tin xuất bản

The Visual Computer

Tập 38

4373-4382

Thông tin tác giả