MeshCNN

ACM Transactions on Graphics - Tập 38 Số 4 - Trang 1-12 - 2019
Rana Hanocka1, Amir Hertz1, Noa Fish1, Raja Giryes1, Shachar Fleishman2, Daniel Cohen‐Or1
1Tel Aviv University
2Amazon

Tóm tắt

Polygonal meshes provide an efficient representation for 3D shapes. They explicitly captureboth shape surface and topology, and leverage non-uniformity to represent large flat regions as well as sharp, intricate features. This non-uniformity and irregularity, however, inhibits mesh analysis efforts using neural networks that combine convolution and pooling operations. In this paper, we utilize the unique properties of the mesh for a direct analysis of 3D shapes using MeshCNN , a convolutional neural network designed specifically for triangular meshes. Analogous to classic CNNs, MeshCNN combines specialized convolution and pooling layers that operate on the mesh edges, by leveraging their intrinsic geodesic connections. Convolutions are applied on edges and the four edges of their incident triangles, and pooling is applied via an edge collapse operation that retains surface topology, thereby, generating new mesh connectivity for the subsequent convolutions. MeshCNN learns which edges to collapse, thus forming a task-driven process where the network exposes and expands the important features while discarding the redundant ones. We demonstrate the effectiveness of MeshCNN on various learning tasks applied to 3D meshes.

Từ khóa


Tài liệu tham khảo

Adobe. 2016. Adobe Fuse 3D Characters. https://www.mixamo.com. Adobe. 2016. Adobe Fuse 3D Characters. https://www.mixamo.com.

10.1145/1186822.1073207

10.5555/3157096.3157320

10.1145/3197517.3201301

10.5555/1370949

10.1109/CVPR.2014.491

Davide Boscaini , Jonathan Masci , Simone Melzi , Michael M Bronstein , Umberto Castellani , and Pierre Vandergheynst . 2015. Learning class-specific descriptors for deformable shapes using localized spectral convolutional networks . In Computer Graphics Forum , Vol. 34 . Wiley Online Library , 13--23. Davide Boscaini, Jonathan Masci, Simone Melzi, Michael M Bronstein, Umberto Castellani, and Pierre Vandergheynst. 2015. Learning class-specific descriptors for deformable shapes using localized spectral convolutional networks. In Computer Graphics Forum, Vol. 34. Wiley Online Library, 13--23.

Davide Boscaini Jonathan Masci Emanuele Rodolà and Michael Bronstein. 2016. Learning shape correspondence with anisotropic convolutional neural networks. In Advances in Neural Information Processing Systems. 3189--3197. Davide Boscaini Jonathan Masci Emanuele Rodolà and Michael Bronstein. 2016. Learning shape correspondence with anisotropic convolutional neural networks. In Advances in Neural Information Processing Systems. 3189--3197.

Mario Botsch Leif Kobbelt Mark Pauly Pierre Alliez and Bruno Lévy. 2010. Polygon mesh processing. AK Peters/CRC Press. Mario Botsch Leif Kobbelt Mark Pauly Pierre Alliez and Bruno Lévy. 2010. Polygon mesh processing. AK Peters/CRC Press.

Darko Bozidar and Tomaz Dobravec . 2015. Comparison of parallel sorting algorithms. CoRR abs/1511.03404 ( 2015 ). Darko Bozidar and Tomaz Dobravec. 2015. Comparison of parallel sorting algorithms. CoRR abs/1511.03404 (2015).

Andrew Brock Theodore Lim J.M. Ritchie and Nick Weston. 2016. Generative and Discriminative Voxel Modeling with Convolutional Neural Networks. In NIPS 3D Deep Learning Workshop. Andrew Brock Theodore Lim J.M. Ritchie and Nick Weston. 2016. Generative and Discriminative Voxel Modeling with Convolutional Neural Networks. In NIPS 3D Deep Learning Workshop.

10.1145/1899404.1899405

10.1109/MSP.2017.2693418

Joan Bruna , Wojciech Zaremba , Arthur Szlam , and Yann LeCun . 2014 . Spectral Networks and Locally Connected Networks on Graphs. In International Conference on Learning Representations (ICLR). Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun. 2014. Spectral Networks and Locally Connected Networks on Graphs. In International Conference on Learning Representations (ICLR).

C. Cangea , P. Velickovic , N. Jovanovic , T. Kipf , and P. Lio . 2018 . Towards Sparse Hierarchical Graph Classifiers. In NeurIPS Workshop on Relational Representation Learning. C. Cangea, P. Velickovic, N. Jovanovic, T. Kipf, and P. Lio. 2018. Towards Sparse Hierarchical Graph Classifiers. In NeurIPS Workshop on Relational Representation Learning.

Liang-Chieh Chen , George Papandreou , Iasonas Kokkinos , Kevin Murphy , and Alan L Yuille . 2018 . Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs . IEEE transactions on pattern analysis and machine intelligence 40, 4 (2018), 834--848. Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. 2018. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE transactions on pattern analysis and machine intelligence 40, 4 (2018), 834--848.

Michaël Defferrard Xavier Bresson and Pierre Vandergheynst. 2016. Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in Neural Information Processing Systems. 3844--3852. Michaël Defferrard Xavier Bresson and Pierre Vandergheynst. 2016. Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in Neural Information Processing Systems. 3844--3852.

10.1111/cgf.13244

10.1145/3130800.3130848

10.1145/258734.258849

Daniela Giorgi , Silvia Biasotti , and Laura Paraboschi . 2007. Shape retrieval contest 2007: Watertight models track. SHREC competition 8, 7 ( 2007 ). Daniela Giorgi, Silvia Biasotti, and Laura Paraboschi. 2007. Shape retrieval contest 2007: Watertight models track. SHREC competition 8, 7 (2007).

10.1109/IJCNN.2017.7965883

Benjamin Graham , Martin Engelcke , and Laurens van der Maaten . 2017. 3D Semantic Segmentation with Submanifold Sparse Convolutional Networks. CoRR abs/1711.10275 ( 2017 ). Benjamin Graham, Martin Engelcke, and Laurens van der Maaten. 2017. 3D Semantic Segmentation with Submanifold Sparse Convolutional Networks. CoRR abs/1711.10275 (2017).

Paul Guerrero , Yanir Kleiman , Maks Ovsjanikov , and Niloy J . Mitra . 2018 . PCPNet: Learning Local Shape Properties from Raw Point Clouds. Computer Graphics Forum Paul Guerrero, Yanir Kleiman, Maks Ovsjanikov, and Niloy J. Mitra. 2018. PCPNet: Learning Local Shape Properties from Raw Point Clouds. Computer Graphics Forum

37, 2 ( 2018 ), 75--85 . 37, 2 (2018), 75--85.

Niv Haim , Nimrod Segol , Heli Ben-Hamu , Haggai Maron , and Yaron Lipman . 2018. Surface Networks via General Covers. CoRR abs/1812.10705 ( 2018 ). Niv Haim, Nimrod Segol, Heli Ben-Hamu, Haggai Maron, and Yaron Lipman. 2018. Surface Networks via General Covers. CoRR abs/1812.10705 (2018).

10.1145/3267347

Mikael Henaff , Joan Bruna , and Yann LeCun . 2015. Deep Convolutional Networks on Graph-Structured Data. CoRR abs/1506.05163 ( 2015 ). Mikael Henaff, Joan Bruna, and Yann LeCun. 2015. Deep Convolutional Networks on Graph-Structured Data. CoRR abs/1506.05163 (2015).

10.1145/258734.258843

Hugues Hoppe . 1999 . New quadric metric for simplifying meshes with appearance attributes. In Visualization'99 . Proceedings. IEEE, 59--510 . Hugues Hoppe. 1999. New quadric metric for simplifying meshes with appearance attributes. In Visualization'99. Proceedings. IEEE, 59--510.

10.1145/166117.166119

Yangqing Jia. 2014. Learning Semantic Image Representations at a Large Scale. (2014). Yangqing Jia. 2014. Learning Semantic Image Representations at a Large Scale. (2014).

Chiyu Max Jiang , Jingwei Huang , Karthik Kashinath , Prabhat, Philip Marcus , and Matthias Niessner . 2019 . Spherical CNNs on Unstructured Grids. In International Conference on Learning Representations. https://openreview.net/forum?id=Bkl-43C9FQ Chiyu Max Jiang, Jingwei Huang, Karthik Kashinath, Prabhat, Philip Marcus, and Matthias Niessner. 2019. Spherical CNNs on Unstructured Grids. In International Conference on Learning Representations. https://openreview.net/forum?id=Bkl-43C9FQ

Evangelos Kalogerakis , Melinos Averkiou , Subhransu Maji , and Siddhartha Chaudhuri . 2017 . 3D shape segmentation with projective convolutional networks . In Proc. CVPR , Vol. 1. 8 . Evangelos Kalogerakis, Melinos Averkiou, Subhransu Maji, and Siddhartha Chaudhuri. 2017. 3D shape segmentation with projective convolutional networks. In Proc. CVPR, Vol. 1. 8.

10.1145/1778765.1778839

I. Kokkinos , M. M. Bronstein , R. Litman , and A. M. Bronstein . 2012. Intrinsic shape context descriptors for deformable shapes . In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 159--166 . I. Kokkinos, M. M. Bronstein, R. Litman, and A. M. Bronstein. 2012. Intrinsic shape context descriptors for deformable shapes. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 159--166.

Ilya Kostrikov , Zhongshi Jiang , Daniele Panozzo , Denis Zorin , and Burna Joan . 2018 . Surface Networks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Ilya Kostrikov, Zhongshi Jiang, Daniele Panozzo, Denis Zorin, and Burna Joan. 2018. Surface Networks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

Alex Krizhevsky Ilya Sutskever and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems. 1097--1105. Alex Krizhevsky Ilya Sutskever and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems. 1097--1105.

10.1109/34.879802

10.1109/34.879802

10.1007/978-3-642-33863-2_51

Yangyan Li , Rui Bu , Mingchao Sun , and Baoquan Chen . 2018. Point CNN. CoRR abs/1801.07791 ( 2018 ). Yangyan Li, Rui Bu, Mingchao Sun, and Baoquan Chen. 2018. PointCNN. CoRR abs/1801.07791 (2018).

Yangyan Li , Soren Pirk , Hao Su , Charles R Qi , and Leonidas J Guibas . 2016 . FPNN: Field probing neural networks for 3D data. In Advances in Neural Information Processing Systems (NIPS). 307--315. Yangyan Li, Soren Pirk, Hao Su, Charles R Qi, and Leonidas J Guibas. 2016. FPNN: Field probing neural networks for 3D data. In Advances in Neural Information Processing Systems (NIPS). 307--315.

Z Lian , A Godil , B Bustos , M Daoudi , J Hermans , S Kawamura , Y Kurita , G Lavoua , and P Dp Suetens . 2011 . Shape retrieval on non-rigid 3D watertight meshes . In Eurographics Workshop on 3D Object Retrieval (3DOR). Z Lian, A Godil, B Bustos, M Daoudi, J Hermans, S Kawamura, Y Kurita, G Lavoua, and P Dp Suetens. 2011. Shape retrieval on non-rigid 3D watertight meshes. In Eurographics Workshop on 3D Object Retrieval (3DOR).

Or Litany Alexander M. Bronstein Michael M. Bronstein and Ameesh Makadia. 2018. Deformable Shape Completion With Graph Convolutional Autoencoders. In CVPR. Or Litany Alexander M. Bronstein Michael M. Bronstein and Ameesh Makadia. 2018. Deformable Shape Completion With Graph Convolutional Autoencoders. In CVPR.

10.1145/3072959.3073616

10.1109/ICCVW.2015.112

10.1109/CVPR.2017.576

Federico Monti , Oleksandr Shchur , Aleksandar Bojchevski , Or Litany , Stephan Gunnemann , and Michael M . Bronstein . 2018 . Dual-Primal Graph Convolutional Networks. CoRR abs/1806.00770 (2018). Federico Monti, Oleksandr Shchur, Aleksandar Bojchevski, Or Litany, Stephan Gunnemann, and Michael M. Bronstein. 2018. Dual-Primal Graph Convolutional Networks. CoRR abs/1806.00770 (2018).

Mathias Niepert , Mohamed Ahmed , and Konstantin Kutzkov . 2016 . Learning Convolutional Neural Networks for Graphs. In International Conference on Machine Learning (ICML). Mathias Niepert, Mohamed Ahmed, and Konstantin Kutzkov. 2016. Learning Convolutional Neural Networks for Graphs. In International Conference on Machine Learning (ICML).

Adam Paszke Sam Gross Soumith Chintala Gregory Chanan Edward Yang Zachary DeVito Zeming Lin Alban Desmaison Luca Antiga and Adam Lerer. 2017. Automatic differentiation in PyTorch. In NIPS-W. Adam Paszke Sam Gross Soumith Chintala Gregory Chanan Edward Yang Zachary DeVito Zeming Lin Alban Desmaison Luca Antiga and Adam Lerer. 2017. Automatic differentiation in PyTorch. In NIPS-W.

10.1145/3272127.3275102

Charles R Qi , Hao Su , Kaichun Mo , and Leonidas J Guibas . 2017 a. Pointnet: Deep learning on point sets for 3d classification and segmentation . Proc. Computer Vision and Pattern Recognition (CVPR), IEEE 1 , 2 (2017), 4 . Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. 2017a. Pointnet: Deep learning on point sets for 3d classification and segmentation. Proc. Computer Vision and Pattern Recognition (CVPR), IEEE 1, 2 (2017), 4.

Charles R. Qi , Hao Su , Matthias Niessner , Angela Dai , Mengyuan Yan , and Leonidas J . Guibas . 2016 . Volumetric and multi-view CNNs for object classification on 3d data. In Computer Vision and Pattern Recognition (CVPR) . 5648--5656. Charles R. Qi, Hao Su, Matthias Niessner, Angela Dai, Mengyuan Yan, and Leonidas J. Guibas. 2016. Volumetric and multi-view CNNs for object classification on 3d data. In Computer Vision and Pattern Recognition (CVPR). 5648--5656.

Charles R. Qi Li Yi Hao Su and Leonidas J Guibas. 2017b. PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space. In Advances in Neural Information Processing Systems (NIPS). 5105--5114. Charles R. Qi Li Yi Hao Su and Leonidas J Guibas. 2017b. PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space. In Advances in Neural Information Processing Systems (NIPS). 5105--5114.

Anurag Ranjan , Timo Bolkart , Soubhik Sanyal , and Michael J . Black . 2018 . Generating 3D faces using Convolutional Mesh Autoencoders. In European Conference on Computer Vision (ECCV). Springer International Publishing , 725--741. Anurag Ranjan, Timo Bolkart, Soubhik Sanyal, and Michael J. Black. 2018. Generating 3D faces using Convolutional Mesh Autoencoders. In European Conference on Computer Vision (ECCV). Springer International Publishing, 725--741.

Gernot Riegler , Ali Osman Ulusoy, and Andreas Geiger . 2017 . OctNet: Learning deep 3D representations at high resolutions. In Computer Vision and Pattern Recognition (CVPR) . Gernot Riegler, Ali Osman Ulusoy, and Andreas Geiger. 2017. OctNet: Learning deep 3D representations at high resolutions. In Computer Vision and Pattern Recognition (CVPR).

10.1007/978-3-319-24574-4_28

10.1145/344779.344940

10.1007/978-3-030-01270-0_5

Pierre Sermanet , David Eigen , Xiang Zhang , Michaël Mathieu , Rob Fergus , and Yann LeCun . 2013 . Overfeat: Integrated recognition, localization and detection using convolutional networks. arXiv preprint arXiv:1312.6229 (2013). Pierre Sermanet, David Eigen, Xiang Zhang, Michaël Mathieu, Rob Fergus, and Yann LeCun. 2013. Overfeat: Integrated recognition, localization and detection using convolutional networks. arXiv preprint arXiv:1312.6229 (2013).

Karen Simonyan and Andrew Zisserman . 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 ( 2014 ). Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).

10.1007/978-3-319-46466-4_14

10.1109/ICCV.2015.114

10.1109/JSTSP.2017.2726981

Marco Tarini , Nico Pietroni , Paolo Cignoni , Daniele Panozzo , and Enrico Puppo . 2010. Practical quad mesh simplification . In Computer Graphics Forum , Vol. 29 . Wiley Online Library , 407--418. Marco Tarini, Nico Pietroni, Paolo Cignoni, Daniele Panozzo, and Enrico Puppo. 2010. Practical quad mesh simplification. In Computer Graphics Forum, Vol. 29. Wiley Online Library, 407--418.

10.1109/CVPR.2018.00409

Lyne P. Tchapmi Christopher B. Choy Iro Armeni JunYoung Gwak and Silvio Savarese. 2017. SEGCloud: Semantic Segmentation of 3D Point Clouds. In 3DV. Lyne P. Tchapmi Christopher B. Choy Iro Armeni JunYoung Gwak and Silvio Savarese. 2017. SEGCloud: Semantic Segmentation of 3D Point Clouds. In 3DV.

Petar Velickovic , Guillem Cucurull , Arantxa Casanova , Adriana Romero , Pietro Lio , and Yoshua Bengio . 2018 . Graph Attention Networks. In International Conference on Learning Representations. Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2018. Graph Attention Networks. In International Conference on Learning Representations.

Nitika Verma E. Boyer and Jakob Verbeek. 2018. FeaStNet: Feature-Steered Graph Convolutions for 3D Shape Analysis. In CVPR. Nitika Verma E. Boyer and Jakob Verbeek. 2018. FeaStNet: Feature-Steered Graph Convolutions for 3D Shape Analysis. In CVPR.

10.1145/1360612.1360696

10.1145/3072959.3073608

10.1145/2366145.2366184

Yue Wang , Yongbin Sun , Ziwei Liu , Sanjay E Sarma , Michael M Bronstein , and Justin M Solomon . 2018a. Dynamic graph CNN for learning on point clouds. arXiv preprint arXiv:1801.07829 ( 2018 ). Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. 2018a. Dynamic graph CNN for learning on point clouds. arXiv preprint arXiv:1801.07829 (2018).

Yue Wang , Yongbin Sun , Ziwei Liu , Sanjay E Sarma , Michael M Bronstein , and Justin M Solomon . 2018b. Dynamic Graph CNN for Learning on Point Clouds. arXiv preprint arXiv:1801.07829 ( 2018 ). Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. 2018b. Dynamic Graph CNN for Learning on Point Clouds. arXiv preprint arXiv:1801.07829 (2018).

Francis Williams , Teseo Schneider , Claudio Silva , Denis Zorin , Joan Bruna , and Daniele Panozzo . 2018. Deep Geometric Prior for Surface Reconstruction. arXiv preprint arXiv:1811.10943 ( 2018 ). Francis Williams, Teseo Schneider, Claudio Silva, Denis Zorin, Joan Bruna, and Daniele Panozzo. 2018. Deep Geometric Prior for Surface Reconstruction. arXiv preprint arXiv:1811.10943 (2018).

Zhirong Wu Shuran Song Aditya Khosla Fisher Yu Linguang Zhang Xiaoou Tang and Jianxiong Xiao. 2015. 3D shapenets: A deep representation for volumetric shapes. In Computer Vision and Pattern Recognition (CVPR). 1912--1920. Zhirong Wu Shuran Song Aditya Khosla Fisher Yu Linguang Zhang Xiaoou Tang and Jianxiong Xiao. 2015. 3D shapenets: A deep representation for volumetric shapes. In Computer Vision and Pattern Recognition (CVPR). 1912--1920.

10.1109/ICCV.2017.294

Li Yi Hao Su Xingwen Guo and Leonidas Guibas. 2017. SyncSpecCNN: Synchronized Spectral CNN for 3D Shape Segmentation. In Computer Vision and Pattern Recognition (CVPR). Li Yi Hao Su Xingwen Guo and Leonidas Guibas. 2017. SyncSpecCNN: Synchronized Spectral CNN for 3D Shape Segmentation. In Computer Vision and Pattern Recognition (CVPR).

Zhitao Ying Jiaxuan You Christopher Morris Xiang Ren Will Hamilton and Jure Leskovec. 2018. Hierarchical Graph Representation Learning with Differentiable Pooling. In Advances in Neural Information Processing Systems. 4805--4815. Zhitao Ying Jiaxuan You Christopher Morris Xiang Ren Will Hamilton and Jure Leskovec. 2018. Hierarchical Graph Representation Learning with Differentiable Pooling. In Advances in Neural Information Processing Systems. 4805--4815.

10.1145/3197517.3201323