CNNs on surfaces using rotation-equivariant features

ACM Transactions on Graphics - Tập 39 Số 4 - 2020
Ruben Wiersma1, Elmar Eisemann1, Klaus Hildebrandt1
1Delft University of Technology

Tóm tắt

This paper is concerned with a fundamental problem in geometric deep learning that arises in the construction of convolutional neural networks on surfaces. Due to curvature, the transport of filter kernels on surfaces results in a rotational ambiguity, which prevents a uniform alignment of these kernels on the surface. We propose a network architecture for surfaces that consists of vector-valued, rotation-equivariant features. The equivariance property makes it possible to locally align features, which were computed in arbitrary coordinate systems, when aggregating features in a convolution layer. The resulting network is agnostic to the choices of coordinate systems for the tangent spaces on the surface. We implement our approach for triangle meshes. Based on circular harmonic functions, we introduce convolution filters for meshes that are rotation-equivariant at the discrete level. We evaluate the resulting networks on shape correspondence and shape classifications tasks and compare their performance to other approaches.

Từ khóa


Tài liệu tham khảo

Adobe. 2016. Adobe Mixamo 3D characters. www.mixamo.com.

10.1145/1073204.1073207

Federica Bogo, Javier Romero, Matthew Loper, and Michael J. Black. 2014. FAUST: Dataset and evaluation for 3D mesh registration. In CVPR. IEEE.

Davide Boscaini Jonathan Masci Emanuele Rodolà and Michael Bronstein. 2016. Learning shape correspondence with anisotropic convolutional neural networks. In NeurIPS. 3189--3197.

10.1145/1899404.1899405

10.1109/MSP.2017.2693418

Joan Bruna Wojciech Zaremba Arthur Szlam and Yann Lecun. 2014. Spectral networks and locally connected networks on graphs. In ICLR.

Taco Cohen, Maurice Weiler, Berkay Kicanaoglu, and Max Welling. 2019. Gauge Equivariant Convolutional Networks and the Icosahedral CNN. In ICML, Vol. 97. 1321--1330.

Taco Cohen and Max Welling. 2016. Group equivariant convolutional networks. In ICML. 2990--2999.

Taco S. Cohen Mario Geiger Jonas Köhler and Max Welling. 2018. Spherical CNNs. In ICLR.

Taco S. Cohen and Max Welling. 2017. Steerable CNNs. In ICLR.

10.1145/2516971.2516977

Michaël Defferrard Xavier Bresson and Pierre Vandergheynst. 2016. Convolutional neural networks on graphs with fast localized spectral filtering. In NeurIPS. 3844--3852.

10.1111/cgf.13244

10.1109/ICPR.2006.1020

Matthias Fey and Jan E. Lenssen. 2019. Fast Graph Representation Learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds.

Matthias Fey, Jan Eric Lenssen, Frank Weichert, and Heinrich Müller. 2018. SplineCNN: Fast Geometric Deep Learning With Continuous B-Spline Kernels. In CVPR. IEEE, 869--877.

Chamberlain Fong. 2015. Analytical Methods for Squaring the Disc. arXiv:1509.06344 [math.HO]

William T. Freeman and Edward H Adelson. 1991. The design and use of steerable filters. IEEE Transactions on Pattern Analysis & Machine Intelligence (1991), 891--906.

10.1111/cgf.13343

Niv Haim, Nimrod Segol, Heli Ben-Hamu, Haggai Maron, and Yaron Lipman. 2019. Surface Networks via General Covers. In ICCV. IEEE, 632--641.

10.1145/3306346.3322959

10.1111/cgf.13607

Geoffrey E Hinton, Alex Krizhevsky, and Sida D Wang. 2011. Transforming autoencoders. In International Conference on Artificial Neural Networks. Springer, 44--51.

Jingwei Huang, Haotian Zhang, Li Yi, Thomas A. Funkhouser, Matthias Nießner, and Leonidas J. Guibas. 2019. TextureNet: Consistent Local Parametrizations for Learning From High-Resolution Signals on Meshes. In CVPR. IEEE, 4440--4449.

Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In ICLR.

Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In ICLR.

Risi Kondor, Zhen Lin, and Shubhendu Trivedi. 2018. Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Network. In NeurIPS 2018. 10138--10147.

Ilya Kostrikov, Zhongshi Jiang, Daniele Panozzo, Denis Zorin, and Joan Bruna. 2018. Surface networks. In CVPR. IEEE, 2540--2548.

Dmitry Laptev, Nikolay Savinov, Joachim M Buhmann, and Marc Pollefeys. 2016. TIPOOLING: Transformation-invariant pooling for feature learning in convolutional neural networks. In CVPR. IEEE, 289--297.

Hugo Larochelle Dumitru Erhan Aaron Courville James Bergstra and Yoshua Bengio. 2007. An empirical evaluation of deep architectures on problems with many factors of variation. In ICML. ACM 473--480.

Yangyan Li Rui Bu Mingchao Sun Wei Wu Xinhan Di and Baoquan Chen. 2018. PointCNN: Convolution On X-Transformed Points. In NeurIPS. 820--830.

Zhouhui Lian, Afzal Godil, Benjamin Bustos, Mohamed Daoudi, Jeroen Hermans, Shun Kawamura, Yukinori Kurita, Guillaume Lavoué, Hien Nguyen, Ryutarou Ohbuchi, Yuki Ohkita, Yuya Ohishi, Fatih Porikli, Martin Reuter, Ivan Sipiran, Dirk Smeets, Paul Suetens, Hedi Tabia, and Dirk Vandermeulen. 2011. SHREC '11 Track: Shape Retrieval on Non-rigid 3D Watertight Meshes. Eurographics Workshop on 3D Object Retrieval, 79--88.

Or Litany, Alexander M. Bronstein, Michael M. Bronstein, and Ameesh Makadia. 2018. Deformable Shape Completion with Graph Convolutional Autoencoders. In CVPR. IEEE, 1886--1895.

Kun Liu Qing Wang Wolfgang Driever and Olaf Ronneberger. 2012. 2d/3d rotationinvariant detection using equivariant filters and kernel weighted mapping. In CVPR. IEEE 917--924.

Diego Marcos, Michele Volpi, and Devis Tuia. 2016. Learning rotation invariant convolutional filters for texture classification. In ICPR. IEEE, 2012--2017.

10.1145/3072959.3073616

Jonathan Masci Davide Boscaini Michael Bronstein and Pierre Vandergheynst. 2015. Geodesic convolutional neural networks on riemannian manifolds. In ICCV. 37--45.

10.1111/j.1467-8659.2012.03187.x

10.1109/CVPR.2017.576

Hao Pan, Shilin Liu, Yang Liu, and Xin Tong. 2018. Convolutional Neural Networks on 3D Surfaces Using Parallel Frames. arXiv: 1808.04952 (2018).

10.1145/3272127.3275102

Adrien Poulenard Marie-Julie Rakotosaona Yann Ponty and Maks Ovsjanikov. 2019. Effective Rotation-Invariant Point CNN with Spherical Harmonics Kernels. In 3DV. 47--56.

Charles Ruizhongtai Qi, Hao Su, Kaichun Mo, and Leonidas J. Guibas. 2017a. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. In CVPR. IEEE, 77--85.

Charles Ruizhongtai Qi Li Yi Hao Su and Leonidas J Guibas. 2017b. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In NeurIPS. 5099--5108.

Anurag Ranjan, Timo Bolkart, Soubhik Sanyal, and Michael J. Black. 2018. Generating 3D faces using Convolutional Mesh Autoencoders. In ECCV. 725--741.

10.1145/1141911.1141930

10.1145/1837026.1837034

Stefan C Schonsheck, Bin Dong, and Rongjie Lai. 2018. Parallel Transport Convolution: A New Tool for Convolutional Neural Networks on Manifolds. arXiv:1805.07857 (2018).

Nicholas Sharp. 2019. Polyscope. www.polyscope.run.

Nicholas Sharp Keenan Crane et al. 2019a. geometry-central. www.geometry-central.net.

10.1145/3243651

Ayan Sinha Jing Bai and Karthik Ramani. 2016. Deep Learning 3D Shape Surfaces Using Geometry Images. In ECCV. 223--240.

Hang Su, Subhransu Maji, Evangelos Kalogerakis, and Erik G. Learned-Miller. 2015. Multi-view Convolutional Neural Networks for 3D Shape Recognition. In ICCV. IEEE, 945--953.

Zhiyu Sun, Jia Lu, and Stephen Baek. 2018. ZerNet: Convolutional Neural Networks on Arbitrary Surfaces via Zernike Local Tangent Space Estimation. arXiv:1812.01082 (2018).

Nathaniel Thomas, Tess Smidt, Steven M. Kearnes, Lusann Yang, Li Li, Kai Kohlhoff, and Patrick Riley. 2018. Tensor Field Networks: Rotation- and Translation-Equivariant Neural Networks for 3D Point Clouds. arXiv:1802.08219 (2018).

Nitika Verma, Edmond Boyer, and Jakob Verbeek. 2018. FeaStNet: Feature-Steered Graph Convolutions for 3D Shape Analysis. In CVPR. IEEE, 2598--2606.

10.1145/1360612.1360696

10.1145/3326362

10.1016/j.cagd.2007.07.006

Maurice Weiler and Gabriele Cesa. 2019. General E(2)-Equivariant Steerable CNNs. In NeurIPS. 14334--14345. http://papers.nips.cc/paper/9580-general-e2-equivariant-steerable-cnns

Daniel E Worrall, Stephan J Garbin, Daniyar Turmukhambetov, and Gabriel J Brostow. 2017. Harmonic networks: Deep translation and rotation equivariance. In CVPR. IEEE, 5028--5037.

Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and Philip S. Yu. 2019. A Comprehensive Survey on Graph Neural Networks. arXiv:1901.00596 (2019).

Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 2015. 3d shapenets: A deep representation for volumetric shapes. In CVPR. IEEE, 1912--1920.

Jie Zhou, Ganqu Cui, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, and Maosong Sun. 2018. Graph Neural Networks: A Review of Methods and Applications. arXiv:abs/1812.08434 (2018).

Received January 2020; revised April 2020; accepted April 2020