Shape google
Tóm tắt
The computer vision and pattern recognition communities have recently witnessed a surge of feature-based methods in object recognition and image retrieval applications. These methods allow representing images as collections of “visual words” and treat them using text search approaches following the “bag of features” paradigm. In this article, we explore analogous approaches in the 3D world applied to the problem of nonrigid shape retrieval in large databases. Using multiscale diffusion heat kernels as “geometric words,” we construct compact and informative shape descriptors by means of the “bag of features” approach. We also show that considering pairs of “geometric words” (“geometric expressions”) allows creating spatially sensitive bags of features with better discriminative power. Finally, adopting metric learning approaches, we show that shapes can be efficiently represented as binary codes. Our approach achieves state-of-the-art results on the SHREC 2010 large-scale shape retrieval benchmark.
Từ khóa
Tài liệu tham khảo
Behmo , R. , Paragios , N. , and Prinet , V . 2008. Graph commute times for image representation . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'08) . Behmo, R., Paragios, N., and Prinet, V. 2008. Graph commute times for image representation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'08).
Belkin , M. , Sun , J. , and Wang , Y . 2009. Constructing Laplace operator from point clouds in Rd . In Proceedings of the Symposium on Discrete Algorithms. 1031--1040 . Belkin, M., Sun, J., and Wang, Y. 2009. Constructing Laplace operator from point clouds in Rd. In Proceedings of the Symposium on Discrete Algorithms. 1031--1040.
Ben-Chen , M. , Weber , O. , and Gotsman , C . 2008. Characterizing shape using conformal factors . In Proceedings of the 3DOR Conference. Ben-Chen, M., Weber, O., and Gotsman, C. 2008. Characterizing shape using conformal factors. In Proceedings of the 3DOR Conference.
Biasotti , S. , Marini , S. , Mortara , M. , and Patané , G . 2003. An overview on properties and efficacy of topological skeletons in shape modeling . In Proceedings of the IEEE International Conference on Shape Modeling and Applications (SMI'03) . 245--254. Biasotti, S., Marini, S., Mortara, M., and Patané, G. 2003. An overview on properties and efficacy of topological skeletons in shape modeling. In Proceedings of the IEEE International Conference on Shape Modeling and Applications (SMI'03). 245--254.
Borg I. and Groenen P. 1997. Modern Multidimensional Scaling -- Theory and Applications. Springer. Borg I. and Groenen P. 1997. Modern Multidimensional Scaling -- Theory and Applications. Springer.
Bronstein , A. M. and Bronstein , M. M . 2010a. Affine-Invariant spatially-sensitive vocabularies . In Proceedings of the European Conference on Computer Vision (ECCV'10) . Bronstein, A. M. and Bronstein, M. M. 2010a. Affine-Invariant spatially-sensitive vocabularies. In Proceedings of the European Conference on Computer Vision (ECCV'10).
Bronstein , A. M. , Bronstein , M. M. , Bustos , B. , Castelani , U. , Crisani , M. , Falcidieno , B. , Guibas , L. J. , Sipiran , I. , Kokkinos , I. , Murino , V. , Ovsjanikov , M. , Patané , G. , Spagnuolo , M. , and Sun , J . 2010a . SHREC 2010: Robust feature detection and description benchmark. In Proceedings of the 3DOR Conference. Bronstein, A. M., Bronstein, M. M., Bustos, B., Castelani, U., Crisani, M., Falcidieno, B., Guibas, L. J., Sipiran, I., Kokkinos, I., Murino, V., Ovsjanikov, M., Patané, G., Spagnuolo, M., and Sun, J. 2010a. SHREC 2010: Robust feature detection and description benchmark. In Proceedings of the 3DOR Conference.
Bronstein , A. M. , Bronstein , M. M. , Castellani , U. , Falcidieno , B. , Fusiello , A. , Godil , A. , Guibas , L. J. , Kokkinos , I. , Lian , Z. , Ovsjanikov , M. , Patané , G. , Spagnuolo , M. , and Toldo , R . 2010b . SHREC 2010: Robust large-scale shape retrieval benchmark. In Proceedings of the 3DOR Conference. Bronstein, A. M., Bronstein, M. M., Castellani, U., Falcidieno, B., Fusiello, A., Godil, A., Guibas, L. J., Kokkinos, I., Lian, Z., Ovsjanikov, M., Patané, G., Spagnuolo, M., and Toldo, R. 2010b. SHREC 2010: Robust large-scale shape retrieval benchmark. In Proceedings of the 3DOR Conference.
Bronstein A. M. Bronstein M. M. and Kimmel R. 2008. Numerical Geometry of Non-Rigid Shapes. Springer. Bronstein A. M. Bronstein M. M. and Kimmel R. 2008. Numerical Geometry of Non-Rigid Shapes. Springer.
Bronstein A. M. Bronstein M. M. and Kimmel R. 2010c. The video genome. arXiv 1003.5320v1. Bronstein A. M. Bronstein M. M. and Kimmel R. 2010c. The video genome. arXiv 1003.5320v1.
Bronstein M. M. and Bronstein A. M. 2009. On a relation between shape recognition algorithms based on distributions of distances. Tech. rep. CIS-2009-14 Department of Computer Science Technion Israel. Bronstein M. M. and Bronstein A. M. 2009. On a relation between shape recognition algorithms based on distributions of distances. Tech. rep. CIS-2009-14 Department of Computer Science Technion Israel.
Bronstein , M. M. , Bronstein , A. M. , Michel , F. , and Paragios , N . 2010e. Data fusion through cross-modality metric learning using similarity-sensitive hashing . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'10) . Bronstein, M. M., Bronstein, A. M., Michel, F., and Paragios, N. 2010e. Data fusion through cross-modality metric learning using similarity-sensitive hashing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'10).
Bronstein , M. M. and Kokkinos , I . 2010. Scale-Invariant heat kernel signatures for non-rigid shape recognition . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'10) . Bronstein, M. M. and Kokkinos, I. 2010. Scale-Invariant heat kernel signatures for non-rigid shape recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'10).
Chazal , F. , Guibas , L. J. , Oudot , S. Y. , and Skraba , P . 2009b. Analysis of scalar fields over point cloud data . In Proceedings of the Annual ACM-SIAM Symposium on Discrete Algorithms (SODA'09) . 1021--1030. Chazal, F., Guibas, L. J., Oudot, S. Y., and Skraba, P. 2009b. Analysis of scalar fields over point cloud data. In Proceedings of the Annual ACM-SIAM Symposium on Discrete Algorithms (SODA'09). 1021--1030.
Chum , O. , Philbin , J. , Sivic , J. , Isard , M. , and Zisserman , A . 2007. Total recall: Automatic query expansion with a generative feature model for object retrieval . In Proceedings of the IEEE International Conference on Computer Vision (ICCV'07) . Chum, O., Philbin, J., Sivic, J., Isard, M., and Zisserman, A. 2007. Total recall: Automatic query expansion with a generative feature model for object retrieval. In Proceedings of the IEEE International Conference on Computer Vision (ICCV'07).
Dubrovina , A. and Kimmel , R . 2010. Matching shapes by eigen-decomposition of the Laplace-Beltrami operator . In Proceedings of the 3DPVT Conference. Dubrovina, A. and Kimmel, R. 2010. Matching shapes by eigen-decomposition of the Laplace-Beltrami operator. In Proceedings of the 3DPVT Conference.
Edelsbrunner , H. , Letscher , D. , and Zomorodian , A . 2000. Topological persistence and simplification . In Proceedings of the IEEE Conference on Foundations of Computer Science. 454--463 . Edelsbrunner, H., Letscher, D., and Zomorodian, A. 2000. Topological persistence and simplification. In Proceedings of the IEEE Conference on Foundations of Computer Science. 454--463.
Elad , A. and Kimmel , R . 2001. Bending invariant representations for surfaces . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'01) . 168--174. Elad, A. and Kimmel, R. 2001. Bending invariant representations for surfaces. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'01). 168--174.
Floater M. S. and Hormann K. 2005. Surface parameterization: A tutorial and survey. Adv. Multiresolut. Geom. Model. 1. Floater M. S. and Hormann K. 2005. Surface parameterization: A tutorial and survey. Adv. Multiresolut. Geom. Model. 1.
Freund , Y. and Schapire , R. E . 1995. A decision-theoretic generalization of on-line learning and an application to boosting . In Proceedings of the European Conference on Computational Learning Theory. Freund, Y. and Schapire, R. E. 1995. A decision-theoretic generalization of on-line learning and an application to boosting. In Proceedings of the European Conference on Computational Learning Theory.
Gelfand , N. , Mitra , N. J. , Guibas , L. J. , and Pottmann , H . 2005. Robust global registration . In Proceedings of the Symposium on Geometry Processing (SGP'05) . Gelfand, N., Mitra, N. J., Guibas, L. J., and Pottmann, H. 2005. Robust global registration. In Proceedings of the Symposium on Geometry Processing (SGP'05).
Glomb , P. 2009 . Detection of interest points on 3D data: Extending the harris operator . In Proceedings of the Conference on Computer Recognition Systems 3. Advances in Soft Computing , Vol. 57 . Springer, 103--111. Glomb, P. 2009. Detection of interest points on 3D data: Extending the harris operator. In Proceedings of the Conference on Computer Recognition Systems 3. Advances in Soft Computing, Vol. 57. Springer, 103--111.
Gromov M. 1981. Structures Metriques Pour les Varietes Rieman- niennes. Number 1 in Textes Mathematiques. Gromov M. 1981. Structures Metriques Pour les Varietes Rieman- niennes. Number 1 in Textes Mathematiques.
Harris , C. and Stephens , M . 1988. A combined corner and edge detection . In Proceedings of the 4th Alvey Vision Conference. 147--151 . Harris, C. and Stephens, M. 1988. A combined corner and edge detection. In Proceedings of the 4th Alvey Vision Conference. 147--151.
Hsu , E. P. 2002. Stochastic Analysis on Manifolds . American Mathematical Society . Hsu, E. P. 2002. Stochastic Analysis on Manifolds. American Mathematical Society.
Jain , P. , Kulis , B. , and Grauman , K . 2008. Fast image search for learned metrics . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recongnition (CVPR'08) . Jain, P., Kulis, B., and Grauman, K. 2008. Fast image search for learned metrics. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recongnition (CVPR'08).
Jiantao , P. , Yi , L. , Guyu , X. , Hongbin , Z. , Weibin , L. , and Uehara , Y . 2004. 3D model retrieval based on 2D slice similarity measurements . In Proceedings of the 3DPVT Conference. 95--101 . Jiantao, P., Yi, L., Guyu, X., Hongbin, Z., Weibin, L., and Uehara, Y. 2004. 3D model retrieval based on 2D slice similarity measurements. In Proceedings of the 3DPVT Conference. 95--101.
Kazhdan , M. , Funkhouser , T. , and Rusinkiewicz , S . 2003. Rotation invariant spherical harmonic representation of 3D shape descriptors . In Proceedings of the Symposium on Geometry Processing (SGP'03) . 156--164. Kazhdan, M., Funkhouser, T., and Rusinkiewicz, S. 2003. Rotation invariant spherical harmonic representation of 3D shape descriptors. In Proceedings of the Symposium on Geometry Processing (SGP'03). 156--164.
Kolomenkin , M. , Shimshoni , I. , and Tal , A . 2009. On edge detection on surfaces ,. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'09) . Kolomenkin, M., Shimshoni, I., and Tal, A. 2009. On edge detection on surfaces,. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'09).
Leibe , B. , Leonardis , A. , and Schiele , B . 2004. Combined object categorization and segmentation with an implicit shape model . In Proceedings of the Workshop Statistical Learning in Computer Vision. 17--32 . Leibe, B., Leonardis, A., and Schiele, B. 2004. Combined object categorization and segmentation with an implicit shape model. In Proceedings of the Workshop Statistical Learning in Computer Vision. 17--32.
Mateus , D. , Horaud , R. P. , Knossow , D. , Cuzzolin , F. , and Boyer , E . 2008. Articulated shape matching using laplacian eigen-functions and unsupervised point registration . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'08) . Mateus, D., Horaud, R. P., Knossow, D., Cuzzolin, F., and Boyer, E. 2008. Articulated shape matching using laplacian eigen-functions and unsupervised point registration. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'08).
Mémoli F. 2007. On the use of Gromov-Hausdorff distances for shape comparison. In Point Based Graphics. Mémoli F. 2007. On the use of Gromov-Hausdorff distances for shape comparison. In Point Based Graphics.
Min P. Kazhdan M. and Funkhouser T. 2004. A comparison of text and shape matching for retrieval of online 3D models. Res. Adv. Technol. Digital Librar. 209--220. Min P. Kazhdan M. and Funkhouser T. 2004. A comparison of text and shape matching for retrieval of online 3D models. Res. Adv. Technol. Digital Librar. 209--220.
Mitra , N. , Bronstein , A. M. , and Bronstein , M. M . 2010. Intrinsic regularity detection in 3D geometry . In Proceedings of the European Conference on Computer Vision (ECCV'10) . Mitra, N., Bronstein, A. M., and Bronstein, M. M. 2010. Intrinsic regularity detection in 3D geometry. In Proceedings of the European Conference on Computer Vision (ECCV'10).
Mitra , N. J. , Guibas , L. J. , Giesen , J. , and Pauly , M . 2006. Probabilistic fingerprints for shapes . In Proceedings of the Symposium on Geometry Processing (SGP'06) . Mitra, N. J., Guibas, L. J., Giesen, J., and Pauly, M. 2006. Probabilistic fingerprints for shapes. In Proceedings of the Symposium on Geometry Processing (SGP'06).
Napoléon , T. , Adamek , T. , Schmitt , F. , and O'Connor , N. E. 2007 . Multi-View 3D retrieval using silhouette intersection and multi-scale contour representation . In Proceedings of the Conference on Shape Modeling and Applications. Napoléon, T., Adamek, T., Schmitt, F., and O'Connor, N. E. 2007. Multi-View 3D retrieval using silhouette intersection and multi-scale contour representation. In Proceedings of the Conference on Shape Modeling and Applications.
Ovsjanikov , M. , Bronstein , A. M. , Bronstein , M. M. , and Guibas , L. J . 2009. Shape Google: A computer vision approach to invariant shape retrieval . In Proceedings of the Workshop on Non-Rigid Shape Analysis and Deformable Image Alignment (NORDIA'09) . Ovsjanikov, M., Bronstein, A. M., Bronstein, M. M., and Guibas, L. J. 2009. Shape Google: A computer vision approach to invariant shape retrieval. In Proceedings of the Workshop on Non-Rigid Shape Analysis and Deformable Image Alignment (NORDIA'09).
Raviv , D. , Bronstein , A. M. , Bronstein , M. M. , and Kimmel , R . 2007. Symmetries of non-rigid shapes . In Proceedings of the NRTL'07 Conference. Raviv, D., Bronstein, A. M., Bronstein, M. M., and Kimmel, R. 2007. Symmetries of non-rigid shapes. In Proceedings of the NRTL'07 Conference.
Raviv , D. , Bronstein , A. M. , Bronstein , M. M. , Kimmel , R. , and Sapiro , G . 2010a. Diffusion symmetries of non-rigid shapes . In Proceedings of the 3DPVT Conference. Raviv, D., Bronstein, A. M., Bronstein, M. M., Kimmel, R., and Sapiro, G. 2010a. Diffusion symmetries of non-rigid shapes. In Proceedings of the 3DPVT Conference.
Rustamov , R. M. 2007 . Laplace-Beltrami eigenfunctions for deformation invariant shape representation . In Proceedings of the Symposium on Geometry Processing (SGP'07) . 225--233. Rustamov, R. M. 2007. Laplace-Beltrami eigenfunctions for deformation invariant shape representation. In Proceedings of the Symposium on Geometry Processing (SGP'07). 225--233.
Shilane , P. , Min , P. , Kazhdan , M. , and Funkhouser , T . 2004. The Princeton shape benchmark . In Proceedings of the the IEEE International Conference on Shape Modeling and Applications (SMI'04) . 167--178. Shilane, P., Min, P., Kazhdan, M., and Funkhouser, T. 2004. The Princeton shape benchmark. In Proceedings of the the IEEE International Conference on Shape Modeling and Applications (SMI'04). 167--178.
Sipiran , I. and Bustos , B . 2010. A robust 3D interest points detector based on Harris operator . In Proceedings of the Eurographics Workshop on 3D Object Retrieval. 7--14 . Sipiran, I. and Bustos, B. 2010. A robust 3D interest points detector based on Harris operator. In Proceedings of the Eurographics Workshop on 3D Object Retrieval. 7--14.
Sivic , J. and Zisserman , A . 2003. Video Google: A text retrieval approach to object matching in videos . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'03) . Sivic, J. and Zisserman, A. 2003. Video Google: A text retrieval approach to object matching in videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'03).
Sun , J. , Ovsjanikov , M. , and Guibas , L. J . 2009. A concise and provably informative multi-scale signature based on heat diffusion . In Proceedings of the Symposium on Geometry Processing (SGP'09) . Sun, J., Ovsjanikov, M., and Guibas, L. J. 2009. A concise and provably informative multi-scale signature based on heat diffusion. In Proceedings of the Symposium on Geometry Processing (SGP'09).
Sundar , H. , Silver , D. , Gagvani , N. , and Dickinson , S . 2003. Skeleton based shape matching and retrieval . In Proceedings of the IEEE Conference on Shape Modeling and Applications (SMI'03) . Vol. 130 . Sundar, H., Silver, D., Gagvani, N., and Dickinson, S. 2003. Skeleton based shape matching and retrieval. In Proceedings of the IEEE Conference on Shape Modeling and Applications (SMI'03). Vol. 130.
Tal , A. , Elad , M. , and Ar , S . 2001. Content based retrieval of VRML objects - An iterative and interactive approach . In Proceedings of the Eurographics Workshop on Multimedia. Tal, A., Elad, M., and Ar, S. 2001. Content based retrieval of VRML objects - An iterative and interactive approach. In Proceedings of the Eurographics Workshop on Multimedia.
Thorstensen , N. and Keriven , R . 2009. Non-Rigid shape matching using geometry and photometry . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'09) . Thorstensen, N. and Keriven, R. 2009. Non-Rigid shape matching using geometry and photometry. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'09).
Tola , E. , Lepetit , V. , and Fua , P . 2008. A fast local descriptor for dense matching . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'08) . Tola, E., Lepetit, V., and Fua, P. 2008. A fast local descriptor for dense matching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'08).
Toldo , R. , Castellani , U. , and Fusiello , A . 2009. Visual vocabulary signature for 3D object retrieval and partial matching . In Proceedings of the 3DOR Conference. Toldo, R., Castellani, U., and Fusiello, A. 2009. Visual vocabulary signature for 3D object retrieval and partial matching. In Proceedings of the 3DOR Conference.
Torralba , A. , Fergus , R. , and Weiss , Y . 2008. Small codes and large image databases for recognition . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'08) . Torralba, A., Fergus, R., and Weiss, Y. 2008. Small codes and large image databases for recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'08).
Veltkamp R. C. and Hagedoorn M. 2001. State of the art in shape matching. In Principles of Visual Information Retrieval. Veltkamp R. C. and Hagedoorn M. 2001. State of the art in shape matching. In Principles of Visual Information Retrieval.
Wang C. Bronstein M. M. and Paragios N. 2010. Discrete minimum-distortion correspondence problems in non-rigid shape analysis. Res. rep. 7333 INRIA. Wang C. Bronstein M. M. and Paragios N. 2010. Discrete minimum-distortion correspondence problems in non-rigid shape analysis. Res. rep. 7333 INRIA.
Wardetzky , M. , Mathur , S. , Kalberer , F. , and Grinspun , E . 2008. Discrete Laplace operators: No free lunch . In Conference on Computer Graphics and Interactive Techniques. Wardetzky, M., Mathur, S., Kalberer, F., and Grinspun, E. 2008. Discrete Laplace operators: No free lunch. In Conference on Computer Graphics and Interactive Techniques.
Zaharescu , A. , Boyer , E. , Varanasi , K. , and Horaud , R . 2009. Surface feature detection and description with applications to mesh matching . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'09) . Zaharescu, A., Boyer, E., Varanasi, K., and Horaud, R. 2009. Surface feature detection and description with applications to mesh matching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'09).
Zeng , Y. , Wang , C. , Wang , Y. , Gu , X. , Samaras , D. , and Paragios , N . 2010. Dense non-rigid surface registration using high-order graph matching . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'10) . Zeng, Y., Wang, C., Wang, Y., Gu, X., Samaras, D., and Paragios, N. 2010. Dense non-rigid surface registration using high-order graph matching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'10).
Zhang , C. and Chen , T . 2001. Efficient feature extraction for 2D/3D objects in mesh representation . In Proceedings of the IEEE International Conference on Image Processing (ICIP'01) . Vol. 3 . Zhang, C. and Chen, T. 2001. Efficient feature extraction for 2D/3D objects in mesh representation. In Proceedings of the IEEE International Conference on Image Processing (ICIP'01). Vol. 3.
Zhang , H. 2004 . Discrete combinatorial Laplacian operators for digital geometry processing . In Proceedings of the Conference on Geometric Design. 575--592 . Zhang, H. 2004. Discrete combinatorial Laplacian operators for digital geometry processing. In Proceedings of the Conference on Geometric Design. 575--592.