CTSN: Predicting cloth deformation for skeleton-based characters with a two-stream skinning network

Springer Science and Business Media LLC
Yudi Li1, Min Tang1, Yun Yang1, Ruofeng Tong1, Shuangcai Yang2, Yao Li2, Bing An2, Qilong Kou2
1College of Computer Science and Technology, Zhejiang University, Hangzhou, 310058, China
2Aurora Studios, Tencent, Shenzhen, 518057, China

Tóm tắt

AbstractWe present a novel learning method using a two-stream network to predict cloth deformation for skeleton-based characters. The characters processed in our approach are not limited to humans, and can be other targets with skeleton-based representations such as fish or pets. We use a novel network architecture which consists of skeleton-based and mesh-based residual networks to learn the coarse features and wrinkle features forming the overall residual from the template cloth mesh. Our network may be used to predict the deformation for loose or tight-fitting clothing. The memory footprint of our network is low, thereby resulting in reduced computational requirements. In practice, a prediction for a single cloth mesh for a skeleton-based character takes about 7 ms on an nVidia GeForce RTX 3090 GPU. Compared to prior methods, our network can generate finer deformation results with details and wrinkles.

Từ khóa


Tài liệu tham khảo

Baraff, D.; Witkin, A. Large steps in cloth simulation. In: Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques, 43–54, 1998.

Provot, X. Deformation constraints in a mass-spring model to describe rigid cloth behavior. In: Proceedings of the Graphics Interface, 147–154, 1995.

Bouaziz, S.; Martin, S.; Liu, T.; Kavan, L.; Pauly, M. Projective dynamics: Fusing constraint projections for fast simulation. ACM Transactions on Graphics Vol. 33, No. 4, Article No. 154, 2014.

Bridson, R.; Fedkiw, R.; Anderson, J. Robust treatment of collisions, contact and friction for cloth animation. ACM Transactions on Graphics Vol. 21, No. 3, 594–603, 2002.

Harmon, D.; Vouga, E.; Tamstorf, R.; Grinspun, E. Robust treatment of simultaneous collisions. ACM Transactions on Graphics Vol. 27, No. 3, 1–4, 2008.

Tang, M.; Tong, R. F.; Wang, Z. D.; Manocha, D. Fast and exact continuous collision detection with Bernstein sign classification. ACM Transactions on Graphics Vol. 33, No. 6, Article No. 186, 2014.

Tang, M.; Wang, T.; Liu, Z.; Tong, R.; Manocha, D. I-cloth: Incremental collision handling for GPU-based interactive cloth simulation. ACM Transactions on Graphics Vol. 37, No. 6, Article No. 204, 2018.

Patel, C.; Liao, Z.; Pons-Moll, G. TailorNet: Predicting clothing in 3D as a function of human pose, shape and garment style. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 7363–7373, 2020.

Bertiche, H.; Madadi, M.; Tylson, E.; Escalera, S. DeePSD: Automatic deep skinning and pose space deformation for 3D garment animation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 5451–5460, 2021.

Santesteban, I.; Otaduy, M. A.; Casas, D. Learning-based animation of clothing for virtual try-on. Computer Graphics Forum Vol. 38, No. 2, 355–366, 2019.

Loper, M.; Mahmood, N.; Romero, J.; Pons-Moll, G.; Black, M. J. SMPL: A skinned multi-person linear model. ACM Transactions on Graphics Vol. 34, No. 6, Article No. 248, 2015.

Wang, T. Y.; Shao, T. J.; Fu, K.; Mitra, N. J. Learning an intrinsic garment space for interactive authoring of garment animation. ACM Transactions on Graphics Vol. 38, No. 6, Article No. 220, 2019.

Gundogdu, E.; Constantin, V.; Seifoddini, A.; Dang, M.; Salzmann, M.; Fua, P. GarNet: A two-stream network for fast and accurate 3D cloth draping. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 8738–8747, 2019.

Gundogdu, E.; Constantin, V.; Parashar, S.; Seifoddini, A.; Dang, M.; Salzmann, M.; Fua, P. GarNet++: Improving fast and accurate static 3D cloth draping by curvature loss. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 44, No. 1, 181–195, 2022.

Vidaurre, R.; Santesteban, I.; Garces, E.; Casas, D. Fully convolutional graph neural networks for parametric virtual try-on. Computer Graphics Forum Vol. 39, No. 8, 145–156, 2020.

Tang, M.; Wang, H. M.; Tang, L.; Tong, R. F.; Manocha, D. CAMA: Contact-aware matrix assembly with unified collision handling for GPU-based cloth simulation. Computer Graphics Forum Vol. 35, No. 2, 511–521, 2016.

Li, C.; Tang, M.; Tong, R.; Cai, M.; Zhao, J.; Manocha, D. P-cloth: Interactive complex cloth simulation on multi-GPU systems using dynamic matrix assembly and pipelined implicit integrators. ACM Transactions on Graphics Vol. 39, No. 6, Article No. 180, 2020.

Li, T.; Shi, R.; Kanai, T. Detail-aware deep clothing animations infused with multi-source attributes. Computer Graphics Forum Vol. 42, 231–244, 2023.

Kavan, L.; Collins, S.; Žára, J.; O’Sullivan, C. Skinning with dual quaternions. In: Proceedings of the Symposium on Interactive 3D Graphics and Games, 39–46, 2007.

Santesteban, I.; Thuerey, N.; Otaduy, M. A.; Casas, D. Self-supervised collision handling via generative 3D garment models for virtual try-on. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 11758–11768, 2021.

Holden, D.; Duong, B. C.; Datta, S.; Nowrouzezahrai, D. Subspace neural physics: Fast data-driven interactive simulation. In: Proceedings of the 18th Annual ACM SIGGRAPH/Eurographics Symposium on Computer Animation, Article No. 6, 2019.

Li, Y.; Tang, M.; Yang, Y.; Huang, Z.; Tong, R.; Yang, S.; Li, Y.; Manocha, D. N-cloth: Predicting 3D cloth deformation with mesh-based networks. Computer Graphics Forum Vol. 41, 547–558, 2022.

Tan, Q. Y.; Pan, Z. R.; Manocha, D. Lcollision: Fast generation of collision-free human poses using learned non-penetration constraints. Proceedings of the AAAI Conference on Artificial Intelligence Vol. 35, No. 5, 3913–3921, 2021.

Tan, Q.; Pan, Z.; Smith, B.; Shiratori, T.; Manocha, D. Active learning of neural collision handler for complex 3D mesh deformations. arXiv preprint arXiv:2110.07727, 2021.

Bertiche, H.; Madadi, M.; Escalera, S. PBNS: Physically based neural simulation for unsupervised garment pose space deformation ACM Transactions on Graphics Vol. 40, No. 610, Article No. 198, 2021.

Pan, X. Y.; Huang, J. C.; Mai, J. M.; Wang, H.; Li, H. L.; Su, T. K.; Wang, W. J.; Jin, X. G. HeterSkinNet: A heterogeneous network for skin weights prediction. Proceedings of the ACM on Computer Graphics and Interactive Techniques Vol. 4, No. 1, Article No. 10, 2021.

Xu, Z.; Zhou, Y.; Kalogerakis, E.; Landreth, C.; Singh, K. Rignet: Neural rigging for articulated characters. arXiv preprint arXiv:2005.00559, 2020.

Liu, L.; Zheng, Y.; Tang, D.; Yuan, Y.; Fan, C.; Zhou, K. NeuroSkinning: Automatic skin binding for production characters with deep graph networks. ACM Transactions on Graphics Vol. 38, No. 4, Article No. 114, 2019.

Magnenat-Thalmann, N.; Laperrière, R.; Thalmann, D. Joint-dependent local deformations for hand animation and object grasping. In: Proceedings on Graphics Interface, 26–33, 1989.

Shi, Y.; Huang, Z.; Feng, S.; Zhong, H.; Wang, W.; Sun, Y. Masked label prediction: Unified message passing model for semi-supervised classification. arXiv preprint arXiv:2009.03509, 2020.

Narain, R.; Samii, A.; O’Brien, J. F. Adaptive anisotropic remeshing for cloth simulation. ACM Transactions on Graphics Vol. 31, No. 6, Article No. 152, 2012.

Narain, R.; Pfaff, T.; O’Brien, J. F. Folding and crumpling adaptive sheets. ACM Transactions on Graphics Vol. 32, No. 4, Article No. 51, 2013.

Pfaff, T.; Narain, R.; de Joya, J. M.; O’Brien, J. F. Adaptive tearing and cracking of thin sheets. ACM Transactions on Graphics Vol. 33, No. 4, Article No. 110, 2014.

Kingma, D. P., Ba, J. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.

Guan, P.; Reiss, L.; Hirshberg, D. A.; Weiss, A.; Black, M. J. DRAPE. ACM Transactions on Graphics Vol. 31, No. 4, Article No. 35, 2012.

Santesteban, I.; Otaduy, M. A.; Casas, D. SNUG: Self-supervised neural dynamic garments. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8130–8140, 2022.

Thông tin
Thông tin xuất bản

Springer Science and Business Media LLC

Thông tin tác giả