DualSmoke: Sketch-based smoke illustration design with two-stage generative model
Springer Science and Business Media LLC - Trang 1-15
Tóm tắt
The dynamic effects of smoke are impressive in illustration design, but it is a troublesome and challenging issue for inexpert users to design smoke effects without domain knowledge of fluid simulations. In this work, we propose DualSmoke, a two-stage global-to-local generation framework for interactive smoke illustration design. In the global stage, the proposed approach utilizes fluid patterns to generate Lagrangian coherent structures from the user’s hand-drawn sketches. In the local stage, detailed flow patterns are obtained from the generated coherent structure. Finally, we apply a guiding force field to the smoke simulator to produce the desired smoke illustration. To construct the training dataset, DualSmoke generates flow patterns using finite-time Lyapunov exponents of the velocity fields. The synthetic sketch data are generated from the flow patterns by skeleton extraction. Our user study verifies that the proposed design interface can provide various smoke illustration designs with good user usability. Our code is available at https://githubcom/shasph/DualSmoke.
Tài liệu tham khảo
Kazi, R. H.; Chevalier, F.; Grossman, T.; Zhao, S. D.; Fitzmaurice, G. Draco: Bringing life to illustrations with kinetic textures. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 351–360, 2014.
Xing, J.; Kazi, R. H.; Grossman, T.; Wei, L. Y.; Stam, J.; Fitzmaurice, G. Energy-brushes: Interactive tools for illustrating stylized elemental dynamics. In: Proceedings of the 29th Annual Symposium on User Interface Software and Technology, 755–766, 2016.
citation_journal_title=ACM Transactions on Graphics; citation_title=Sketch-based dynamic illustration of fluid systems; citation_author=B Zhu, M Iwata, R Haraguchi, T Ashihara, N Umetani, T Igarashi, K Nakazawa; citation_volume=30; citation_issue=6; citation_publication_date=2011; citation_pages=1-8; citation_doi=10.1145/2070781.2024168; citation_id=CR3
citation_journal_title=Computer Animation and Virtual Worlds; citation_title=Sketch2VF: Sketch-based flow design with conditional generative adversarial network; citation_author=Z Y Hu, H R Xie, T Fukusato, T Sato, T Igarashi; citation_volume=30; citation_issue=3–4; citation_publication_date=2019; citation_pages=e1889; citation_doi=10.1002/cav.1889; citation_id=CR4
Rasmussen, N.; Enright, D.; Nguyen, D.; Marino, S.; Sumner, N.; Geiger, W.; Hoon, S.; Fedkiw, R. Directable photorealistic liquids. In: Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 193–202, 2004.
citation_journal_title=ACM Transactions on Graphics; citation_title=Keyframe control of smoke simulations; citation_author=A Treuille, A McNamara, Z Popović, J Stam; citation_volume=22; citation_issue=3; citation_publication_date=2003; citation_pages=716-723; citation_doi=10.1145/882262.882337; citation_id=CR6
citation_journal_title=Computer Graphics Forum; citation_title=Improved variational guiding of smoke animations; citation_author=M B Nielsen, B B Christensen; citation_volume=29; citation_issue=2; citation_publication_date=2010; citation_pages=705-712; citation_doi=10.1111/j.1467-8659.2009.01640.x; citation_id=CR7
Sato, S.; Dobashi, Y.; Kim, T. Stream-guided smoke simulations. ACM Transactions on Graphics Vol. 40, No. 4, Article No. 161, 2021.
Kim, Y.; Machiraju, R.; Thompson, D. Path-based control of smoke simulations. In: Proceedings of the ACM SIGGRAPH/Eurographics symposium on Computer animation, 33–42, 2006.
citation_journal_title=Computer Graphics Forum; citation_title=Deep reconstruction of 3D smoke densities from artist sketches; citation_author=B Kim, X C Huang, L Wuelfroth, J W Tang, G Cordonnier, M Gross, B Solenthaler; citation_volume=41; citation_issue=2; citation_publication_date=2022; citation_pages=97-110; citation_doi=10.1111/cgf.14461; citation_id=CR10
citation_journal_title=Computer Graphics Forum; citation_title=A survey on sketch based content creation: From the desktop to virtual and augmented reality; citation_author=S Bhattacharjee, P Chaudhuri; citation_volume=39; citation_issue=2; citation_publication_date=2020; citation_pages=757-780; citation_doi=10.1111/cgf.14024; citation_id=CR11
Igarashi, T.; Matsuoka, S.; Tanaka, H. Teddy: A sketching interface for 3D freeform design. In: Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, 409–416, 1999.
Guérin, É.; Digne, J.; Galin, É.; Peytavie, A.; Wolf, C.; Benes, B.; Martinez, B. Interactive example-based terrain authoring with conditional generative adversarial networks. ACM Transactions on Graphics Vol. 36, No. 6, Article No. 228, 2017.
Brodt, K.; Bessmeltsev, M. Sketch2Pose: Estimating a 3D character pose from a bitmap sketch. ACM Transactions on Graphics Vol. 41, No. 4, Article No. 85, 2022.
citation_journal_title=Computer Graphics Forum; citation_title=Controllable dendritic crystal simulation using orientation field; citation_author=B Ren, J H Huang, M C Lin, S M Hu; citation_volume=37; citation_issue=2; citation_publication_date=2018; citation_pages=485-495; citation_doi=10.1111/cgf.13378; citation_id=CR15
Guay, M.; Ronfard, R.; Gleicher, M.; Cani, M. P. Space-time sketching of character animation. ACM Transactions on Graphics Vol. 34, No. 4, Article No. 118, 2015.
Wu, L. Q.; Liu, S. G.; Yu, Z. J.; Sun, H. Q.; Zhang, M. Dynamic fluids mixed with local-control effects. In: Proceedings of the 7th International Conference on Image and Graphics, 694–700, 2013.
Yan, G. W.; Chen, Z. L.; Yang, J. M.; Wang, H. M. Interactive liquid splash modeling by user sketches. ACM Transactions on Graphics Vol. 39, No. 6, Article No. 165, 2020.
citation_journal_title=Computational Visual Media; citation_title=DualFace: Two-stage drawing guidance for freehand portrait sketching; citation_author=Z Y Huang, Y C Peng, T Hibino, C Q Zhao, H R Xie, T Fukusato, K Miyata; citation_volume=8; citation_issue=1; citation_publication_date=2022; citation_pages=63-77; citation_doi=10.1007/s41095-021-0227-7; citation_id=CR19
Peng, Y. C.; Zhao, C. Q.; Huang, Z. Y.; Fukusato, T.; Xie, H. R.; Miyata, K. Two-stage motion editing interface for character animation. In: Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 1–2, 2021.
Stam, J. Stable fluids. In: Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, 121–128, 1999.
Fedkiw, R.; Stam, J.; Jensen, H. W. Visual simulation of smoke. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, 15–22, 2001.
citation_journal_title=ACM Transactions on Graphics; citation_title=A vortex particle method for smoke, water and explosions; citation_author=A Selle, N Rasmussen, R Fedkiw; citation_volume=24; citation_issue=3; citation_publication_date=2005; citation_pages=910-914; citation_doi=10.1145/1073204.1073282; citation_id=CR23
citation_journal_title=ACM Transactions on Graphics; citation_title=Feedback control of cumuliform cloud formation based on computational fluid dynamics; citation_author=Y Dobashi, K Kusumoto, T Nishita, T Yamamoto; citation_volume=27; citation_issue=3; citation_publication_date=2008; citation_pages=1-8; citation_doi=10.1145/1360612.1360693; citation_id=CR24
citation_journal_title=Computer Graphics Forum; citation_title=Honey, I shrunk the domain: Frequency-aware force field reduction for efficient fluids optimization; citation_author=J W Tang, V C Azevedo, G Cordonnier, B Solenthaler; citation_volume=40; citation_issue=2; citation_publication_date=2021; citation_pages=339-353; citation_doi=10.1111/cgf.142637; citation_id=CR25
citation_journal_title=ACM Transactions on Graphics; citation_title=Pattern-guided smoke animation with Lagrangian coherent structure; citation_author=Z Yuan, F Chen, Y Zhao; citation_volume=30; citation_issue=6; citation_publication_date=2011; citation_pages=1-8; citation_doi=10.1145/2070781.2024170; citation_id=CR26
Huang, R. G.; Melek, Z.; Keyser, J. Preview-based sampling for controlling gaseous simulations. In: Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 177–186, 2011.
Ladický, L.; Jeong, S.; Solenthaler, B.; Pollefeys, M.; Gross, M. Data-driven fluid simulations using regression forests. ACM Transactions on Graphics Vol. 34, No. 6, Article No. 199, 2015.
Chu, M. Y.; Thuerey, N. Data-driven synthesis of smoke flows with CNN-based feature descriptors. ACM Transactions on Graphics Vol. 36, No. 4, Article No. 69, 2017.
Xie, Y.; Franz, E.; Chu, M. Y.; Thuerey, N. tempoGAN: A temporally coherent, volumetric GAN for superresolution fluid flow. ACM Transactions on Graphics Vol. 37, No. 4, Article No. 95, 2018.
Chu, M. Y.; Thuerey, N.; Seidel, H. P.; Theobalt, C.; Zayer, R. Learning meaningful controls for fluids. ACM Transactions on Graphics Vol. 40, No. 4, Article No. 100, 2021.
citation_journal_title=Journal of Scientific Computing; citation_title=An unconditionally stable MacCormack method; citation_author=A Selle, R Fedkiw, B Kim, Y J Liu, J Rossignac; citation_volume=35; citation_issue=2–3; citation_publication_date=2008; citation_pages=350-371; citation_doi=10.1007/s10915-007-9166-4; citation_id=CR32
citation_journal_title=Physica D: Nonlinear Phenomena; citation_title=Definition and properties of Lagrangian coherent structures from finite-time Lyapunov exponents in two-dimensional aperiodic flows; citation_author=S C Shadden, F Lekien, J E Marsden; citation_volume=212; citation_issue=3–4; citation_publication_date=2005; citation_pages=271-304; citation_doi=10.1016/j.physd.2005.10.007; citation_id=CR33
citation_journal_title=IEEE Transactions on Visualization and Computer Graphics; citation_title=Interactive separating streak surfaces; citation_author=F Ferstl, K Burger, H Theisel, R Westermann; citation_volume=16; citation_issue=6; citation_publication_date=2010; citation_pages=1569-1577; citation_doi=10.1109/TVCG.2010.169; citation_id=CR34
citation_journal_title=Computer Graphics Forum; citation_title=Mean curvature skeletons; citation_author=A Tagliasacchi, I Alhashim, M Olson, H Zhang; citation_volume=31; citation_issue=5; citation_publication_date=2012; citation_pages=1735-1744; citation_doi=10.1111/j.1467-8659.2012.03178.x; citation_id=CR35
Au, O. K. C.; Tai, C. L.; Chu, H. K.; Cohen-Or, D.; Lee, T. Y. Skeleton extraction by mesh contraction. In: Proceedings of the ACM SIGGRAPH Papers, 1–10, 2008.
citation_journal_title=Computers & Graphics; citation_title=2D skeleton extraction based on heat equation; citation_author=F Y Gao, G S Wei, S Q Xin, S S Gao, Y F Zhou; citation_volume=74; citation_publication_date=2018; citation_pages=99-108; citation_doi=10.1016/j.cag.2018.05.005; citation_id=CR37
Isola, P.; Zhu, J. Y.; Zhou, T. H.; Efros, A. A. Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 5967–5976, 2017.
citation_title=U-net: Convolutional networks for biomedical image segmentation; citation_inbook_title=Medical Image Computing and Computer-Assisted Intervention - MICCAI 2015; citation_publication_date=2015; citation_pages=234-241; citation_id=CR39; citation_author=O Ronneberger; citation_author=P Fischer; citation_author=T Brox; citation_publisher=Springer
Pfaff, T.; Thuerey, N. mantaflow. Available at
http://mantaflow.com/
Brooke, J. SUS: A’ quick and dirty’ usability scale. In: Usability Evaluation in Industry. CRC Press, 207–212, 1996.
citation_journal_title=Advances in Psychology; citation_title=Development of NASATLX (task load index): Results of empirical and theoretical research; citation_author=S G Hart, L E Staveland; citation_volume=52; citation_publication_date=1988; citation_pages=139-183; citation_doi=10.1016/S0166-4115(08)62386-9; citation_id=CR42
citation_journal_title=Journal of Usability Studies; citation_title=Determining what individual SUS scores mean: Adding an adjective rating scale; citation_author=A Bangor, P Kortum, J T Miller; citation_volume=4; citation_issue=3; citation_publication_date=2009; citation_pages=114-123; citation_id=CR43
Su, W. C.; Du, D.; Yang, X.; Zhou, S. Z.; Fu, H. B. Interactive sketch-based normal map generation with deep neural networks. Proceedings of the ACM on Computer Graphics and Interactive Techniques Vol. 1, No. 1, Article No. 22, 2018.
He, Y.; Xie, H. R.; Zhang, C.; Yang, X.; Miyata, K. Sketch-based normal map generation with geometric sampling. In: Proceedings of the SPIE 11766, International Workshop on Advanced Imaging Technology, 261–266, 2021.
citation_journal_title=IEEE Transactions on Visualization and Computer Graphics; citation_title=Efficient computation and visualization of coherent structures in fluid flow applications; citation_author=C Garth, F Gerhardt, X Tricoche, H Hans; citation_volume=13; citation_issue=6; citation_publication_date=2007; citation_pages=1464-1471; citation_doi=10.1109/TVCG.2007.70551; citation_id=CR46