3D corrective nose reconstruction from a single image
Tóm tắt
There is a steadily growing range of applications that can benefit from facial reconstruction techniques, leading to an increasing demand for reconstruction of high-quality 3D face models. While it is an important expressive part of the human face, the nose has received less attention than other expressive regions in the face reconstruction literature. When applying existing reconstruction methods to facial images, the reconstructed nose models are often inconsistent with the desired shape and expression. In this paper, we propose a coarse-to-fine 3D nose reconstruction and correction pipeline to build a nose model from a single image, where 3D and 2D nose curve correspondences are adaptively updated and refined. We first correct the reconstruction result coarsely using constraints of 3D-2D sparse landmark correspondences, and then heuristically update a dense 3D-2D curve correspondence based on the coarsely corrected result. A final refinement step is performed to correct the shape based on the updated 3D-2D dense curve constraints. Experimental results show the advantages of our method for 3D nose reconstruction over existing methods.
Tài liệu tham khảo
Samad, M. D.; Iftekharuddin, K. M. Frenet frame-based generalized space curve representation for pose-invariant classification and recognition of 3-D face. IEEE Transactions on Human-Machine Systems Vol. 46, No. 4, 522–533, 2016.
Werghi, N.; Tortorici, C.; Berretti, S.; Del Bimbo, A. Boosting 3D LBP-based face recognition by fusing shape and texture descriptors on the mesh. IEEE Transactions on Information Forensics and Security Vol. 11, No. 5, 964–979, 2016.
Li, H.; Yu, J. H.; Ye, Y. T.; Bregler, C. Realtime facial animation with on-the-fly correctives. ACM Transactions on Graphics Vol. 32, No. 4, Article No. 42, 2013.
Li, Y.; Ma, L. Q.; Fan, H. Q.; Mitchell, K. Feature-preserving detailed 3D face reconstruction from a single image. In: Proceedings of the 15th ACM SIGGRAPH European Conference on Visual Media Production, Article No. 1, 2018.
Jiang, L.; Zhang, J. Y.; Deng, B. L.; Li, H.; Liu, L. G. 3D face reconstruction with geometry details from a single image. IEEE Transactions on Image Processing Vol. 27, No. 10, 4756–4770, 2018.
Tang, Y. L.; Han, X. G.; Li, Y.; Ma, L. Q.; Tong, R. F. Expressive facial style transfer for personalized memes mimic. The Visual Computer Vol. 35, 783–795, 2019.
Kass, M.; Witkin, A.; Terzopoulos, D. Snakes: Active contour models. International Journal of Computer Vision Vol. 1, No. 4, 321–331, 1988.
Tang, Y. L.; Tong, R. F.; Tang, M.; Zhang, Y. Depth incorporating with color improves salient object detection. The Visual Computer Vol. 32, 111–121, 2016.
Blanz, V.; Vetter, T. A morphable model for the synthesis of 3D faces. In: Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, 187–194, 1999.
Paysan, P.; Knothe, R.; Amberg, B.; Romdhani, S.; Vetter, T. A 3D face model for pose and illumination invariant face recognition. In: Proceedings of the 6th IEEE International Conference on Advanced Video and Signal Based Surveillance, 296–301, 2009.
Booth, J.; Roussos, A.; Zafeiriou, S.; Ponniah, A.; Dunaway, D. A 3D morphable model learnt from 10,000 faces. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 5543–5552, 2016.
Zhu, X. Y.; Lei, Z.; Liu, X. M.; Shi, H. L.; Li, S. Z. Face alignment across large poses: A 3D solution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 146–155, 2016.
Cao, C.; Weng, Y. L.; Zhou, S.; Tong, Y. Y.; Zhou, K. FaceWarehouse: A 3D facial expression database for visual computing. IEEE Transactions on Visualization and Computer Graphics Vol. 20, No. 3, 413–425, 2014.
Feng, Y.; Wu, F.; Shao, X. H.; Wang, Y. F.; Zhou, X. Joint 3D face reconstruction and dense alignment with position map regression network. In: Computer Vision-ECCV 2018. Lecture Notes in Computer Science, Vol.11218. Ferrari, V.; Hebert, M.; Sminchisescu, C.; Weiss, Y. Eds. Springer Cham, 557–574, 2018.
Sanyal, S.; Bolkart, T.; Feng, H. W.; Black, M. J. Learning to regress 3D face shape and expression from an image without 3D supervision. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 7755–7764, 2019.
Deng, Y.; Yang, J. L.; Xu, S. C.; Chen, D.; Jia, Y. D.; Tong, X. Accurate 3D face reconstruction with weakly-supervised learning: From single image to image set. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 285–295, 2019.
Garrido, P.; Zollhöfer, M.; Casas, D.; Valgaerts, L.; Varanasi, K.; Pérez, P.; Theobalt, C. Reconstruction of personalized 3D face rigs from monocular video. ACM Transactions on Graphics Vol. 35, No. 3, Article No. 28, 2016.
Sorkine, O.; Cohen-Or, D.; Lipman, Y.; Alexa, M.; Rössl, C.; Seidel, H. P. Laplacian surface editing. In: Proceedings of the Eurographics/ACM SIGGRAPH symposium on Geometry processing, 175–184, 2004.
Bouaziz, S.; Wang, Y. G.; Pauly, M. Online modeling for realtime facial animation. ACM Transactions on Graphics Vol. 32, No. 4, Article No. 40, 2013.
Vallet, B.; Lévy, B. Spectral geometry processing with manifold harmonics. Computer Graphics Forum Vol. 27, No. 2, 251–260, 2008.
Wen, Q.; Xu, F.; Lu, M.; Yong, J. H. Real-time 3D eyelids tracking from semantic edges. ACM Transactions on Graphics Vol. 36, No. 6, Article No. 193, 2017.
Garrido, P.; Zollhöfer, M.; Wu, C. L.; Bradley, D.; Pérez, P.; Beeler, T.; Theobalt, C. Corrective 3D reconstruction of lips from monocular video. ACM Transactions on Graphics Vol. 35, No. 6, Article No. 219, 2016.
Dinev, D.; Beeler, T.; Bradley, D.; Bächer, M.; Xu, H.; Kavan, L. User-guided lip correction for facial performance capture. Computer Graphics Forum Vol. 37, No. 8, 93–101, 2018.
Feng, Z. H.; Huber, P.; Kittler, J.; Hancock, P.; Wu, X. J.; Zhao, Q. J.; Koppen, P.; Raetsch, M. Evaluation of dense 3D reconstruction from 2D face images in the wild. In: Proceedings of the 13th IEEE International Conference on Automatic Face & Gesture Recognition, 780–786, 2018.
Yin, L. J.; Wei, X. Z.; Sun, Y.; Wang, J.; Rosato, M. J. A 3D facial expression database for facial behavior research. In: Proceedings of the 7th International Conference on Automatic Face and Gesture Recognition, 211–216, 2006.
Thies, J.; Zollhöfer, M.; Stamminger, M.; Theobalt, C.; Nießner, M. Face2Face: Real-time face capture and reenactment of RGB videos. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2387–2395, 2016.
Canny, J. A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. PAMI-8, No. 6, 679–698, 1986.