3D face recognition: a survey
Tóm tắt
3D face recognition has become a trending research direction in both industry and academia. It inherits advantages from traditional 2D face recognition, such as the natural recognition process and a wide range of applications. Moreover, 3D face recognition systems could accurately recognize human faces even under dim lights and with variant facial positions and expressions, in such conditions 2D face recognition systems would have immense difficulty to operate. This paper summarizes the history and the most recent progresses in 3D face recognition research domain. The frontier research results are introduced in three categories: pose-invariant recognition, expression-invariant recognition, and occlusion-invariant recognition. To promote future research, this paper collects information about publicly available 3D face databases. This paper also lists important open problems.
Từ khóa
Tài liệu tham khảo
Xu C, Wang Y, Tan T, Quan L (2004) Depth vs. intensity: which is more important for face recognition? Int Conf Pattern Recognit 4:342–345
Bowyer KW, Chang PFK (2006) A survey of approaches and challenges in 3d and multi-modal 3d + 2d face recognition. Comput Vis Image Underst 101:1–15
Zhu X, Lei Z, Yan J, Yi D, Li SZ (2015) High-fidelity pose and expression normalization for face recognition in the wild. Computer vision and pattern recognition. pp 787–796
Patil H, Kothari KBA (2015) 3-d face recognition: features, databases, algorithms and challenges. Artif Intell Rev 44:393–441
Zaharescu A, Boyer E, Varanasi K, Horaud R (2009) Surface feature detection and description with applications to mesh matching. In: Proc. IEEE Conf. on Comput
Kittler J, Hilton MH (2005) A survey of 3d imaging, modelling and recognition approachest. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops. pp 114–114
Bennamoun M, Guo Y, Sohel F (2015) Feature selection for 2d and 3d face recognition. Research Gate, vol 17
Guo Y, Zhang MLJWYM J (2014) Benchmark datasets for 3d computer vision. In: Industrial electronics and applications. pp 1846–1851
Soltanpour S, Boufama QJWB (2017) A survey of local feature methods for 3d face recognition. Pattern Recognit 72:391–406
Zhu J, San-Segundo R, Pardo JM (2017) Feature extraction for robust physical activity recognition. Human-centric Comput Inform Sci 7(1):16
Huang D, Zhang G, Ardabilian M, Wang Y, Chen L (2010) 3d face recognition using distinctiveness enhanced facial representations and local feature hybrid matching. In: Biometrics: theory, applications and systems
Bledsoe WW (1966) The model method in facial recognition. Panoramic Research
Pentland MT (1991) Face recognition using eigenfaces. Computer vision and pattern recognition. pp 586–591
Belhumeur PN, Hespanha DJKJP (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. Trans Pattern Anal Mach Intell 19:711–720
Frey BJ, Colmenarez TSH A (1998) Mixtures of local linear subspaces for face recognition. In: Computer vision and pattern recognition
Wiskott L, Fellous JM, Kruger N, Von Der Malsburg C (1997) Face recognition by elasric bunch graph matching. PAMI 17:775–779
Mpiperis I, Malassiotis MGSS (2007) 3-d face recognition with the geodesic polar representation. Inform Forensics Secur 5:537–547
Abate AF, Nappi DRGSM (2007) 2d and 3d face recognition: a survey. Pattern Recognit Lett 28:1885–1906
Cartoux JY, LaPreste JT, Richetin M (1989) Face authentication or recognition by profile extraction from range images. Interpretation of 3D Scenes. pp 194–199
Gordon G (1995) Face recognition from frontal and profile views. In: International workshop on automatic face and gesture recognition pp 47–52
Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems. pp 1106–1114
Kim D, Hernandez JC M, Medioni, G (2017) Deep 3d face identification. arXiv:1703.10714 http://arxiv.org/abs/1703.10714
Gilani SZ, Mian A (2017) Learning from millions of 3d scans for large-scale 3d face recognition. ArXiv e-prints. arXiv:1711.05942
Prabhu U, Heo MSJ (2011) Unconstrained poseinvariant face recognition using 3d generic elastic models. Pattern Anal Mach Intell 33:1952–1961
Blanz V, Vetter T (1999) A morphable model for the synthesis of 3d faces. In: Computer graphics, annual conference series. pp 187–194
Lee KC, Ho J, Yang MH, Kriegman D (2003) Video-based face recognition using probabilistic appearance manifolds. In: Computer vision and pattern recognition
Hu Y, Jiang D, Yan S, Zhang L (2004) Automatic 3d reconstruction for face recognition. In: FG pp 843–848
Arandjelovic O, Shakhnarovich G, Fisher J, Cipolla R, Darrell T (2005) Face recognition with image sets using manifold density divergence. In: Computer vision and pattern recognition pp 581–588
Vetter VBT (2003) Face recognition based on fitting a 3d morphable model. Pattern Anal Mach Intell 25:1063–1074
Pan G, Wu YPZ (2003) Automatic 3d face verification from range data. Acoustics Speech Sign Process 3:193–196
Chua CS, Han F, Ho YK (2000) 3d human face recognition using point signature. In: Autom. Face Gesture Recogn. pp 233–238
Hesher C, Srivastava A, Erlebacher G (2003) A novel technique for face recognition using range imaging. In: Signal processing and its applications. pp 201–204
Min J, Flynn PJ, Bowyer KW (2003) Using multiple gallery and probe images per person to improve performance of face recognition. In: Notre Dame computer science and engineering technical report
Moreno AB, Sanchez A, Velez JF, Diaz FJ (2003) Face recognition using 3d surface extracted descriptors. In: Proceedings of the Irish machine vision and image processing
Martínez AM (2000) Recognition of partially occluded and/or imprecisely localized faces using a probabilistic approach. In: Computer vision and pattern recognition. pp 712–717
Tan X, Chen ZHZFZS (2005) Recognizing partially occluded, expression variant faces from single training image per person with som and soft k-nn ensemble. Vision 16:875–886
Colombo A, Cusano C, Schettini R (2006) Detection and restoration of occlusions for 3d face recognition. In: Multimedia and Expo. pp 1541–1544
Phillips PJ, Flynn PJ, Scruggs T, Bowyer KW, Chang J, Hoffman K, Marques J, Min J, Worek W (2005) Overview of face recognition grand challenge. In: Computer vision and pattern recognition. pp 947–954
Yin L, Wei X, Sun Y, Wang J, Rosato MJ (2006) A 3d facial expression database for facial behavior research. In: Automatic face and gesture recognition. pp 211–216
Savran A, Alyuz N, Dibeklioglu H, Celiktutan O, Gokberk B, Sankur B, Akarun L (2008) Bosphorus database for 3d face analysis. In: Biometrics and identity management
Faltemier TC, Bowyer KW, Flynn PJ (2007) Using a multi-instance enrollment representation to improve 3d face recognition. In: Biometrics: theory, applications, and systems. pp 1–6
Gupta S, Castleman KR, Markey MK, Bovik AC (2010) Texas 3d face recognition database. In: Southwest Symp. image analysis interpretation. pp 97–100
Xu C, Tan T, Li S, Wang Y, Zhong C (2006) Learning effective intrinsic features to boost 3d-based face recognition. In: Computer vision. pp 416–427
Vijayan V, Bowyer KW, Flynn PJ, Huang D, Chen L, Hansen M, Ocegueda O, Shah SK, Kakadiaris IA (2011) Twins 3d face recognition challenge. In: International joint conferenceon biometrics
Guo Y, Sohel FA, Bennamoun M, Wan J, Lu M (2013) Rops: a local feature descriptor for 3d rigid objects based on rotational projection statistics. In: Signal processing
Guo Y, Wan J, Lu M, Niu W (2013) A parts-based method for articulated target recognition in laser radar data. Optik 124:2727–2733
Esteban CH, Schmitt F (2002) Multi-stereo 3d object reconstruction. In: 3D data processing visualization and transmission. pp 159–166
Liang Y, Zhang XXZY (2017) Pose-invariant 3d face recognition using half face. Sign Process 57:94
Song H, Yang KSU (2004) 3d face recognition under pose varying environments. Lect Notes Comput Sci 2908:333–347
Passalis G, Perakis P, Theoharis T, Kakadiaris IA (2011) Perakis: Using facial symmetry to handle pose variations in real-world 3d face recognition. Pattern Anal Mach Intell 33:1938–1951
Perakis P, Passalis G, Theoharis T, Toderici G, Kakadiaris IA (2009) Partial matching of interpose 3d facial data for face recognition. In: Biometrics: theory, applications, and systems. pp 439–446
Berretti S, Del Bimbo PPA (2013) Sparse matching of salient facial curves for recognition of 3d faces with missing parts. Forensics Secur 8:374–389
Drira H, Amor BB, Srivastava A, Daoudi M, Slama R (2013) 3d face recognition under expressions, occlusions, and pose variations. Pattern Anal Mach Intell 35:2270–2283
Mahmood SA, Ghani RF, Kerim AA (2014) 3d face recognition using pose invariant nose region detector. In: Computer science and electronic engineering conference
Hua WG (2009) Implicit elastic matching with random projections for pose-variant face recognition. In: Comput. Vis. Pattern Recognit. pp 1502–1509
Lu X, Jain AK (2006) Automatic feature extraction for multiview 3d face recognition. In: FG
Dibeklioglu H (2008) Part-based, 3d face recognition under pose and expression variations. Master’s thesis. Bogazici University
Dibeklioglu H, Salah AA, Akarun L (2008) 3d facial landmarking under expression, pose and occlusion variations. In: Biometrics theory, applications and systems pp 1–6
Blanz V, Scherbaum K, Seidel HP (2007) Fitting a morphable model to 3d scans of faces. In: Computer vision. pp 1–8
Mian AS, Bennamoun ROM (2007) An efficient multimodal 2d–3d hybrid approach to automatic face recognition. Pattern Anal Mach Intell 29:1584–1601
Segundo MP, Queirolo C, Bellon OR, Silva L (2007) Automatic 3d facial segmentation and landmark detection. In: Image analysis and processing. pp 431–436
Wei X, P.L., Yin L (2007) Automatic facial pose determination of 3d dynamic range data for face model and expression identification. In: IEEE/IAPR 2nd international conference on biometrics
Al-Osaimi F, Bennamoun M, Mian A (2009) An expression deformation approach to non-rigid 3D face recognition. Int J Comput Vis. 81(3):302–316
Faltemier TC, Bowyer PJFKW (2008) Using multi-instance enrollment to improve performance of 3d face recognition, computer vision and image understanding. Vis Image Understanding 112:114–125
Mian A, Bennamoun M, Owens R (2006) Automatic 3d face detection, normalization and recognition. In: 3D data processing, visualization and transmission
Bronstein AM, Bronstein MM, Kimmel R (2005) Three-dimensional face recognition. Int J Comput Vis. 64(1):5–10
Kakadiaris IA, Passalis GTMNMYLNKTTG (2007) Three-dimensional face recognition in the presence of facial expressions: an annotated deformable model approach. Pattern Anal Mach Intell 29:640–649
Lu X, Jain A (2006) Deformation modeling for robust 3d face matching. In: Computer vision and pattern recognition. pp 1377–1383
Amor BB, Ardabilian M, Chen L (2008) Toward a regionbased 3d face recognition approach. In: Multimedia and expo, hannover. pp 101–104
Queirolo CC, Silva L, Bellon OR, Segundo MP (2010) 3d face recognition using simulated annealing and the surface interpenetration measure. Pattern Anal Mach Intell 32:206–219
Bornak B, Rafiei S, Sarikhani A, Babaei A (2010) 3d face recognition by used region-based with facial expression variation. In: Signal processing systems
Erdogmus N, Daniel L, Dugelay JL (2012) Probabilistic fusion of regional scores in 3d face recognition. Image processing
Miao S, Krim H (2011) Robustness and expression independence in 3d face recognition. In: Signal processing systems
Samir C, Srivastava A, Daoudi M (2006) Three-dimensional facerecognition using shapes of facial curves. Pattern Anal Mach Intell. 28:1858–1863
Klassen E, Srivastava A, Mio M, Joshi SH (2004) Analysis of planar shapes using geodesic paths on shape spaces. IEEE Trans Pattern Analy Mach Intell 26(3):372–383
Wong KC, Lin YHHNBXZWY (2007) Optimal linear combination of facial regions for improving identification performance. Syst Man Cybern B 37:1138–1148
Fels M, Olver PJ (1997) Moving coframes. I. A practical algorithm. Acta Appl Math 51:99–136
Faltemier TC, Bowyer KW, Flynn PJ (2008) A region ensemble for 3d face recognition. In: Information forensics and security 3:62–73
Li H, Huang PLJ-MMLCD (2011) Expression robust 3d face recognition via mesh-based histograms of multiple order surface differential quantities. Image Process (ICIP):3053–3056
Smeets D, Keustermans J, Vandermeulen D, Suetens P (2013) meshSIFT: Local surface features for 3D face recognition under expression variations and partial data. Comput Vis Image Underst 117:158–69
Berretti S, Werghi N, Del Bimbo A, Pala P (2014) Selecting stable keypoints and local descriptors for person identification using 3D face scans. Vis Comput 30(11):1275–92
Tang H, Yin B, Sun Y, Hu Y (2013) 3D face recognition using local binary patterns. Sign Process 93(8):2190–8
Lei Y, Bennamoun MHYGM (2014) An efficient 3d face recognition approach using local geometrical signatures. Pattern Recognit 47:509–524
Ming Y (2015) Robust regional bounding spherical descriptor for 3d face recognition and emotion analysis. Image Vis Comput 35:14–22
Li H, Huang D, Morvan JM, Wang Y, Chen L (2014) Towards 3d face recognition in the real: a registration-free approach using fine-grained matching of 3d keypoint. Int J Comput Vis 113:1–14
Boehnen C, Peters T, Flynn PJ (2009) 3d signatures for fast 3d face recognition. In: IAPR/IEEE Int’l Conf. Biometrics. pp 12–21
Li X, Da F (2012) Efficient 3d face recognition handling facial expression and hair occlusion. Image Vis Comput 30:668–679
Lei Y, Bennamoun M, El-Sallam AA (2013) An efficient 3d face recognition approach based on the fusion of novel local low-level features. Pattern Recognit 46:24–37
Li H, Huang J-MMLCYWD (2014) Expression-robust 3d face recognition via weighted sparse representation of multi-scale and multi-component local normal patterns. Neurocomputing 133:179–193
Smeets D, Fabry T, Hermans J, Vandermeulen D, Suetens P (2010) Fusion of an isometricdeformation modeling approach using spectraldecomposition and a region-based approach using icp for expression invariant 3d face recognition. In: International conference on pattern recognition
Miao S, Krim H (2010) 3d face recognition based on evolution of iso-geodesic distance curves. In: Acoustics, speech, and signal processing. pp 1134–1137
Feng S, Krim H, Kogan IA (2007) 3d face recognition using euclidean integral invariants signature. In: Statistical signal processing, pp 156–160
Berretti S, Del Bimbo PPA (2010) 3d face recognition using isogeodesic stripes. Pattern Anal Mach Intell 32:2162–2177
Ballihi L, Amor MDASDABB (2012) Boosting 3-d-geometric features for efficient face recognition and gender classification. Forensics Secur 7:1766–1779
Mpiperis I, Malassiotis MGSS (2008) Bilinear models for 3-d face and facial expression recognition. Inform Forensics Secur 3:498–511
Amberg B, Knothe R, Vetter T (2008) Expression invariant 3d face recognition with a morphable model. In: FG’08 7:1766
Al-Osaimi F, Bennamoun M, Mian A (2009) An expression deformation approach to non-rigid 3D face recognition. Int J Comput Vis 81(3):302–16
ter Haar RC, Velkamp F (2010) Expression modeling for expression-invariant face recognition. Comput Graph 34:231–241
Alyuz N, Gokberk B, Akarun L (2008) A 3d face recognition system for expression and occlusion invariance. In: Biometrics: theory, applications and systems
Mayo M, Zhang E (2009) 3d face recognition using multiview key point matching. Advanced video and signal based surveillance. pp 290–295
Alyuz N, Gokberk LSRVLAB (2012) Robust 3d face recognition in the presence of realistic occlusions. Biometrics (ICB):111–118
Alyuz N, Gokberk B, Akarun L (2013) 3-d face recognition under occlusion using masked projection. Inform Forensics Secur 8:789–802
Hariri W, Tabia H, Farah N, Benouareth A, Declercq D (2017) 3d facial expression recognition using kernel methods on riemannian manifold. Eng Appl Artif Intell 64:25–32
Zeng Z, Pantic GRTHM (2009) A survey of affect recognition methods: audio. Patt Analy Mach Intell 31:39–58
Nicolaou M, Gunes MPH (2011) Continuous prediction of spontaneous affect from multiple cues and modalities in valencearousal space. Affect Comput 2:92–105
Vinciarelli A, Pantic DHCPIPFDMSM (2012) Bridging the gap between social animal and unsocial machine: a survey of social signal processing. Forensics 3:69–87
Kemelmacher-Shlizerman I, Basri R (2011) 3d face reconstruction from a single image using a single reference face shape. Pattern Anal Mach Intell 33:394–405
Lanitis A, Taylor CJ (2000) Robust face recognition using automaticage normalization. In: Mediterranean electrotechnical conference Vol.2, pp 478–481
Lanitis A, Taylor TCC (2002) Toward automatic simulation of aging effects on face images. Pattern Anal Mach Intell 24:442–455
Boussaad L, Benmohammed M, Benzid R (2016) Age invariant face recognition based on dct feature extraction and kernel fisher analysis. J Inform Process Syst 12(3):392–409