3D face recognition: a survey

Song Zhou1, Shuangjiu Xiao1
1College of Computer Science and Electronic Engineering, Hunan University, South Lushan Road, Yuelu District, Changsha, 410082, China

Tóm tắt

Abstract

3D face recognition has become a trending research direction in both industry and academia. It inherits advantages from traditional 2D face recognition, such as the natural recognition process and a wide range of applications. Moreover, 3D face recognition systems could accurately recognize human faces even under dim lights and with variant facial positions and expressions, in such conditions 2D face recognition systems would have immense difficulty to operate. This paper summarizes the history and the most recent progresses in 3D face recognition research domain. The frontier research results are introduced in three categories: pose-invariant recognition, expression-invariant recognition, and occlusion-invariant recognition. To promote future research, this paper collects information about publicly available 3D face databases. This paper also lists important open problems.

Từ khóa


Tài liệu tham khảo

Zhang X, Gao Y (2009) Face recognition across pose: a review. Pattern recognit 42:2876–2896

Chellappa R, Wilson SSC (1995) Human and machine recognition of faces: a survey. IEEE 83:705–740

Xu C, Wang Y, Tan T, Quan L (2004) Depth vs. intensity: which is more important for face recognition? Int Conf Pattern Recognit 4:342–345

Bowyer KW, Chang PFK (2006) A survey of approaches and challenges in 3d and multi-modal 3d + 2d face recognition. Comput Vis Image Underst 101:1–15

Zhu X, Lei Z, Yan J, Yi D, Li SZ (2015) High-fidelity pose and expression normalization for face recognition in the wild. Computer vision and pattern recognition. pp 787–796

Patil H, Kothari KBA (2015) 3-d face recognition: features, databases, algorithms and challenges. Artif Intell Rev 44:393–441

Zaharescu A, Boyer E, Varanasi K, Horaud R (2009) Surface feature detection and description with applications to mesh matching. In: Proc. IEEE Conf. on Comput

Kittler J,  Hilton MH (2005) A survey of 3d imaging, modelling and recognition approachest. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops. pp 114–114

Bennamoun M, Guo Y, Sohel F (2015) Feature selection for 2d and 3d face recognition. Research Gate, vol 17

Guo Y, Zhang MLJWYM J (2014) Benchmark datasets for 3d computer vision. In: Industrial electronics and applications. pp 1846–1851

Soltanpour S, Boufama QJWB (2017) A survey of local feature methods for 3d face recognition. Pattern Recognit 72:391–406

Zhu J, San-Segundo R, Pardo JM (2017) Feature extraction for robust physical activity recognition. Human-centric Comput Inform Sci 7(1):16

Huang D, Zhang G, Ardabilian M, Wang Y, Chen L (2010) 3d face recognition using distinctiveness enhanced facial representations and local feature hybrid matching. In: Biometrics: theory, applications and systems

Zhao W, Chellappa PJPARR (2003) Face recognition: a literature survey. ACM Comput Surv 35:399–485

Bledsoe WW (1966) The model method in facial recognition. Panoramic Research

Pentland MT (1991) Face recognition using eigenfaces. Computer vision and pattern recognition. pp 586–591

Belhumeur PN, Hespanha DJKJP (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. Trans Pattern Anal Mach Intell 19:711–720

Frey BJ, Colmenarez TSH A (1998) Mixtures of local linear subspaces for face recognition. In: Computer vision and pattern recognition

Moghaddam B, Jebara APT (2000) Bayesian face recognition. Pattern Recognit 33:1771–1782

Wiskott L, Fellous JM, Kruger N, Von Der Malsburg C (1997) Face recognition by elasric bunch graph matching. PAMI 17:775–779

Mpiperis I, Malassiotis MGSS (2007) 3-d face recognition with the geodesic polar representation. Inform Forensics Secur 5:537–547

Abate AF, Nappi DRGSM (2007) 2d and 3d face recognition: a survey. Pattern Recognit Lett 28:1885–1906

Cartoux JY, LaPreste JT, Richetin M (1989) Face authentication or recognition by profile extraction from range images. Interpretation of 3D Scenes. pp 194–199

Gordon G (1995) Face recognition from frontal and profile views. In: International workshop on automatic face and gesture recognition pp 47–52

Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems. pp 1106–1114

LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436–444

Kim D, Hernandez JC M, Medioni, G (2017) Deep 3d face identification. arXiv:1703.10714 http://arxiv.org/abs/1703.10714

Gilani SZ, Mian A (2017) Learning from millions of 3d scans for large-scale 3d face recognition. ArXiv e-prints. arXiv:1711.05942

Prabhu U, Heo MSJ (2011) Unconstrained poseinvariant face recognition using 3d generic elastic models. Pattern Anal Mach Intell 33:1952–1961

Blanz V, Vetter T (1999) A morphable model for the synthesis of 3d faces. In: Computer graphics, annual conference series. pp 187–194

Lee KC, Ho J, Yang MH, Kriegman D (2003) Video-based face recognition using probabilistic appearance manifolds. In: Computer vision and pattern recognition

Hu Y, Jiang D, Yan S, Zhang L (2004) Automatic 3d reconstruction for face recognition. In: FG pp 843–848

Arandjelovic O, Shakhnarovich G, Fisher J, Cipolla R, Darrell T (2005) Face recognition with image sets using manifold density divergence. In: Computer vision and pattern recognition pp 581–588

Vetter VBT (2003) Face recognition based on fitting a 3d morphable model. Pattern Anal Mach Intell 25:1063–1074

Pan G, Wu YPZ (2003) Automatic 3d face verification from range data. Acoustics Speech Sign Process 3:193–196

Chua CS, Han F, Ho YK (2000) 3d human face recognition using point signature. In: Autom. Face Gesture Recogn. pp 233–238

Hesher C, Srivastava A, Erlebacher G (2003) A novel technique for face recognition using range imaging. In: Signal processing and its applications. pp 201–204

Min J, Flynn PJ, Bowyer KW (2003) Using multiple gallery and probe images per person to improve performance of face recognition. In: Notre Dame computer science and engineering technical report

Moreno AB, Sanchez A, Velez JF, Diaz FJ (2003) Face recognition using 3d surface extracted descriptors. In: Proceedings of the Irish machine vision and image processing

Martínez AM (2000) Recognition of partially occluded and/or imprecisely localized faces using a probabilistic approach. In: Computer vision and pattern recognition. pp 712–717

Tan X, Chen ZHZFZS (2005) Recognizing partially occluded, expression variant faces from single training image per person with som and soft k-nn ensemble. Vision 16:875–886

Colombo A, Cusano C, Schettini R (2006) Detection and restoration of occlusions for 3d face recognition. In: Multimedia and Expo. pp 1541–1544

Phillips PJ, Flynn PJ, Scruggs T, Bowyer KW, Chang J, Hoffman K, Marques J, Min J, Worek W (2005) Overview of face recognition grand challenge. In: Computer vision and pattern recognition. pp 947–954

Yin L, Wei X, Sun Y, Wang J, Rosato MJ (2006) A 3d facial expression database for facial behavior research. In: Automatic face and gesture recognition. pp 211–216

Savran A, Alyuz N, Dibeklioglu H, Celiktutan O, Gokberk B, Sankur B, Akarun L (2008) Bosphorus database for 3d face analysis. In: Biometrics and identity management

Faltemier TC, Bowyer KW, Flynn PJ (2007) Using a multi-instance enrollment representation to improve 3d face recognition. In: Biometrics: theory, applications, and systems. pp 1–6

Gupta S, Castleman KR, Markey MK, Bovik AC (2010) Texas 3d face recognition database. In: Southwest Symp. image analysis interpretation. pp 97–100

Xu C, Tan T, Li S, Wang Y, Zhong C (2006) Learning effective intrinsic features to boost 3d-based face recognition. In: Computer vision. pp 416–427

Vijayan V, Bowyer KW, Flynn PJ, Huang D, Chen L, Hansen M, Ocegueda O, Shah SK, Kakadiaris IA (2011) Twins 3d face recognition challenge. In: International joint conferenceon biometrics

Guo Y, Sohel FA, Bennamoun M, Wan J, Lu M (2013) Rops: a local feature descriptor for 3d rigid objects based on rotational projection statistics. In: Signal processing

Guo Y, Wan J, Lu M, Niu W (2013) A parts-based method for articulated target recognition in laser radar data. Optik 124:2727–2733

Esteban CH, Schmitt F (2002) Multi-stereo 3d object reconstruction. In: 3D data processing visualization and transmission. pp 159–166

Liang Y, Zhang XXZY (2017) Pose-invariant 3d face recognition using half face. Sign Process 57:94

Song H, Yang KSU (2004) 3d face recognition under pose varying environments. Lect Notes Comput Sci 2908:333–347

Passalis G, Perakis P, Theoharis T, Kakadiaris IA (2011) Perakis: Using facial symmetry to handle pose variations in real-world 3d face recognition. Pattern Anal Mach Intell 33:1938–1951

Perakis P, Passalis G, Theoharis T, Toderici G, Kakadiaris IA (2009) Partial matching of interpose 3d facial data for face recognition. In: Biometrics: theory, applications, and systems. pp 439–446

Berretti S, Del Bimbo PPA (2013) Sparse matching of salient facial curves for recognition of 3d faces with missing parts. Forensics Secur 8:374–389

Drira H, Amor BB, Srivastava A, Daoudi M, Slama R (2013) 3d face recognition under expressions, occlusions, and pose variations. Pattern Anal Mach Intell 35:2270–2283

Mahmood SA, Ghani RF, Kerim AA (2014) 3d face recognition using pose invariant nose region detector. In: Computer science and electronic engineering conference

Hua WG (2009) Implicit elastic matching with random projections for pose-variant face recognition. In: Comput. Vis. Pattern Recognit. pp 1502–1509

Lu X, Jain AK (2006) Automatic feature extraction for multiview 3d face recognition. In: FG

Dibeklioglu H (2008) Part-based, 3d face recognition under pose and expression variations. Master’s thesis. Bogazici University

Dibeklioglu H, Salah AA, Akarun L (2008) 3d facial landmarking under expression, pose and occlusion variations. In: Biometrics theory, applications and systems pp 1–6

Blanz V, Scherbaum K, Seidel HP (2007) Fitting a morphable model to 3d scans of faces. In: Computer vision. pp 1–8

Mian AS, Bennamoun ROM (2007) An efficient multimodal 2d–3d hybrid approach to automatic face recognition. Pattern Anal Mach Intell 29:1584–1601

Segundo MP, Queirolo C, Bellon OR, Silva L (2007) Automatic 3d facial segmentation and landmark detection. In: Image analysis and processing. pp 431–436

Wei X, P.L., Yin L (2007) Automatic facial pose determination of 3d dynamic range data for face model and expression identification. In: IEEE/IAPR 2nd international conference on biometrics

Al-Osaimi F, Bennamoun M, Mian A (2009) An expression deformation approach to non-rigid 3D face recognition. Int J Comput Vis. 81(3):302–316

Faltemier TC, Bowyer PJFKW (2008) Using multi-instance enrollment to improve performance of 3d face recognition, computer vision and image understanding. Vis Image Understanding 112:114–125

Mian A, Bennamoun M, Owens R (2006) Automatic 3d face detection, normalization and recognition. In: 3D data processing, visualization and transmission

Bronstein AM, Bronstein MM, Kimmel R (2005) Three-dimensional face recognition. Int J Comput Vis. 64(1):5–10

Kakadiaris IA, Passalis GTMNMYLNKTTG (2007) Three-dimensional face recognition in the presence of facial expressions: an annotated deformable model approach. Pattern Anal Mach Intell 29:640–649

Lu X, Jain A (2006) Deformation modeling for robust 3d face matching. In: Computer vision and pattern recognition. pp 1377–1383

Amor BB, Ardabilian M, Chen L (2008) Toward a regionbased 3d face recognition approach. In: Multimedia and expo, hannover. pp 101–104

Queirolo CC, Silva L, Bellon OR, Segundo MP (2010) 3d face recognition using simulated annealing and the surface interpenetration measure. Pattern Anal Mach Intell 32:206–219

Bornak B, Rafiei S, Sarikhani A, Babaei A (2010) 3d face recognition by used region-based with facial expression variation. In: Signal processing systems

Erdogmus N, Daniel L, Dugelay JL (2012) Probabilistic fusion of regional scores in 3d face recognition. Image processing

Miao S, Krim H (2011) Robustness and expression independence in 3d face recognition. In: Signal processing systems

Samir C, Srivastava A, Daoudi M (2006) Three-dimensional facerecognition using shapes of facial curves. Pattern Anal Mach Intell. 28:1858–1863

Klassen E, Srivastava A, Mio M, Joshi SH (2004) Analysis of planar shapes using geodesic paths on shape spaces. IEEE Trans Pattern Analy Mach Intell 26(3):372–383

Wong KC, Lin YHHNBXZWY (2007) Optimal linear combination of facial regions for improving identification performance. Syst Man Cybern B 37:1138–1148

Fels M, Olver PJ (1997) Moving coframes. I. A practical algorithm. Acta Appl Math 51:99–136

Faltemier TC, Bowyer KW, Flynn PJ (2008) A region ensemble for 3d face recognition. In: Information forensics and security 3:62–73

Li H, Huang PLJ-MMLCD (2011) Expression robust 3d face recognition via mesh-based histograms of multiple order surface differential quantities. Image Process (ICIP):3053–3056

Smeets D, Keustermans J, Vandermeulen D, Suetens P (2013) meshSIFT: Local surface features for 3D face recognition under expression variations and partial data. Comput Vis Image Underst 117:158–69

Berretti S, Werghi N, Del Bimbo A, Pala P (2014) Selecting stable keypoints and local descriptors for person identification using 3D face scans. Vis Comput 30(11):1275–92

Tang H, Yin B, Sun Y, Hu Y (2013) 3D face recognition using local binary patterns. Sign Process 93(8):2190–8

Lei Y, Bennamoun MHYGM (2014) An efficient 3d face recognition approach using local geometrical signatures. Pattern Recognit 47:509–524

Ming Y (2015) Robust regional bounding spherical descriptor for 3d face recognition and emotion analysis. Image Vis Comput 35:14–22

Li H, Huang D, Morvan JM, Wang Y, Chen L (2014) Towards 3d face recognition in the real: a registration-free approach using fine-grained matching of 3d keypoint. Int J Comput Vis 113:1–14

Boehnen C, Peters T, Flynn PJ (2009) 3d signatures for fast 3d face recognition. In: IAPR/IEEE Int’l Conf. Biometrics. pp 12–21

Spreeuwers L (2011) Fast and accurate 3d face recognition. Int J Comput Vis 93(3):389–414

Li X, Da F (2012) Efficient 3d face recognition handling facial expression and hair occlusion. Image Vis Comput 30:668–679

Lei Y, Bennamoun M, El-Sallam AA (2013) An efficient 3d face recognition approach based on the fusion of novel local low-level features. Pattern Recognit 46:24–37

Li H, Huang J-MMLCYWD (2014) Expression-robust 3d face recognition via weighted sparse representation of multi-scale and multi-component local normal patterns. Neurocomputing 133:179–193

Smeets D, Fabry T, Hermans J, Vandermeulen D, Suetens P (2010) Fusion of an isometricdeformation modeling approach using spectraldecomposition and a region-based approach using icp for expression invariant 3d face recognition. In: International conference on pattern recognition

Miao S, Krim H (2010) 3d face recognition based on evolution of iso-geodesic distance curves. In: Acoustics, speech, and signal processing. pp 1134–1137

Feng S, Krim H, Kogan IA (2007) 3d face recognition using euclidean integral invariants signature. In: Statistical signal processing, pp 156–160

Berretti S, Del Bimbo PPA (2010) 3d face recognition using isogeodesic stripes. Pattern Anal Mach Intell 32:2162–2177

Ballihi L, Amor MDASDABB (2012) Boosting 3-d-geometric features for efficient face recognition and gender classification. Forensics Secur 7:1766–1779

Mpiperis I, Malassiotis MGSS (2008) Bilinear models for 3-d face and facial expression recognition. Inform Forensics Secur 3:498–511

Amberg B, Knothe R, Vetter T (2008) Expression invariant 3d face recognition with a morphable model. In: FG’08 7:1766

Al-Osaimi F, Bennamoun M, Mian A (2009) An expression deformation approach to non-rigid 3D face recognition. Int J Comput Vis 81(3):302–16

ter Haar RC, Velkamp F (2010) Expression modeling for expression-invariant face recognition. Comput Graph 34:231–241

Alyuz N, Gokberk B, Akarun L (2008) A 3d face recognition system for expression and occlusion invariance. In: Biometrics: theory, applications and systems

Mayo M, Zhang E (2009) 3d face recognition using multiview key point matching. Advanced video and signal based surveillance. pp 290–295

Alyuz N, Gokberk LSRVLAB (2012) Robust 3d face recognition in the presence of realistic occlusions. Biometrics (ICB):111–118

Alyuz N, Gokberk B, Akarun L (2013) 3-d face recognition under occlusion using masked projection. Inform Forensics Secur 8:789–802

Hariri W, Tabia H, Farah N, Benouareth A, Declercq D (2017) 3d facial expression recognition using kernel methods on riemannian manifold. Eng Appl Artif Intell 64:25–32

Zeng Z, Pantic GRTHM (2009) A survey of affect recognition methods: audio. Patt Analy Mach Intell 31:39–58

Nicolaou M, Gunes MPH (2011) Continuous prediction of spontaneous affect from multiple cues and modalities in valencearousal space. Affect Comput 2:92–105

Vinciarelli A, Pantic DHCPIPFDMSM (2012) Bridging the gap between social animal and unsocial machine: a survey of social signal processing. Forensics 3:69–87

Kemelmacher-Shlizerman I, Basri R (2011) 3d face reconstruction from a single image using a single reference face shape. Pattern Anal Mach Intell 33:394–405

Park U, Tong AKJY (2010) Age invariant face recognition. Pattern Anal Mach Intell 32:947–954

Lanitis A, Taylor CJ (2000) Robust face recognition using automaticage normalization. In: Mediterranean electrotechnical conference Vol.2, pp 478–481

Lanitis A, Taylor TCC (2002) Toward automatic simulation of aging effects on face images. Pattern Anal Mach Intell 24:442–455

Boussaad L, Benmohammed M, Benzid R (2016) Age invariant face recognition based on dct feature extraction and kernel fisher analysis. J Inform Process Syst 12(3):392–409