Automatic Face Recognition (FR) presents a challenging task in the field of pattern recognition and despite the huge research in the past several decades; it still remains an open research problem. This is primarily due to the variability in the facial images, such as non-uniform illuminations, low resolution, occlusion, and/or variation in poses. Due to its non-intrusive nature, the FR is an attractive biometric modality and has gained a lot of attention in the biometric research community. Driven by the enormous number of potential application domains, many algorithms have been proposed for the FR. This paper presents an overview of the state-of-the-art FR algorithms, focusing their performances on publicly available databases. We highlight the conditions of the image databases with regard to the recognition rate of each approach. This is useful as a quick research overview and for practitioners as well to choose an algorithm for their specified FR application. To provide a comprehensive survey, the paper divides the FR algorithms into three categories: (1) intensity-based, (2) video-based, and (3) 3D based FR algorithms. In each category, the most commonly used algorithms and their performance is reported on standard face databases and a brief critical discussion is carried out.

Keywords:

References

1. W. Zhao, R. Chellappa, P. J. Phillips and A. Rosenfeld, Face recognition: A literature survey ACM Comput. Surveys 35(4) (2003) 399–458. Crossref, Web of Science, Google Scholar
2. Z. Hailing, M. Ajmal and W. Lei, Recent advances on singlemodal and multimodal face recognition: A survey IEEE Trans. Human Mach. Syst. 44(6) (2014) 701–716. Crossref, Web of Science, Google Scholar
3. Sun et al., Hybrid deep learning for face verification, in Proc. IEEE Int. Conf. Computer Vision (2013), pp. 1489–1496. Google Scholar
4. H. Moon, Biometrics person authentication using projection-based face recognition system in verification scenario, Int. Conf. on Bioinformatics and Its Applications, Hong Kong, China (2004), pp. 207–213. Google Scholar
5. T. J. Kavitha and A. Suruliandi, Performance evaluation of various color local texture models for face recognition, Int. Conf. Circuits, Power and Computing Technologies (IEEE, 2013), pp. 1206–1210. Google Scholar
6. T. Ali, L. J. Spreeuwers and R. N. Veldhuis, Forensic face recognition: A survey, face recognition: Methods, applications, and technology Computer Sci. Technol. Appl. 9 (2012) 187–200. Google Scholar
7. T. Ali, L. J. Spreeuwers and R. N. Veldhuis, Towards automatic forensic face recognition, Vol. 252, ICIEIS, Malaysia, CCIS (Springer Verlag, 2011), pp. 47–55. Google Scholar
8. M. Grgic, K. Delac and S. Grgic, Surveillance cameras face database Multimedia Tools Appl. J. 51 (2011) 863–879. Crossref, Web of Science, Google Scholar
9. Parkhi et al., Deep face recognition, Br. Machine Vision Conf. 1 (2015) 1–12. Google Scholar
10. A. Bansal, K. Mehta and S. Arora, Face recognition using PCA & LDA algorithms, 2nd Int. Conf. Advanced Computing & Communication Technologies (2012), pp. 251–254. Google Scholar
11. Dan et al., Facial biometrics using non-tensor product wavelet and 2D discriminant techniques, Int. J. Pattern Recognit. Artif. Intell. 23(3) (2009) 1–21. Web of Science, Google Scholar
12. D. J. Kim and S. H. Lee, Face recognition in unconstrained environments, Int. Conf. Consumer Electronics (2012), pp. 143–144. Google Scholar
13. R. Min, A. Hadid and J. L. Dugelay, Improving the recognition of faces occluded by facial accessories, Int. Conf. Automatic Face & Gesture Recognition and Workshops (ICAFGR) (2011), pp. 442–447. Google Scholar
14. Y. Deng and Q. Dai, Graph laplace for occluded face completion and recognition IEEE Trans. Image Process. 20(8) (2011) 2329–2338. Crossref, Web of Science, Google Scholar
15. L. Ding and X. Ding, Continuous pose normalization for pose-robust face recognition IEEE Signal Process. Lett. 19(11) (2012) 721–724. Crossref, Web of Science, Google Scholar
16. X. Zhang and Y. Ghao, Heterogeneous specular and diffuse 3D surface approximation for face recognition across pose IEEE Trans. Inform. Forensics Security 7(2) (2012) 506–517. Crossref, Web of Science, Google Scholar
17. S. Lee and H. Li, Face identification system using single hidden markov model and single sample image per person, Int. Joint Conf. Neural Networks (2004), pp. 455–459. Google Scholar
18. S. Klum, H. Han and A. K. Jain, Sketch based face recognition: Forensic vs. composite sketches, Int. Conf. Biometrics (2013), pp. 1–8. Google Scholar
19. Thanh et al., Face recognition using multi-modal binary patterns, 21st Int. Conf. Pattern Recognition (2012), pp. 2343–2346. Google Scholar
20. H. Mohammad and M. Mohamed, A multimodal approach for face modeling and recognition IEEE Trans. Inform. Forensics Security 3(3) (2008) 431–440. Crossref, Web of Science, Google Scholar
21. Z. Mahmood, T. Ali, S. Khattak, L. Hassan and S. U. Khan, Automatic player detection and identification for sports entertainment applications Pattern Anal. Appl. 18(4) (2015) 971–982. Crossref, Web of Science, Google Scholar
22. Z. Mahmood, T. Ali and S. Khattak, Automatic player detection and recognition in images using AdaBoost, 9th Int. Bhurban Conf., Applied Sciences and Technology (IBCAST) (IEEE, 2012), pp. 64–69. Google Scholar
23. R. Chuan-Xian et al., Enhanced local gradient order features and discriminant analysis for face recognition, IEEE Trans. Cybern. 46(1) (2016) 2656–2669. Google Scholar
24. M. C. Pinho and N. M. Ribeiro, Automatic detection of human faces in uncontrolled environments: Identification of direction and movement, Int. Conf. Information Systems and Technologies (CISTI), Iberian, (2011), pp. 1–7. Google Scholar
25. Z. Si and S. C. Zhu, Learning AND-OR templates for object recognition and detection IEEE Trans. Pattern Anal. Mach. Intell. 35(9) (2013) 2189–2205. Crossref, Web of Science, Google Scholar
26. S. Zafeiriou, G. A. Atkinson, M. F. Hansen and A. P. Smith, Face recognition and verification using photometric stereo, The photoface database and a comprehensive evaluation IEEE Trans. Inform. Forensics Security 8(1) (2013) 121–135. Crossref, Web of Science, Google Scholar
27. S. Lawrence and G. J. Bala, Time complexity for face recognition under varying pose, illumination, and facial expressions based on sparse representation Int. J. Computer Appl. 51(12) (2012) 7–10. Google Scholar
28. http://www.cognitec-systems.de/FaceVACS-SDK.19.0.html. Google Scholar
29. D. Huang and C. Shan, Local binary patterns and its application to facial image analysis: A survey IEEE Trans. Syst. Man. Cybern. 41(6) (2011) 765–781. Crossref, Web of Science, Google Scholar
30. B. Thomas and B. Peter, POOF: Part-based one-vs.-one features for fine-grained categorization, face verification, and attribute estimation, Computer Vision and Pattern Recognition (CVPR) (2013), pp. 955–962. Google Scholar
31. W. R. Schwartz, H. Guo, J. Choi and L. S. Davis, Face identification using large feature sets IEEE Trans. Image Process. 21(4) (2012) 2245–2255. Crossref, Web of Science, Google Scholar
32. R. Sawalha and I. A. Doush, Face recognition using harmony search-based selected features Int. J. Hybrid Inform. Technol. 5(2) (2012) 1–16. Google Scholar
33. S. Kakarwal and R. Deshmukh, Wavelet transform based feature extraction for face recognition Int. J. Computer Sci. Appl. 1(1) (2010) 100–104. Google Scholar
34. A. Y. Yang and Z. Zhou, Fast l-1-minimization algorithms for robust face recognition IEEE Trans. Image Process. 22(8) (2013) 1598–1603. Crossref, Web of Science, Google Scholar
35. O. Deniz, G. Bueno, J. Salido and F. D. Torre, Face recognition using histograms of oriented gradients Pattern Recognit. Lett. 32(12) (2011) 1598–1603. Crossref, Web of Science, Google Scholar
36. H. Huang and H. He, Super-resolution method for face recognition using nonlinear mappings on coherent features IEEE Trans. Neural Netw. 22(1) (2011) 121–130. Crossref, Web of Science, Google Scholar
37. T. H. Le and L. Bui, Face recognition based on SVM and 2-DPCA Int. J. Signal Process. Pattern Recognit. 4(3) (2011) 85–94. Google Scholar
38. M. P. Satone and G. K. Kharate, Face detection and recognition in colored images Int. J. Computer Sci. 8(2) (2011) 467–471. Google Scholar
39. L. V. Fang and P. Tao, Realization of face recognition system based on gabor wavelet and elastic bunch graph matching, 25th, Control and Decision Conference (CCDC) (2013), pp. 3384–3386. Google Scholar
40. M. C. Leong, Y. T. Lee and F. Fang, A search-and-validate method for face identification from single line drawings IEEE Trans. Pattern Anal. Mach. Intell. 35(11) (2013) 2576–2591. Crossref, Web of Science, Google Scholar
41. D. Monzo, A. Albiol and J. M. Mossi, Color HOG-EBGM for face recognition, 18th IEEE Int. Conf. Image Process. (2011), pp. 785–788. Google Scholar
42. C. Zhang, Z. Zhou, H. Sun and F. Dong, Comparison of three face recognition algorithms, Int. Conf. Systems and Informatics (ICSAI) (2012), pp. 1896–1900. Google Scholar
43. R. Jafri and H. R. Arabnia, A survey of face recognition techniques J. Inform. Process. Syst. 5(2) (2009) 41–68. Crossref, Google Scholar
44. Chen et al., Blessing of dimensionality: High-dimensional feature and its efficient compression for face verification, IEEE Conf. Computer Vision and Pattern Recognition (2013), pp. 3025–3032. Google Scholar
45. B. Klare, Z. Li and A. K. Jain, Matching forensic sketches to mug Shot photos IEEE Trans. Pattern Anal. Mach. Intell. 33(3) (2011) 639–646. Crossref, Web of Science, Google Scholar
46. Cui et al., Fusing robust face region descriptors via multiple metric learning for face recognition in the wild, Computer Vision and Pattern Recognition (CVPR) (2013), pp. 3554–3561. Google Scholar
47. Zhen et al., Learning discriminant face descriptor, IEEE Trans. Pattern Anal. Mach. Intell. 36(2) (2014) 289–302. Crossref, Web of Science, Google Scholar
48. Q. Cao, Y. Yiming and P. Li, Similarity metric learning for face recognition, Computer Vision (ICCV), (2013), pp. 2408–2415. Google Scholar
49. Barkan et al., Fast high dimensional vector multiplication face recognition, Computer Vision (ICCV) (2013), pp. 1960–1964. Google Scholar
50. Simonyan et al., Fisher vector faces in the wild, Proc. BMVC 1 (2013) 1–13. Google Scholar
51. P. Sharma, R. Yadav and K. Arya, Pose-invariant face recognition using curvelet neural network IET Biometrics 3(3) (2014) 128–138. Crossref, Web of Science, Google Scholar
52. H. Boughrara, M. Chtourou, C. Amar and L. Chen, Face recognition based on perceived facial images and multilayer perceptron neural network using constructive training algorithm IET Computer Vis. 8(6) (2014) 729–739. Crossref, Web of Science, Google Scholar
53. A. Stuhlsatz, J. Lippel and T. Zielke, Feature extraction with deep neural networks by generalized discriminant analysis IEEE Trans. Neural Netw. Learning Syst 23(4) (2012) 596–608. Crossref, Web of Science, Google Scholar
54. M. Marsico, M. Nappi, D. Riccio and H. Wechsler, Robust face recognition for uncontrolled pose and illumination changes IEEE Trans. Syst. Man. Cybern. 43(1) (2013) 149–163. Crossref, Web of Science, Google Scholar
55. B. Martinez and M. F. Valstar, Local evidence aggregation for regression-based facial point detection IEEE Trans. Pattern Anal. Mach. Intell. 35(5) (2013) 1149–1163. Crossref, Web of Science, Google Scholar
56. X. Lu, Y. Wang and A. K. Jain, Combining classifiers for face recognition, Proc. IEEE Int. Conf. Multimedia & Expo (ICME 2003), Baltimore, MD (2003), pp. 13–16. Google Scholar
57. Jianwu et al., Pairwise costs in semisupervised discriminant analysis for face recognition, IEEE Trans. Inform. Forensics Security 9(10) (2014) 1569–1580. Crossref, Web of Science, Google Scholar
58. Vuong et al., Expression recognition from 3D dynamic faces using robust spatiotemporal shape features, Int. Conf. Automatic Face and Gesture Recognition and Workshops (2011), pp. 414–421. Google Scholar
59. L. I. Kuncheva and W. J. Faithfull, PCA feature extraction for change detection in multidimensional unlabeled data IEEE Trans. Neural Netw. Learning Syst. 25(1) (2014) 69–80. Crossref, Web of Science, Google Scholar
60. M. Mohamed, Effective two-step method for face hallucination based on sparse compensation on over-complete patches IET Image Process. 7(6) (2013) 624–632. Crossref, Web of Science, Google Scholar
61. Y. Rahulamathavan, C. W. Raphael, P. Jonathon and A. Chambers, Facial expression recognition in the encrypted domain based on local fisher discriminant analysis IEEE Trans. Affective Comput. 4(1) (2013) 83–92. Crossref, Web of Science, Google Scholar
62. Xiaohua et al., Spatiotemporal local monogenic binary patterns for facial expression recognition, IEEE Signal Process. Lett. 19(5) (2012) 243–246. Crossref, Web of Science, Google Scholar
63. L. An, M. Kafai and B. Bhanu, Dynamic Bayesian network for unconstrained face recognition in surveillance camera networks IEEE J. Emerg. Sel. Topics Circuits Syst. 3(2) (2013) 155–164. Crossref, Web of Science, Google Scholar
64. Siti et al., Spatio-temporal texture-based feature extraction for spontaneous facial expression recognition, 14th IAPR International Conference on Machine Vision Applications (MVA) (2015), pp. 467–470. Google Scholar
65. B. Xiong and X. Fan, Face region based conversational video coding IEEE Trans. Circuits Syst. Video Technol. 21(7) (2011) 917–931. Crossref, Web of Science, Google Scholar
66. X. Tang and L. Zhifeng, Audio-guided video-based face recognition IEEE Trans. Circuits Syst. Video Technol. 19(7) (2009) 955–964. Crossref, Web of Science, Google Scholar
67. S. Yang and B. Bhanu, Understanding discrete facial expressions in video using an emotion avatar image IEEE Trans. Syst. Cybern. 42(4) (2012) 980–992. Crossref, Web of Science, Google Scholar
68. B. Kamgar and W. Lawson, Toward development of a face recognition system for watchlist surveillance IEEE Trans. Pattern Anal. Mach. Intell. 33(10) (2011) 1925–1937. Crossref, Web of Science, Google Scholar
69. X. Yong and D. Zhang, A two-phase test sample sparse representation method for use with face recognition IEEE Trans. Circuits Syst. Video Technol. 21(9) (2011) 1255–1262. Crossref, Web of Science, Google Scholar
70. Y. Kim, J. H. Yoo and K. Choi, A motion and similarity-based fake detection method for biometric face recognition systems IEEE Trans. Consumer Electron. 57(2) (2011) 756–762. Crossref, Web of Science, Google Scholar
71. John et al., Customized orthogonal locality preserving projections with soft-margin maximization for face recognition, IEEE Trans. Instrum. Meas. 64(9) (2015) 2417–2426. Crossref, Web of Science, Google Scholar
72. C. Tian, G. Fian and X. Gao, Multiview face recognition: From TensorFace to V-TensorFace and K-TensorFace IEEE Trans. Syst. ManCybern. 42(2) (2012) 320–333. Crossref, Web of Science, Google Scholar
73. H. Drira and B. B. Amor, 3D face recognition under expressions, occlusions, and pose variations, IEEE Trans. Pattern Anal. Mach. Intell. 35(9) (2013) 2270–2283. Crossref, Web of Science, Google Scholar
74. Duc et al., 3D-model-based video analysis for computer generated faces identification, IEEE Trans. Inform. Forensics Security 10(8) (2015) 1752–1763. Crossref, Web of Science, Google Scholar
75. P. Liu and Y. Wang, Learning the spherical harmonic features for 3D face recognition IEEE Trans. Image Process. 22(3) (2013) 914–925. Crossref, Web of Science, Google Scholar
76. H. Tang, B. Yin and Y. Sun, 3D face recognition using local binary patterns, Signal Process. 93 (2012) 2190–2198. Crossref, Web of Science, Google Scholar
77. Y. Jin, Y. Wang, Q. Ruan and X. Wang, A new scheme for 3D face recognition based on 2-D gabor wavelet transform plus LBP, 6th Int. Conf. Computer Science & Education (ICCSE 2011), IEEE, Singapore (2011), pp. 860–865. Google Scholar
78. P. Perakis and G. Passalis, 3-D facial landmark detection under large yaw and expression variations, IEEE Trans. Pattern Anal. Mach. Intell. 35(7) (2013) 1552–1564. Crossref, Web of Science, Google Scholar
79. N. Alyuz, B. Gokberk and L. Akarun, 3-D face recognition under occlusion using masked projection, IEEE Trans. Inform. Forensics Security 8(5) (2013) 789–802. Crossref, Web of Science, Google Scholar
80. S. Barretti, A. D. Bimbo and P. Pala, Sparse matching of salient facial curves for recognition of 3-D faces with missing parts IEEE Trans. Inform. Forensics Security 8(2) (2013) 374–389. Crossref, Web of Science, Google Scholar
81. M. Hoda and H. Dimitrios, Iterative closest normal point for 3D face recognition IEEE Trans. Pattern Anal. Mach. Intell. 35(2) (2013) 381–397. Crossref, Web of Science, Google Scholar
82. Stefano et al., Face recognition by super-resolved 3D models from consumer depth cameras, IEEE Trans. Inform. Forensics Security 9(9) (2014) 1436–1449. Crossref, Web of Science, Google Scholar
83. S. Jahanbin, H. Choi and A. C. Bovik, Passive multimodal 2-D+3-D face recognition using gabor features and landmark distances IEEE Trans. Information Forensics Security 6(4) (2011) 1287–1304. Crossref, Web of Science, Google Scholar
84. P. S. Hiremath and M. Hiremath, Depth and intensity gabor features based 3-D face recognition using symbolic LDA and AdaBoost Int. J. Image Graphics Signal Process. 1 (2014) 32–39. Google Scholar
85. D. Huang, W. B. Soltana and M. Ardabilian, Extured 3-D face recognition using biological vision-based facial representation and optimized weighted sum fusion, IEEE Int. Conf. Computer Vision and Pattern Recognition Workshops (CVPRW) (2011), pp. 1–8. Google Scholar
86. B. Pradeep et al., Physiology-based face recognition in the thermal infrared spectrum, IEEE Trans. Pattern Anal. Mach. Intell. 29(4) (2007) 613–626. Crossref, Web of Science, Google Scholar
87. R. S. Ghiass et al., Illumination-invariant face recognition from a single image across extreme pose using a dual dimension AAM ensemble in the thermal infrared spectrum, Int. Joint Conf. Neural Networks, Dallas (2013), pp. 1–10. Google Scholar
88. Z. Naser, Thermal face recognition using moments invariants Int. J. Signal Process. Syst. 3(2) (2015) 94–99. Google Scholar
89. A. Seal et al., Minutiae based thermal face recognition using blood perfusion data, International Conference on Image Information Processing (ICIIP) (2011), pp. 1–4. Google Scholar
90. H. S. Jagadeesh and B. K. Suresh, Face recognition based on DWT and DBC, Int. Conf. Computer Communications and Informatics, India (2012), pp. 1–6. Google Scholar
91. F. Juefei, K. Luu, M. Savvides et al., Investigating age invariant face recognition based on periocular biometrics, Int. Joint Conf. Biometrics Compendium (IJCB) (IEEE, Washington DC, USA, 2011), pp. 1–7. Google Scholar
92. Z. Mahmood, T. Ali, S. Khattak and S. U. Khan, A comparative study of baseline algorithms of face recognition, 12th International Conference on Frontiers of Information Technology (FIT) (2014), pp. 263–268. Google Scholar
93. A. M. Guzman et al., Thermal imaging as a biometrics approach to facial signature authentication, IEEE Trans. Inform. Technol. 17(1) (2013) 214–222. Google Scholar
94. N. F. I. Addendum 1, internal document, General method of facial comparison. Google Scholar
95. S. H. Lee and J. Y. Choi, Local color vector binary patterns from multichannel face images for face recognition IEEE Trans. Image Process. 21(4) (2012) 2347–2353. Crossref, Web of Science, Google Scholar
96. W. Ou et al., Robust face recognition via occlusion dictionary learning, Pattern Recognit. 47(4) (2014) 1559–1572. Crossref, Web of Science, Google Scholar
97. Y. Wang et al., Robust 3D face recognition by local shape difference boosting, IEEE Trans. Pattern Anal. Mach. Intell. 32(10) (2010) 1858–1870. Crossref, Web of Science, Google Scholar
98. Y. Sun et al., Deepid3: Face recognition with very deep neural networks, arXiv preprint arXiv. (2015) 1–5. Google Scholar
99. Z. Zhu et al., Deep learning identity-preserving face space, in Proc. IEEE Int. Conf. Computer Vision (2013), pp. 113–120. Google Scholar
100. Y. Sun et al., Deeply learned face representations are sparse, selective, and robust, in Proc. IEEE Conference on Computer Vision and Pattern Recognition (2015), pp. 2892–2900. Google Scholar
101. Z. Zhu, P. Luo, X. Wang and X. Tang, Multi-view perceptron: A deep model for learning face identity and view representations, in Advances in Neural Information Processing Systems (NIPS) (2014), Vol. 12, pp. 217–225. Google Scholar
102. Z. Mahmood, T. Ali and S. U. Khan, Effects of pose and image resolution on automatic face recognition IET Biometrics 5(2) (2016) 111–119. Crossref, Web of Science, Google Scholar