基于多尺度細節(jié)增強的面部表情識別方法
doi: 10.11999/JEIT181088
-
1.
首都師范大學信息工程學院 北京 100048
-
2.
北京師范大學虛擬現(xiàn)實應用教育部工程研究中心 北京 100875
-
3.
首都師范大學電子系統(tǒng)可靠性與數(shù)理交叉學科國家國際科技合作示范型基地 ??北京 ??100048
-
4.
首都師范大學北京成像理論與技術高精尖創(chuàng)新中心 ??北京 ??100048
Facial Expression Recognition Method Based on Multi-scale Detail Enhancement
-
1.
College of Information Engineering, Capital Normal University, Beijing 100048, China
-
2.
College of Information Science and Technology, Beijing Normal University, Beijing 100875, China
-
3.
Beijing Key Laboratory of Electronic System Reliability and Prognostics, Capital Normal University, Beijing 100048, China
-
4.
Beijing Advanced Innovation Center for Imaging Technology, Capital Normal University, Beijing 100048, China
-
摘要: 人類面部表情是其心理情緒變化的最直觀刻畫,不同人的面部表情具有很大差異,現(xiàn)有表情識別方法均利用面部統(tǒng)計特征區(qū)分不同表情,其缺乏對于人臉細節(jié)信息的深度挖掘。根據(jù)心理學家對面部行為編碼的定義可以看出,人臉的局部細節(jié)信息決定了其表情意義。因此該文提出一種基于多尺度細節(jié)增強的面部表情識別方法,針對面部表情受圖像細節(jié)影響較大的特點,提出利用高斯金字塔提取圖像細節(jié)信息,并對圖像進行細節(jié)增強,從而強化人臉表情信息。針對面部表情的局部性特點,提出利用層次結構的局部梯度特征計算方法,描述面部特征點局部形狀特征。最后,使用支持向量機(SVM)對面部表情進行分類。該文在CK+表情數(shù)據(jù)庫中的實驗結果表明,該方法不僅驗證了圖像細節(jié)對面部表情識別過程的重要作用,而且在小規(guī)模訓練數(shù)據(jù)下也能夠得到非常好的識別結果,表情平均識別率達到98.19%。Abstract: Facial expression is the most intuitive description of changes in psychological emotions, and different people have great differences in facial expressions. The existing facial expression recognition methods use facial statistical features to distinguish among different expressions, but these methods are short of deep exploration for facial detail information. According to the definition of facial behavior coding by psychologists, it can be seen that the local detail information of the face determines the meaning of facial expression. Therefore, a facial expression recognition method based on multi-scale detail enhancement is proposed, because facial expression is much more affected by the image details than other information, the method proposed in this paper extracts the image detail information with the Gaussian pyramid firstly, thus the image is enhanced in detail to enrich the facial expression information. Secondly, for the local characteristics of facial expressions, a local gradient feature calculation method is proposed based on hierarchical structure to describe the local shape features of facial feature points. Finally, facial expressions are classified using a Support Vector Machine (SVM). The experimental results in the CK+ expression database show that the method not only proves the important role of image detail in facial expression recognition, but also obtains very good recognition results under small-scale training data. The average recognition rate of expressions reaches 98.19%.
-
表 1 高斯模糊半徑取值的最優(yōu)識別率(%)
高斯核 中性 憤怒 厭惡 恐懼 微笑 悲傷 驚訝 整體 K1 98.00 99.33 100 96.67 100 94.67 96.00 97.81 K2 96.00 98.67 100 95.33 100 96.67 98.00 97.81 下載: 導出CSV
表 2 表情識別率分布表(%)
中性 憤怒 厭惡 恐懼 高興 悲傷 驚訝 中性 98.00 0 0 0 0 2.00 0 憤怒 0 100.00 0 0 0 0 0 厭惡 0 0 99.33 0 0 0.67 0 恐懼 0 0 0 95.33 2.67 0 2.00 高興 0 0 0 0 100 0 0 悲傷 3.33 0 0 0 0 96.67 0 驚訝 1.33 0 0 0.67 0 0 98.00 下載: 導出CSV
-
MEHRABIAN A. Communication without words[J]. Psychology Today, 1968, 2(4): 53–56. GUO Zhenhua, ZHANG Lei, and ZHANG D. A completed modeling of local binary pattern operator for texture classification[J]. IEEE Transactions on Image Processing, 2010, 19(6): 1657–1663. doi: 10.1109/TIP.2010.2044957 王瑋, 黃非非, 李見為, 等. 采用LBP金字塔的人臉描述與識別[J]. 計算機輔助設計與圖形學學報, 2009, 21(1): 94–100, 106.WANG Wei, HUANG Feifei, LI Jianwei, et al. Face description and recognition by LBP pyramid[J]. Journal of Computer-Aided Design &Computer Graphics, 2009, 21(1): 94–100, 106. 鐘思志. 人臉面部表情識別算法研究[D]. [碩士論文], 華東師范大學, 2015.ZHONG Sizhi. Research on facial expression recognition[D]. [Master dissertation], East China Normal University, 2015. 楊凡, 張磊. 基于Gabor參數(shù)矩陣與改進Adaboost的人臉表情識別[J]. 計算機應用, 2014, 34(4): 1134–1138.YANG Fan and ZHANG Lei. Facial expression recognition based on Gabor parameters matrix and improved Adaboost[J]. Journal of Computer Applications, 2014, 34(4): 1134–1138. 童瑩. 基于空間多尺度HOG特征的人臉表情識別方法[J]. 計算機工程與設計, 2014, 35(11): 3918–3922, 3979. doi: 10.3969/j.issn.1000-7024.2014.11.041TONG Ying. Facial expression recognition algorithm based on spatial multi-scaled HOG feature[J]. Computer Engineering and Design, 2014, 35(11): 3918–3922, 3979. doi: 10.3969/j.issn.1000-7024.2014.11.041 TURAN C and LAM K M. Region-based feature fusion for facial-expression recognition[C]. Proceedings of 2014 IEEE International Conference on Image Processing, Paris, France, 2014: 5966–5970. SAEED A, AL-HAMADI A, and NIESE R. The effectiveness of using geometrical features for facial expression recognition[C]. Proceedings of 2013 IEEE International Conference on Cybernetics, Lausanne, Switzerland, 2013: 122–127. CHEN Junkai, CHEN Zenghai, CHI Zheru, et al. Facial expression recognition in video with multiple feature fusion[J]. IEEE Transactions on Affective Computing, 2018, 9(1): 38–50. doi: 10.1109/TAFFC.2016.2593719 任福繼, 于曼麗, 胡敏, 等. 融合表情和BVP生理信號的雙模態(tài)視頻情感識別[J]. 中國圖象圖形學報, 2018, 23(5): 688–697.REN Fuji, YU Manli, HU Min, et al. Dual-modality video emotion recognition based on facial expression and BVP physiological signal[J]. Journal of Image and Graphics, 2018, 23(5): 688–697. LOPES A T, DE AGUIAR E, DE SOUZA A F, et al. Facial expression recognition with convolutional neural networks: Coping with few data and the training sample order[J]. Pattern Recognition, 2017, 61: 610–628. doi: 10.1016/j.patcog.2016.07.026 ZHANG Chongsheng, WANG Pengyou, CHEN Ke, et al. Identity-aware convolutional neural networks for facial expression recognition[J]. Journal of Systems Engineering and Electronics, 2017, 28(4): 784–792. LUCEY P, COHN J F, KANADE T, et al. The extended cohn-kanade dataset (CK+): A complete dataset for action unit and emotion-specified expression[C]. Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops, San Francisco, USA, 2010: 94–101. YAO Yongqiang, HUANG Di, YANG Xudong, et al. Texture and geometry scattering representation-based facial expression recognition in 2D+3D videos[J]. ACM Transactions on Multimedia Computing, Communications, and Applications, 2018, 14(1S): 18. 李勇, 林小竹, 蔣夢瑩. 基于跨連接LeNet-5網(wǎng)絡的面部表情識別[J]. 自動化學報, 2018, 44(1): 176–182.LI Yong, LIN Xiaozhu, and JIANG Mengying. Facial expression recognition with cross-connect LeNet-5 network[J]. Acta Automatica Sinica, 2018, 44(1): 176–182. JIA Qi, GAO Xinkai, GUO He, et al. Multi-layer sparse representation for weighted LBP-patches based facial expression recognition[J]. Sensors, 2015, 15(3): 6719–6739. doi: 10.3390/s150306719 ZHOU Jun, ZHANG Sue, MEI Hongyan, et al. A method of facial expression recognition based on Gabor and NMF[J]. Pattern Recognition and Image Analysis, 2016, 26(1): 119–124. doi: 10.1134/S1054661815040070 -