基于多尺度細節(jié)增強的面部表情識別方法

譚小慧; 李昭偉; 樊亞春

doi:10.11999/JEIT181088

基于多尺度細節(jié)增強的面部表情識別方法

doi: 10.11999/JEIT181088

譚小慧^{1, 3},
李昭偉^{1, 4},
樊亞春^2, ,

1.
首都師范大學信息工程學院北京 100048
2.
北京師范大學虛擬現(xiàn)實應用教育部工程研究中心北京 100875
3.
首都師范大學電子系統(tǒng)可靠性與數(shù)理交叉學科國家國際科技合作示范型基地 ??北京 ??100048
4.
首都師范大學北京成像理論與技術高精尖創(chuàng)新中心 ??北京 ??100048

基金項目: 國家重點研發(fā)計劃項目(2017YFB1002804)，國家自然科學基金項目(61602324)，浙江大學CAD&CG國家重點實驗室開放課題(A1914)

詳細信息

作者簡介:
譚小慧：女，1977年生，博士，副教授，研究方向為計算機圖形圖像，虛擬現(xiàn)實

李昭偉：男，1992年生，碩士生，研究方向為情感計算

樊亞春：女，1978年生，博士，副教授，研究方向為計算機圖形圖像，人機交互

通訊作者:
樊亞春　fanyachun@hotmail.com

中圖分類號: TP391.41
計量
- 文章訪問數(shù): 4037
- HTML全文瀏覽量: 1641
- PDF下載量: 143
- 被引次數(shù): 0
出版歷程
- 收稿日期: 2018-11-26
- 修回日期: 2019-02-27
- 網(wǎng)絡出版日期: 2019-05-20
- 刊出日期: 2019-11-01

Facial Expression Recognition Method Based on Multi-scale Detail Enhancement

Xiaohui TAN^{1, 3},
Zhaowei LI^{1, 4},
Yachun FAN^{2
, ,}

1.
College of Information Engineering, Capital Normal University, Beijing 100048, China
2.
College of Information Science and Technology, Beijing Normal University, Beijing 100875, China
3.
Beijing Key Laboratory of Electronic System Reliability and Prognostics, Capital Normal University, Beijing 100048, China
4.
Beijing Advanced Innovation Center for Imaging Technology, Capital Normal University, Beijing 100048, China

Funds: The National Key R&D Program of China(2017YFB1002804), The National Natural Science Foundation of China(61602324), The Open Project of State Key Lab of CAD & CG, Zhejiang University(A1914)

摘要

摘要: 人類面部表情是其心理情緒變化的最直觀刻畫，不同人的面部表情具有很大差異，現(xiàn)有表情識別方法均利用面部統(tǒng)計特征區(qū)分不同表情，其缺乏對于人臉細節(jié)信息的深度挖掘。根據(jù)心理學家對面部行為編碼的定義可以看出，人臉的局部細節(jié)信息決定了其表情意義。因此該文提出一種基于多尺度細節(jié)增強的面部表情識別方法，針對面部表情受圖像細節(jié)影響較大的特點，提出利用高斯金字塔提取圖像細節(jié)信息，并對圖像進行細節(jié)增強，從而強化人臉表情信息。針對面部表情的局部性特點，提出利用層次結構的局部梯度特征計算方法，描述面部特征點局部形狀特征。最后，使用支持向量機(SVM)對面部表情進行分類。該文在CK+表情數(shù)據(jù)庫中的實驗結果表明，該方法不僅驗證了圖像細節(jié)對面部表情識別過程的重要作用，而且在小規(guī)模訓練數(shù)據(jù)下也能夠得到非常好的識別結果，表情平均識別率達到98.19%。
- 表情識別 /
- 圖像金字塔 /
- 高斯差分 /
- 細節(jié)增強 /
- 支持向量機
Abstract: Facial expression is the most intuitive description of changes in psychological emotions, and different people have great differences in facial expressions. The existing facial expression recognition methods use facial statistical features to distinguish among different expressions, but these methods are short of deep exploration for facial detail information. According to the definition of facial behavior coding by psychologists, it can be seen that the local detail information of the face determines the meaning of facial expression. Therefore, a facial expression recognition method based on multi-scale detail enhancement is proposed, because facial expression is much more affected by the image details than other information, the method proposed in this paper extracts the image detail information with the Gaussian pyramid firstly, thus the image is enhanced in detail to enrich the facial expression information. Secondly, for the local characteristics of facial expressions, a local gradient feature calculation method is proposed based on hierarchical structure to describe the local shape features of facial feature points. Finally, facial expressions are classified using a Support Vector Machine (SVM). The experimental results in the CK+ expression database show that the method not only proves the important role of image detail in facial expression recognition, but also obtains very good recognition results under small-scale training data. The average recognition rate of expressions reaches 98.19%.
- Expression recognition /
- Image pyramid /
- Gauss difference /
- Detail enhancement /
- Support Vector Machine(SVM)

HTML全文

圖 1 多尺度細節(jié)增強人臉表情識別流程

下載: 全尺寸圖片幻燈片

圖 2 特征點分布圖

下載: 全尺寸圖片幻燈片

圖 3 高斯圖像差分

下載: 全尺寸圖片幻燈片

圖 4 細節(jié)特征疊加圖像

下載: 全尺寸圖片幻燈片

圖 5 細節(jié)增強前后對比

下載: 全尺寸圖片幻燈片

圖 6 像素區(qū)域

下載: 全尺寸圖片幻燈片

圖 7 特征點及區(qū)域圖

下載: 全尺寸圖片幻燈片

圖 8 CK+表情庫7種表情示例圖像

下載: 全尺寸圖片幻燈片

圖 9 實驗結果比較

下載: 全尺寸圖片幻燈片

圖 10 P-R曲線

下載: 全尺寸圖片幻燈片

圖 11 多尺度細節(jié)增強效果對比圖

下載: 全尺寸圖片幻燈片

表 1 高斯模糊半徑取值的最優(yōu)識別率(%)

高斯核	中性	憤怒	厭惡	恐懼	微笑	悲傷	驚訝	整體
K₁	98.00	99.33	100	96.67	100	94.67	96.00	97.81
K₂	96.00	98.67	100	95.33	100	96.67	98.00	97.81

下載: 導出CSV

表 2 表情識別率分布表(%)

	中性	憤怒	厭惡	恐懼	高興	悲傷	驚訝
中性	98.00	0	0	0	0	2.00	0
憤怒	0	100.00	0	0	0	0	0
厭惡	0	0	99.33	0	0	0.67	0
恐懼	0	0	0	95.33	2.67	0	2.00
高興	0	0	0	0	100	0	0
悲傷	3.33	0	0	0	0	96.67	0
驚訝	1.33	0	0	0.67	0	0	98.00

下載: 導出CSV

表 3 本文方法與CNN與LeNet-5方法比較(%)

方法	中性	憤怒	厭惡	恐懼	微笑	悲傷	驚訝	整體
CNN^[11]	95.15	91.11	99.44	92.00	100	82.14	98.80	95.75
LeNet-5^[15]	65.37	76.30	87.59	80.92	94.26	82.23	94.55	83.74
本文方法	98.00	100	99.33	95.33	100	96.67	98.00	98.19

下載: 導出CSV

表 4 不同方法識別率對比(%)

方法	識別率
LBP^[16]	92.30
Gabor^[17]	98.10
CNN^[11]	95.75
I²CNN^[12]	96.20
LeNet-5^[15]	83.74
本文方法	98.19

下載: 導出CSV

參考文獻(17)

MEHRABIAN A. Communication without words[J]. Psychology Today, 1968, 2(4): 53–56.

GUO Zhenhua, ZHANG Lei, and ZHANG D. A completed modeling of local binary pattern operator for texture classification[J]. IEEE Transactions on Image Processing, 2010, 19(6): 1657–1663. doi: 10.1109/TIP.2010.2044957

王瑋, 黃非非, 李見為, 等. 采用LBP金字塔的人臉描述與識別[J]. 計算機輔助設計與圖形學學報, 2009, 21(1): 94–100, 106.

WANG Wei, HUANG Feifei, LI Jianwei, et al. Face description and recognition by LBP pyramid[J]. Journal of Computer-Aided Design &Computer Graphics, 2009, 21(1): 94–100, 106.

鐘思志. 人臉面部表情識別算法研究[D]. [碩士論文], 華東師范大學, 2015.

ZHONG Sizhi. Research on facial expression recognition[D]. [Master dissertation], East China Normal University, 2015.

楊凡, 張磊. 基于Gabor參數(shù)矩陣與改進Adaboost的人臉表情識別[J]. 計算機應用, 2014, 34(4): 1134–1138.

YANG Fan and ZHANG Lei. Facial expression recognition based on Gabor parameters matrix and improved Adaboost[J]. Journal of Computer Applications, 2014, 34(4): 1134–1138.

童瑩. 基于空間多尺度HOG特征的人臉表情識別方法[J]. 計算機工程與設計, 2014, 35(11): 3918–3922, 3979. doi: 10.3969/j.issn.1000-7024.2014.11.041

TONG Ying. Facial expression recognition algorithm based on spatial multi-scaled HOG feature[J]. Computer Engineering and Design, 2014, 35(11): 3918–3922, 3979. doi: 10.3969/j.issn.1000-7024.2014.11.041

TURAN C and LAM K M. Region-based feature fusion for facial-expression recognition[C]. Proceedings of 2014 IEEE International Conference on Image Processing, Paris, France, 2014: 5966–5970.

SAEED A, AL-HAMADI A, and NIESE R. The effectiveness of using geometrical features for facial expression recognition[C]. Proceedings of 2013 IEEE International Conference on Cybernetics, Lausanne, Switzerland, 2013: 122–127.

CHEN Junkai, CHEN Zenghai, CHI Zheru, et al. Facial expression recognition in video with multiple feature fusion[J]. IEEE Transactions on Affective Computing, 2018, 9(1): 38–50. doi: 10.1109/TAFFC.2016.2593719

任福繼, 于曼麗, 胡敏, 等. 融合表情和BVP生理信號的雙模態(tài)視頻情感識別[J]. 中國圖象圖形學報, 2018, 23(5): 688–697.

REN Fuji, YU Manli, HU Min, et al. Dual-modality video emotion recognition based on facial expression and BVP physiological signal[J]. Journal of Image and Graphics, 2018, 23(5): 688–697.

LOPES A T, DE AGUIAR E, DE SOUZA A F, et al. Facial expression recognition with convolutional neural networks: Coping with few data and the training sample order[J]. Pattern Recognition, 2017, 61: 610–628. doi: 10.1016/j.patcog.2016.07.026

ZHANG Chongsheng, WANG Pengyou, CHEN Ke, et al. Identity-aware convolutional neural networks for facial expression recognition[J]. Journal of Systems Engineering and Electronics, 2017, 28(4): 784–792.

LUCEY P, COHN J F, KANADE T, et al. The extended cohn-kanade dataset (CK+): A complete dataset for action unit and emotion-specified expression[C]. Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops, San Francisco, USA, 2010: 94–101.

YAO Yongqiang, HUANG Di, YANG Xudong, et al. Texture and geometry scattering representation-based facial expression recognition in 2D+3D videos[J]. ACM Transactions on Multimedia Computing, Communications, and Applications, 2018, 14(1S): 18.

李勇, 林小竹, 蔣夢瑩. 基于跨連接LeNet-5網(wǎng)絡的面部表情識別[J]. 自動化學報, 2018, 44(1): 176–182.

LI Yong, LIN Xiaozhu, and JIANG Mengying. Facial expression recognition with cross-connect LeNet-5 network[J]. Acta Automatica Sinica, 2018, 44(1): 176–182.

JIA Qi, GAO Xinkai, GUO He, et al. Multi-layer sparse representation for weighted LBP-patches based facial expression recognition[J]. Sensors, 2015, 15(3): 6719–6739. doi: 10.3390/s150306719

ZHOU Jun, ZHANG Sue, MEI Hongyan, et al. A method of facial expression recognition based on Gabor and NMF[J]. Pattern Recognition and Image Analysis, 2016, 26(1): 119–124. doi: 10.1134/S1054661815040070

施引文獻

資源附件(0)

訪問統(tǒng)計