融合時空特征的視頻序列表情識別
doi: 10.11999/JEIT170592
基金項目:
國家自然科學(xué)基金(61672202, 61432004, 61300119)，國家自然科學(xué)基金深圳聯(lián)合基金重點項目(U1613217)，江蘇省物聯(lián)網(wǎng)移動互聯(lián)技術(shù)工程實驗室開放課題(JSWLW-2017-017)
Facial Expression Recognition Based on the Fusion of Spatio-temporal Features in Video Sequences
Funds:
The National Natural Science Foundation of China (61672202, 61432004, 61300119), The National Natural Science Foundation of China -Shenzhen Joint Foundation (Key Project) (U1613217), Open foundation of ?The Laboratory for Internet of Things and Mobile Internet Technology of Jiangsu Province (JSWLW-2017-017)
-
摘要: 針對視頻表情識別，靜態(tài)特征不能有效描述人臉區(qū)域沿時間軸動態(tài)變化信息的局限，該文提出一種融合動態(tài)紋理信息和運動信息的表情識別方法，借鑒LBP-TOP原理，提出具有時空域描述能力的時空韋伯局部描述子(STWLD)來提取動態(tài)紋理信息，同時采用分塊光流直方圖(BHOF)描述運動信息，最后利用SVM對融合后的紋理和運動信息完成表情分類。在CK+和MMI表情數(shù)據(jù)庫上的交叉實驗結(jié)果表明，相比基于單一特征的識別方法，所提方法取得了更好的效果；與其他相關(guān)方法的對比實驗也驗證了該方法的優(yōu)越性。Abstract: For facial expression recognition based on video sequences, the changing information of facial regions along the time axis can be described by dynamic descriptors more effectively than static descriptors. This paper proposes an expression recognition method based on the dynamic texture and motion information, learning from the principle of Local Binary Pattern on Three Orthogonal Planes (LBP-TOP), Spatio-Temporal Weber Local Descriptor (STWLD) is proposed to describe the dynamic texture feature information of the facial expression sequence. Moreover, using Block-based Histogram of Optical Flow features (BHOF), the motion information can be described. Through the combination of the dynamic texture and motion information, and finally SVM is applied to complete the expression classification. The results of the cross experiments on the CK + and MMI expression database show that the method achieves better performance than methods using the single descriptors. The comparison experiments with other related methods also prove the superiority of the method.
-
CHEON Y and KIM D. Natural facial expression recognition using differential-AAM and manifold learning[J]. Pattern Recognition, 2009, 42(7): 1340-1350. doi: 10.1016/j.patcog. 2008.10.010. PAN Z, POLCEANU M, and LISETTI C. On constrained local model feature normalization for facial expression recognition[C]. International Conference on Intelligent Virtual Agents. Los Angeles, CA, USA, 2016: 369-372. doi: 10.1007/978-3-319-47665-0_35. ZHU X and RAMANAN D. Face detection, pose estimation, and landmark localization in the wild[C]. 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, USA, 2012: 2879-2886. doi: 10.1109 /CVPR.2012.6248014. ZHAO L, WANG Z, and ZHANG G. Facial expression recognition from video sequences based on spatial-temporal motion local binary pattern and Gabor multiorientation fusion histogram[J]. Mathematical Problems in Engineering, 2017, (1): 1-12. doi: 10.1155/2017/7206041. ZHOU J, ZHANG S, MEI H, et al. A method of facial expression recognition based on Gabor and NMF[J]. Pattern Recognition and Image Analysis, 2016, 26(1): 119-124. doi: 10.1134/S1054661815040070. CHEN J, SHAN S, HE C, et al. WLD: A robust local image descriptor[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(9): 1705-1720. doi: 10.1109/ TPAMI.2009.155. ZHAO G and PIETIKAINEN M. Dynamic texture recognition using local binary patterns with an application to facial expressions[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29(6): 915-928. doi: 10.1109/ TPAMI.2007.111. 付曉峰, 付曉鵑, 李建軍, 等. 視頻序列中基于多尺度時空局部方向角模式直方圖映射的表情識別[J]. 計算機輔助設(shè)計與圖形學(xué)學(xué)報, 2015, 27(6): 1060-1066. FU Xiaofeng, FU Xiaojuan, LI Jianjun, et al. Facial expression recognition using multi-scale spatiotemporal local orienta-tional pattern histogram projection in video sequences[J]. Journal of Computer Aided Design Computer Graphics, 2015, 27(6): 1060-1066. KAMAROL S K A, JAWARD M H, PARKKINEN J, et al. Spatiotemporal feature extraction for facial expression recognition[J]. IET Image Processing, 2016, 10(7): 534-541. doi: 10.1049/iet-ipr.2015.0519. MEINHARDT-Llopis E, P?REZ J S, and KONDERMANN D. Horn-schunck optical flow with a multi-scale strategy[J]. Image Processing on Line, 2013, 20: 151-172. doi: 10.5201/ ipol.2013.20. 張軒閣, 田彥濤, 顏飛, 等. 基于全局光流特征的微表情識別[J]. 模式識別與人工智能, 2016, 29(8): 760-768. doi: 10.16451 /j.cnki.issn1003-6059.201608011. ZHANG Xuange, TIAN Yantao, YAN Fei, et al. Micro- expression recognition based on global optical flow feature[J]. Pattern Recognition and Artificial Intelligence. 2016, 29(8): 760-768. doi: 10.16451/j.cnki.issn1003-6059.201608011. YACOOB Y and DAVIS L S. Recognizing human facial expressions from long image sequences using optical flow[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1996, 18(6): 636-642. doi: 10.1109/34.506414. LUCEY P, COHN J F, KANADE T, et al. The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression[C]. 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), California, USA, 2010: 94-101. doi: 10.1.1.182.3759. PANTIC M, VALSTAR M, RADEMAKER R, et al. Web-based database for facial expression analysis[C]. IEEE International Conference on Multimedia and Expo, Amsterdam, The Netherlands, 2005: 317-321. doi: 10.1109/ ICME.2005.1521424. 邱玉, 趙杰煜, 汪燕芳. 結(jié)合運動時序性的人臉表情識別方法[J]. 電子學(xué)報, 2016, 44(6): 1307-1313. doi: 10.3969/j.issn. 0372-2112.2016.06.007. QIU Yu, ZHAO Jieyu, and WANG Yanfang. Facial expression recognition using temporal relations among facial movements[J]. Acta Electronica Sinica, 2016, 44(6): 1307-1313. doi: 10.3969/j.issn.0372-2112.2016.06.007. FAN X and TJAHJADI T. A spatial-temporal framework based on histogram of gradients and optical flow for facial expression recognition in video sequences[J]. Pattern Recognition, 2015, 48(11): 3407-3416. doi: 10.1016/j.patcog. 2015.04.025. LONG F and BARTLETT M S. Video-based facial expression recognition using learned spatiotemporal pyramid sparse coding features[J]. Neurocomputing, 2016, 173: 2049-2054. doi: 10.1016/j.neucom.2015.09.049 GUPTA O, RAVIV D, and RASKAR R. Multi-velocity neural networks for facial expression recognition in videos[J]. IEEE Transactions on Affective Computing, 1949, 99: 1. FANG H, MAC Parthalin N, AUBREY A J, et al. Facial expression recognition in dynamic sequences: An integrated approach[J]. Pattern Recognition, 2014, 47(3): 1271-1281. doi: 10.1016/j.patcog.2013.09.023. WANG Z, WANG S, and Ji Q. Capturing complex spatio- temporal relations among facial muscles for facial expression recognition[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA. 2013: 3422-3429. doi: 10.1109/CVPR.2013.439. JUNG H, LEE S, YIM J, et al. Joint fine-tuning in deep neural networks for facial expression recognition[C]. Proceedings of the IEEE International Conference on Computer Vision. Santiago, Chile, 2015: 2983-2991. doi: 10.1109/ICCV.2015.341. -
計量
- 文章訪問數(shù): 1003
- HTML全文瀏覽量: 174
- PDF下載量: 301
- 被引次數(shù): 0
下載:
