基于區(qū)域協(xié)方差的視頻顯著度局部空時優(yōu)化模型
doi: 10.11999/JEIT151122
基金項(xiàng)目:
國家自然科學(xué)基金青年基金(61501509)
A Local Spatiotemporal Optimization Framework for Video Saliency Detection Using Region Covariance
Funds:
The National Natural Science Youth Foundation of China (61501509)
-
摘要: 顯著度檢測在計(jì)算機(jī)視覺中應(yīng)用非常廣泛，圖像級的顯著度檢測研究已較為成熟，但視頻顯著度因其高度挑戰(zhàn)性研究相對較少。該文借鑒圖像級顯著度算法的思想，提出一種通用的空時特征提取與優(yōu)化模型來檢測視頻顯著度。首先利用區(qū)域協(xié)方差矩陣構(gòu)造視頻的空時特征描述子，然后計(jì)算對比度得出初始顯著圖，最后通過聯(lián)合前后幀的局部空時優(yōu)化模型得到最終的顯著圖。在2個公開視頻顯著性數(shù)據(jù)集上的實(shí)驗(yàn)結(jié)果表明，所提算法性能優(yōu)于目前的主流算法，同時具有良好的擴(kuò)展性。
-
關(guān)鍵詞:
- 視頻顯著度 /
- 區(qū)域協(xié)方差 /
- 局部對比度 /
- 局部空時優(yōu)化
Abstract: Visual saliency is widely applied to computer vision. Image saliency detection has been extensively studied, while there are only a few effective methods of computing saliency for videos owing to its high challenge. Inspired by image saliency methods, this paper proposes a unified spatiotemporal feature extraction and optimization framework for video saliency. First, the spatiotemporal feature descriptor is constructed via region covariance. Then, initial saliency map is computed by the local contrast of the descriptor. Finally, a local spatiotemporal optimization framework considering the previous and next frames of the current one is modeled to obtain the final saliency map. Extensive experiments on two public datasets demonstrate that the proposed algorithm not only outperforms the state-of-the-art methods, but also is of great extendibility.-
Key words:
- Video saliency /
- Region covariance /
- Local contrast /
- Local spatiotemporal optimization
-
BORJI A, CHENG M, JIANG H, et al. Salient object detection: A benchmark[J]. IEEE Transactions on Image Processing, 2015, 24(12): 5706-5722. doi: 10.1109/ TIP.2015.2487833. ROTHER C, KOLMOGOROV V, and BLAKE A. Grabcut: Interactive foreground extraction using iterated graph cuts[J]. ACM Transactions on Graphics, 2004, 23(1): 309-314. doi: 10.1145/1186562.1015720. DING Y, XIAO J, and YU J. Importance filtering for image retargeting[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Colorado Springs, USA, 2011: 89-96. doi: 10.1109/CVPR.2011.5995445. MAHADEVAN V and VASOONCEIOS N. Saliency-based discriminant tracking[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Miami, USA, 2009: 1007-1013. doi: 10.1109/CVPR.2009.5206573. SHARMA G, JURIE F, and SCHMID C. Discriminative spatial saliency for image classification[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Rhode Island, USA, 2012: 3506-3513. doi: 10.1109/ CVPR.2012.6248093. HADIZADEH H and BAJIC I. Saliency-aware video compression[J]. IEEE Transactions on Image Processing, 2014, 23(1): 19-33. doi: 10.1109/TIP.2013.2282897. CHENG M, ZHANG G, MIERA N, et al. Global contrast based salient region detection[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Colorado Springs, USA, 2011: 409-416. doi: 10.1109/ CVPR.2011.5995344. PERAZZI F, KRAHENBUHL P, PRITCH Y, et al. Saliency filters: Contrast based filtering for salient region detection[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Rhode Island, USA, 2012: 733-740. doi: 10.1109/CVPR.2012.6247743. YANG C, ZHANG L, LU H, et al. Saliency detection via graph-based manifold ranking[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Portland, USA, 2013: 3166-3173. doi: 10.1109/CVPR. 2013.407. ZHU W, LIANG S, WEI Y, et al. Saliency optimization from robust background detection[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, USA, 2014: 2814-2821. doi: 10.1109/ CVPR. 2014.360. QIN Y, LU H, XU Y, et al. Saliency detection via cellular automata[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, USA, 2015: 111-119. doi: 10.1109/CVPR.2015.7298606. 蔣寓文, 譚樂怡, 王守覺. 選擇性背景優(yōu)先的顯著性檢測模型[J]. 電子與信息學(xué)報(bào), 2015, 37(1): 130-136. doi: 10.11999/JEIT140119. JIANG Yuwen, TAN Leyi, and WANG Shoujue. Saliency detected model based on selective edges prior[J]. Journal of Electronics Information Technology, 2015, 37(1): 130-136. doi: 10.11999/JEIT140119. 呂建勇, 唐振民. 一種基于圖的流形排序的顯著性目標(biāo)檢測改進(jìn)方法[J]. 電子與信息學(xué)報(bào), 2015, 37(11): 2555-2563. doi: 10.11999/JEIT150619. LV Jianyong and TANG Zhenmin. An improved graph-based manifold ranking for salient object detection[J]. Journal of Electronics Information Technology, 2015, 37(11): 2555-2563. doi: 10.11999/JEIT150619. MAHADEVAN V and VASCONCELOS N. Spatiotemporal saliency in dynamic scenes[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(1): 171-177. doi: 10.1109/TPAMI.2009.112. ZHOU F, KANG S B, and COHEN M F. Time-mapping using space-time saliency[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, USA, 2014: 3358-3365. doi: 10.1109/CVPR.2014.429. HUANG C, CHANG Y, YANG Z, et al. Video saliency map detection by dominant camera motion removal[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2014, 24(8): 1336-1349. doi: 10.1109/TCSVT.2014.2308652. LIU Z, ZHANG X, LUO S, et al. Superpixel-based spatiotemporal saliency detection[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2014, 24(9): 1522-1540. doi: 10.1109/TCSVT.2014.2308642. WANG W, SHEN J, and SHAO L. Consistent video saliency using local gradient flow optimization and global refinement[J]. IEEE Transactions on Image Processing, 2015, 24(10): 1-12. doi: 10.1109/TIP.2015.2460013. MUTHUSWAMY K and RAJAN D. Particle filter framework for salient object detection in videos[J]. IET Computer Vision, 2015, 9(3): 428-438. doi: 10.1049/ iet-cvi.2013.0298. KIM H, KIM Y, SIM J, et al. Spatiotemporal saliency detection for video sequences based on random walk with restart[J]. IEEE Transactions on Image Processing, 2015, 24(8): 2552-2564. doi: 10.1109/TIP.2015.2425544. WANG W, SHEN J, and PORIKLI F. Saliency-aware geodesic video object segmentation[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, USA, 2015: 3395-3402. doi: 10.1109/CVPR.2015.7298961. KIM W and HAN J. Video saliency detection using contrast of spatiotemporal directional coherence[J]. IEEE Signal Processing Letters, 2014, 21(10): 1250-1254. doi: 10.1109/ LSP.2014.2332213. ERDEM E and ERDEM A. Visual saliency estimation by nonlinearly integrating features using region covariances[J]. Journal of Vision, 2013, 13(4): 1-20. doi: 10.1167/13.4.11. KOCAK A, CIZMECILERr K, ERDEM A, et al. Top down saliency estimation via superpixel-based discriminative dictionaries[C]. British Machine Vision Conference (BMVC), Nottingham, UK, 2014: 1-10. doi: 10.5244/C.28.73. CHANG K, LIU Y, CHEN H, et al. Fusing generic objectness and visual saliency for salient object detection[C]. IEEE Conference on Computer Vision (ICCV), Barcelona, Spain, 2011: 914-921. doi: 10.1109/ICCV.2011.6126333. LI J, TIAN Y, DUAN L, et al. Estimating visual saliency through single image optimization[J]. IEEE Signal Processing Letters, 2013, 20(9): 845-848. doi: 10.1109/LSP. 2013.2268868. TUAEL O, PORIKLI F, and MEER P. Region covariance: A fast descriptor for detection and classification[C]. European Conference on Computer Vision, Graz, Austria, 2006: 589-600. doi: 10.1007/11744047_45. ACHANTA R, SHAJI A, SMITH K, et al. Slic superpixels compared to state-of-the-art superpixel methods[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(11): 2274-2281. doi: 10.1109/TPAMI.2012.120. BROX T and MALIK J. Large displacement optical flow: Descriptor matching in variational motion estimation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 33(3): 500-513. doi: 10.1109/TPAMI. 2010.143. HONG X, CHANG H, and SHAN S. Sigma set: A small second order statistical region descriptor[C]. IEEE Conference on Computer Vision and Pattern Recognition, Miami, USA, 2009: 1802-1809. doi: 10.1109/CVPR. 2009.5206742. TSAI D, FLAGG M, and REHG J M. Motion coherent tracking with multilabel MRF optimization[C]. British Machine Vision Conference (BMVC), Aberystwyth, UK, 2010: 1-11. doi: 10.5244/C.24.56. -
計(jì)量
- 文章訪問數(shù): 1420
- HTML全文瀏覽量: 147
- PDF下載量: 665
- 被引次數(shù): 0
下載:
