基于空間分布分析的混合失真無參考圖像質(zhì)量評價

陳勇; 朱凱欣; 房昊; 劉煥淋

doi:10.11999/JEIT190721

基于空間分布分析的混合失真無參考圖像質(zhì)量評價

doi: 10.11999/JEIT190721

陳勇^1, ,,
朱凱欣¹,
房昊¹,
劉煥淋²

1.
重慶郵電大學工業(yè)物聯(lián)網(wǎng)與網(wǎng)絡化控制教育部重點實驗室重慶 400065
2.
重慶郵電大學通信與信息工程學院重慶 400065

基金項目: 國家自然科學基金(51977021)

詳細信息

作者簡介:
陳勇：男，1963年生，博士，教授，研究方向為圖像處理

朱凱欣：女，1994年，碩士生，研究方向為無參考圖像質(zhì)量評價和立體圖像質(zhì)量評價

房昊：男，1993年，碩士，研究方向為無參考圖像質(zhì)量評價

劉煥淋：女，1970年生，博士，教授，研究方向為信號處理等方面

通訊作者:
陳勇　chenyong@cqupt.edu.cn

中圖分類號: TN911.73; TP391.41
計量
- 文章訪問數(shù): 2930
- HTML全文瀏覽量: 957
- PDF下載量: 103
- 被引次數(shù): 0
出版歷程
- 收稿日期: 2019-09-17
- 修回日期: 2020-02-16
- 網(wǎng)絡出版日期: 2020-03-09
- 刊出日期: 2020-10-13

No-reference Image Quality Evaluation for Multiply-distorted Images Based on Spatial Domain Coding

1.
Key Laboratory of Industrial Internet of Things & Network Control, Ministry of Education, Chongqing University of Posts and Telecommunications, Chongqing 400065, China
2.
School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, China

Funds: The National Natural Science Foundation of China (51977021)

摘要

摘要: 針對難以準確有效地提取混合失真圖像質(zhì)量特征的問題，該文提出一種基于空間分布分析的圖像質(zhì)量評價方法。首先將圖像進行亮度系數(shù)歸一化處理，然后將圖像進行分塊，利用卷積神經(jīng)網(wǎng)絡(CNN)進行端對端的深度學習，采用多層次卷積核堆疊的方法獲取圖像的質(zhì)量感知特征，并通過全連接層將特征映射到圖像塊的質(zhì)量分數(shù)。再將塊質(zhì)量分數(shù)匯總獲取質(zhì)量池，通過對質(zhì)量池中局部質(zhì)量的空間分布情況進行分析，提取能夠表征其空間分布情況的特征，然后采用神經(jīng)網(wǎng)絡建立局部質(zhì)量到整體質(zhì)量的映射模型，將圖像的局部質(zhì)量進行匯總。最后在MLIVE, MDID2013, MDID2016混合失真圖像庫中進行性能測試以及與相關的對比算法進行比較，驗證了該算法的有效性。
- 圖像質(zhì)量評價 /
- 無參考 /
- 卷積神經(jīng)網(wǎng)絡
Abstract: Considering the problem that it is difficult to accurately and effectively extract the quality features of mixed distortion image, an image quality assessment method based on spatial distribution analysis is proposed. Firstly, the brightness coefficients of the image are normalized, and the image is divided into blocks. While the Convolutional Neural Network (CNN) is used for end-to-end depth learning, the multi-level stacking of convolution cores is applied to acquire image quality perception features. The feature is mapped to the mass fraction of the image block through the full connection layer, then the quality pool is obtained by aggregating the quality of the block. Through the analysis of the spatial distribution of local quality in the quality pool, the features that can represent its spatial distribution are extracted, and then the mapping model from local quality to overall quality is established by the neural network to aggregate the local quality of the image. Finally, the effectiveness of the algorithm is verified by the performance tests in MLIVE, MDID2013 and MDID2016 mixed distortion image databases.
- Image quality assessment /
- No-reference /
- Convolutional Neural Network (CNN)

HTML全文

圖 1 CNN中各層網(wǎng)絡結構

下載: 全尺寸圖片幻燈片

圖 2 “baby girl”失真圖像與其可視化質(zhì)量池

下載: 全尺寸圖片幻燈片

圖 3 質(zhì)量池與其直方圖統(tǒng)計

下載: 全尺寸圖片幻燈片

圖 4 基于空間分布分析的圖像質(zhì)量評價方法流程圖

下載: 全尺寸圖片幻燈片

圖 5 算法的收斂性

下載: 全尺寸圖片幻燈片

圖 6 MLIVE失真圖像的客觀評價值與DMOS的散點圖

下載: 全尺寸圖片幻燈片

表 1 混合失真圖像庫描述

圖像庫	參考圖像	失真類型	圖像數(shù)	主觀評分
MLIVE	15	模糊+噪聲/模糊+JPEG壓縮	450	0-100(DMOS)
MDID2013	12	模糊+噪聲+JPEG壓縮	324	0-1(DMOS)
MDID2016	20	模糊+噪聲+對比度+JPEG壓縮+JP2K壓縮	1600	0-8(MOS)

下載: 導出CSV

表 2 MLIVE圖像庫中各特征性有效性實驗

特征	PLCC	SROCC	KROCC
均值	0.951	0.941	0.753
方差	0.795	0.740	0.625
偏斜度	0.570	0.472	0.334
峰度	0.461	0.493	0.348
整體評價	0.961	0.951	0.781

下載: 導出CSV

表 3 不同圖像庫中算法性能測試

圖像庫	PLCC	SROCC	KROCC	RMSE
MLIVE(Part1)	0.969	0.956	0.822	4.502
MLIVE(Part2)	0.957	0.942	0.784	4.944
MLIVE(All)	0.961	0.951	0.781	4.831
MDID2013	0.935	0.922	0.755	0.017
MDID2016	0.921	0.917	0.749	0.756

下載: 導出CSV

表 4 算法性能對比實驗

算法		MLIVE(450 images)			MDID2013(324 images)
算法		PLCC	SROCC	RMSE	PLCC	SROCC	RMSE
PSNR	FR	0.740	0.677	12.724	0.561	0.560	0.042
SSIM	FR	0.926	0.902	6.797	0.457	0.450	0.045
VIF^[18]	FR	0.932	0.915	6.761	0.915	0.905	0.020
BRISQUE^[5]	NR	0.924	0.900	7.143	0.833	0.819	0.027
NFERM^[6]	NR	0.917	0.898	7.459	0.871	0.855	0.024
GWH-LBP^[7]	NR	0.949	0.944	8.873	0.913	0.908	0.019
HOSA^[8]	NR	0.926	0.902	6.974	0.892	0.872	0.021
Zhou^[10]	NR	0.951	0.943	5.747	0.919	0.907	0.019
CORNIA^[11]	NR	0.916	0.900	7.586	0.904	0.898	0.020
NIQE^[19]	NR	0.839	0.775	10.294	0.563	0.545	0.042
SISBLM^[20]	NR	0.895	0.878	8.439	0.814	0.808	0.030
本文算法	NR	0.961	0.951	4.831	0.935	0.922	0.017

下載: 導出CSV

表 5 各算法時間復雜度對比實驗(s)

算法	SSIM	VIF	GWH-LBP	SISBLM	本文算法
時間	0.102	5.236	0.657	2.486	0.842

下載: 導出CSV

參考文獻(20)

GU Ke, TAO Dacheng, QIAO Junfei, et al. Learning a no-reference quality assessment model of enhanced images with big data[J]. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(4): 1301–1313. doi: 10.1109/TNNLS.2017.2649101

FREITAS P G, AKAMINE W Y L, and FARIAS M C Q. No-Reference image quality assessment using orthogonal color planes patterns[J]. IEEE Transactions on Multimedia, 2018, 20(12): 3353–3360. doi: 10.1109/TMM.2018.2839529

張敏輝, 楊劍. 評價SAR圖像去噪效果的無參考圖像質(zhì)量指標[J]. 重慶郵電大學學報: 自然科學版, 2018, 30(4): 530–536. doi: 10.3979/j.issn.1673-825X.2018.04.014

ZHANG Minhui and YANG Jian. A new referenceless image quality index to evaluate denoising performance of SAR images[J]. Journal of Chongqing University of Posts and Telecommunications:Natural Science Edition, 2018, 30(4): 530–536. doi: 10.3979/j.issn.1673-825X.2018.04.014

徐弦秋, 劉宏清, 黎勇, 等. 基于RGB通道下模糊核估計的圖像去模糊[J]. 重慶郵電大學學報: 自然科學版, 2018, 30(2): 216–221. doi: 10.3979/j.issn.1673-825X.2018.02.009

XU Xianqiu, LIU Hongqing, LI Yong, et al. Image deblurring with blur kernel estimation in RGB channels[J]. Journal of Chongqing University of Posts and Telecommunications:Natural Science Edition, 2018, 30(2): 216–221. doi: 10.3979/j.issn.1673-825X.2018.02.009

MITTAL A, MOORTHY A K, and BOVIK A C. No-reference image quality assessment in the spatial domain[J]. IEEE Transactions on Image Processing, 2012, 21(12): 4695–4708. doi: 10.1109/TIP.2012.2214050

GU Ke, ZHAI Guangtao, YANG Xiaokang, et al. Using free energy principle for blind image quality assessment[J]. IEEE Transactions on Multimedia, 2015, 17(1): 50–63. doi: 10.1109/TMM.2014.2373812

LI Qiaohong, LIN Weisi, and FANG Yuming. No-reference quality assessment for multiply-distorted images in gradient domain[J]. IEEE Signal Processing Letters, 2016, 23(4): 541–545. doi: 10.1109/LSP.2016.2537321

DAI Tao, GU Ke, NIU Li, et al. Referenceless quality metric of multiply-distorted images based on structural degradation[J]. Neurocomputing, 2018, 290: 185–195. doi: 10.1016/j.neucom.2018.02.050

JIA Sen and ZHANG Yang. Saliency-based deep convolutional neural network for no-reference image quality assessment[J]. Multimedia Tools and Applications, 2018, 77(12): 14859–14872. doi: 10.1007/s11042-017-5070-6

ZHOU Wujie, YU Lu, QIAN Yaguan, et al. Deep blind quality evaluator for multiply distorted images based on monogenic binary coding[J]. Journal of Visual Communication and Image Representation, 2019, 60: 305–311. doi: 10.1016/j.jvcir.2019.03.001

YE Peng, KUMAR J, KANG Le, et al. Unsupervised feature learning framework for no-reference image quality assessment[C]. 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, USA, 2012: 1098–1105. doi: 10.1109/CVPR.2012.6247789.

BOUREAU Y L, BACH F, LECUN Y, et al. Learning mid-level features for recognition[C]. 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, USA, 2010: 2559–2566. doi: 10.1109/CVPR.2010.5539963.

孫婭楠, 林文斌. 梯度下降法在機器學習中的應用[J]. 蘇州科技大學學報: 自然科學版, 2018, 35(2): 26–31. doi: 10.12084/j.issn.2096-3289.2018.02.006

SUN Yanan and LIN Wenbin. Application of gradient descent method in machine learning[J]. Journal of Suzhou University of Science and Technology:Natural Science, 2018, 35(2): 26–31. doi: 10.12084/j.issn.2096-3289.2018.02.006

JAYARAMAN D, MITTAL A, MOORTHY A K, et al. Objective quality assessment of multiply distorted images[C]. 2012 Conference Record of the Forty Sixth Asilomar Conference on Signals, Systems and Computers, Pacific Grove, USA, 2012: 1693–1697. doi: 10.1109/ACSSC.2012.6489321.

GU Ke, ZHAI Guangtao, YANG Xiaokang, et al. Hybrid no-reference quality metric for singly and multiply distorted images[J]. IEEE Transactions on Broadcasting, 2014, 60(3): 555–567. doi: 10.1109/TBC.2014.2344471

SUN Wen, ZHOU Fei, and LIAO Qingmin. MDID: A multiply distorted image database for image quality assessment[J]. Pattern Recognition, 2017, 61: 153–168. doi: 10.1016/j.patcog.2016.07.033

ZHANG Min, MURAMATSU C, ZHOU Xiangrong, et al. Blind image quality assessment using the joint statistics of generalized local binary pattern[J]. IEEE Signal Processing Letters, 2015, 22(2): 207–210. doi: 10.1109/LSP.2014.2326399

SHEIKH H R and BOVIK A C. Image information and visual quality[J]. IEEE Transactions on Image Processing, 2006, 15(2): 430–444. doi: 10.1109/TIP.2005.859378

MITTAL A, SOUNDARARAJAN R, and BOVIK A C. Making a "completely blind" image quality analyzer[J]. IEEE Signal Processing Letters, 2013, 20(3): 209–212. doi: 10.1109/LSP.2012.2227726

LI Qiaohong, LIN Weisi, XU Jingtao, et al. Blind image quality assessment using statistical structural and luminance features[J]. IEEE Transactions on Multimedia, 2016, 18(12): 2457–2469. doi: 10.1109/TMM.2016.2601028

施引文獻

資源附件(0)

訪問統(tǒng)計