基于多通道多尺度卷積神經(jīng)網(wǎng)絡(luò)的單幅圖像去雨方法
doi: 10.11999/JEIT190755
-
1.
哈爾濱理工大學(xué)電氣與電子工程學(xué)院 哈爾濱 150000
-
2.
哈爾濱工程大學(xué)信息與通信工程學(xué)院 哈爾濱 150000
基金項目: 國家自然科學(xué)基金(51779050)
Research on Rain Removal Method for Single Image Based on Multi-channel and Multi-scale CNN
-
1.
School of Electrical and Electronic Engineering, Harbin University of Science and Technology, Harbin 150000, China
-
2.
School of Information and Communication Engineering, Harbin Engineering University, Harbin 150000, China
Funds: The National Natural Science Foundation of China(51779050)
-
摘要: 雨天等惡劣天氣會嚴重影響到圖像成像質(zhì)量，從而影響到視覺處理算法的性能。為了改善雨天圖像的成像質(zhì)量，該文提出一種基于多通道多尺度卷積神經(jīng)網(wǎng)絡(luò)的去雨算法，建立了多通道多尺度卷積神經(jīng)網(wǎng)絡(luò)結(jié)構(gòu)來提取雨線特征。首先利用小波閾值引導(dǎo)的雙邊濾波將有雨圖像進行分解，得到高頻雨線圖像和輪廓保持度高的低頻背景圖像。然后為了使圖像高頻部分的雨線信息更為明顯，減少雨線特征學(xué)習(xí)時高頻圖像中的背景誤判，將得到的高頻雨線圖像再一次通過濾波器得到減弱背景信息同時增強雨線信息的到更高頻雨線圖像。其次針對低頻背景圖像上也殘留了大量雨痕，該文提出將低頻背景圖像和更高頻雨線圖像一起送入卷積神經(jīng)網(wǎng)絡(luò)進行特征學(xué)習(xí)，其中對圖像提取的是多尺度特征信息，最后得到雨線去除更徹底的復(fù)原圖像。同時在構(gòu)造網(wǎng)絡(luò)模型時利用空洞卷積代替標準卷積來提取圖像的特征信息，得到更豐富的圖像特征，提高了算法的去雨性能。從實驗結(jié)果可以看出去雨之后的圖像清晰，細節(jié)保持度較高。
-
關(guān)鍵詞:
- 深度學(xué)習(xí) /
- 空洞卷積 /
- 圖像分解 /
- 多尺度提取特征
Abstract: Rainy days and other severe weather will seriously affect the image quality, thus affecting the performance of vision processing algorithms. In order to improve the imaging quality of rain images, a rain removal algorithm based on multi-channel multi-scale convolution neural network to extract rain line features is proposed. Firstly, the rain images are decomposed by wavelet threshold-guided bilateral filtering to obtain high-frequency rain line images and low-frequency background images with high contour preservation. Then, in order to make the rain line information in the high-frequency part of the image more obvious and reduce the background misjudgment in the high-frequency image during the rain line feature learning, the obtained high-frequency rain line image is passed through a filter again to obtain a higher-frequency rain line image with reduced background information and enhanced rain line information. Secondly, in view of the large amount of raindrop imprint left on the low-frequency background image, it is proposed to send the low-frequency background image and the higher-frequency rain line image together into the convolution neural network for feature learning, in which multi-scale feature information is extracted from the image, finally, a more complete restoration image with rain line removal is obtained. At the same time, when constructing the network model, hole convolution is used instead of standard convolution to extract the feature information of the image, thus obtaining richer image features and improving the rain removal performance of the algorithm. From the experimental results, after removing rain, the image is clear and the detail retention is high. -
表 3 圖像質(zhì)量指標對比
圖像 指標 單尺度卷積
5 × 5單尺度卷積
7 × 7多尺度卷積 第1幅 PSNR 21.253 20.197 23.128 SSIM 0.9109 0.9114 0.9236 第2幅 PSNR 25.017 25.714 26.821 SSIM 0.9211 0.9237 0.9420 第3幅 PSNR 31.581 30.336 34.460 SSIM 0.9382 0.9369 0.9448 300張平均 PSNR 25.973 25.285 27.794 SSIM 0.9350 0.9337 0.9434 
下載: 導(dǎo)出CSV
-
郭智, 宋萍, 張義, 等. 基于深度卷積神經(jīng)網(wǎng)絡(luò)的遙感圖像飛機目標檢測方法[J]. 電子與信息學(xué)報, 2018, 40(11): 2684–2690. doi: 10.11999/JEIT180117GUO Zhi, SONG Ping, ZHANG Yi, et al. Aircraft detection method based on deep convolutional neural network for remote sensing images[J]. Journal of Electronics &Information Technology, 2018, 40(11): 2684–2690. doi: 10.11999/JEIT180117 孫彥景, 石韞開, 云霄, 等. 基于多層卷積特征的自適應(yīng)決策融合目標跟蹤算法[J]. 電子與信息學(xué)報, 2019, 41(10): 2464–2470. doi: 10.11999/JEIT180971SUN Yanjing, SHI Yunkai, YUN Xiao, et al. Adaptive strategy fusion target tracking based on multi-layer convolutional features[J]. Journal of Electronics &Information Technology, 2019, 41(10): 2464–2470. doi: 10.11999/JEIT180971 GARG K and NAYAR S K. Detection and removal of rain from videos[C]. 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Washington DC, USA, 2004. doi: 10.1109/CVPR.2004.1315077. BARNUM P C, NARASIMHAN S, and KANADE T. Analysis of rain and snow in frequency space[J]. International Journal of Computer Vision, 2010, 86(2): 256–274. doi: 10.1007/s11263-008-0200-2 LIU Jiaying, YANG Wenhan, YANG Shuai, et al. Erase or fill? Deep joint recurrent rain removal and reconstruction in videos[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 3233–3242. doi: 10.1109/CVPR.2018.00341. CHEN Jie, TAN C H, HOU Junhui, et al. Robust video content alignment and compensation for rain removal in a CNN framework[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 6286–6295. doi: 10.1109/CVPR.2018.00658. KIM J H, LEE C, SIM J Y, et al. Single-image deraining using an adaptive nonlocal means filter[C]. 2013 IEEE International Conference on Image Processing, Melbourne, Australia, 2013: 914–917. doi: 10.1109/ICIP.2013.6738189. HUANG Dean, KANG Liwei, WANG Y C F, et al. Self-learning based image decomposition with applications to single image denoising[J]. IEEE Transactions on Multimedia, 2014, 16(1): 83–93. doi: 10.1109/TMM.2013.2284759 LUO Yu, XU Yong, and JI Hui. Removing rain from a single image via discriminative sparse coding[C]. 2015 IEEE International Conference on Computer Vision, Santiago, Chile, 2015: 3397–3405. doi: 10.1109/ICCV.2015.388. KANG Liwei, LIN C W, and FU Y H. Automatic single-image-based rain streaks removal via image decomposition[J]. IEEE Transactions on Image Processing, 2012, 21(4): 1742–1755. doi: 10.1109/TIP.2011.2179057 QIAN Rui, TAN R T, YANG Wenhan, et al. Attentive generative adversarial network for raindrop removal from a single image[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 2482–2491. doi: 10.1109/CVPR.2018.00263. HE Zhang and PATEL V M. Density-aware single image de-raining using a multi-stream dense network[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 695–704. doi: 10.1109/CVPR.2018.00079. FU Xueyang, HUANG Jiabin, DING Xinghao, et al. Clearing the skies: A deep network architecture for single-image rain removal[J]. IEEE Transactions on Image Processing, 2017, 26(6): 2944–2956. doi: 10.1109/TIP.2017.2691802 肖進勝, 李文昊, 姜紅, 等. 基于雙域濾波的三維塊匹配視頻去噪算法[J]. 通信學(xué)報, 2015, 36(9): 91–97. doi: 10.11959/j.issn.1000-436x.2015245XIAO Jinsheng, LI Wenhao, JIANG Hong, et al. Three dimensional block-matching video denoising algorithm based on dual-domain filtering[J]. Journal on Communications, 2015, 36(9): 91–97. doi: 10.11959/j.issn.1000-436x.2015245 XIAO Jinsheng, LI Wenhao, LIU Guoxiong, et al. Hierarchical tone mapping based on image colour appearance model[J]. IET Computer Vision, 2014, 8(4): 358–364. doi: 10.1049/iet-cvi.2013.0230 YU Hancheng, LI Zhao, and WANG Haixian. Image denoising using trivariate shrinkage filter in the wavelet domain and joint bilateral filter in the Spatial Domain[J]. IEEE Transactions on Image Processing, 2009, 18(10): 2364–2369. doi: 10.1109/TIP.2009.2026685 HUYNH-THU Q and GHANBARI M. Scope of validity of PSNR in image/video quality assessment[J]. Electronics Letters, 2008, 44(13): 800–801. doi: 10.1049/el:20080522 WANG Zhou, BOVIK A C, SHEIKH H R, et al. Image quality assessment: From error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600–612. doi: 10.1109/TIP.2003.819861 -
圖(7) / 表(3)
計量
- 文章訪問數(shù): 3022
- HTML全文瀏覽量: 924
- PDF下載量: 179
- 被引次數(shù): 0
下載:
