利用近清圖像空間搜索的深度圖像先驗降噪模型

徐少平; 熊明海; 周常飛

doi:10.11999/JEIT240114

利用近清圖像空間搜索的深度圖像先驗降噪模型

doi: 10.11999/JEIT240114

南昌大學(xué)數(shù)學(xué)與計算機(jī)學(xué)院南昌 330031

基金項目: 國家自然科學(xué)基金(62162043)

詳細(xì)信息

作者簡介:
徐少平：男，博士，教授，博士生導(dǎo)師，研究方向為數(shù)字圖像處理、機(jī)器視覺、虛擬手術(shù)模擬

熊明海：男，碩士生，研究方向為數(shù)字圖像處理、機(jī)器視覺

周常飛：男，碩士生，研究方向為數(shù)字圖像處理、機(jī)器視覺

通訊作者:
徐少平　xushaoping@ncu.edu.cn

中圖分類號: TN911.73; TP391.4
計量
- 文章訪問數(shù): 185
- HTML全文瀏覽量: 74
- PDF下載量: 17
- 被引次數(shù): 0
出版歷程
- 收稿日期: 2024-02-28
- 修回日期: 2024-09-07
- 網(wǎng)絡(luò)出版日期: 2024-09-29
- 刊出日期: 2024-11-10

Deep Image Prior Denoising Model Using Relatively Clean Image Space Search

School of Mathematics and Computer Sciences, Nanchang University, Nanchang 330031, China

Funds: The National Natural Science Foundation of China (62162043)

摘要

摘要: 鑒于深度圖像先驗(DIP)降噪模型的性能高度依賴于目標(biāo)圖像所確定的搜索空間，該文提出一種新的基于近清圖像空間搜索策略的改進(jìn)降噪模型。首先，使用當(dāng)前兩種主流有監(jiān)督降噪模型對同一場景下兩張噪聲圖像分別進(jìn)行降噪，所獲得兩張降噪后圖像稱為近清圖像；其次，采用隨機(jī)采樣融合法將兩張近清圖像融合后作為網(wǎng)絡(luò)輸入，同時以兩張近清圖像替換噪聲圖像作為雙目標(biāo)圖像以更好地約束搜索空間，進(jìn)而在更為接近參考圖像的空間范圍內(nèi)搜索可能的圖像作為降噪后圖像；最后，將原DIP模型的多尺度UNet網(wǎng)絡(luò)簡化為單尺度模式，同時引入Transformer模塊以增強(qiáng)網(wǎng)絡(luò)對長距離像素點(diǎn)之間的建模能力，從而在保證網(wǎng)絡(luò)搜索能力的基礎(chǔ)上提升模型的執(zhí)行效率。實(shí)驗結(jié)果表明：所提改進(jìn)模型在降噪效果和執(zhí)行效率兩個方面顯著優(yōu)于原DIP模型，在降噪效果方面也超過了主流有監(jiān)督降噪模型。
- 深度圖像先驗 /
- 降噪性能 /
- 近清圖像 /
- 隨機(jī)采樣融合 /
- 雙目標(biāo)圖像 /
- Transformer
Abstract: Given that the performance of the Deep Image Prior (DIP) denoising model highly depends on the search space determined by the target image, a new improved denoising model called RS-DIP (Relatively clean image Space-based DIP) is proposed by comprehensively improving its network input, backbone network, and loss function.Initially, two state-of-the-art supervised denoising models are employed to preprocess two noisy images from the same scene, which are referred to as relatively clean images. Furthermore, these two relatively clean images are combined as the network input using a random sampling fusion method. At the same time, the noisy images are replaced with two relatively clean images, which serve as dual-target images. This strategy narrows the search space, allowing exploration of potential images that closely resemble the ground-truth image. Finally, the multi-scale U-shaped backbone network in the original DIP model is simplified to a single scale. Additionally, the inclusion of Transformer modules enhances the network’s ability to effectively model distant pixels. This augmentation bolsters the model’s performance while preserving the network’s search capability. Experimental results demonstrate that the proposed denoising model exhibits significant advantages over the original DIP model in terms of both denoising effectiveness and execution efficiency. Moreover, regarding denoising effectiveness, it surpasses mainstream supervised denoising models.
- Deep Image Prior (DIP) /
- Denoising performance /
- Relatively clean images /
- Random sampling fusion /
- Dual-target image /
- Transformer

HTML全文

圖 1 RS-DIP降噪模型降噪原理的可視化示意圖

下載: 全尺寸圖片幻燈片

圖 2 RS-DIP網(wǎng)絡(luò)模型框架圖

下載: 全尺寸圖片幻燈片

圖 3 各對比方法在PolyU圖像上的降噪效果對比

下載: 全尺寸圖片幻燈片

表 1 不同降噪模型組合對降噪性能的影響(dB)

算法組合	Restormer+ DnCNN	Restormer+ FFDNet	Restormer+ DAGL	DAGL+ DnCNN	DAGL+ FFDNet	DAGL+ SwinIR	Restormer+ SwinIR
1	36.33	36.29	35.79	36.19	35.77	36.08	36.44
2	36.74	36.64	36.06	36.43	36.14	36.48	36.84
3	35.25	34.78	34.78	34.78	35.57	35.28	35.03
4	31.33	31.49	31.48	31.55	31.56	31.62	31.58
5	38.41	37.00	37.57	38.21	36.86	37.30	37.31
6	38.51	40.11	38.53	38.40	39.72	39.82	40.23
7	30.19	30.27	30.02	29.58	29.78	29.77	30.32
8	34.52	34.69	34.17	33.78	33.98	34.03	34.70
9	35.83	35.41	34.79	34.53	34.81	34.80	35.50
10	36.65	36.38	36.04	36.14	35.85	35.94	36.48
平均值	35.27	35.31	34.98	34.96	35.00	35.11	35.44

下載: 導(dǎo)出CSV

表 2 隨機(jī)采樣操作對模型降噪性能的影響比較(dB)

對比算法	RS-DIP-1	RS-DIP
1	36.42	36.44
2	36.90	36.84
3	34.93	35.03
4	31.59	31.58
5	37.18	37.31
6	40.14	40.23
7	30.30	30.32
8	34.70	34.70
9	35.49	35.50
10	36.36	36.48
平均值	35.40	35.44

下載: 導(dǎo)出CSV

表 3 簡化骨干網(wǎng)絡(luò)對模型降噪性能的影響(dB)

對比算法	RS-DIP-2	RS-DIP
1	36.22	36.44
2	36.79	36.84
3	35.07	35.03
4	31.48	31.58
5	36.83	37.31
6	40.21	40.23
7	30.31	30.32
8	34.54	34.70
9	35.43	35.50
10	36.20	36.48
平均值	35.31	35.44

下載: 導(dǎo)出CSV

表 4 各對比方法在各真實(shí)噪聲數(shù)據(jù)集上所獲得的PSNR值比較(dB)

數(shù)據(jù)集	PolyU	NIND	SIDD
BM3D	33.90	32.87	38.00
DnCNN	33.28	31.33	32.84
FFDNet	34.25	32.72	37.54
DAGL	33.44	32.10	42.47
Restormer*	33.47	35.20	44.17
SwinIR*	34.45	33.84	37.24
DIP	34.43	33.79	39.42
RS-DIP	35.44	35.46	44.60
*表示該模型被用于處理噪聲圖像，在NIND和PolyU數(shù)據(jù)集上分別使用Restormer和SwinIR作為預(yù)處理算法處理不同的噪聲圖像，而在SIDD數(shù)據(jù)集上僅使用Restormer作為預(yù)處理算法處理兩張噪聲圖像。

下載: 導(dǎo)出CSV

參考文獻(xiàn)(20)

[1]	JIN Xin, ZHANG Li, SHAN Chaowei, et al. Dual prior learning for blind and blended image restoration[J]. IEEE Transactions on Image Processing, 2022, 31: 1042–1056. doi: 10.1109/TIP.2021.3135482.
[2]	白勇強(qiáng), 禹晶, 李一秾, 等. 基于深度先驗的盲圖像去模糊算法[J]. 電子學(xué)報, 2023, 51(4): 1050–1067. doi: 10.12263/DZXB.20211483. BAI Yongqiang, YU Jing, LI Yinong et al. Deep prior-based blind image deblurring[J]. Acta Electronica Sinica, 2023, 51(4): 1050–1067. doi: 10.12263/DZXB.20211483.
[3]	DABOV K, FOI A, KATKOVNIK V, et al. Image denoising by sparse 3-D transform-domain collaborative filtering[J]. IEEE Transactions on Image Processing, 2007, 16(8): 2080–2095. doi: 10.1109/TIP.2007.901238.
[4]	GOYAL B, DOGRA A, AGRAWAL S, et al. Image denoising review: From classical to state-of-the-art approaches[J]. Information Fusion, 2020, 55: 220–244. doi: 10.1016/j.inffus.2019.09.003.
[5]	周建新, 周鳳祺. 基于改進(jìn)協(xié)同量子粒子群的小波去噪分析研究[J]. 電光與控制, 2022, 29(1): 47–50. doi: 10.3969/j.issn.1671-637X.2022.01.010. ZHOU Jianxin and ZHOU Fengqi. Wavelet denoising analysis based on cooperative quantum-behaved particle swarm optimization[J]. Electronics Optics & Control, 2022, 29(1): 47–50. doi: 10.3969/j.issn.1671-637X.2022.01.010.
[6]	曾理, 熊西林, 陳偉. 低劑量CT圖像降噪的深度圖像先驗的目標(biāo)偏移加速算法[J]. 電子與信息學(xué)報, 2023, 45(6): 2188–2196. doi: 10.11999/JEIT220551. ZENG Li, XIONG Xilin, and CHEN Wei. Deep image prior acceleration method for target offset in low-dose CT images denoising[J]. Journal of Electronics & Information Technology, 2023, 45(6): 2188–2196. doi: 10.11999/JEIT220551.
[7]	ZHANG Kai, ZUO Wangmeng, CHEN Yunjin, et al. Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising[J]. IEEE Transactions on Image Processing, 2017, 26(7): 3142–3155. doi: 10.1109/TIP.2017.2662206.
[8]	ZHANG Kai, ZUO Wangmeng, and ZHANG Lei. FFDNet: Toward a fast and flexible solution for CNN-based image denoising[J]. IEEE Transactions on Image Processing, 2018, 27(9): 4608–4622. doi: 10.1109/TIP.2018.2839891.
[9]	GUO Shi, YAN Zifei, ZHANG Kai, et al. Toward convolutional blind denoising of real photographs[C]. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, USA, 2019: 1712–1722. doi: 10.1109/CVPR.2019.00181.
[10]	ANWAR S and BARNES N. Real image denoising with feature attention[C]. 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Korea (South), 2019: 3155–3164. doi: 10.1109/ICCV.2019.00325.
[11]	LEHTINEN J, MUNKBERG J, HASSELGREN J, et al. Noise2Noise: Learning image restoration without clean data[C]. Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden, 2018: 2965–2974.
[12]	KRULL A, BUCHHOLZ T O, and JUG F. Noise2void-learning denoising from single noisy images[C]. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, USA, 2019: 2124–2132. doi: 10.1109/CVPR.2019.00223.
[13]	HUANG Tao, LI Songjiang, JIA Xu, et al. Neighbor2Neighbor: Self-supervised denoising from single noisy images[C]. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, USA, 2021: 14776–14785. doi: 10.1109/CVPR46437.2021.01454.
[14]	LEMPITSKY V, VEDALDI A, and ULYANOV D. Deep image prior[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 9446–9454. doi: 10.1109/CVPR.2018.00984.
[15]	HENDRYCKS D and GIMPEL K. Gaussian error linear units (GELUs)[EB/OL]. https://arxiv.org/abs/1606.08415, 2023.
[16]	XU Jun, LI Hui, LIANG Zhetong, et al. Real-world noisy image denoising: A new benchmark[EB/OL]. https://arxiv.org/abs/1804.02603, 2018.
[17]	YANG Yu, XU Haifeng, QI Bin, et al. Stroke screening data modeling based on openEHR and NINDS Stroke CDE[C]. 2020 IEEE International Conference on BioInformatics and BioMedicine (BIBM), Seoul, Korea (South), 2020: 2147–2152. doi: 10.1109/BIBM49941.2020.9313127.
[18]	ABDELHAMED A, LIN S, and BROWN M S. A high-quality denoising dataset for smartphone cameras[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 1692–1700. doi: 10.1109/CVPR.2018.00182.
[19]	徐勝軍, 楊華, 李明海, 等. 基于雙頻域特征聚合的低照度圖像增強(qiáng)[J]. 光電工程, 2023, 50(12): 230225. doi: 10.12086/oee.2023.230225. XU Shengjun, YANG Hua, LI Minghai, et al. Low-light image enhancement based on dual-frequency domain feature aggregation[J]. Opto-Electronic Engineering, 2023, 50(12): 230225. doi: 10.12086/oee.2023.230225.
[20]	SOH J W, CHO S, and CHO N I. Meta-transfer learning for zero-shot super-resolution[C]. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2020: 3513–3522. doi: 10.1109/CVPR42600.2020.00357.