多級跳線連接的深度殘差網(wǎng)絡(luò)超分辨率重建
doi: 10.11999/JEIT190036
-
1.
蘭州理工大學(xué)電氣工程與信息工程學(xué)院 ??蘭州 ??730050
-
2.
甘肅省工業(yè)過程先進控制重點實驗室 ??蘭州 ??730050
-
3.
蘭州理工大學(xué)國家級電氣與控制工程實驗教學(xué)中心 ??蘭州 ??730050
基金項目: 國家科學(xué)自然基金(61763029, 61873116)
Super-Resolution Reconstruction of Deep Residual Network with Multi-Level Skip Connections
-
1.
College of Electrical Engineering and Information Engineering, Lanzhou University of Technology, Lanzhou 730050, China
-
2.
Key Laboratory of Gansu Advanced Control for Industrial Processes, Lanzhou University of Technology, Lanzhou 730050, China
-
3.
National Experimental Teaching Center of Electrical and Control Engineering, Lanzhou University of Technology, Lanzhou 730050, China
Funds: The National Natural Science Foundation of China (61763029, 61873116)
-
摘要: 由于快速的卷積神經(jīng)網(wǎng)絡(luò)超分辨率重建算法(FSRCNN)卷積層數(shù)少、相鄰卷積層的特征信息之間缺乏關(guān)聯(lián)性，因此難以提取到圖像深層信息導(dǎo)致圖像超分辨率重建效果不佳。針對此問題，該文提出多級跳線連接的深度殘差網(wǎng)絡(luò)超分辨率重建方法。首先，該方法設(shè)計了多級跳線連接的殘差塊，在多級跳線連接的殘差塊基礎(chǔ)上構(gòu)造了多級跳線連接的深度殘差網(wǎng)絡(luò)，解決相鄰卷積層的特性信息缺乏關(guān)聯(lián)性的問題；然后，使用隨機梯度下降法(SGD)以可調(diào)節(jié)的學(xué)習(xí)率策略對多級跳線連接的深度殘差網(wǎng)絡(luò)進行訓(xùn)練，得到該網(wǎng)絡(luò)超分辨率重建模型；最后，將低分辨率圖像輸入到多級跳線連接的深度殘差網(wǎng)絡(luò)超分辨率重建模型中，通過多級跳線連接的殘差塊得到預(yù)測的殘差特征值，再將殘差圖像和低分辨率圖像組合在一起轉(zhuǎn)化為高分辨率圖像。該文方法與bicubic, A+, SRCNN, FSRCNN和ESPCN算法在Set5和Set14測試集上進行了對比測試，在視覺效果和評價指標數(shù)值上該方法都優(yōu)于其它對比算法。
-
關(guān)鍵詞:
- 超分辨率重建 /
- 深度殘差網(wǎng)絡(luò) /
- 多級跳線連接的殘差塊 /
- 隨機梯度下降法
Abstract: The Fast Super-Resolution Convolutional Neural Network algorithm (FSRCNN) is difficult to extract deep image information due to the small number of convolution layers and the correlation lack between the feature information of adjacent convolutional layers. To solve this problem, a deep residual network super-resolution reconstruction method with multi-level skip connections is proposed. Firstly, a residual block with multi-level skip connections is designed to solve the problem that the characteristic information of adjacent convolutional layers lacks relevance. A deep residual network with multi-level skip connections is constructed on the basis of the residual block. Then, the deep residual network connected to the multi-level skip is trained by using the adaptive gradient rate strategy of Stochastic Gradient Descent (SGD) method and the network super-resolution reconstruction model is obtained. Finally, the low-resolution image is input into the deep residual network super-resolution reconstruction model with the multi-level skip connections, and the residual eigenvalue is obtained by the residual block connected the multi-level skip connections. The residual eigenvalue and the low resolution image are combined and converted into a high resolution image. The proposed method is compared with the bicubic, A+, SRCNN, FSRCNN and ESPCN algorithms in the Set5 and Set14 test sets. The proposed method is superior to other comparison algorithms in terms of visual effects and evaluation index values. -
表 2 在Set14測試集上的測得的PSNR(dB)/ SSIM值
放大因子 Bicubic A+ SRCNN FSRCNN ESPCN 本文方法 2 30.24/0.8688 32.28/0.9056 32.42/0.9063 32.63/0.9088 32.75/0.9098 33.34/0.9143 3 27.55/0.7742 29.13/0.8188 29.28/0.8209 29.43/0.8242 29.49/0.8271 30.09/0.8512 4 26.00/0.7027 27.32/0.7491 27.49/0.7503 27.59/0.7535 27.73/0.7637 28.26/0.7893 
下載: 導(dǎo)出CSV
-
THORNTON M W, ATKINSON P M, and HOLLAND D A. Sub-pixel mapping of rural land cover objects from fine spatial resolution satellite sensor imagery using super-resolution pixel-swapping[J]. International Journal of Remote Sensing, 2006, 27(3): 473–491. doi: 10.1080/01431160500207088 PELED S and YESHURUN Y. Superresolution in MRI: Application to human white matter fiber tract visualization by diffusion tensor imaging[J]. Magnetic Resonance in Medicine, 2001, 45(1): 29–35. doi: 10.1002/1522-2594(200101)45 ZOU W W W and YUEN P C. Very low resolution face recognition problem[J]. IEEE Transactions on Image Processing, 2012, 21(1): 327–340. doi: 10.1109/TIP.2011.2162423 LU Huimin, LI Yujie, CHEN Min, et al. Brain intelligence: Go beyond artificial intelligence[J]. Mobile Networks and Applications, 2018, 23(2): 368–375. doi: 10.1007/s11036-017-0932-8 KOCH M. Artificial intelligence is becoming natural[J]. Cell, 2018, 173(3): 531–533. doi: 10.1016/j.cell.2018.04.007 LEO M, MEDIONI G, TRIVEDI M, et al. Computer vision for assistive technologies[J]. Computer Vision and Image Understanding, 2017, 154: 1–15. doi: 10.1016/j.cviu.2016.09.001 ZHU Hong, TANG Xinming, XIE Junfeng, et al. Spatio-temporal super-resolution reconstruction of remote-sensing images based on adaptive multi-scale detail enhancement[J]. Sensors, 2018, 18(2): 498. doi: 10.3390/s18020498 SHI Jun, LIU Qingping, WANG Chaofeng, et al. Super-resolution reconstruction of MR image with a novel residual learning network algorithm[J]. Physics in Medicine & Biology, 2018, 63(8): 085011. doi: 10.1088/1361-6560/aab9e9 SU Heng, ZHOU Jie, and ZHANG Zhihao. Survey of super-resolution image reconstruction methods[J]. Acta Automatica Sinica, 2013, 39(8): 1202–1213. doi: 10.3724/SP.J.1004.2013.01202 GRIBBON K T and BAILEY D G. A novel approach to real-time bilinear interpolation[C]. The 2nd IEEE International Workshop on Electronic Design, Test and Applications, Perth, Australia, 2004: 126–131. doi: 10.1109/DELTA.2004.10055. FRITSCH F N and CARLSON R E. Monotone piecewise cubic interpolation[J]. SIAM Journal on Numerical Analysis, 1980, 17(2): 238–246. doi: 10.1137/0717021 STARK H and OSKOUI P. High-resolution image recovery from image-plane arrays, using convex projections[J]. Journal of the Optical Society of America A, 1989, 6(11): 1715–1726. doi: 10.1364/JOSAA.6.001715 PATANAVIJIT V and JITAPUNKUL S. An iterative super-resolution reconstruction of image sequences using fast affine block-based registration with BTV regularization[C]. Proceedings of 2006 IEEE Asia Pacific Conference on Circuits and Systems, Singapore, 2006: 1717–1720. doi: 10.1109/APCCAS.2006.342128. ZHOU Fei, YANG Wenming, and LIAO Qingmin. Interpolation-based image super-resolution using multisurface fitting[J]. IEEE Transactions on Image Processing, 2012, 21(7): 3312–3318. doi: 10.1109/TIP.2012.2189576 LIN Zhouchen and SHUM H Y. Fundamental limits of reconstruction-based superresolution algorithms under local translation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(1): 83–97. doi: 10.1109/TPAMI.2004.1261081 YOUNG T, HAZARIKA D, PORIA S, et al. Recent trends in deep learning based natural language processing[J]. IEEE Computational Intelligence Magazine, 2018, 13(3): 55–75. doi: 10.1109/MCI.2018.2840738 KERMANY D S, GOLDBAUM M, CAI Wenjia, et al. Identifying medical diagnoses and treatable diseases by image-based deep learning[J]. Cell, 2018, 172(5): 1122–1131.e9. doi: 10.1016/j.cell.2018.02.010 DONG Chao, LOY C C, HE Kaiming, et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2): 295–307. doi: 10.1109/TPAMI.2015.2439281 DONG Chao, LOY C C, and TANG Xiaoou. Accelerating the super-resolution convolutional neural network[C]. The 14th European Conference on Computer Vision, Amsterdam, The Netherlands, 2016: 391–407. doi: 10.1007/978-3-319-46475-6_25. KIM J, LEE J K, and LEE K M. Deeply-recursive convolutional network for image super-resolution[C]. 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 1637–1645. doi: 10.1109/CVPR.2016.181. SHI Wenzhe, CABALLERO J, HUSZáR F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]. 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 1874–1883. doi: 10.1109/CVPR.2016.207. LEDIG C, THEIS L, HUSZáR F, et al. Photo-realistic single image super-resolution using a generative adversarial network[C]. 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 4681–4690. doi: 10.1109/CVPR.2017.19. HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]. 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 770–778. doi: 10.1109/CVPR.2016.90. KIM J, LEE J K, and LEE K M. Accurate image super-resolution using very deep convolutional networks[C]. 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 1646–1654. doi: 10.1109/CVPR.2016.182. YUAN Fei, HUANG Lianfen, and YAO Yan. An improved PSNR algorithm for objective video quality evaluation[C]. 2007 Chinese Control Conference, Hunan, China, 2007: 376–380. doi: 10.1109/CHICC.2006.4347144. WANG Zhou, BOVIK A C, SHEIKH H R, et al. Image quality assessment: From error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600–612. doi: 10.1109/TIP.2003.819861 GAO Shengkui and GRUEV V. Bilinear and bicubic interpolation methods for division of focal plane polarimeters[J]. Optics Express, 2011, 19(27): 26161–26173. doi: 10.1364/OE.19.026161 TIMOFTE R, DE SMET V, and VAN GOOL L. A+: Adjusted anchored neighborhood regression for fast super-resolution[C]. The 12th Asian Conference on Computer Vision, Singapore, 2014: 111–126. doi: 10.1007/978-3-319-16817-3_8. ZEYDE R, ELAD M, and PROTTER M. On single image scale-up using sparse-representations[C]. The 7th International Conference on Curves and Surfaces, Avignon, France, 2010: 711–730. doi: 10.1007/978-3-642-27413-8_47. -
圖(6) / 表(2)
計量
- 文章訪問數(shù): 3093
- HTML全文瀏覽量: 1238
- PDF下載量: 112
- 被引次數(shù): 0
下載:
