基于多通道憶阻脈沖耦合神經(jīng)網(wǎng)絡(luò)的多幀圖像超分辨率重建算法

董哲康; 杜晨杰; 林輝品; 賴俊昇; 胡小方; 段書凱

doi:10.11999/JEIT190868

基于多通道憶阻脈沖耦合神經(jīng)網(wǎng)絡(luò)的多幀圖像超分辨率重建算法

doi: 10.11999/JEIT190868

董哲康^{1, 2},
杜晨杰¹,
林輝品^{1, 2, ,},
賴俊昇³,
胡小方⁴,
段書凱⁴

1.
杭州電子科技大學(xué)電子信息學(xué)院杭州 310018
2.
浙江大學(xué)電氣工程學(xué)院杭州 310027
3.
牛津大學(xué)工程科學(xué)系英國牛津 OX1 3PJ
4.
西南大學(xué)人工智能學(xué)院重慶 400715

基金項(xiàng)目: 國家自然科學(xué)基金(61571394, 61601376)，浙江省屬高?；究蒲袠I(yè)務(wù)費(fèi)項(xiàng)目(GK199900299012-010)

詳細(xì)信息

作者簡介:
董哲康：男，1989年生，副教授，主要研究方向?yàn)閼涀枥碚?、基于憶阻器的神?jīng)形態(tài)系統(tǒng)研究

杜晨杰：男，1990年生，博士生，研究方向?yàn)閼涀枥碚摗⒒趹涀杵鞯纳窠?jīng)形態(tài)系統(tǒng)研究

林輝品：男，1987年生，講師，主要研究方向?yàn)閼涀枥碚?、基于憶阻器的神?jīng)形態(tài)系統(tǒng)研究

賴俊昇：男，1991年生，助理教授，主要研究方向?yàn)閼涀枥碚?、基于憶阻器的神?jīng)形態(tài)系統(tǒng)研究

胡小方：女，1984年生，副教授，主要研究方向?yàn)閼涀杵骼碚摗⒒趹涀杵鞯姆蔷€性系統(tǒng)研究

段書凱：男，1973年生，教授，主要研究方向?yàn)閼涀杵骼碚?、微納系統(tǒng)研究

通訊作者:
林輝品　linhuipin@hdu.edu.cn

中圖分類號: TN601; TN911.73
計(jì)量
- 文章訪問數(shù): 6171
- HTML全文瀏覽量: 1894
- PDF下載量: 170
- 被引次數(shù): 0
出版歷程
- 收稿日期: 2019-11-01
- 修回日期: 2020-01-12
- 網(wǎng)絡(luò)出版日期: 2020-02-12
- 刊出日期: 2020-06-04

Multi-channel Memristive Pulse Coupled Neural Network Based Multi-frame Images Super-resolution Reconstruction Algorithm

1.
School of Electronic Information, Hangzhou Dianzi University, Hangzhou 310018, China
2.
College of Electrical Engineering, Zhejiang University, Hangzhou 310027, China
3.
Department of Engineering Science, University of Oxford, Oxford OX1 3PJ, United Kingdom
4.
College of Artificial Intelligence, Southwest University, Chongqing 400715, China

Funds: The National Natural Science Foundation of China (61571394, 61601376), The Fundamental Research Funds for the Provincial Universities (GK199900299012-010)

摘要

摘要: 高清晰度的圖像是信息獲取和精確分析的前提，研究多幀圖像的超分辨率重建能夠有效解決因外部拍攝環(huán)境引起的圖像細(xì)節(jié)丟失、邊緣模糊等問題。該文基于納米級憶阻器，設(shè)計(jì)一種多通道憶阻脈沖耦合神經(jīng)網(wǎng)絡(luò)模型(MMPCNN)，能夠有效模擬網(wǎng)絡(luò)中連接系數(shù)的動態(tài)變化，解決神經(jīng)網(wǎng)絡(luò)中固有的參數(shù)估計(jì)問題。同時，將提出的網(wǎng)絡(luò)應(yīng)用于多幀圖像超分辨率重建中，實(shí)現(xiàn)低分辨率配準(zhǔn)圖像的融合操作，并通過基于稀疏編碼的單幀圖像超分辨率重構(gòu)算法對獲得的初始高分辨率圖像進(jìn)行優(yōu)化。最終，一系列計(jì)算機(jī)仿真及分析(主觀/客觀分析)驗(yàn)證了該文提出方案的正確性和有效性。
- 憶阻器 /
- 脈沖耦合神經(jīng)網(wǎng)絡(luò) /
- 多幀圖像 /
- 超分辨率重建
Abstract: The high-resolution image is the prerequisite of information acquisition and precise analysis. Multi-frame super-resolution images reconstruction technologies are able to address many image degraded issues (caused by external shooting environment), such as detail information lost, blurred edges, and so forth. According to the nanoscale memristor, a Multi-channel Memristive Pulse Coupled Neural Network (MMPCNN) model is proposed. This model is able to simulate the adaptive-variable linking coefficient in pulse coupled neural network. Meanwhile, the proposed network is applied to the multi-frame super resolution reconstruction for fusing the registered low resolution images. Furthermore, the sparse coding based super resolution method is performed to improve the original high-resolution image. Finally, a series of computer experiments and the relevant subjective/objective analysis jointly illustrate the validity and effectiveness of the entire scheme.
- Memristor /
- Pulse coupled neural network /
- Multi-frame images /
- Super-resolution reconstruction

HTML全文

圖 1 VTEAM憶阻器仿真結(jié)果

下載: 全尺寸圖片幻燈片

圖 2 多通道憶阻脈沖耦合神經(jīng)網(wǎng)絡(luò)MMPCNN

下載: 全尺寸圖片幻燈片

圖 3 圖像降質(zhì)模型

下載: 全尺寸圖片幻燈片

圖 4 提出的圖像超分辨率算法流程圖

下載: 全尺寸圖片幻燈片

圖 5 多幀圖像超分辨重構(gòu)的實(shí)現(xiàn)流程

下載: 全尺寸圖片幻燈片

圖 6 對比實(shí)驗(yàn)仿真結(jié)果

下載: 全尺寸圖片幻燈片

圖 7 對比實(shí)驗(yàn)仿真結(jié)果

下載: 全尺寸圖片幻燈片

參考文獻(xiàn)(20)

PARK S C, PARK M K, and KANG M G. Super-resolution image reconstruction: A technical overview[J]. IEEE Signal Processing Magazine, 2003, 20(3): 21–36. doi: 10.1109/MSP.2003.1203207

趙小強(qiáng), 宋昭漾. 多級跳線連接的深度殘差網(wǎng)絡(luò)超分辨率重建[J]. 電子與信息學(xué)報(bào), 2019, 41(10): 2501–2508. doi: 10.11999/JEIT190036

ZHAO Xiaoqiang and SONG Zhaoyang. Super-resolution reconstruction of deep residual network with multi-level skip connections[J]. Journal of Electronics &Information Technology, 2019, 41(10): 2501–2508. doi: 10.11999/JEIT190036

DONG Zhekang, LAI C S, XU Zhao, et al. Single image super-resolution via the implementation of the hardware-friendly sparse coding[C]. The 37th Chinese Control Conference, Wuhan, China, 2018: 8132–8137.

HUI Zheng, WANG Xiumei, and GAO Xinbo. Fast and accurate single image super-resolution via information distillation network[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 723–731.

BAKER S and KANADE T. Super-resolution optical flow[R]. Technical Report CMU-RI-TR-36, 1999.

NING Beijia and GAO Xinbo. Multi-frame image super-resolution reconstruction using sparse co-occurrence prior and sub-pixel registration[J]. Neurocomputing, 2013, 117: 128–137. doi: 10.1016/j.neucom.2013.01.019

XU Jieping, LIANG Yonghui, LIU Jin, et al. Online multi-frame super-resolution of image sequences[J]. EURASIP Journal on Image and Video Processing, 2018(1): 136–10. doi: 10.1186/s13640-018-0376-5

AIZAWA K, KOMATSU T, SAITO T, et al. Subpixel registration for a high resolution imaging scheme using multiple imagers[C]. 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing, Minneapolis, USA 1993: 133–136.

SINHA A and WU Xiaolin. Fast generalized motion estimation and superresolution[C]. 2007 IEEE International Conference on Image Processing, San Antonio, USA, 2007: V–413–V–416.

ARYA M S and JAIN P. Fifth-level Second-generation Wavelet-based Image Fusion Algorithm for Visual Quality Enhancement of Digital Image Data[M]. MISHRA D K, AZAR A T, and JOSHI A. Information and Communication Technology. Singapore: Springer, 2018: 139–149.

WU Zhiliang, HUANG Yongdong, and ZHANG Kang. Remote sensing image fusion method based on PCA and Curvelet transform[J]. Journal of the Indian Society of Remote Sensing, 2018, 46(5): 687–695. doi: 10.1007/s12524-017-0736-0

DONG Zhekang, LAI C S, QI Donglian, et al. A general memristor-based pulse coupled neural network with variable linking coefficient for multi-focus image fusion[J]. Neurocomputing, 2018, 308: 172–183. doi: 10.1016/j.neucom.2018.04.066

李志軍, 向林波, 肖文潤. 一種通用的記憶器件模擬器及在串聯(lián)諧振電路中的應(yīng)用[J]. 電子與信息學(xué)報(bào), 2017, 39(7): 1626–1633. doi: 10.11999/JEIT161060

LI Zhijun, XIANG Linbo, and XIAO Wenrun. Universal mem-elements emulator and its application in RLC circuit[J]. Journal of Electronics &Information Technology, 2017, 39(7): 1626–1633. doi: 10.11999/JEIT161060

KVATINSKY S, RAMADAN M, FRIEDMAN E G, et al. VTEAM: A general model for voltage-controlled memristors[J]. IEEE Transactions on Circuits and Systems II: Express Briefs, 2015, 62(8): 786–790. doi: 10.1109/TCSII.2015.2433536

KUPFER B, NETANYAHU N S, and SHIMSHONI I. An efficient SIFT-based mode-seeking algorithm for sub-pixel registration of remotely sensed images[J]. IEEE Geoscience and Remote Sensing Letters, 2015, 12(2): 379–383. doi: 10.1109/LGRS.2014.2343471

QU Xiaobo, HU Changwei, and YAN Jingwen. Image fusion algorithm based on orientation information motivated pulse coupled neural networks[C]. The 7th World Congress on Intelligent Control and Automation, Chongqing, China, 2008: 2437–2441.

CHENG Xuan, ZENG Ming, and LIU Xinguo. Feature-preserving filtering with L₀ gradient minimization[J]. Computers & Graphics, 2014, 38: 150–157.

TIMOFTE R, DE V, and VAN GOOL L. Anchored neighborhood regression for fast example-based super-resolution[C]. 2013 IEEE International Conference on Computer Vision, Sydney, Austrlia, 2013: 1920–1927.

DONG Chao, LOY C C, HE Kaiming, et al. Learning a deep convolutional network for image super-resolution[C]. The 13th European Conference on Computer Vision, Zurich, Switzerland, 2014: 184–199.

https://github.com/tingfengainiaini/sparseCodingSuperResolution, 2019.

相關(guān)文章

施引文獻(xiàn)

資源附件(0)

訪問統(tǒng)計(jì)