基于高層特征圖組合及池化的高分辨率遙感圖像檢索
doi: 10.11999/JEIT190017
-
1.
南昌航空大學軟件學院 南昌 330063
-
2.
南昌大學信息工程學院 南昌 330031
基金項目: 國家自然科學基金(41801288, 41261091, 61662044, 61663031, 61762067)
The Combination and Pooling Based on High-level Feature Map for High-resolution Remote Sensing Image Retrieval
-
1.
School of Software, Nanchang Hangkong University, Nanchang 330063, China
-
2.
School of Information Engineering, Nanchang University, Nanchang 330031, China
Funds: The National Natural Science Foundation of China (41801288, 41261091, 61662044, 61663031, 61762067)
-
摘要: 高分辨率遙感圖像內(nèi)容復雜，提取特征來準確地表達圖像內(nèi)容是提高檢索性能的關(guān)鍵。卷積神經(jīng)網(wǎng)絡(CNN)遷移學習能力強，其高層特征能夠有效遷移到高分辨率遙感圖像中。為了充分利用高層特征的優(yōu)點，該文提出一種基于高層特征圖組合及池化的方法來融合不同CNN中的高層特征。首先將高層特征作為特殊的卷積層特征，進而在不同輸入尺寸下保留高層輸出的特征圖；然后將不同高層輸出的特征圖組合成一個更大的特征圖，以綜合不同CNN學習到的特征；接著采用最大池化的方法對組合特征圖進行壓縮，提取特征圖中的顯著特征；最后，采用主成分分析(PCA)來降低顯著特征的冗余度。實驗結(jié)果表明，與現(xiàn)有檢索方法相比，該方法提取的特征在檢索效率和準確率上都有優(yōu)勢。Abstract: High-resolution remote sensing images have complex visual contents, and extracting feature to represent image content accurately is the key to improving image retrieval performance. Convolutional Neural Networks (CNN) have strong transfer learning ability, and the high-level features of CNN can be efficiently transferred to high-resolution remote sensing images. In order to make full use of the advantages of high-level features, a combination and pooling method based on high-level feature maps is proposed to fuse high-level features from different CNNs. Firstly, the high-level features are adopted as special convolutional features to preserve the feature maps of the high-level outputs under different input sizes, and then the feature maps are combined into a larger feature map to integrate the features learned by different CNNs. The combined feature map is compressed by max-pooling method to extract salient features. Finally, the Principal Component Analysis (PCA) is utilized to reduce the redundancy of the salient features. The experimental results show that compared with the existing retrieval methods, the features extracted by this method have advantages in retrieval efficiency and precision.
-
Key words:
- Remote sensing image retrieval /
- Transfer learning /
- High-level feature map /
- Combination /
- Pooling
-
表 1 不同輸入圖像尺寸下高層CNN特征的輸出值
輸入圖像尺寸 fc G-pool5 R-pool5 默認尺寸 1×1×4096 1×1×1024 1×1×2048 256×256×3(UC-Merced) 2×2×4096 2×2×1024 2×2×2048 600×600×3(WHU-RS) 13×13×4096 12×12×1024 13×13×2048 
下載: 導出CSV
表 2 UC-Merced中特征的相關(guān)系數(shù)
特征 A M 16 19 G M –0.0037 16 0.0006 0.0028 19 –0.0023 0.0053 0.4817 G 0.0012 –0.0063 –0.0086 –0.0100 R –0.0100 0.0008 –0.0060 –0.0021 0.1175
下載: 導出CSV
表 3 WHU-RS中特征的相關(guān)系數(shù)
特征 A M 16 19 G M –0.0080 16 –0.0009 0.0027 19 0.0001 0.0051 0.4762 G –0.0024 –0.0038 –0.0110 –0.0093 R –0.0045 0.0084 –0.0069 –0.0022 0.1138
下載: 導出CSV
表 4 不同輸入尺寸CoP特征檢索結(jié)果比較
數(shù)據(jù)集 特征 默認尺寸 原始尺寸 ANMRR mAP ANMRR mAP UC-Merced CoP(16_G) 0.2898 0.6411 0.2880 0.6446 CoP(16_G_M) 0.2834 0.6485 0.2832 0.6504 CoP(16_G_M_19) 0.2834 0.6496 0.2805 0.6544 WHU-RS CoP(A_16) 0.2007 0.7466 0.2330 0.7116 CoP(A_16_G) 0.1891 0.7582 0.2319 0.7125 CoP(A_16_G_19) 0.1875 0.7610 0.2318 0.7124
下載: 導出CSV
表 5 UC-Merced中微調(diào)CoP特征檢索結(jié)果比較
數(shù)據(jù)集 特征 默認尺寸 原始尺寸 ANMRR mAP ANMRR mAP UC-Merced CoP(16_G)-FT 0.2738 0.6602 0.2777 0.6566 CoP(16_G_M)-FT 0.2642 0.6716 0.2678 0.6683 CoP(16_G_M_19)-FT 0.2604 0.6767 0.2561 0.6822 WHU-RS CoP(A_16)-FT 0.1723 0.7809 0.1975 0.7501 CoP(A_16_G)-FT 0.1582 0.7971 0.1924 0.7559 CoP(A_16_G_19)-FT 0.1519 0.8048 0.1879 0.7615
下載: 導出CSV
表 6 UC-Merced中CoP特征與其他特征檢索結(jié)果比較
特征 ANMRR 維數(shù) 淺層特征 VLAD[2] 0.4604 16384 3層圖[4] 0.4317 – CNN特征 VGGM-fc[14] 0.3780 4096 VGGM-conv5-IFK[15] 0.4580 102400 VGG16-fc[15] 0.3940 4096 VGG16-conv5-IFK[15] 0.4070 102400 LDCNN[15] 0.4390 30 GoogLeNet-MultiPatch-FT[16] 0.3140 1024 GoogLeNet-MultiPatch-FT-PCA[16] 0.2850 32 CoP(16_G) 0.2880 4096 CoP(16_G_M_19) 0.2805 4096 CoP(16_G_M_19)-FT 0.2561 4096 CoP(16_G_M_19)-PCA 0.2577 32
下載: 導出CSV
-
DEMIR B and BRUZZONE L. A novel active learning method in relevance feedback for content-based remote sensing image retrieval[J]. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(5): 2323–2334. doi: 10.1109/TGRS.2014.2358804 ?ZKAN S, ATE? T, TOLA E, et al. Performance analysis of state-of-the-art representation methods for geographical image retrieval and categorization[J]. IEEE Geoscience and Remote Sensing Letters, 2014, 11(11): 1996–2000. doi: 10.1109/LGRS.2014.2316143 陸麗珍, 劉仁義, 劉南. 一種融合顏色和紋理特征的遙感圖像檢索方法[J]. 中國圖象圖形學報, 2004, 9(3): 328–333. doi: 10.3969/j.issn.1006-8961.2004.03.013LU Lizhen, LIU Renyi, and LIU Nan. Remote sensing image retrieval using color and texture fused features[J]. Journal of Image and Graphics, 2004, 9(3): 328–333. doi: 10.3969/j.issn.1006-8961.2004.03.013 WANG Yuebin, ZHANG Liqiang, TONG Xiaohua, et al. A three-layered graph-based learning approach for remote sensing image retrieval[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(10): 6020–6034. doi: 10.1109/TGRS.2016.2579648 郭智, 宋萍, 張義, 等. 基于深度卷積神經(jīng)網(wǎng)絡的遙感圖像飛機目標檢測方法[J]. 電子與信息學報, 2018, 40(11): 2684–2690. doi: 10.11999/JEIT180117GUO Zhi, SONG Ping, ZHANG Yi, et al. Aircraft detection method based on deep convolutional neural network for remote sensing images[J]. Journal of Electronics &Information Technology, 2018, 40(11): 2684–2690. doi: 10.11999/JEIT180117 YE Famao, SU Yanfei, XIAO Hui, et al. Remote sensing image registration using convolutional neural network features[J]. IEEE Geoscience and Remote Sensing Letters, 2018, 15(2): 232–236. doi: 10.1109/LGRS.2017.2781741 KRIZHEVSKY A, SUTSKEVER I, and HINTON G E. ImageNet classification with deep convolutional neural networks[C]. The 25th International Conference on Neural Information Processing Systems, Nevada, USA, 2012: 1097–1105. CHATFIELD K, SIMONYAN K, VEDALDI A, et al. Return of the devil in the details: Delving deep into convolutional networks[C]. The 25th British Machine Vision Conference, Nottingham, UK, 2014. SIMONYAN K and ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]. The 3rd International Conference on Learning Representations, San Diego, USA, 2015. SZEGEDY C, LIU Wei, JIA Yangqing, et al. Going deeper with convolutions[C]. 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, USA, 2015: 1–9. HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]. 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 770–778. CASTELLUCCIO M, POGGI G, SANSONE C, et al. Land use classification in remote sensing images by convolutional neural networks[J]. Acta Ecologica Sinica, 2015, 28(2): 627–635. ALIAS B, KARTHIKA R, and PARAMESWARAN L. Content based image retrieval of remote sensing images using deep learning with different distance measures[J]. Journal of Advanced Research in Dynamical and Control Systems, 2018, 10(3): 664–674. NAPOLETANO P. Visual descriptors for content-based retrieval of remote-sensing Images[J]. International Journal of Remote Sensing, 2018, 39(5): 1343–1376. doi: 10.1080/01431161.2017.1399472 ZHOW Weixun, NEWSAM S, LI Congmin, et al. Learning low dimensional convolutional neural networks for high-resolution remote sensing image retrieval[J]. Remote Sensing, 2017, 9(5): 489. doi: 10.3390/rs9050489 HU Fan, TONG Xinyi, XIA Guisong, et al. Delving into deep representations for remote sensing image retrieval[C]. The IEEE 13th International Conference on Signal Processing, Chengdu, China, 2016: 198–203. SHELHAMER E, LONG J, and DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640–651. doi: 10.1109/TPAMI.2016.2572683 VEDALDI A and LENC K. MatConvNet: Convolutional neural networks for MATLAB[C]. The 23rd ACM International Conference on Multimedia, Brisbane, Australia, 2015: 689–692. HU Fan, XIA Guisong, HU Jingwen, et al. Transferring deep convolutional neural networks for the scene classification of high-resolution remote sensing imagery[J]. Remote Sensing, 2015, 7(11): 14680–14707. doi: 10.3390/rs71114680 ZOU Qin, NI Lihao, ZHANG Tong, et al. Deep learning based feature selection for remote sensing scene classification[J]. IEEE Geoscience and Remote Sensing Letters, 2015, 12(11): 2321–2325. doi: 10.1109/LGRS.2015.2475299 -
圖(5) / 表(6)
計量
- 文章訪問數(shù): 2848
- HTML全文瀏覽量: 1130
- PDF下載量: 65
- 被引次數(shù): 0
下載:
