采用自適應(yīng)預(yù)篩選的遙感圖像目標(biāo)開集檢測研究

黨思航; 李曉哲; 夏召強(qiáng); 蔣曉悅; 桂術(shù)亮; 馮曉毅

doi:10.11999/JEIT231426

采用自適應(yīng)預(yù)篩選的遙感圖像目標(biāo)開集檢測研究

doi: 10.11999/JEIT231426

1.
西北工業(yè)大學(xué)上海閔行協(xié)同創(chuàng)新中心上海 201108
2.
西北工業(yè)大學(xué)電子信息學(xué)院西安 710072
3.
重慶郵電大學(xué)通信與信息工程學(xué)院重慶 400065

基金項(xiàng)目: 國家自然科學(xué)基金(62201461, 62301101)，上海市2022年度“科技創(chuàng)新行動計劃”啟明星培育(揚(yáng)帆專項(xiàng))項(xiàng)目(22YF1452100)，陜西省科技廳秦創(chuàng)原項(xiàng)目(QCYRCXM-2022-325)，陜西省重點(diǎn)研發(fā)計劃(2023-ZDLGY-16, 2023-ZDLGY-44, 2023-ZDLGY-12, 2021-ZDLGY15-01, 2021-ZDLGY09-04, 2021GY-004, 2022-ZDLGY06-07)，重慶市博士“直通車”科研項(xiàng)目(sl202100000315)

詳細(xì)信息

作者簡介:
黨思航：男，副教授，研究方向?yàn)槔走_(dá)目標(biāo)識別、增量學(xué)習(xí)

李曉哲：男，碩士，研究方向?yàn)槟繕?biāo)檢測、開集識別

夏召強(qiáng)：男，副教授，研究方向?yàn)閳D像處理、計算機(jī)視覺

蔣曉悅：女，副教授，研究方向?yàn)閳D像處理、計算機(jī)視覺

桂術(shù)亮：男，講師，研究方向?yàn)槔走_(dá)信號處理、目標(biāo)檢測

馮曉毅：女，教授，研究方向?yàn)閳D像處理、計算機(jī)視覺

通訊作者:
夏召強(qiáng)　zxia@nwpu.edu.cn

中圖分類號: TP75
計量
- 文章訪問數(shù): 361
- HTML全文瀏覽量: 164
- PDF下載量: 72
- 被引次數(shù): 0
出版歷程
- 收稿日期: 2023-12-02
- 修回日期: 2024-07-04
- 網(wǎng)絡(luò)出版日期: 2024-07-25
- 刊出日期: 2024-10-30

Research on Open-Set Object Detection in Remote Sensing Images Based on Adaptive Pre-Screening

1.
Collaborative Innovation Center of NPU, Shanghai, Northwestern Polytechnical University, Shanghai 201108, China
2.
School of Electronics and Information, Northwestern Polytechnical University, Xi’an 710072, China
3.
School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, China

Funds: The National Natural Science Foundation of China(62201461, 62301101), Shanghai Sailing Program (22YF1452100), The QINCHUANGYUAN Program (QCYRCXM-2022-325), The Key Research and Development Program of Shaanxi (2023-ZDLGY-16, 2023-ZDLGY-44, 2023-ZDLGY-12, 2021-ZDLGY15-01, 2021-ZDLGY09-04, 2021GY-004, 2022-ZDLGY06-07), Chongqing Doctoral Direct Train Research Project (sl202100000315)

摘要

摘要: 開放動態(tài)環(huán)境下目標(biāo)類別不斷豐富，遙感目標(biāo)檢測問題不能局限于已知類目標(biāo)的鑒別，還需要對未知類目標(biāo)做出有效判決。該文設(shè)計一種基于自適應(yīng)預(yù)篩選的遙感開集目標(biāo)檢測網(wǎng)絡(luò)，首先，提出面向目標(biāo)候選框的自適應(yīng)預(yù)篩選模塊，依據(jù)篩選出的候選框坐標(biāo)得到具有豐富語義信息和空間特征的查詢傳遞至解碼器。然后，結(jié)合原始圖像中目標(biāo)邊緣信息提出一種偽標(biāo)簽選取方法，并以開集判決為目的構(gòu)造損失函數(shù)，提高網(wǎng)絡(luò)對未知新類特征的學(xué)習(xí)能力。最后，采用MAR20飛機(jī)目標(biāo)識別數(shù)據(jù)集模擬不同的開放動態(tài)遙感目標(biāo)檢測環(huán)境，通過廣泛的對比實(shí)驗(yàn)和消融實(shí)驗(yàn)，驗(yàn)證了該文方法能夠?qū)崿F(xiàn)對已知類目標(biāo)的可靠檢測和未知類目標(biāo)的有效檢出。
- 遙感目標(biāo)檢測 /
- 候選框篩選 /
- 開集識別
Abstract: In open, dynamic environments where the range of object categories continually expands, the challenge of remote sensing object detection is to detect a known set of object categories while simultaneously identifying unknown objects. To this end, a remote sensing open-set object detection network based on adaptive pre-screening is proposed. Firstly, an adaptive pre-screening module is proposed for object region proposals. Based on the coordinates of the selected region proposals, queries with rich semantic information and spatial features are generated and passed to the decoder. Subsequently, a pseudo-label selection method is devised based on object edge information, and loss functions are constructed with the aim of open set classification to enhance the network’s ability to learn knowledge of unknown classes. Finally, the Military Aircraft Recognition (MAR20) dataset is used to simulate various dynamic environments. Extensive comparative experiments and ablation experiments show that the proposed method can achieve reliable detection of known and unknown objects.
- Remote sensing object detection /
- Region proposal selection /
- Open set recognition

HTML全文

圖 1 網(wǎng)絡(luò)總體結(jié)構(gòu)

下載: 全尺寸圖片幻燈片

圖 2 自適應(yīng)預(yù)篩選模塊

下載: 全尺寸圖片幻燈片

圖 3 MAR20數(shù)據(jù)集部分圖像展示

下載: 全尺寸圖片幻燈片

圖 4 MAR20測試集示例圖像定性結(jié)果

下載: 全尺寸圖片幻燈片

1 基于圖像邊緣信息的偽標(biāo)簽選取算法

輸入：當(dāng)前迭代$ t $條件下：對應(yīng)特征圖$ \boldsymbol{A} $；經(jīng)過DDETR匹配機(jī)制剩余的預(yù)測候選框$ {{\boldsymbol}}_{i}^{{\mathrm{F}}} $；基于圖像邊緣信息生成的候選框$ {{\boldsymbol}}_{j}^{{\mathrm{E}}} $；損失存儲隊(duì) 　列$ {L}_{m} $；微調(diào)參數(shù)$ {\lambda }_{p} $和$ {\lambda }_{n} $；權(quán)重更新迭代次數(shù)$ {T}_{w} $；權(quán)重值$ {w}_{1} $和$ {w}_{2} $；偽標(biāo)簽個數(shù)$ u $
輸出：當(dāng)前迭代$ t $條件下：圖像的偽標(biāo)簽
1. while train do:
2. 式(1)初步得到基于卷積特征的目標(biāo)置信度得分$ F\left({{\boldsymbol}}_{i}^{{\mathrm{F}}}\right) $；
3. 式(3)得到基于圖像底層邊緣信息的目標(biāo)置信度得分S$ \left({{\boldsymbol}}_{i}^{\mathrm{{E}}}\right) $；
4. if $ t\mathrm{\%}{T}_{w}==0 $ then:
5. 使用式(7)和$ {L}_{m} $計算$ \Delta l $；
6. 使用式(8)計算$ \Delta w $；
7. 使用式(5)更新權(quán)重值$ {w}_{1} $和$ {w}_{2} $；
8. end if
9. 使用式(4)得到剩余的預(yù)測候選框$ {{\boldsymbol}}_{i}^{{\mathrm{F}}} $的最終目標(biāo)置信度分?jǐn)?shù)$ {F}_{i}^{{\mathrm{new}}} $；
10. 對$ {F}_{i}^{\mathrm{n}\mathrm{e}\mathrm{w}} $從大到小排序，選取前$ {u} $個候選框標(biāo)記“未知類”。

下載: 導(dǎo)出CSV

表 1 MAR20數(shù)據(jù)集圖像數(shù)量分布情況

A1	A2	A3	A4	A5	A6	A7	A8	A9	A10
168	16	150	70	247	31	100	142	146	146

A11	A12	A13	A14	A15	A16	A17	A18	A19	A20
86	66	212	252	108	265	173	37	129	130
含多類	1017
總計	3842

下載: 導(dǎo)出CSV

表 2 開集目標(biāo)檢測任務(wù)

實(shí)驗(yàn)編號	未知類	目標(biāo)類別		訓(xùn)練與測試比例		總計
實(shí)驗(yàn)編號	#已知類+#未知類	已知類	未知類	訓(xùn)練	測試	總計
任務(wù)1	0.75	A1～A5	A6～A20	644	161	805
任務(wù)2	0.5	A1～A10	A11～A20	736	185	921
任務(wù)3	0.25	A1～A15	A16～A20	764	192	956

下載: 導(dǎo)出CSV

表 3 網(wǎng)絡(luò)檢測結(jié)果對比(%)

任務(wù)編號	任務(wù)1		任務(wù)2		任務(wù)3
任務(wù)編號	已知類mAP	未知類召回率	已知類mAP	未知類召回率	已知類mAP	未知類召回率
Faster-RCNN	73.95	–	77.84	–	88.18	–
YOLOv3	88.02	–	88.40	–	88.86	–
DDETR	84.30	–	87.60	–	88.95	–
OW-DETR	82.52	17.66	87.90	29.68	87.66	30.41
CAT	77.40	21.21	83.78	36.09	85.05	53.42
本文算法	89.09	38.67	90.35	47.17	90.38	61.20

下載: 導(dǎo)出CSV

表 4 模塊驗(yàn)證實(shí)驗(yàn)結(jié)果(%)

模塊			任務(wù)1消融實(shí)驗(yàn)		任務(wù)2消融實(shí)驗(yàn)		任務(wù)3消融實(shí)驗(yàn)
基準(zhǔn) 模型	自適應(yīng)預(yù)篩選	基于邊緣信息的偽標(biāo)簽選取策略	已知類mAP	未知類召回率	已知類mAP	未知類召回率	已知類mAP	未知類召回率
√			82.52	17.66	87.90	29.68	87.66	30.41
√	√		89.34	2.43	90.83	5.96	89.74	13.33
√		√	83.28	45.81	87.54	63.01	87.69	54.80
√	√	√	89.09	38.67	90.35	47.17	90.38	61.20

下載: 導(dǎo)出CSV

參考文獻(xiàn)(24)

[1]	ZAIDI S S A, ANSARI M S, ASLAM A, et al. A survey of modern deep learning based object detection models[J]. Digital Signal Processing, 2022, 126: 103514. doi: 10.1016/j.dsp.2022.103514.
[2]	ZOU Zhengxia, CHEN Keyan, SHI Zhenwei, et al. Object detection in 20 years: A survey[J]. Proceedings of the IEEE, 2023, 111(3): 257–276. doi: 10.1109/JPROC.2023.3238524.
[3]	呂進(jìn)東, 王彤, 唐曉斌. 基于圖注意力網(wǎng)絡(luò)的半監(jiān)督SAR艦船目標(biāo)檢測[J]. 電子與信息學(xué)報, 2023, 45(5): 1541–1549. doi: 10.11999/JEIT220139. Lü Jindong, WANG Tong, and TANG Xiaobin. Semi-supervised SAR ship target detection with graph attention network[J]. Journal of Electronics & Information Technology, 2023, 45(5): 1541–1549. doi: 10.11999/JEIT220139.
[4]	王璽坤, 姜宏旭, 林珂玉. 基于改進(jìn)型YOLO算法的遙感圖像艦船檢測[J]. 北京航空航天大學(xué)學(xué)報, 2020, 46(6): 1184–1191. doi: 10.13700/j.bh.1001-5965.2019.0394. WANG Xikun, JIANG Hongxu, and LIN Keyu. Remote sensing image ship detection based on modified YOLO algorithm[J]. Journal of Beijing University of Aeronautics and Astronautics, 2020, 46(6): 1184–1191. doi: 10.13700/j.bh.1001-5965.2019.0394.
[5]	AI Jiaqiu, TIAN Ruitian, LUO Qiwu, et al. Multi-scale rotation-invariant Haar-like feature integrated CNN-based ship detection algorithm of multiple-target environment in SAR imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(12): 10070–10087. doi: 10.1109/TGRS.2019.2931308.
[6]	黃玉玲, 陶昕辰, 朱濤, 等. 殘差對抗目標(biāo)檢測算法的遙感圖像檢測[J]. 電光與控制, 2023, 30(7): 63–67. doi: 10.3969/j.issn.1671-637X.2023.07.011. HUANG Yuling, TAO Xinchen, ZHU Tao, et al. A remote sensing image detection method based on residuals adversarial object detection algorithm[J]. Electronics Optics & Control, 2023, 30(7): 63–67. doi: 10.3969/j.issn.1671-637X.2023.07.011.
[7]	馬梁, 茍于濤, 雷濤, 等. 基于多尺度特征融合的遙感圖像小目標(biāo)檢測[J]. 光電工程, 2022, 49(4): 210363. doi: 10.12086/oee.2022.210363. MA Liang, GOU Yutao, LEI Tao, et al. Small object detection based on multi-scale feature fusion using remote sensing images[J]. Opto-Electronic Engineering, 2022, 49(4): 210363. doi: 10.12086/oee.2022.210363.
[8]	REN Shaoqing, HE Kaiming, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137–1149. doi: 10.1109/TPAMI.2016.2577031.
[9]	REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: Unified, real-time object detection[C]. The 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 779–788. doi: 10.1109/CVPR.2016.91.
[10]	REDMON J and FARHADI A. YOLOv3: An incremental improvement[EB/OL]. https://arxiv.org/abs/1804.02767, 2018.
[11]	邵延華, 張鐸, 楚紅雨, 等. 基于深度學(xué)習(xí)的YOLO目標(biāo)檢測綜述[J]. 電子與信息學(xué)報, 2022, 44(10): 3697–3708. doi: 10.11999/JEIT210790. SHAO Yanhua, ZHANG Duo, CHU Hongyu, et al. A review of YOLO object detection based on deep learning[J]. Journal of Electronics & Information Technology, 2022, 44(10): 3697–3708. doi: 10.11999/JEIT210790.
[12]	CARION N, MASSA F, SYNNAEVE G, et al. End-to-end object detection with transformers[C]. The 16th European Conference on Computer Vision, Glasgow, UK, 2020: 213–229. doi: 10.1007/978-3-030-58452-8_13.
[13]	GENG Chuanxing, HUANG Shengjun, and CHEN Songcan. Recent advances in open set recognition: A survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(10): 3614–3631. doi: 10.1109/TPAMI.2020.2981604.
[14]	DANG Sihang, CAO Zongjie, CUI Zongyong, et al. Open set incremental learning for automatic target recognition[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(7): 4445–4456. doi: 10.1109/TGRS.2019.2891266.
[15]	DANG Sihang, XIA Zhaoqiang, JIANG Xiaoyue, et al. Inclusive consistency-based quantitative decision-making framework for incremental automatic target recognition[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 5215614. doi: 10.1109/TGRS.2023.3312330.
[16]	JOSEPH K J, KHAN S, KHAN F S, et al. Towards open world object detection[C]. The 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, USA, 2021: 5826–5836. doi: 10.1109/CVPR46437.2021.00577.
[17]	GUPTA A, NARAYAN S, JOSEPH KJ, et al. OW-DETR: Open-world detection transformer[C]. The 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, USA, 2022: 9225–9234. doi: 10.1109/CVPR52688.2022.00902.
[18]	MA Shuailei, WANG Yuefeng, WEI Ying, et al. CAT: LoCalization and identification cascade detection transformer for open-world object detection[C]. The 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, Canada, 2023: 19681–19690. doi: 10.1109/CVPR52729.2023.01885.
[19]	UIJLINGS J R R, VAN DE SANDE K E A, GEVERS T, et al. Selective search for object recognition[J]. International Journal of Computer Vision, 2013, 104(2): 154–171. doi: 10.1007/s11263-013-0620-5.
[20]	ZOHAR O, WANG K C, and YEUNG S. PROB: Probabilistic objectness for open world object detection[C]. The 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, Canada, 2023: 11444–11453. doi: 10.1109/CVPR52729.2023.01101.
[21]	CHENG Gong, XIE Xingxing, HAN Junwei, et al. Remote sensing image scene classification meets deep learning: Challenges, methods, benchmarks, and opportunities[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13: 3735–3756. doi: 10.1109/JSTARS.2020.3005403.
[22]	ZITNICK C L and DOLLáR P. Edge boxes: Locating object proposals from edges[C]. The 13th European Conference on Computer Vision, Zurich, Switzerland, 2014: 391–405. doi: 10.1007/978-3-319-10602-1_26.
[23]	禹文奇, 程塨, 王美君, 等. MAR20: 遙感圖像軍用飛機(jī)目標(biāo)識別數(shù)據(jù)集[J]. 遙感學(xué)報, 2023, 27(12): 2688–2696. doi: 10.11834/jrs.20222139. YU Wenqi, CHENG Gong, WANG Meijun, et al. MAR20: A benchmark for military aircraft recognition in remote sensing images[J]. National Remote Sensing Bulletin, 2023, 27(12): 2688–2696. doi: 10.11834/jrs.20222139.
[24]	ZHU Xizhou, SU Weijie, LU Lewei, et al. Deformable DETR: Deformable transformers for end-to-end object detection[C]. 9th International Conference on Learning Representations, 2021.