利用非局部上下文信息的遥感图像小目标检测

doi:10.19665/j.issn1001-2400.2022.05.014

摘要/Abstract

摘要：

近年来,基于深度学习的目标检测方法取得了令人瞩目的成果,并被成功应用到遥感领域。但是由于遥感图像覆盖面广,而小目标的有效信息少且定位困难,想要精确地检测出小目标并不容易。针对这一问题,利用图像中的非局部特征以及上下文信息来提高小目标检测的质量。首先采用改善特征金字塔网络和跨层注意力网络的组合结构作为主干网络,改善特征金字塔网络用于提取小目标丰富的特征信息,跨层注意力网络用来提取非局部信息并均衡分配给各层;其次使用上下文转移模块,将包含了非局部特征的上下文信息传递给对应的感兴趣区域;最后采取级联网络作为检测网络,改善小目标定位框质量。在Small-DOTA、DIOR和OHD-SJTU-S遥感图像数据集上进行了实验。实验结果表明,所提方法在3个数据集的平均精度均值都达到了最高;在DIOR数据集中包含小目标比较多的船、车辆和风车3个类别上,该算法的平均精度也是最高。这说明与现有方法相比,所提方法能进一步改善遥感图像小目标检测性能。

关键词: 遥感, 目标检测, 图像处理, 非局部信息, 上下文

Abstract:

In recent years,the object detection method based on deep learning has achieved remarkable results and has been successfully applied to remote sensing.However,due to the wide coverage of remote sensing images,and the small object with less effective information and which is difficult to locate,it is challenging to accurately detect a small object from remote sensing images.To solve this problem,non-local information and context are utilized to improve the quality of small object detection in this paper.First,the detector uses a combination of Refine Feature Pyramid Networks(Refine FPN)and Cross-layer Attention Network(CA-Net)as the backbone,where the Refine FPN obtains richer feature information of a small object,and the CA-Net extracts non-local information and distributes it to each layer evenly.Second,the context transfer module is proposed to transfer the non-local context information to the corresponding region of interest.Finally,the cascade network is used as the detection network to improve the quality of the bounding box of the small object.Experiments are carried out on three remote sensing image datasets,Small-DOTA,DIOR,and OHD-SJTU-S.Experimental results show that the mean average precision(mAP)of the detector proposed in this paper reaches the highest in the three datasets.Among the three categories of ships,vehicles,and windmills that contain more small targets in the DIOR,the average precision(AP)of the detector in this paper is also the highest.This shows that compared with the existing methods,the method in this paper can further improve the detection performance of the small object in remote sensing images.

Key words: remote sensing, object detection, image processing, non-local information, context

中图分类号:

TP75

李阳阳, 毛鹤亭, 张小龙, 陈彦桥, 柴兴华. 利用非局部上下文信息的遥感图像小目标检测[J]. 西安电子科技大学学报, 2022, 49(5): 117-124.

LI Yangyang, MAO Heting, ZHANG Xiaolong, CHEN Yanqiao, CHAI Xinghua. Small object detection in remote sensing images using non-local context information[J]. Journal of Xidian University, 2022, 49(5): 117-124.

图/表 7

图1

图2

图3

表1

表2

表3

图4

参考文献 19

[1]	戴昌达, 姜小光, 唐伶俐, 等. 遥感图像应用处理与分析[M]. 北京: 清华大学出版社, 2004:219.
[2]	LI Y Y, MAO H T, LIU R J, et al. A Lightweight Keypoint-Based Oriented Object Detection of Remote Sensing Images[J]. Remote Sensing, 2021, 13 (13):2459-2478. doi: 10.3390/rs13132459
[3]	REN S, HE K, GIRSHICK R, et al. Faster R-CNN:Towards Real-Time Object Detection with Region Proposal Networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 39(6):1137-1149. doi: 10.1109/TPAMI.2016.2577031
[4]	DAI J, LI Y, HE K, et al. R-FCN:Object Detection via Region-Based Fully Convolutional Networks[C]//Advances in Neural Information Processing Systems. New York: ACM, 2016:379-387.
[5]	REDMON J, DIVVALA S, GIRSHICK R, et al. You Only Look Once:Unified,Real-Time Object Detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016:779-788.
[6]	LIN T Y, GOYAL P, GIRSHICK R, et al. Focal Loss for Dense Object Detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(2):318-327. doi: 10.1109/TPAMI.2018.2858826
[7]	LA W, HDENG J. CornerNet:Detecting Objects as Paired Keypoints[J]. International Journal of Computer Vision, 2020, 128 (3):642-656. doi: 10.1007/s11263-019-01204-1
[8]	LI Y Y, HUANG Q, PEI X, et al. RADet:Refine Feature Pyramid Network and Multi-Layer Attention Network for Arbitrary-Oriented Object Detection of Remote Sensing Images[J]. Remote Sensing, 2020, 12 (3):389-408. doi: 10.3390/rs12030389
[9]	LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature Pyramid Networks for Object Detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017:2117-2125.
[10]	LI Y Y, HUANG Q, PEI X, et al. Cross-layer Attention Network for Small Object Detection in Remote Sensing Imagery[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 14:2148-2161. doi: 10.1109/JSTARS.2020.3046482
[11]	CAI Z, VASCONCELOS N. Cascade R-CNN:Delving into High Quality Object Detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018:6154-6162.
[12]	HE K, ZHANG X, REN S, et al. Deep Residual Learning for Image Recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016:770-778.
[13]	WANG X, GIRSHICK R, GUPTA A, et al. Non-local Neural Networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018:7794-7803.
[14]	CAO Y, XU J, LIN S, et al. GCNet:Non-local Networks Meet Squeeze-Excitation Networks and Beyond[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops. Piscataway: IEEE, 2019:1971-1980.
[15]	DENG J, DONG W, SOCHER R, et al. Imagenet:A Large-Scale Hierarchical Image Database[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2009:248-255.
[16]	HE K, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN[C]//IEEE Transactions on Pattern Analysis & Machine Intelligence. Piscataway: IEEE, 2017:2961-2969.
[17]	XIA G S, BAI X, DING J, et al. DOTA:A Large-Scale Dataset for Object Detection in Aerial Images[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018:3974-3983.
[18]	LI K, WAN G, CHENG G, et al. Object Detection in Optical Remote Sensing Images:a Survey and A New Benchmark[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 159:296-307. doi: 10.1016/j.isprsjprs.2019.11.023
[19]	YANG X, YAN J. On the Arbitrary-Oriented Object Detection:Classification Based Approaches Revisited[EB/OL].[2022-03-23]. https://arxiv.org/pdf/2003.05597.pdf .

方法	小型车辆	船	储油罐	大型车辆	mAP
Faster R-CNN	81.22	80.38	87.87	73.57	80.76
FPN	83.15	80.74	88.03	74.75	81.67
Refine FPN	85.17	89.20	88.11	75.07	84.39
Cascade R-CNN	84.46	89.24	88.18	74.66	84.14
CA-Net^*	84.85	89.10	88.41	78.68	85.26
文中算法	85.71	89.53	88.95	79.81	86.00

对比方法	飞机	船	mAP
Faster R-CNN	86.09	74.58	80.34
FPN	85.47	78.76	82.12
Refine FPN	86.62	79.90	83.26
Cascade R-CNN	89.93	82.31	86.12
CA-Net^*	88.52	80.72	84.62
文中算法	90.17	83.83	87.00

类别	Faster R-CNN	FPN	Refine FPN	Cascade R-CNN	CA-Net^*	文中算法
飞机	52.3	62.2	62.8	62.8	69.8	63.0
机场	66.4	73.4	73.7	81.3	78.9	83.7
棒球场	67.2	71.7	71.6	71.9	77.5	72.0
篮球场	80.5	81.0	87.5	88.2	87.2	88.3
桥梁	34.3	44.9	50.7	52.2	52.2	52.9
烟囱	71.8	72.5	79.2	79.5	79.6	79.7
水坝	54.2	58.8	64.8	64.8	65.2	67.9
高速公路服务区	62.5	76.8	78.4	79.0	84.0	83.9
高尔夫球场	69.6	72.8	75.0	77.7	78.3	82.1
田径场	69.1	76.7	82.4	83.3	82.6	82.2
港口	48.6	43.5	55.3	59.5	55.7	59.7
立交桥	52.4	57.4	62.1	63.2	61.7	64.5
船	69.3	71.9	80.9	81.0	80.9	81.1
体育馆	62.3	68.6	73.4	74.2	75.3	72.6
储油罐	57.8	62.3	62.7	62.9	62.5	70.7
网球场	79.8	81.3	81.4	81.5	88.2	81.5
火车站	51.9	56.9	62.3	62.7	64.4	67.0
车辆	38.6	49.7	50.8	50.8	51.1	51.4
风车	78.0	80.6	88.6	89.1	89.1	89.2
mAP	60.8	66.7	71.0	71.8	73.0	73.5