融合注意力机制的多尺度显著性目标检测网络

doi:10.19665/j.issn1001-2400.2022.04.014

摘要/Abstract

摘要：

目前大多数显著性目标检测方法会受到图像复杂背景的干扰并且会出现检测结果亮度不均匀、边缘模糊的现象。针对上述问题,提出一种融合注意力机制的多尺度显著性目标检测网络方法。首先,网络以编码器-解码器架构为基础,并采用在编解码过程中连接相邻层特征的多尺度特征融合方法,以便于捕捉到图像中不同尺度的显著性目标;其次,在网络中融合注意力机制,用以关注特征的空间信息和通道信息,目的是得到均匀完整且边缘更加清晰的显著性目标检测结果;最后,在编码器与解码器之间使用一种并行多分支结构即上下文特征提取模块实现不同感受野下的特征提取,进一步提升显著性目标检测性能。实验结果表明,在ECSSD显著性目标检测数据集上检测平均绝对误差MAE和F-measure指标,相较于对比网络至少有10%和0.7%的提高。所提网络不仅能准确定位显著性目标并使其均匀显示,而且在复杂背景下能够精确预测显著性目标边缘。

关键词: 显著性目标检测, 注意力机制, 多尺度特征融合, 深度学习, 图像处理

Abstract:

At present,most salient object detection algorithms are disturbed by the complex background of the image,and the detection results show the phenomena of uneven brightness and blurred edges.To address the above issues,a salient object detection network combining attention mechanism and multi-scale feature fusion is proposed.First,the network is based on the encoder-decoder architecture and the features from adjacent layers are connected in the encoding and decoding process,which captures the multi-scale salient objects in the image.Second,the attention mechanism is integrated in the network to focus on the spatial information and channel information of features,with the purpose of obtaining uniform and complete salient object detection results with clear edges.Finally,a parallel multi-branch structure,named Context Feature Extraction Module,is used to extract features under different receptive fields to improve the performance of salient object detection.Experimental results show that the proposed method can not only accurately locate and highlight the salient objects,but also accurately predict the edge of the salient object in the complex background.Compared with the contrast methods,the average absolute error of MAE and F-Measure on the salient object detection dataset ECSSD can be improved by at least 10% and 0.7%,respectively.

Key words: salient object detection, attention mechanism, multi-scale feature fusion, deep learning, image processing

中图分类号:

TP391.4

刘迪,郭继昌,汪昱东,张怡. 融合注意力机制的多尺度显著性目标检测网络[J]. 西安电子科技大学学报, 2022, 49(4): 118-126.

LIU Di,GUO Jichang,WANG Yudong,ZHANG Yi. Multi-scale salient object detection network combining an attention mechanism[J]. Journal of Xidian University, 2022, 49(4): 118-126.

图/表 9

图1

图2

图3

图4

图5

表1

图6

表2

图7

参考文献 24

[1]	FENG T, ABHAYARATNE C. Visual Saliency Guided High Dynamic Range Image Compression[C]// 2018 26th European Signal Processing Conference.Piscataway:IEEE, 2018:166-170.
[2]	LIU H, CHU W, WANG H. Automatic Segmentation Algorithm of Ultrasound Heart Image Based on Convolutional Neural Network and Image Saliency[J]. IEEE Access, 2020, 8:104445-104457. doi: 10.1109/ACCESS.2020.2989819
[3]	HORBERT E, GARCÍA G M, FRINTROP S, et al. Sequence-Level Object Candidates Based on Saliency for Generic Object Recognition on Mobile Systems[C]// 2015 IEEE International Conference on Robotics and Automation.Piscataway:IEEE, 2015:127-134.
[4]	LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-Based Learning Applied to Document Recognition[J]. Proceedings of the IEEE, 1998, 86(11):2278-2324. doi: 10.1109/5.726791
[5]	孙豪杰, 李苗钰, 章盼盼, 等. 用于面瘫分级的自监督非对称特征学习方法[J]. 西安电子科技大学学报, 2021, 48(3):115-122.
	SUN Haojie, LI Miaoyu, ZHANG Panpan, et al. Self-Supervised Facial Asymmetry Learning for Automatic Evaluation of Facial Paralysis[J]. Journal of Xidian University, 2021, 48(3):115-122.
[6]	BHUVANESWARI M. Automatic Segmenting Technique of Brain Tumors with Convolutional Neural Networks in MRI Images[C]// 2021 6th International Conference on Inventive Computation Technologies.Piscataway:IEEE, 2021:759-764.
[7]	ZHANG P, WANG D, LU H, et al. Amulet:Aggregating Multi-Level Convolutional Features for Salient Object Detection[C]// Proceedings of the 2017 IEEE International Conference on Computer Vision.Piscataway:IEEE, 2017:202-211.
[8]	CHEN S, TAN X, WANG B, et al. Reverse Attention for Salient Object Detection[C]// Proceedings of the European Conference on Computer Vision.Heidelberg:Springer, 2018:234-250.
[9]	WANG W, ZHAO S, SHEN J, et al. Salient Object Detection with Pyramid Attention and Salient Edges[C]// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2019:1448-1457.
[10]	张晴, 左保川, 石艳娇, 等. 多尺度卷积神经网络显著物体检测[J]. 中国图象图形学报, 2020, 25(6):1116-1129.
	ZHANG Qing, ZUO Baochuan, SHI Yanjiao, et al. A Multi-Scale Convolutional Neural Network for Salient Object Detection[J]. Journal of Image and Graphics, 2020, 25(6):1116-1129.
[11]	WOO S, PARK J, LEE J Y, et al. CBAM:Convolutional Block Attention Module[C]// Proceedings of the European Conference on Computer Vision.Heidelberg:Springer, 2018:3-19.
[12]	WANG L, LU H, WANG Y, et al. Learning to Detect Salient Objects with Image-Level Supervision[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2017:136-145.
[13]	YANG C, ZHANG L, LU H. Graph-Regularized Saliency Detection with Convex-Hull-Based Center Prior[J]. IEEE Signal Processing Letters, 2013, 20(7):637-640. doi: 10.1109/LSP.2013.2260737
[14]	CHEN L C, ZHU Y, PAPANDREOU G, et al. Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation[C]// Proceedings of the European Conference on Computer Vision.Heidelberg:Springer, 2018:801-818.
[15]	QIN X, ZHANG Z, HUANG C, et al. BASNeT:Boundary-Aware Salient Object Detection[C]// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2019:7479-7489.
[16]	PANG Y, ZHAO X, ZHANG L, et al. Multi-Scale Interactive Network for Salient Object Detection[C]// Proceedings of the 2020 IEEE /CVF Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2020:9413-9422.
[17]	时斐斐, 张松龙, 彭力. 结合边缘特征先验引导的深度卷积显著性检测[J]. 计算机工程与应用, 2020, 56(14):199-206.
	SHI Feifei, ZHANG Songlong, PENG Li, et al. Deep Convolution Saliency Detection Combined with Edge Feature Prior-Based Inspection[J]. Computer Engineering and Applications, 2020, 56(14):199-206.
[18]	RONNEBERGER O, FISCHER P, BROX T. U-Net:Convolutional Networks for Biomedical Image Segmentation[C]// International Conference on Medical Image Computing and Computer-Assisted Intervention.Heidelberg:Springer, 2015:234-241.
[19]	DE BOER P T, KROESE D P, MANNOR S, et al. A Tutorial on The Cross-Entropy Method[J]. Annals of Operations Research, 2005, 134(1):19-67. doi: 10.1007/s10479-005-5724-z
[20]	WANG Y, GUO J, GAO H, et al. UIE$\hat{C}$2-Net:CNN-Based Underwater Image Enhancement Using Two Color Space[J]. Signal Processing:Image Communication, 2021, 96:116250. doi: 10.1016/j.image.2021.116250
[21]	HOU Q, CHENG M M, HU X, et al. Deeply Supervised Salient Object Detection with Short Connections[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2017:3203-3212.
[22]	LIU N, HAN J, YANG M H. PicaNet:Learning Pixel-Wise Contextual Attention for Saliency Detection[C]// Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2018:3089-3098.
[23]	LIU J J, HOU Q, CHENG M M, et al. A Simple Pooling-Based Design for Real-Time Salient Object Detection[C]// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2019:3917-3926.
[24]	CHEN Z, XU Q, CONG R, et al. Global Context-Aware Progressive Aggregation Network for Salient Object Detection[C]// Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto: AAAI, 2020:10599-10606.

评价指标	U-Net	DSS	PicaNet	PoolNet	GCPANet	AMNet
MAE	0.095	0.088	0.097	0.052	0.049	0.044
F-measure	0.793	0.839	0.811	0.893	0.901	0.908

评价指标	Baseline	Baseline +CASA	Baseline +CFEM	AMNet
MAE	0.095	0.082	0.075	0.044
F-measure	0.793	0.848	0.849	0.908