一种用于自动驾驶场景的轻量级语义分割网络

doi:10.19665/j.issn1001-2400.2023.01.014

Abstract

Abstract:

In the autonomous driving scenario,aiming at the problem of limited memory and insufficient computing power when the semantic segmentation model is deployed in vehicle hardware devices,it is necessary to design a semantic segmentation model that can balance efficiency and accuracy.In this paper,a single-branch network structure is used to design a lightweight multi-scale bidirectional attention network.To achieve efficient feature extraction,a lightweight convolutional unit is designed to form the feature extraction backbone of the network.In order to better locate and segment objects with large scale differences in road scenes,a multi-scale bidirectional attention module is proposed which has a global multi-scale receptive field and encodes channel attention in one direction while preserving spatial location information in the other.Based on this attention module,a skip attention connection module and a feature attention fusion module are designed,so that the output features have both detailed information and semantic information.On the Cityscapes dataset,the model in this paper achieves an MIoU(Mean Intersection over Union) of 71.86% with a parameter size of 0.9M,and achieves an inference speed of 88FPS(Frames Per Second) under a single RTX2080Ti GPU.The test results on public datasets show that the model achieves a high segmentation accuracy and is suitable for deployment and application under in-vehicle hardware,which is of certain practical value.

Key words: automatic driving, semantic segmentation, lightweight network, attention mechanism, deep learning

CLC Number:

TP391.4

LIU Bochong, CAI Huaiyu, YANG Shiyuan, LI Haotian, WANG Yi, CHEN Xiaodong. Lightweight semantic segmentation network for autonomous driving scenarios[J].Journal of Xidian University, 2023, 50(1): 118-128.

Figures/Tables 11

References 31

[1]	汪梓艺, 苏育挺, 刘艳艳, 等. 一种改进DeeplabV3网络的烟雾分割算法[J]. 西安电子科技大学学报, 2019, 46(6):52-59.
	WANG Ziyi, SU Yuting, LIU Yanyan, et al. An Improved DeeplabV3 Network Smoke Segmentation Algorithm[J]. Journal of Xidian University, 2019, 46(6):52-59.
[2]	回海生, 张雪英, 吴泽林, 等. 一种主辅路径注意力补偿的脑卒中病灶分割方法[J]. 西安电子科技大学学报, 2021, 48(4):200-208.
	HUI Haisheng, ZHANG Xueying, WU Zelin, et al. Lightweight Image Super-Resolution with the Adaptive Weight Learning Network[J]. Journal of Xidian University, 2021, 48(4):200-208.
[3]	SHELHAMER E, LONG J, DARRELL T. Fully Convolutional Networks for Semantic Segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39:640-651. doi: 10.1109/TPAMI.2016.2572683 pmid: 27244717
[4]	LI Y, SHI T, ZHANG Y, et al. Learning Deep Semantic Segmentation Network under Multiple Weakly-Supervised Constraints for Cross-Domain Remote Sensing Image Semantic Segmentation[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 175:20-33. doi: 10.1016/j.isprsjprs.2021.02.009
[5]	YUAN X, SHI J, GU L. A Review of Deep Learning Methods for Semantic Segmentation of Remote Sensing Imagery[J]. Expert Systems with Applications, 2021, 169:114417. doi: 10.1016/j.eswa.2020.114417
[6]	WANG Y, ZHOU Q, LIU J, et al. LEDNet:A Lightweight Encoder-Decoder Network for Real-Time Semantic Segmentation[C]// Proceedings of the 2019 IEEE International Conference on Image Processing(ICIP).Piscataway:IEEE, 2019:1860-1864.
[7]	WANG Y, ZHOU Q, XIONG J, et al. ESNet:An Efficient Symmetric Network for Real-Time Semantic Segmentation[C]// Proceedings of the Chinese Conference on Pattern Recognition and Computer Vision(PRCV).Cham:Springer, 2019:41-52.
[8]	ROMERA E, ALVAREZ J M, BERGASA L M, et al. ERFNet:Efficient Residual Factorized Convnet for Real-Time Semantic Segmentation[J]. IEEE Transactions on Intelligent Transportation Systems, 2017, 19(1):263-272. doi: 10.1109/TITS.2017.2750080
[9]	WANG K, YANG J, YUAN S, et al. A Lightweight Network with Attention Decoder for Real-Time Semantic Segmentation[J]. The Visual Computer, 2021, 38(7):2329-2339. doi: 10.1007/s00371-021-02115-4
[10]	ZHUANG M, ZHONG X, GU D, et al. LRDNet:A Lightweight and Efficient Network with Refined Dual Attention Decoder for Real-Time Semantic Segmentation[J]. Neurocomputing, 2021, 459:349-360. doi: 10.1016/j.neucom.2021.07.019
[11]	YU C, WANG J, PENG C, et al. BiSeNet:Bilateral Segmentation Network for Real-Time Semantic Segmentation[C]// Proceedings of the European Conference on Computer Vision(ECCV).Cham:Springer, 2018:334-349.
[12]	WU T, TANG S, ZHANG R, et al. CGNet:A Light-weight Context Guided Network for Semantic Segmentation[J]. IEEE Transactions on Image Processing, 2021, 30:1169-1179. doi: 10.1109/TIP.83
[13]	GAO G, XU G, YU Y, et al. MSCFNet:A Lightweight Network with Multi-Scale Context Fusion for Real-Time Semantic Segmentation[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(12):25489-25499. doi: 10.1109/TITS.2021.3098355
[14]	YANG Q, CHEN T, FAN J, et al. EADNet:Efficient Asymmetric Dilated Network for Semantic Segmentation[C]// Proceedings of the ICASSP 2021-2021 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP).Piscataway:IEEE, 2021:2315-2319.
[15]	HE K, ZHANG X, REN S, et al. Identity Mappings in Deep Residual Networks[C]// Proceedings of the European Conference on Computer Vision.Cham:Springer, 2016:630-645.
[16]	CHOLLET F. Xception:Deep Learning with Depthwise Separable Convolutions[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2017:1800-1807.
[17]	PASZKE A, CHAURASIA A, KIM S, et al. ENet:A Deep Neural Network Architecture for Real-Time Semantic Segmentation[J/OL]. [2016-06-07] https://doi.org/10.48550/arXiv.1606.02147.
[18]	ZHANG X, ZHOU X, LIN M, et al. ShuffleNet:An Extremely Efficient Convolutional Neural Network for Mobile Devices[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2018:6848-6856.
[19]	成磊, 王玥, 田春娜. 一种添加残差注意力机制的视觉目标跟踪算法[J]. 西安电子科技大学学报, 2020, 47(6):148-157.
	CHENG Lei, WANG Yue, TIAN Chunna. A Visual Object Tracking Algorithm with Residual Attention Mechanism[J]. Journal of Xidian University, 2020, 47(6):148-157.
[20]	宋建锋, 苗启广, 王崇晓, 等. 注意力机制的多尺度单目标跟踪算法[J]. 西安电子科技大学学报, 2021, 48(5):110-116.
	SONG Jianfeng, MIAO Qiguang, WANG Chongxiao, et al. Multi-Scale Single Target Tracking Algorithm Based on Attention Mechanism[J]. Journal of Xidian University, 2021, 48(5):110-116.
[21]	CAO Y, XU J, LIN S, et al. GCNet:Non-Local Networks Meet Squeeze-Excitation Networks and Beyond[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops.Piscataway:IEEE, 2019:1971-1980.
[22]	常新旭, 张杨, 杨林, 等. 融合多头自注意力机制的语音增强方法[J]. 西安电子科技大学学报, 2020, 47(1):104-110
	CHANG Xinxu, ZHANG Yang, YANG Lin, et al. A Speech Enhancement Method Integrating Multi-Head Self-Attention Mechanism[J]. Journal of Xidian University, 2020, 47(1):104-110.
[23]	WOO S, PARK J, LEE J Y, et al. CBAM:Convolutional Block Attention Module[C]// Proceedings of the European Conference on Computer Vision(ECCV).Cham:Springer, 2018:3-19.
[24]	HU J, SHEN L, SUN G. Squeeze-and-Excitation Networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8):2011-2023. doi: 10.1109/TPAMI.2019.2913372 pmid: 31034408
[25]	TIAN Z, HE T, SHEN C, et al. Decoders Matter for Semantic Segmentation:Data-Dependent Decoding Enables Flexible Feature Aggregation[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2019:3121-3130.
[26]	CORDTS M, OMRAN M, RAMOS S, et al. The Cityscapes Dataset for Semantic Urban Scene Understanding[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2016:3213-3223.
[27]	KINGMA D P, BA J. Adam:A Method for Stochastic Optimization[J/OL].[2017-01-30]. https://doi.org/10.48550/arXiv.1412.6980.
[28]	REZATOFIGHI H, TSOI N, GWAK J, et al. Generalized Intersection over Union:A Metric and a Loss for Bounding Box Regression[C] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2019:658-666.
[29]	LI H, XIONG P, FAN H, et al. DFANet:Deep Feature Aggregation for Real-Time Semantic Segmentation[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2019:9514-9523.
[30]	ZHAO H, QI X, SHEN X, et al. ICNet for Real-Time Semantic Segmentation on High-Resolution Images[C]// Proceedings of the European Conference on Computer Vision(ECCV).Cham:Springer, 2018:418-434.
[31]	HE K, ZHANG X, REN S, et al. Deep Residual Learning for Image Recognition[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2016:770-778.

阶段1	阶段2	阶段3	平均交并比/%	参数量/M	浮点运算数每秒/G
1	4	8	68.92	0.895	10.25
2	4	8	71.86	0.900	10.87
3	4	8	71.15	0.902	11.48
2	3	8	70.76	0.880	10.29
2	5	8	70.37	0.915	11.44
2	4	7	71.01	0.830	10.31
2	4	9	71.41	0.965	11.42

SACM	FAFM	平均交并比/%	参数量/M	浮点运算数每秒/G
—	—	69.94	0.832	10.05
—	√	70.80_+0.86	0.864	10.27
√	—	70.77_+0.83	0.871	10.77
√	√	71.86_+1.92	0.900	10.87

r	平均交并比/%	参数量/M	浮点运算数每秒/G
4	71.01	0.95	10.88
8	71.86	0.90	10.87
16	69.57	0.87	10.86

SACM	FAFM	平均交并比/%	参数量/M	浮点运算数每秒/G
—	—	70.37	0.848	10.85
—	SE	70.81_+0.42	0.851	10.85
SE	—	70.76_+0.37	0.851	10.85
SE	SE	71.20_+0.81	0.853	10.85
—	MBA	71.08_+0.69	0.874	10.86
MBA	—	71.13_+0.74	0.874	10.86
MBA	MBA	71.86_+1.47	0.900	10.87

算法	Backbone	平均交并比/%	参数量/M	浮点运算数每秒/G	帧数每秒
CGNet^[12]	—	65.99	0.50	6.98	83
LEDNet^[6]	—	69.02	0.92	11.46	71
ESNet^[7]	—	69.10	1.66	23.24	67
ERFNet^[8]	—	68.13	2.06	25.79	55
BiSeNet^[11]	Xception39^[15]	69.00	5.80	2.90	71
DFANet^[29]	Xception A	70.30	7.80	2.10	77
ICNet^[30]	Resnet50^[31]	69.50	26.13	9.63	37
LMBANet	—	71.86	0.90	10.87	88

Lightweight semantic segmentation network for autonomous driving scenarios

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 11

References 31

Related Articles 15

Metrics

Comments

Recommended 10

[1]	ZHANG Yali,LI Wenyuan,LI Changlu,DING Shaobo. Method for enhancement of the multi-scale low-light image by combining an attention guidance [J]. Journal of Xidian University, 2023, 50(1): 129-136.
[2]	CUI Shaoguo,CHEN Siqi,DU Xing. Dual graph attention networks model for target sentiment analysis [J]. Journal of Xidian University, 2023, 50(1): 137-148.
[3]	ZHANG Qiang, YANG Xinpeng, ZHAO Shixiang, WEI Dongdong, HAN Zhen. Vehicle-target detection network for SAR images based on the attention mechanism [J]. Journal of Xidian University, 2023, 50(1): 36-47.
[4]	LIU Xiaowen, GUO Jichang, ZHENG Sida. Weakly-supervised salient object detection with the multi-scale progressive network [J]. Journal of Xidian University, 2023, 50(1): 48-57.
[5]	ZHANG Zehuan, LIU Qiang, GUO Difei. High efficient framework for large-scale zero-shot image recognition [J]. Journal of Xidian University, 2022, 49(6): 103-110.
[6]	LI Jiaojiao, LIU Zhiqiang, SONG Rui, LI Yunsong. Algorithm for segmentation of remote sensing imagery using the improved Unet [J]. Journal of Xidian University, 2022, 49(6): 67-75.
[7]	ZHANG Zhaoyu,TIAN Chunna,ZHOU Heng,TIAN Xilan. Online classification jointed RGBT tracking based on the dual attention Siamese network [J]. Journal of Xidian University, 2022, 49(6): 76-85.
[8]	LIU Shigang,ZHANG Tong,YANG Jiangong,GE Bao. Progressive dialtion residual network for deep binocular stereo matching [J]. Journal of Xidian University, 2022, 49(5): 175-180.
[9]	WANG Kan, WANG Mengyang, LIU Xin, TIAN Guoqiang, LI Chuan, LIU Wei. Event detection by combining self-attention and CNN-BiGRU [J]. Journal of Xidian University, 2022, 49(5): 181-188.
[10]	QI Peihan,LI Bing,XIE Aiping,GAO Xianglan. Deep learning reconstruction algorithm for incomplete samples of frequency hopping communication signals [J]. Journal of Xidian University, 2022, 49(4): 1-7.
[11]	MA Lun,LIU Xin,ZHAO Bin,WANG Ruiping,LIAO Guisheng,ZHANG Yajing. Impaired behavior recognition by using the multi-head-siamese neural network [J]. Journal of Xidian University, 2022, 49(4): 100-108.
[12]	JING Peiguang,LI Yaxin,SU Yuting. Micro-video multi-label classification method based on multi-modal feature encoding [J]. Journal of Xidian University, 2022, 49(4): 109-117.
[13]	LIU Di,GUO Jichang,WANG Yudong,ZHANG Yi. Multi-scale salient object detection network combining an attention mechanism [J]. Journal of Xidian University, 2022, 49(4): 118-126.
[14]	GAO Deyong,KANG Zibing,WANG Song,WANG Yangping. Method to recognize human action by using the convolutional block attention mechanism [J]. Journal of Xidian University, 2022, 49(4): 144-155.
[15]	SHI Jiarong,LI Jinhong. Novel deep matrix factorization and its application in the recommendation system [J]. Journal of Xidian University, 2022, 49(3): 171-182.