一种基于CA-Net的面部口罩分割方法

doi:10.16180/j.cnki.issn1007-7820.2023.06.010

摘要/Abstract

摘要：

佩戴不同材质、型号的口罩对病毒传播的防控效果各不相同,对图像数据集中的口罩进行分割,将有助分析不同类型的口罩在防控效果上的差异。当前,面部口罩检测算法较多,但缺乏进一步的分割,为此文中提出一种基于图像处理和深度学习的面部口罩分割方法。文中所提方法是一种改进的对比度自适应直方图均衡化预处理方法,其通过亮度自适应调整减少因部分图像过暗导致的传统预处理效果不佳的影响。以SSD(Single Shot MultiBox Detector)进行口罩预检测,对结果以CA-Net(Comprehensive Attention Convolutional Neural Networks)进行口罩分割。CA-Net以U-Net为骨干网络,增加了空间注意模块、通道注意模块和尺度注意模块以便同时实现关于特征地图的空间、通道和比例的综合注意力引导,突出空间位置、通道和尺度。使用该方法初步分割结果的Dice系数评价指标可以达到79.18%±3.44%;增加预处理和后处理操作后,Dice系数可提升至84.03%±2.81%,表明文中所提方法能够明显改善视觉分割结果。

关键词: 目标检测, HSV空间转换, 亮度自适应调整, 图像分割, 深度学习, CA-Net, 闭运算, SSD

Abstract:

Wearing medical masks of various materials and types will lead to different prevention and control the effect of virus spread. Segmentation to masks in image data sets will benefit analyzing the differences in prevention and control effects of different types of masks. Currently, there are many facial mask detection algorithms, but there is a lack of further segmentation. Therefore, this study proposes a facial mask segmentation method based on image processing and deep learning. An improved contrast-limited adaptive histogram equalization preprocessing method is proposed to reduce the poor effect of traditional preprocessing caused by dark part of the image through brightness adaptive adjustment. SSD network is used for mask pre-detection, and CA- Net is used for mask segmentation. With U-Net as the backbone network, CA-Net adds spatial attention module, channel attention module and scale attention module to realize the comprehensive attention guidance of space, channel and scale of feature map at the same time. The segmentation result is processed by binary image closing operation to get the final segmentation results. The Dice evaluation of preliminarily segmentation index can reach to 79.18%±3.44%, and the Dice coefficient is increased to 84.03%±2.81% by adding preprocessing and post-processing operations, indicating that the proposed method could significantly improve the visual segmentation results.

Key words: target detection, HSV space conversion, brightness adaptive adjustment, image segmentation, deep learning, CA-Net, closed operation, SSD

中图分类号:

TP391

李珂然,陈胜,柯盼盼. 一种基于CA-Net的面部口罩分割方法[J]. 电子科技, 2023, 36(6): 64-71.

LI Keran,CHEN Sheng,KE Panpan. A Method of Facial Mask Segmentation Based on CA-Net[J]. Electronic Science and Technology, 2023, 36(6): 64-71.

图/表 11

图1

图2

图3

图4

表1

图5

表2

表3

表4

图6

表5

参考文献 28

[1]	党永强. 基于深度学习的口罩疵点检测与分类算法研究[D]. 西安: 西安工程大学, 2018:42-64.
	Dang Yongqiang. The research on defect detection and classification algorithm of masks based on deep learning[D]. Xi'an: Xi'an Polytechnic University, 2018:42-64.
[2]	刘贺鹏, 吕建平. 基于形态学分水岭的Normalized Cut图像分割方法[J]. 电子科技, 2016, 29(9):12-14.
	Liu Hepeng, Lü Jianping. A Normalized Cut image segmentation method based on morphological watersheds[J]. Electronic Science and Technology, 2016, 29(9):12-14.
[3]	邬春学, 刘训洋. 改进粒子群结合K-均值聚类的图像分割算法[J]. 电子科技, 2016, 29(8):93-95.
	Wu Chunxue, Liu Xunyang. A modified PSO combined K-means clustering algorithm for image segmentation[J]. Electronic Science and Technology, 2016, 29(8):93-95.
[4]	Zhou Z W, Siddiquee M M R, Tajbakhsh N, et al. U-Net++: Redesigning skip connections to exploit multiscale features in image segmentation[J]. IEEE Transactions on Medical Imaging, 2020, 39(6):1856-1867. doi: 10.1109/TMI.42
[5]	贝琛圆, 于海滨, 潘勉, 等. 基于改进U-Net网络的腺体细胞图像分割算法[J]. 电子科技, 2019, 32(11):18-22.
	Bei Chenyuan, Yu Haibin, Pan Mian, et al. Gland cell image segmentation algorithm based on improved U-Net network[J]. Electronic Science and Technology, 2019, 32(11):18-22.
[6]	司明明, 陈玮, 胡春燕, 等. 融合Resnet50和U-Net的眼底彩色血管图像分割[J]. 电子科技, 2021, 34(8):19-24.
	Si Mingming, Chen Wei, Hu Chunyan, et al. Fundus blood vessel image segmentation combining Resnet50 and U-Net[J]. Electronic Science and Technology, 2021, 34(8):19-24.
[7]	Gu R, Wang G T, Tao S, et al. CA-Net: Comprehensive attention convolutional neural networks for explainable medical image segmentation[J]. IEEE Transactions on Medical Imaging, 2021, 40(2):699-711. doi: 10.1109/TMI.2020.3035253 pmid: 33136540
[8]	王利娟. 低照度彩色图像增强方法研究[D]. 银川: 北方民族大学, 2021:35-68.
	Wang Lijuan. Research on enhancement methods of low-light color image[D]. Yinchuan: North Minzu University, 2021:35-68.
[9]	王春江, 张猛, 张建飞. 基于网络摄像头的活体人脸识别系统设计[J]. 电子科技, 2020, 33(6):63-68.
	Wang Chunjiang, Zhang Meng, Zhang Jianfei. Design of living face recognition system based on web camera[J]. Electronic Science and Technology, 2020, 33(6):63-68.
[10]	胡少聪. 基于深度学习的人脸识别方法研究[J]. 电子科技, 2019, 32(6):82-86.
	Hu Shaocong. Research on face recognition based on deep learning[J]. Electronic Science and Technology, 2019, 32(6):82-86.
[11]	张莹, 刘子龙, 万伟. 基于Faster R-CNN的无人机车辆目标检测[J]. 电子科技, 2021, 34(11):11-20.
	Zhang Ying, Liu Zilong, Wan Wei. UAV vehicle target detection based on Faster R-CNN[J]. Electronic Science and Technology, 2021, 34(11):11-20.
[12]	张震, 李孟洲, 李浩方, 等. 改进的SSD算法及其在目标检测中的应用[J]. 计算机应用与软件, 2021, 38(9):226-231.
	Zhang Zhen, Li Mengzhou, Li Haofang, et al. Improved SSD algorithm and its application in target detection[J]. Computer Applications and Software, 2021, 38(9):226-231.
[13]	Zhang H, Hong X G, Zhu L. Detecting small objects in thermal images using single-shot detector[J]. Automatic Control and Computer Sciences, 2021, 55(2):202-211. doi: 10.3103/S0146411621020097
[14]	Sun P, Zhao Y, Zhu S. An approach to improve SSD through mask prediction of multi-scale feature maps[J]. Pattern Analysis and Applications, 2021, 24(3):1357-1366. doi: 10.1007/s10044-021-00993-x
[15]	吉鑫, 陈剑云, 完颜幸幸. 用于接触网可视化接地识别的改进VGG-16模型[J]. 传感器与微系统, 2021, 40(2):58-60.
	Ji Xin, Chen Jianyun, Wanyan Xingxing, Improved VGG-16 model for visual OCS earthing recognition[J]. Transducer and Microsystem Technologies, 2021, 40(2):58-60.
[16]	Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation[C]. Munich: Proceedings of the Medical Image Computing and Computer-Assisted Intervention, 2015:234-241.
[17]	Wang X L, Girshick R, Gupta A, et al. Non-local neural net-works[C]. Salt Lake City: Proceedings of the Conference on Computer Vision and Pattern Recognition, 2018:7794-7803.
[18]	Oktay O, Schlemper J, Folgoc L L, et al. Attention U-Net: Learning where to look for the pancreas[C]. Amsterdam: Proceedings of the Medical Imaging with Deep Learning, 2018:1-12.
[19]	Hu J, Shen L, Sun G, et al. Squeeze-and-excitation networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 42(8):2011-2023. doi: 10.1109/TPAMI.34
[20]	Woo S, Park J, Lee J Y, et al. CBAM: Convolutional block attention module[C]. Munich: Proceedings of the European Conference on Computer Vision, 2018:3-19.
[21]	苏烨, 李筠, 杨海马, 等. 基于Faster R-CNN的钢轨表面缺陷识别研究[J]. 电子科技, 2020, 33(9):63-68.
	Su Ye, Li Jun, Yang Haima, et al. Research on rail surface defect recognition based on Faster R-CNN[J]. Electronic Science and Technology, 2020, 33(9):63-68.
[22]	Zabihollahy F, Ukwatta E, White J A. Fully automated segmentation of left ventricular myocardium from 3D late gadolinium enhancement magnetic resonance images using a U-Net convolutional neural network-based model[C]. San Diego: Proceedings of the International Society for Optical Engineering Medical Imaging, 2019:832-839.
[23]	Zabihollahy F, White J A, Ukwatta E. Myocardial scar segmentation from magnetic resonance images using convolutional neural network[C]. Houston: Proceedings of the Computer-Aided Diagnosis, 2018:1-10.
[24]	Sara M, Riccardo B, Chiara M, et al. Development and testing of a deep learning based strategy for scar segmentation on CMR LGE images[J]. Magnetic Resonance Materials in Physics,Biology and Medicine, 2019, 32(2):187-195. doi: 10.1007/s10334-018-0718-4
[25]	张灿, 陈玮, 尹钟. 基于弱监督宫颈细胞图像的语义分割方法[J]. 电子科技, 2021, 34(12):68-74.
	Zhang Can, Chen Wei, Yin Zhong. Semantic segmentation of cervical cell image based on weak supervision[J]. Electronic Science and Technology, 2021, 34(12):68-74.
[26]	伍济钢, 成远, 邵俊, 等. 面向智能手机玻璃盖板缺陷检测的YOLOv3改进和应用[J]. 液晶与显示, 2021, 36(12):1728-1736. doi: 10.37188/CJLCD.2021-0172
	Wu Jigang, Cheng Yuan, Shao Jun, et al. Improvement and application of YOLOv3 for defect detection of smart phone glass covers[J]. Chinese Journal of Liquid Crystals and Displays, 2021, 36(12):1728-1736. doi: 10.37188/CJLCD.2021-0172
[27]	梁铭峰, 李蠡. 基于对抗神经网络的增强YOLOv3模糊目标检测[J]. 计算机应用与软件, 2021, 38(10):221-228.
	Liang Mingfeng, Li Li. Enhanced YOLOv3 blur target detection based on generative adversarial network[J]. Computer Applications and Software, 2021, 38(10):221-228.
[28]	夏禹, 肖金球, 翁玉尚. 基于改进Faster-RCNN的偏光片表面缺陷检测[J]. 光学技术, 2021, 47(6):695-702.
	Xia Yu, Xiao Jinqiu, Weng Yushang. Surface defect detection of polarizer based on improved Faeter-RCNN[J]. Optical Technique, 2021, 47(6):695-702.

检测网络	灵敏度结果/%
SSD	97.84
YOLOv3	95.75
Faster R-CNN	96.63

预处理方法	Dice系数/%
无	79.18±3.44
CLAHE	81.43±2.36
改进的CLAHE	83.86±2.74

训练所用模型	Dice系数/%	分割所用时间/ms
U-Net	76.37±2.53	2.4
U-Net++	79.43±2.56	2.9
CA-Net	83.86±2.74	2.1

Epochs	Batch Size
Epochs	2	4	8	16
100	67.32±2.41	65.24±2.93	62.24±3.35	59.89±3.12
150	73.18±3.06	72.57±3.12	71.34±3.10	71.53±3.27
200	78.23±2.71	78.12±2.61	75.67±2.66	72.45±2.36
250	82.35±2.48	82.23±2.32	81.49±2.88	80.92±3.27
300	83.83±1.87	83.86±2.74	83.51±2.66	81.23±3.07
350	83.71±2.03	83.79±2.85	82.68±2.44	81.36±3.12

形态学后处理方法	Dice系数/%
不做后处理	83.86±2.74
闭运算	84.03±2.81
开运算	83.92±2.54