采用元学习的多场景教室学生姿态检测方法

doi:10.19665/j.issn1001-2400.2021.05.008

西安电子科技大学学报 ›› 2021, Vol. 48 ›› Issue (5): 58-67.doi: 10.19665/j.issn1001-2400.2021.05.008

采用元学习的多场景教室学生姿态检测方法

钱志华^1,²(),高陈强^1,²(),叶盛^1,²()

1.重庆邮电大学通信与信息工程学院,重庆 400065
2.信号与信息处理重庆市重点实验室,重庆 400065

收稿日期:2021-03-18 出版日期:2021-10-20 发布日期:2021-11-09
通讯作者: 高陈强
作者简介:钱志华(1997—),男,重庆邮电大学硕士研究生,E-mail: s190101008@stu.cqupt.edu.cn|叶盛(1995—),女,重庆邮电大学硕士研究生,E-mail: s180101011@stu.cqupt.edu.cn
基金资助:
国家自然科学基金(61571071);国家自然科学基金(61906025);重庆市科委自然科学基金(cstc2018jcyjAX0227);重庆市科委自然科学基金(cstc2020jcyj-msxmX0835);重庆市教委科学技术研究项目(KJQN201900607);重庆市教委科学技术研究项目(KJQN202000647)

Method for detection of a student’s pose in a multi-scene classroom based on meta-learning

QIAN Zhihua^1,²(),GAO Chenqiang^1,²(),YE Sheng^1,²()

1. School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications,Chongqing 400065,China
2. Chongqing Key Laboratory of Signal and Information Processing,Chongqing 400065,China

Received:2021-03-18 Online:2021-10-20 Published:2021-11-09
Contact: Chenqiang GAO

摘要/Abstract

摘要：

针对不同教学场景图像的数据分布差异较大造成的跨域偏移问题,提出了一种采用元学习的多场景学生姿态检测方法。该方法设计了姿态检测元模型和一种参数可学习的域适应优化器。通过离线学习和在线学习相结合的方式,实现特定教学场景姿态检测模型的快速域适应。离线学习阶段,该方法通过双层训练模拟姿态检测模型在各类教学场景上域适应的过程,对姿态检测元模型以及域适应优化器参数进行训练;在线学习阶段,只需特定教学场景下少量有标签的样本数据,元模型就能在域适应优化器的引导下快速适应该场景的数据分布。另外,在离线学习阶段,还提出了一种能使双层训练更加稳定的外部优化器。实验结果表明,该方法在多场景姿态检测数据集上的综合检测精度要优于目前主流的目标检测模型,对有标签图像较少的新场景也有较好的域适应效果。

关键词: 姿态检测, 域适应, 小样本学习, 元模型, 域适应优化器

Abstract:

To solve the problem of domain shift in different classroom scenes,this paper proposes a multi-scene classroom pose detection method based on meta-learning.In this method,a pose detection meta-model and a domain adaptive optimizer with learnable parameters are designed.Besides,the offline learning mode and online learning mode are combined to realize the fast domain adaptation of the detection model in a specific classroom scene.In the offline learning stage,the method trains the parameters of the pose detection meta-model and the adaptive domain optimizer through two-layer training.In the online learning stage,guided by the adaptive domain optimizer,the meta-model can quickly adapt to the data distribution of the scene with a few labeled images.In addition,this paper also proposes an external training optimizer which can make the double-layer training more stable.Experiments show that the detection accuracy of this method in multi-scene classroom pose detection dataset is better than that of the current popular object detection models,and that it also has a good domain adaptation effect for new scenes with a few labeled images.

Key words: pose detection, domain adaptation, few shot learning, meta-model, adaptive domain optimizer

中图分类号:

TP391

钱志华,高陈强,叶盛. 采用元学习的多场景教室学生姿态检测方法[J]. 西安电子科技大学学报, 2021, 48(5): 58-67.

QIAN Zhihua,GAO Chenqiang,YE Sheng. Method for detection of a student’s pose in a multi-scene classroom based on meta-learning[J]. Journal of Xidian University, 2021, 48(5): 58-67.

图/表 10

图1

图2

图3

图4

图5

表1

表2

图6

表3

表4

参考文献 21

[1]	TANG L, GAO C Q, CHEN X, et al. Pose Detection in Complex Classroom Environment Based on Improved Faster R-CNN[J]. IET Image Processing, 2019, 13(3):451-457. doi: 10.1049/ipr2.v13.3
[2]	CHEN Y H, LI W, CHRISTOS S, et al. Domain Adaptive Faster R-CNN for Object Detection in the Wild[C]// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2018:3339-3348.
[3]	SAITO K, USHIKU Y, HARADA T, et al. Strong-Weak Distribution Alignment for Adaptive Object Detection[C]// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2019:6956-6965.
[4]	ZHU X G, PANG J M, YANG C Y, et al. Adapting Object Detectors via Selective Cross-Domain Alignment[C]// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2019:687-696.
[5]	XU C D, ZHAO X R, JIN X, et al. Exploring Categorical Regularization for Domain Adaptive Object Detection[C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2020:11724-11733.
[6]	WANG X D, CAI Z W, GAO D S, et al. Towards Universal Object Detection by Domain Attention[C]// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2019:7289-7298.
[7]	WANG T, ZHENG X P, YUAN L, et al. Few-Shot Adaptive Faster R-CNN[C]// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition,Piscataway:IEEE, 2019:7173-7182.
[8]	HE K M, GEORGIA G, PIOTER D, et al. Mask R-CNN[C]// Proceedings of the IEEE International Conference on Computer Vision.Piscataway:IEEE, 2017:2980-2988.
[9]	REN S Q, HE K M, GIRSHICK R B, et al. Faster RCNN:Towards Real-Time Object Detection with Region Proposal Networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6):1137-1149. doi: 10.1109/TPAMI.2016.2577031
[10]	HE K M, ZHANG X Y, REN S Q, et al. Deep Residual Learning for Image Recognition[C]// Proceedings of the 2016 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2016:770-778.
[11]	LIN T, DOLLAR P, GIRSHICK R B, et al. Feature Pyramid Networks for Object Detection[C]// Proceedings of the 2017 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2017:936-944.
[12]	FINN C, ABBEEL P, LEVINE S. Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks[C]// Proceedings of the 34th International Conference on Machine Learning,New York:ACM, 2017:1126-1135.
[13]	CHELSEA F. Learning to Learn with Gradients[D]. Berkeley:University of California, 2018.
[14]	LI Z G, ZHOU F W, CHEN F, et al. Meta-SGD:Learning to Learn Quickly for Few Shot Learning(2017)[EB/OL]. [2017-9-28]. https://arxiv.org/pdf/1707.09835.pdf.
[15]	HANSON S J, PRATT L Y. Comparing Biases for Minimal Network Construction with Back-Propagation[C]//Proceedings of the 1st International Conference on Neural Information Processing Systems, Cambridge: MIT Press, 1988:177-185.
[16]	KINGMA D P, BA J. Adam:A Method for Stochastic Optimization(2017)[EB/OL]. [2017-1-30] https://arxiv.org/pdf/1412.6980.pdf.
[17]	LOSHCHILOV I, HUTTER F. Decoupled Weight Decay Regularization(2017)[J/OL]. [2017-11-14] https://arxiv.org/pdf/1711.05101.pdf.
[18]	许悦雷, 朱明明, 马时平, 等. 迁移学习结合难分样本挖掘的机场目标检测[J]. 西安电子科技大学学报, 2018, 45(5):190-196.
	XU Yuelei, ZHU Mingming, MA Shiping, et al. Airport Object Detection Combining Transfer Learning and Hard Example Mining[J]. Journal of Xidian University, 2018, 45(5):190-196.
[19]	NIE J, ANWER R M, CHOLAKKAL H, et al. Enriched Feature Guided Refinement Network for Object Detection[C]// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision.Piscataway:IEEE, 2019:9537-9546.
[20]	BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4:Optimal Speed and Accuracy of Object Detection(2020)[EB/OL]. [2020-4-23]. https://arxiv.org/pdf/2004.10934v1.pdf.
[21]	LIN T Y, GOYAL P, GIRSHICK R, et al. Focal Loss for Dense Object Detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(2):318-327. doi: 10.1109/TPAMI.34

模型	特征提取网络	坐着	站着	趴着	mAP
CCE-FRCNN^[1]	VGG16	79.27	81.32	66.03	75.54
EFGRNet^[19]	VGG16	87.41	86.52	79.78	84.57
YOLOv4^[20]	CSP-Darknet	87.25	88.80	77.78	84.61
RetinaNet^[21]	ResNet-50-FPN	93.73	91.52	76.38	87.21
Mask R-CNN^[8]	ResNet-50-FPN	93.28	91.40	75.91	86.86
TUOD^[6]	ResNet-50-FPN	93.57	92.10	76.42	87.36
文中方法	ResNet-50-FPN	95.24	96.67	80.82	90.91

方法	域适应	模型	坐着	站着	趴着	mAP
Mask R-CNN	×	ψ	93.28	91.40	75.91	86.86
	√	ψ'	93.26	91.93	76.20	87.13
文中方法	×	元模型	92.10	92.60	67.42	84.04
	√	域适应模型	95.24	96.67	80.82	90.91

方法	坐着	站着	趴着	mAP
Meta-SGD	93.71	94.54	75.83	88.02
MAML	92.11	92.27	74.96	86.45
MAML+ETO	92.53	95.35	75.49	87.79
MAML+DAO	96.55	95.31	77.09	89.64
笔者方法	95.24	96.67	80.82	90.91

方法	坐着	站着	趴着	mAP
Mask R-CNN	53.98	28.45	19.94	34.12
DA^[2]	57.61	30.77	22.88	37.09
SW-DA^[3]	58.89	31.43	24.11	38.14
SC-DA^[4]	59.14	30.96	23.32	37.81
SW-DA-ICR-CCR^[5]	59.86	31.94	24.73	38.84
FAFRCNN^[7]	68.23	49.44	42.29	53.32
文中方法	72.32	55.43	48.16	58.64

采用元学习的多场景教室学生姿态检测方法

Method for detection of a student’s pose in a multi-scene classroom based on meta-learning

RichHTML

PDF (PC)

赞

可视化

摘要/Abstract

引用本文

使用本文

图/表 10

参考文献 21

相关文章 2

Metrics

本文评价

推荐阅读 10

[1]	郭泽坤,田隆,韩宁,王鹏辉,刘宏伟,陈渤. 采用CNN-SSD的雷达HRRP小样本目标识别方法[J]. 西安电子科技大学学报, 2021, 48(2): 7-14.
[2]	杨利霞;葛德彪;白剑;张世田. 三角面元数据模型FDTD网格生成技术 [J]. J4, 2007, 34(2): 298-302.