互惠双向生成对抗网络用于跨模态行人重识别

doi:10.19665/j.issn1001-2400.2021.02.026

Abstract

Abstract:

To improve the accuracy of cross-modal pedestrian re-identification,a reciprocal bi-directional generative adversarial network-based method is proposed.First,we build two generative adversarial networks to generate cross-modal heterogeneous images.Second,an associated loss is designed to pull close the distribution of features in latent space during the image translation between visible and infrared images so as to help the networks generate fake heterogeneous images that have high similarity with the real images.Finally,by concatenating the original and generated heterogeneous pedestrian images into the discriminative feature extraction network,images from different modalities can be unified into a common modality,thus suppressing the cross-modal gap.Representation learning and metric learning are utilized to achieve more discriminative pedestrian features.Comparative experiments are conducted on SYSU-MM01 and RegDB datasets to analyze the accuracy with different loss functions.Compared with other state-of-the-art cross-modal pedestrian re-identification methods,the proposed method achieves a higher accuracy and stronger robustness.

Key words: generative adversarial networks, image translation, feature extraction, cross-modal pedestrianre-identification

CLC Number:

TN911.73

WEI Ziyu,YANG Xi,WANG Nannan,YANG Dong,GAO Xinbo. Reciprocal bi-directional generative adversarial network for cross-modal pedestrian re-identification[J].Journal of Xidian University, 2021, 48(2): 205-212.

Figures/Tables 7

References 15

[1]	罗浩, 姜伟, 范星, 等. 基于深度学习的行人重识别研究进展[J]. 自动化学报, 2019,45(11):2032-2049.
	LUO Hao, JIANG Wei, FAN Xing, et al. A Survey on Deep Learning Based Person Re-identification[J]. Acta Automatica Sinica, 2019,45(11):2032-2049.
[2]	朱莉, 赵俊, 傅应锴, 等. 一种红外热图像目标区域分割的深度学习算法[J]. 西安电子科技大学学报, 2019,46(4):107-114.
	ZHU Li, ZHAO Jun, FU Yingkai, et al. Deep Learning Algorithm for the Segmentation of the Interested Region of an Infrared Thermal Image[J]. Journal of Xidian University, 2019,46(4):107-114.
[3]	WU A, ZHENG W S, YU H X, et al. RGB-Infrared Cross-Modality Person Re-Identification[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway:IEEE, 2017: 5390-5399.
[4]	NGUYEN D T, HONG H G, KIM K W, et al. Person Recognition System Based on a Combination of Body Images from Visible Light and Thermal Cameras[J]. Sensors, 2017,17(3):605.
[5]	YE M, WANG Z, LAN X, et al. Visible Thermal Person Re-Identification via Dual-Constrained Top-Ranking[C]//Proceedings of the 2018 International Joint Conferences on Artificial Intelligence. Stockholm:International Joint Conferences on Artificial Intelligence, 2018: 1092-1099.
[6]	HAO Y, WANG N, LI J, et al. HSME:Hypersphere Manifold Embedding for Visible Thermal Person Re-identification[C]//Proceedings of the 2019 33rd AAAI Conference on Artificial Intelligence. Palo Alto:AAAI Press, 2019: 8385-8392.
[7]	申海杰, 边倩, 陈晓范, 等. 生成对抗网络用于视频去模糊[J]. 西安电子科技大学学报, 2019,46(6):112-117.
	SHEN Haijie, BIAN Qian, CHEN Xiaofan, et al. Video Deblurring Using the Generative Adversarial Network[J]. Journal of Xidian University, 2019,46(6):112-117.
[8]	KNIAZ V V, KNYAZ V A, HLADUVKA J, et al. ThermalGAN:Multimodal Color-to-Thermal Image Translation for Person Re-Identification in Multispectral Dataset[C]//Lecture Notes in Computer Science:11134. Heidelberg:Springer Verlag, 2019: 606-624.
[9]	DAI P, JI R, WANG H, et al. Cross-Modality Person Re-Identification with Generative Adversarial Training[C]// Proceedings of the 2018 International Joint Conferences on Artificial Intelligence. Stockholm:International Joint Conferences on Artificial Intelligence, 2018: 677-683.
[10]	杜鹏, 宋永红, 张鑫瑶. 基于自注意力模态融合网络的跨模态行人再识别方法研究[J]. 自动化学报, 2019: 1-12.
	DU Peng, SONG Yonghong, ZHANG Xinyao. Self-Attention Cross-Modality Fusion Network for Cross-Modality Person Re-identification[J]. Acta Automatica Sinica, 2019: 1-12.
[11]	ZHU J Y, PARK T, ISOLA P, et al. Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway:IEEE, 2017: 2242-2251.
[12]	WANG Z, WANG Z, ZHENG Y, et al. Learning to Reduce Dual-level Discrepancy for Infrared-visible Person Re-identification[C]//Proceedings of the 2019 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington:IEEE Computer Society, 2019: 618-626.
[13]	ISOLA P, ZHU J Y, ZHOU T, et al. Image-to-image Translation with Conditional Adversarial Networks[C]//Proceedings of the 2017 30th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2017: 1125-1134.
[14]	LIAO S, HU Y, ZHU X, et al. Person Re-identification by Local Maximal Occurrence Representation and Metric Learning[C]//Proceedings of the 2015 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington:IEEE Computer Society, 2015: 2197-2206.
[15]	DALAL N, TRIGGS B. Histograms of Oriented Gradients for Human Detection[C]// Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington:IEEE Computer Society, 2005: 886-893.

Λ_rec	Λ_cyc	Λ_ass	Λ_id	Λ_metric	CMC-1	CMC-10	CMC-20	mAP
√	×	√	√	√	34.65	76.15	87.04	48.02
√	√	×	√	√	39.44	79.36	88.59	52.27
√	√	√	×	√	1.05	10.86	21.40	5.44
√	√	√	√	×	32.29	72.68	83.12	45.48
√	√	√	√	√	43.04	81.85	90.77	55.58

方法	SYSU-MM01				RegDB
方法	CMC-1	CMC-10	CMC-20	mAP	CMC-1	CMC-10	CMC-20	mAP
LOMO	1.75	14.14	26.63	3.48	0.85	2.47	4.10	2.28
HOG	2.76	18.25	31.91	4.24	13.49	33.22	43.66	10.31
One-stream	12.04	49.68	66.74	13.67	13.11	32.98	42.51	14.02
Two-stream	11.65	47.99	65.50	12.85	12.43	30.36	40.96	13.42
Zero-padding	14.80	54.12	71.33	15.59	17.75	34.21	44.35	18.90
BDTR	17.01	55.43	71.96	19.66	33.47	58.42	67.52	31.83
D-HSME	20.68	62.74	77.95	23.12	50.85	73.36	81.66	47.00
cmGAN	26.97	67.51	80.56	27.80
D²RL	28.90	70.60	82.40	29.20	43.40	66.10	76.30	44.10
自注意力模态融合	33.31	74.51	85.79	33.18
互惠双向网络(Ours)	43.04	81.85	90.77	55.58	73.79	89.81	93.25	68.96

Reciprocal bi-directional generative adversarial network for cross-modal pedestrian re-identification

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 7

References 15

Related Articles 15

Metrics

Comments

Recommended 10

[1]	SUN Yanjing,WEI Li,ZHANG Nianlong,YUN Xiao,DONG Kaiwen,GE Min,CHENG Xiaozhou,HOU Xiaofeng. Person re-identification method combining the DD-GAN and Global feature in a coal mine [J]. Journal of Xidian University, 2021, 48(5): 201-211.
[2]	LIU Jieyi,GONG Maoguo,ZHAN Tao,LI Hao,ZHANG Mingyang. Method for discrimination of false targets in multistation radar systems based on the deep neural network [J]. Journal of Xidian University, 2021, 48(2): 133-138.
[3]	GUO Zekun,TIAN Long,HAN Ning,WANG Penghui,LIU Hongwei,CHEN Bo. Radar HRRP based few-shot target recognition with CNN-SSD [J]. Journal of Xidian University, 2021, 48(2): 7-14.
[4]	HAN Zhuoxi,WANG Feng,CHEN Pei,LI Zhuolun. Fuzzy data association algorithm assisted by historical features [J]. Journal of Xidian University, 2021, 48(2): 92-98.
[5]	WANG Xiaohua,YAO Pengchao,MA Liping,WANG Wenjie,ZHANG Lei. Algorithm for extraction of features of robot speech control in the factory environment [J]. Journal of Xidian University, 2020, 47(2): 16-22.
[6]	ZHANG Zhiyuan,DIAO Yinghua. Pedestrian trajectory prediction model with social features and attention [J]. Journal of Xidian University, 2020, 47(1): 10-17.
[7]	YANG Xiaoli,LIN Suzhen. Method for multi-band image feature-level fusion based on the attention mechanism [J]. Journal of Xidian University, 2020, 47(1): 120-127.
[8]	WANG Hongyan,ZHU Limin,ZHANG Panjie,LI Jinping. Method for the detection of the piston side defect based on external contour registration [J]. Journal of Xidian University, 2019, 46(5): 75-83.
[9]	MO Jianwen,XU Kailiang,LIN Leping,OUYANG Ning. Text-to-image generation combined with mutual information maximization [J]. Journal of Xidian University, 2019, 46(5): 180-188.
[10]	GUO Rui,ZANG Bo,PENG Shuming,XING Mengdao. Extraction of features of the urban high-rise building from high resolution InSAR data [J]. Journal of Xidian University, 2019, 46(4): 137-143.
[11]	LIU Daohua,CUI Yushuang,ZHAO Yansong,SONG Yuting,WANG Jinghui. Method for retrieving the teaching image based on the improved convolutional neural network [J]. Journal of Xidian University, 2019, 46(3): 52-58.
[12]	YANG Jiangong,WANG Xili,LIU Shigang. Spectral-spatial classification of hyperspectral images using deep Boltzmann machines [J]. Journal of Xidian University, 2019, 46(3): 109-115.
[13]	WANG Xinyuan,XIAO Song,LI Lei,JIAO Lingling. Robust target tracking algorithm based on the ELM and discriminative correlation filter [J]. Journal of Xidian University, 2019, 46(1): 57-63.
[14]	WANG Xing;HUANG Xiaoyu;LIU Xuanpu;KONG Xianguang;NIU Meng. Multi-layer incremental feature extraction method for industrial big data [J]. Journal of Xidian University, 2018, 45(4): 106-111.
[15]	HE Wangpeng;SUN Wei;SU Bo;YANG Yunyi;GUO Baolong. Sparse feature extraction technique and its applications to machinery fault diagnosis [J]. Journal of Xidian University, 2018, 45(2): 154-159.