结合社会特征和注意力的行人轨迹预测模型

doi:10.19665/j.issn1001-2400.2020.01.002

Abstract

Abstract:

To address the problems that the pedestrian interaction feature of the Social GAN is simple and that it cannot make full use of the most of pedestrian interaction information, this paper proposes a pedestrian trajectory prediction model with social features and attention mechanism. This model adapts the structure of generative adversarial networks. The generator adapts an encoder-decoder model and the attention model is put between encoder and decoder. Three social features are set to enrich pedestrian interaction information which assists the attention module to make full use of the most of pedestrian interaction information by allocating the influence of pedestrians in the scene, so that the accuracy of the model is improved. Experimental results on multiple datasets show that the accuracy of this model in the pedestrian trajectory prediction task is increased by 15% compared with the previous pedestrian trajectory prediction model based on the pooling module. The improvement effect is most obvious in scenes with dense pedestrians and lots of non-straight tracks, with the accuracy increased by 34%.

Key words: trajectory generation, generative adversarial networks, attention mechanism, long short-term memory, pedestrian interaction

CLC Number:

TP391

ZHANG Zhiyuan,DIAO Yinghua. Pedestrian trajectory prediction model with social features and attention[J].Journal of Xidian University, 2020, 47(1): 10-17.

Figures/Tables 7

References 17

[1]	LUO Y, CAI P, BERA A , et al. PORCA: Modeling and Planning for Autonomous Driving among Many Pedestrians[J]. IEEE Robotics and Automation Letters, 2018,3(4):3418-3425.
[2]	CHENG J, CHENG H, MENG M Q H , et al. Autonomous Navigation by Mobile Robots in Human Environments: a Survey [C]//Proceedings of the 2018 IEEE International Conference on Robotics and Biomimetics. Piscataway: IEEE, 2018: 1981-1986.
[3]	CHOI W, SAVARESE S . A Unified Framework for Multi-target Tracking and Collective Activity Recognition [C]//Lecture Notes in Computer Science: 7575. Heidelberg: Springer Verlag, 2012: 215-230.
[4]	LIPTON Z C, BERKOWITZ J, ELKAN C . A Critical Review of Recurrent Neural Networks for Sequence Learning[J/OL].[2019-7-16]. Computer Science, 2015.
[5]	CHUNG J, KASTNER K, DINH L , et al. A Recurrent Latent Variable Model for Sequential Data [C]//Advances in Neural Information Processing Systems 28: 2015. Vancouver: Neural Information Processing Systems Foundation, 2015: 2980-2988.
[6]	LIU J, SHAHROUDY A, XU D , et al. Spatio-temporal LSTM with Trust Gates for 3d Human Action Recognition [C]//Lecture Notes in Computer Science: 9907. Heidelberg: Springer Verlag, 2016: 816-833.
[7]	ALAHI A, GOEL K, RAMANATHAN V , et al. Social LSTM: Human Trajectory Prediction in Crowded Spaces[C]//Proceedings of the 2016 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2016: 961-971.
[8]	VEMULA A, MUELLING K, OH J . Social Attention: Modeling Attention in Human Crowds[C]//Proceedings of the 2018 IEEE International Conference on Robotics and Automation. Piscataway: IEEE, 2018: 4601-4607.
[9]	GOODFELLOW I J, POUGET-ABADIE J, MIRZA M , et al. Generative Adversarial Nets[C]//Advances in Neural Information Processing Systems 27: 3. Vancouver: Neural Information Processing Systems Foundation, 2014: 2672-2680.
[10]	陈晓范, 申海杰, 边倩 , 等. 结合注意力机制的人脸超分辨率重建[J]. 西安电子科技大学学报, 2019,46(3):148-153.
	CHEN Xiaofan, SHEN Haijie, BIAN Qian , et al. Face Image Super-resolution with an Attention Mechanism[J]. Journal of Xidian University, 2019,46(3):148-153.
[11]	YU L, ZHANG W, WANG J , et al. Seqgan:Sequence Generative Adversarial Nets with Policy Gradient[C]//Proceedings of the 2017 31st AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2017: 2852-2858.
[12]	GUPTA A, JOHNSON J, FEI-FEI L , et al. Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks[C]//Proceedings of the 2018 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2018: 2255-2264.
[13]	VASWANI A, SHAZEER N, PARMAR N , et al. Attention is All You Need[C]//Advances in Neural Information Processing Systems 30: 2017. Vancouver: Neural Information Processing Systems Foundation, 2017: 5998-6008.
[14]	LUONG M T, PHAM H, MANNING C D . Effective Approaches to Attention-based Neural Machine Translation[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. New York:Association for Computational Linguistics, 2015: 1412-1421.
[15]	PELLEGRINI S, ESS A, VAN GOOL L . Improving Data Association by Joint Modeling of Pedestrian Trajectories and Groupings[C]//Lecture Notes in Computer Science: 6311. Heidelberg: Springer Verlag, 2010: 452-465.
[16]	LERNER A, CHRYSANTHOU Y, LISCHINSKI D . Crowds by Example[J]. Computer Graphics Forum, 2007,26(3):655-664.
[17]	LEE N, CHOI W, VERNAZA P , et al. Desire: Distant Future Prediction in Dynamic Scenes with Interacting Agents[C]//Proceedings of the 2017 30th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 2165-2174.

数据集	ETH	HOTEL	UNIV	ZARA1	ZARA2
数据量	5492	6543	21813/17953	5153	9722
截取帧数	876	1168	444/541	872	1052
行人总数	360	389	415/434	148	204
一帧最大行人数	27	18	75/52	20	20
一帧最小行人数	1	1	21/13	1	1
每帧平均行人数	6.27	5.60	49.13/33.18	5.91	9.24
单个行人存在最大帧数	114	100	352/289	197	584
单个行人存在最小帧数	2	2	4/5	10	7
行人存在平均帧数	15.26	16.82	52.56/41.37	34.82	47.66

评价指标	数据集	Linear	LSTM	S-LSTM	SGAN	+社会特征	+注意力模块	文中方法
ADE	ETH	0.84	0.70	0.73	0.69	0.67	0.65	0.63
	HOTEL	0.35	0.55	0.49	0.49	0.46	0.35	0.34
	UNIV	0.56	0.36	0.41	0.40	0.38	0.36	0.33
	ZARA1	0.41	0.25	0.27	0.22	0.21	0.21	0.20
	ZARA2	0.53	0.31	0.33	0.22	0.22	0.21	0.20
平均ADE		0.54	0.43	0.45	0.42	0.39	0.36	0.34

评价指标	数据集	Linear	LSTM	S-LSTM	SGAN	+社会特征	+注意力模块	文中方法
FDE	ETH	1.60	1.45	1.48	1.45	1.33	1.38	1.24
	HOTEL	0.60	1.17	1.01	1.01	0.86	0.70	0.62
	UNIV	1.01	0.77	0.84	0.82	0.78	0.69	0.69
	ZARA1	0.74	0.53	0.56	0.43	0.41	0.41	0.40
	ZARA2	0.95	0.65	0.70	0.44	0.43	0.42	0.42
平均FDE		0.98	0.91	0.91	0.83	0.76	0.72	0.67

Pedestrian trajectory prediction model with social features and attention

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 7

References 17

Related Articles 12

Metrics

Comments

Recommended 10

[1]	DONG Ruchan,JIAO Licheng,ZHAO Jin,SHEN Weiyan. Application of the deep fusion mechanism in object detection of remote sensing images [J]. Journal of Xidian University, 2021, 48(5): 128-138.
[2]	SUN Yanjing,WEI Li,ZHANG Nianlong,YUN Xiao,DONG Kaiwen,GE Min,CHENG Xiaozhou,HOU Xiaofeng. Person re-identification method combining the DD-GAN and Global feature in a coal mine [J]. Journal of Xidian University, 2021, 48(5): 201-211.
[3]	WEI Ziyu,YANG Xi,WANG Nannan,YANG Dong,GAO Xinbo. Reciprocal bi-directional generative adversarial network for cross-modal pedestrian re-identification [J]. Journal of Xidian University, 2021, 48(2): 205-212.
[4]	CHENG Lei,WANG Yue,TIAN Chunna. Residual attention mechanism for visual tracking [J]. Journal of Xidian University, 2020, 47(6): 148-157.
[5]	DU Lizhao,XU Yan,ZHANG Wei. Phased smoke detection algorithm using dual network fusion [J]. Journal of Xidian University, 2020, 47(4): 141-148.
[6]	YANG Xiaoli,LIN Suzhen. Method for multi-band image feature-level fusion based on the attention mechanism [J]. Journal of Xidian University, 2020, 47(1): 120-127.
[7]	CAO Weidong,LI Jiaqi,WANG Huaichao. Analysis of targeted sentiment by the attention gated convolutional network model [J]. Journal of Xidian University, 2019, 46(6): 30-36.
[8]	WANG Ziyi,SU Yuting,LIU Yanyan,ZHANG Wei. Algorithm for segmentation of smoke using the improved DeeplabV3 network [J]. Journal of Xidian University, 2019, 46(6): 52-59.
[9]	MO Jianwen,XU Kailiang,LIN Leping,OUYANG Ning. Text-to-image generation combined with mutual information maximization [J]. Journal of Xidian University, 2019, 46(5): 180-188.
[10]	CHEN Xiaofan,SHEN Haijie,BIAN Qian,WANG Zhenduo,TIAN Xinzhi. Face image super-resolution with an attention mechanism [J]. Journal of Xidian University, 2019, 46(3): 148-153.
[11]	MA Shulei,ZHANG Guobin,JIAO Yang,SHI Guangming. Improved method for image caption with global attention mechanism [J]. Journal of Xidian University, 2019, 46(2): 17-22.
[12]	XU Bin,CHEN Bo,LIU Jiaqi,WANG Penghui,LIU Hongwei. Radar HRRP target recognition by the bidirectional LSTM model [J]. Journal of Xidian University, 2019, 46(2): 29-34.