基于多Agent强化学习的危险车辆预警算法

doi:10.16180/j.cnki.issn1007-7820.2020.09.008

摘要/Abstract

摘要：

针对目前行人易受到车辆撞击,且缺乏主动保护手段的问题,文中设计了一个包括雷达等模块的智能可穿戴设备来保护行人免受车辆的冲击。在此基础上,提出了基于模糊综合评价的安全智能算法,从行人的角度出发,综合考虑将雷达探测的车辆数据、当地道路交通状况、天气、行人状态等多种影响因素作为评价指标。为提高算法的准确性和适应性,提出了基于BP神经网络和多Agent强化学习的方法赋予模糊综合评价的各指标动态权重。仿真验证结果显示,相较于AHP等取权重方法,该预警算法的警报准确率提高了55%以上;相较单Agent强化学习,该方法学习效率提高了近28倍,说明该智能穿戴设备可以对车辆撞击行人进行有效地预测和警告。

关键词: 多Agent强化学习, 危险车辆预警, 主动保护, 智能穿戴设备, 预警算法, 模糊综合评价

Abstract:

In view of the current problem that pedestrians are vulnerable to vehicle impact and lack of active protection means, an intelligent wearable device including radar module to protect pedestrians from vehicle impact is proposed in this study. On this basis, a safety intelligent algorithm based on fuzzy comprehensive evaluation is proposed. From the perspective of pedestrians, radar-detected vehicle data, local road traffic conditions, weather, pedestrian status and other factors are considered as evaluation indicators. In order to improve the accuracy and adaptability of the algorithm, a method based on BP neural network and multi-agent reinforcement learning is proposed to give dynamic weights to each index of fuzzy comprehensive evaluation. The simulation results shows that the alarm accuracy of the algorithm is more than 55% higher than that of the weighted method such as AHP, and the learning efficiency of the algorithm is nearly 28 times higher than that of the single agent reinforcement learning, which indicated that intelligent wearing equipment can effectively predict and warn the impact of vehicles on pedestrians.

Key words: multi-agent enhancement learning, dangerous vehicle early warning, active protection, intelligent wearing equipment, early warning algorithm, fuzzy comprehensive evaluation

中图分类号:

TN957.52+4

王泽学,万启东,秦杨梅,樊森清,肖泽仪. 基于多Agent强化学习的危险车辆预警算法[J]. 电子科技, 2020, 33(9): 44-49.

WANG Zexue,WAN Qidong,QIN Yangmei,FAN Senqing,XIAO Zeyi. Dangerous Vehicle Early Warning Algorithms Based on Multi-Agent Enhancement Learning[J]. Electronic Science and Technology, 2020, 33(9): 44-49.

图/表 8

图1

图2

图3

图4

表1

图5

图6

图7

参考文献 19

[1]	张盈盈, 刘帅, 邱志军, 等. 行人车辆冲突参数分析与安全评价[J]. 哈尔滨工业大学学报, 2012,44(12):123-128. doi: 10.11918/j.issn.0367-6234.2012.12.022
	Zhang Yingying, Liu Shuai, Qiu Zhijun, et al. Pedestrian-vehicle conflict parameter analysis and safety evaluation[J]. Journal of Harbin University of Technology, 2012,44(12):123-128. doi: 10.11918/j.issn.0367-6234.2012.12.022
[2]	王群山, 苗强, 刘成晔. 汽车一体化安全与行人保护研究[J]. 中国安全科学学报, 2011,21(2):36-40.
	Wang Qunshan, Miao Qiang, Liu Chengye. Research on integrated automobile safety and pedestrian protection[J]. Chinese Journal of Safety Sciences, 2011,21(2):36-40.
[3]	吕能超, 旷权, 谭青山, 等. 基于车路协同的行人车辆碰撞风险识别与决策方法[J]. 中国安全科学学报, 2015,25(1):60-66.
	Lü Nengchao, Kuang Quan, Tan Qingshan, et al. Pedestrian-vehicle collision risk identification and decision-making method based on vehicle-road coordination[J]. Chinese Journal of Safety Sciences, 2015,25(1):60-66.
[4]	韩永刚, 赵巍凯. 基于单目视觉的行人检测研究[J]. 电子科技, 2014,27(8):22-25.
	Han Yonggang, Zhao Weikai. Research on pedestrian detection based on monocular vision[J]. Electronic Science and Technology, 2014,27(8):22-25.
[5]	高华, 邬春学, 鲁俊. 基于动态加权可变形部件模型的行人检测[J]. 电子科技, 2016,29(9):1-3.
	Gao Hua, Hu Chunxue, Lu Jun. Pedestrian detection based on dynamic weighted deformable component model[J]. Electronic Science and Technology, 2016,29(9):1-3.
[6]	崔春宽, 何少斌, 李东浩, 等. 77GHz毫米波雷达传感器技术研究[J]. 汽车实用技术, 2018(1):17-20.
	Cui Chunkuan, He Shaobin, Li Donghao, et al. 77GHz millimeter wave radar sensor technology research[J]. Automotive Practical Technology, 2018(1):17-20.
[7]	戴作宁, 张兴敢, 唐岚, 等. 基于多通道补偿的毫米波雷达高速目标检测方法[J]. 南京大学学报(自然科学版), 2014,50(3):356-360.
	Dai Zuoning, Zhang Xinggan, Tang Lan, et al. High-speed target detection method for millimeter wave radar based on multi-channel compensation[J]. Journal of Nanjing University (Natural Science Edition), 2014,50(3):356-360.
[8]	张雅科. 基于模糊权重相似性的协同过滤算法研究[J]. 电子科技, 2015,28(7):111-114.
	Zhang Yake. Collaborative filtering algorithm based on similarity of fuzzy weights[J]. Electronic Science and Technology, 2015,28(7):111-114.
[9]	高丽媛, 董甲. 基于改进BP神经网络的洪峰预测模型[J]. 电子科技, 2015,28(3):41-43,54.
	Gao Liyuan, Dong Jia. Flood peak prediction model based on improved BP neural network[J]. Electronic Science and Technology, 2015,28(3):41-43,54.
[10]	张天云, 陈奎, 魏伟, 等. BP神经网络法确定工程材料评价指标的权重[J]. 材料导报, 2012,26(2):159-163.
	Zhang Tianyun, Chen Kui, Wei Wei, et al. BP neural network method to determine the weight of evaluation index of engineering materials[J]. Material Report, 2012,26(2):159-163.
[11]	李明, 刘玮, 张彦铎. 基于改进合同网协议的多Agent动态任务分配[J]. 山东大学学报(工学版), 2016,46(2):51-56,63. doi: 10.6040/j.issn.1672-3961.1.2015.218
	Li Ming, Liu Wei, Zhang Yanduo. Multi-Agent dynamic task assignment based on improved contract network protocol[J]. Journal of Shandong University (Engineering Edition), 2016,46(2):51-56,63. doi: 10.6040/j.issn.1672-3961.1.2015.218
[12]	张书钦, 李凯江, 张露, 等. 基于Q-learning机制的攻击图生成技术研究[J]. 电子科技, 2018,31(10):6-10.
	Zhang Shuqin, Li Kaijiang, Zhang Lu, et al. Research on attack graph generation technology based on Q-learning mechanism[J]. Electronic Science and Technology, 2018,31(10):6-10.
[13]	Busoniu L, Babuska R, De Schutter B. A comprehensive survey of multiagent reinforcement learning[J]. IEEE Transactions on Systems Man and Cybernetics Part C-Applications and Reviews, 2008,38(2):156-172. doi: 10.1109/TSMCC.5326
[14]	战忠丽, 王强, 陈显亭. 强化学习的模型、算法及应用[J]. 电子科技, 2011,24(1):47-49.
	Zhan Zhongli, Wang Qiang, Chen Xianting. Model, algorithm and application of reinforcement learning[J]. Electronic Science and Technology, 2011,24(1):47-49.
[15]	Bellman R. A Markovian decision process[J]. Journal of Mathematics and Mechanics, 1957,6(5):679-684.
[16]	Watkins C, Dayan P. Q-learning[J]. Machine Learning, 1992,8(3-4):279-292.
[17]	刘全, 翟建伟, 章宗长, 等. 深度强化学习综述[J]. 计算机学报, 2018,41(1):1-27.
	Liu Quan, Zhai Jianwei, Zhang Zongchang, et al. A review of deep intensive learning[J]. Journal of Computer Science, 2018,41(1):1-27.
[18]	陈国华, 吴武生, 徐三元, 等. 基于WBS-RBS与AHP的跨海桥梁工程施工HSE风险评价[J]. 中国安全科学学报, 2013,23(9):51-57.
	Chen Guohua, Wu Wusheng, Xu Sanyuan, et al. HSE risk assessment of cross-sea bridge construction based on WBS-RBS and AHP[J]. Chinese Journal of Safety Sciences, 2013,23(9):51-57.
[19]	孙秀娟, 陆新秀, 刘法胜, 等. 基于熵权法的交通流组合预测模型研究[J]. 山东科技大学学报(自然科学版), 2018,37(4):111-117.
	Sun Xiujuan, Lu Xinxiu, Liu Fasheng, et al. Research on traffic flow combination forecasting model based on entropy weight method[J]. Journal of Shandong University of Science and Technology(Natural Science Edition), 2018,37(4):111-117.

权重选取方法	合格次数	合格率
AHP法	14	28%
熵权法	16	32%
经验为学习样本的BP神经网络	18	36%
强化学习训练结果为学习样本的BP神经网络	28	56%