基于YOLO的无约束场景中文车牌检测与识别

doi:10.16180/j.cnki.issn1007-7820.2023.10.001

摘要/Abstract

摘要：

传统中文车牌识别方法对场景约束有要求,算法实时性差,且无法被部署在边缘设备上。针对上述问题,文中提出一种基于YOLO(You Only Look Once)的无约束场景中文车牌检测与识别方法。该方法分为车牌检测和车牌字符识别两个模块。在车牌检测部分,使用改进的YOLOv5模型,在预测目标候选区域的基础上多预测4组关键点用于车牌矫正,并使用在COCO数据集上训练的预训练模型进行训练,减少了由环境复杂引起的误检问题,具有高实时性。在车牌字符识别部分,改进了CRNN(Convolutional Recurrent Neural Network)模型,减少了算法的参数量和计算量,使其能成功部署于各类边缘设备。实验结果表明所提出的车牌识别方法能在复杂环境中高效检测并识别车牌。文中提出的车牌检测模型在车牌检测数据集上的map值相较Retina-face提升了3.0%,车牌字符识别模型在车牌识别数据集上精确度相比LPR-Net提升了4.2%。

关键词: 车牌检测, 车牌识别, 神经网络, 深度学习, 文字识别, 目标检测, 数据集, 机翼损失

Abstract:

In view of the problems of traditional Chinese license plate recognition methods, such as the requirement of scenes, poor real-time performance, and inability to deploy on edge devices, this study proposes a Chinese license plate detection and recognition method based on YOLO(You Only Look Once) in unconstrained scenes. This method is divided into two modules: license plate detection and license plate character recognition. In the license plate detection part, the improved YOLOv5 model is used to predict four groups of key points for license plate correction based on the prediction of target candidate regions, and the pre-training model trained on the COCO data set is used for training, which reduces the error detection problem caused by the complex environment and has high real-time performance. In the license plate character recognition part, the CRNN(Convolutional Recurrent Neural Network) model is improved, which greatly reduces the parameters and computation of the algorithm, so that it can be successfully deployed in various edge devices. Experimental results show that the proposed method can efficiently detect and recognize license plates in complex environments. The map value of the proposed license plate detection model is 3.0% higher than that of Retina-face in the license plate detection data set. Compared with LPR-Net, the accuracy of license plate character recognition model in license plate recognition data set is improved by 4.2%.

Key words: license plate detection, license plate recognition, neural network, deep learning, character recognition, object detection, data set, wing loss

中图分类号:

TP183

陈子昂,刘娜,袁野,李清都,万里红. 基于YOLO的无约束场景中文车牌检测与识别[J]. 电子科技, 2023, 36(10): 1-8.

CHEN Ziang,LIU Na,YUAN Ye,LI Qingdu,WAN Lihong. Chinese License Plate Detection and Recognition in Unconstrained Scenarios Based on YOLO[J]. Electronic Science and Technology, 2023, 36(10): 1-8.

图/表 11

图1

图2

图3

图4

图5

图6

图7

表1

表2

表3

图8

参考文献 23

[1]	张长青, 杨楠. 基于混合神经网络的车牌字符识别技术[J]. 电子科技, 2019, 32(9):51-54.
	Zhang Changqing, Yang Nan. License plate character r-ecognition technology based on hybrid neural network[J]. Electronic Science and Technology, 2019, 32(9):51-54.
[2]	马艳娥, 宋金沙, 张甜. 车牌图像定位识别的研究与应用[J]. 中国新通信, 2019, 21(10):127-128.
	Ma Yan'e, Song Jinsha, Zhang Tian, et al. Research and application of license plate image localization and recognition[J]. China New Telecommunications, 2019, 21(10):127-128.
[3]	王能, 赵红立, 琚生根. 一种改进的Canny边缘检测自适应算法[J]. 四川大学学报(自然科学版), 2014, 51(3):479-482.
	Wang Neng, Zhao Hongli, Ju Shenggen, et al. An improved adaptive Canny edge detection algorithm[J]. Journal of Sichuan University(Natural Scinece Edition), 2014, 51(3):479-482.
[4]	谭司庭, 胡志坤. 基于HSV色空间的车牌定位综合方法[J]. 计算机与应用化学, 2011, 28(7):903-906.
	Tan Siting, Hu Zhikun. An effective integration method for license plate location based on HSV color space[J]. Computers and Applied Chemistry, 2011, 28(7):903-906.
[5]	朱晓红, 张永来, 刘冬林. 非正交 haar 小波变换的车牌定位技术研究[J]. 计算机工程与应用, 2009, 45(13):175-175. doi: 10.3778/j.issn.1002-8331.2009.13.051
	Zhu Xiaohong, Zhang Yonglai, Liu Donglin, et al. License plate location technology based on non-orthogonal haar wavelet transformation[J]. Computer Engineering and Applications, 2009, 45(13):175-175 doi: 10.3778/j.issn.1002-8331.2009.13.051
[6]	陈墨林. 基于卷积神经网络的免字符分割车牌识别算法[J]. 智能计算机与应用, 2021, 11(11):125-127,130.
	Chen Molin. License plate recognition algorithm without character segmentation based on convolutional neural network[J]. Intelligent Computer and Applications, 2021, 11(11) :125-127,130.
[7]	林哲聪. 基于卷积神经网络的车牌识别系统设计和算法实现[D]. 杭州: 浙江工业大学, 2018:2-4.
	Lin Zhecong. Design and algorithm implementation of license plate recognition system based on convolutional neural network[D]. Hangzhou: Zhejiang University of Technology, 2018:2-4.
[8]	Xu Z, Yang W, Meng A, et al. Towards end-to-end lice-nse plate detection and recognition:A large dataset and baseline[C]. Munich: Proceedings of the European Conference on Computer Vision, 2018:255-271.
[9]	Redmon J, Divvala S, Girshick R, et al. You only look once:Unified,real-time object detection[C]. Las Vegas: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016:779-788.
[10]	Redmon J, Farhadi A. YOLO9000:Better,faster,stronger[C]. Holunono: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017:7263-7271.
[11]	Zhao Z Q, Zheng P, Xu S, et al. Object detection with deep learning:A review[J]. IEEE Transactions on Neural Networks and Learning Systems, 2019, 30(11):3212-3232. doi: 10.1109/TNNLS.5962385
[12]	Lin T Y, Dollár P, Girshick R, et al. Feature pyramid networks for object detection[C]. Holunono: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017:2117-2125.
[13]	Xiao Y, Tian Z, Yu J, et al. A review of object detection based on deep learning[J]. Multimedia Tools and Applications, 2020, 79(33):23729-23791. doi: 10.1007/s11042-020-08976-6
[14]	赵崇, 迟蒙蒙, 储聪. 导盲犬行走机构运动仿真及其视觉识别算法研究[J]. 电子科技, 2021, 34(9):66-72.
	Zhao Chong, Chi Mengmeng, Chu Cong, et al. Research on motion simulation and visual recognition algorithm of guide dog walking mechanism[J]. Electronic Scienceand Technology, 2021, 34(9):66-72.
[15]	Feng Z H, Kittler J, Awais M, et al. Wing loss for robust facial landmark localisation with convolutional neural networks[C]. Salt Lake City: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018:2235-2245.
[16]	Shi B, Bai X, Yao C. An end-to-end trainable neural n-etwork for image-based sequence recognition and its application to scene text recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 39(11):2298-2304. doi: 10.1109/TPAMI.2016.2646371
[17]	Pak M, Kim S. A review of deep learning in image recognition[C]. Bali Island: The Fourth International Conference on Computer Applications and Information Processing Technology, 2017:1-3.
[18]	Howard A, Sandler M, Chu G, et al. Searching for mobilenetv3[C]. Seoul: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019:1314-1324.
[19]	Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C]. Salt Lake City: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018:7132-7141.
[20]	Zhang S, Zheng D, Hu X, et al. Bidirectional long short-term memory networks for relation classification[C]. Shanghai: Proceedings of the Twenty-ninth Pacific Asia Conference on Language,Information and Computation, 2015:73-78.
[21]	Graves A, Fernández S, Gomez F, et al. Connectionist temporal classification:Labelling unsegmented sequence data with recurrent neural networks[C]. Pittsburgh: Proceedings of the Twenty-third International Conferenceon Machine Learning, 2006:369-376.
[22]	Deng J, Guo J, Ververas E, et al. Retinaface:Single-shot multi-level face localisation in the wild[C]. Seattle: Pr-oceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020:5203-5212.
[23]	Bian L, Wang H, Zhu C, et al. Image-free multi-character recognition[J]. Optics Letters, 2022, 47(6):1343-1346. doi: 10.1364/OL.451777 pmid: 35290309

省份	普通真实车牌数/张	普通生成车牌数/张	新能源真实车牌数/张	新能源生成车牌数/张	共计
京	711	4 000	143	1 000	5 854
沪	5 060	0	1 330	0	6 390
津	256	4 000	24	1 000	5 280
渝	225	4 000	27	1 000	5 252
冀	502	4 000	8	1 000	5 510
晋	261	4 000	0	1 000	5 261
蒙	62	1 000	0	0	1 062
辽	175	4 000	13	1 000	5 188
吉	22	4 000	5	1 000	5 027
黑	95	4 000	16	1 000	5 111
苏	5 119	0	839	700	6 658
浙	2 455	2 500	961	600	6 516
皖	108 045	0	9 042	0	117 087
闽	7 560	0	502	1 500	9 562
赣	678	4 000	78	1 000	5 756
鲁	752	4 000	55	1 000	5 807
豫	969	10 000	149	3 000	14 118
鄂	726	4 000	106	1 000	5 832
湘	406	4 000	53	1 000	5 459
粤	835	4 000	151	1 000	5 986
桂	28	4 000	8	1 000	5 036
琼	12	4 000	7	1 000	5 019
川	264	4 000	36	1 000	5 300
贵	39	4 000	14	1 000	5 053
云	90	4 000	7	1 000	5 097
藏	3	1 000	0	0	1 003
陕	345	4 000	30	1 000	5 375
甘	56	1 000	0	0	1 056
青	25	1 000	0	0	1 025
宁	124	900	0	0	1 024
新	39	1 000	0	0	1 039

模型	mAP/%	Recall/%	FPS/ frame·s^-1
Retina-face	96.8	96.4	32
本文	99.8	99.8	45

模型	Acuracy/%	Parameter/MB
LPR-Net	94.4	4.8
CRNN	96.8	138.0
本文	98.6	6.2