双向长短期记忆网络的时间序列预测方法

doi:10.19665/j.issn1001-2400.20231205

摘要/Abstract

摘要：

时间序列预测即利用历史时间序列数据,预测未来一段时间内的数据信息,以便提前制定相应策略。目前,时间序列的类别复杂繁多,而现有的时间序列预测模型面对多种类型数据时无法取得稳定预测的结果,进而难以同时满足对现实中多种复杂的时序数据预测的应用需求。针对上述问题,提出了一种基于时间注意力机制双向长短期记忆网络的时间序列预测方法。笔者提出的网络模型采用改进的正向和反向传播机制提取时序信息并通过自适应权重分配策略推理未来的时序信息。具体来说,设计了一个改进的双向长短期记忆网络,通过结合双向长短期记忆和长短期记忆网络提取深度时间序列特征,挖掘上下文的时序依赖关系。在此基础上,融合所提出的时间注意力机制,实现对深度时间序列特征进行自适应加权,提升深度时序特征的显著性表达能力。通过与同类代表性方法在多个不同类别数据集上的客观定量对比,实验结果表明,该方法能够在多种类别的复杂时间序列数据上更优的预测性能。

关键词: 时间序列, 双向长短期记忆网络, 长短期记忆网络, 注意力机制, 深度学习

Abstract:

Time series prediction means the use of historical time series to predict a period of time in the future,so as to formulate corresponding strategies in advance.At present,the categories of time series are complex and diverse.However,existing time series prediction models cannot achieve stable prediction results when faced with multiple types of time series data.The application requirements of complex time series data prediction in reality are difficult to simultaneously meet.To address the problem,a time series prediction method is proposed based on the Bidirectional Long and Short-term Memory(BLSTM) with the attention mechanism.The improved forward and backward propagation mechanisms are used to extract temporal information.The future temporal information is inferred through an adaptive weight allocation strategy.Specifically,an improved BLSTM is proposed to extract deep time series features and explore temporal dependencies of context by combining BLSTM and Long Short-term Memory(LSTM) networks,on the basis of which the proposed temporal attention mechanism is fused to achieve adaptive weighting of deep time series features,which improves the saliency expression ability of deep time series features.Experimental results demonstrate that the proposed method has a superior prediction performance in comparison with some representative methods in multiple time series datasets of different categories.

Key words: time series, Bidirectional Long Short-Term Memory, Long Short-Term Memory, attention mechanism, deep learning

中图分类号:

TP391.41

管业鹏, 苏光耀, 盛怡. 双向长短期记忆网络的时间序列预测方法[J]. 西安电子科技大学学报, 2024, 51(3): 103-112.

GUAN Yepeng, SU Guangyao, SHENG Yi. Time series prediction method based on the bidirectional long short-term memory network[J]. Journal of Xidian University, 2024, 51(3): 103-112.

图/表 12

图1

图2

图3

表1

表2

图4

图5

图6

表3

图7

表4

表5

参考文献 22

[1]	KOWSARI K, MEIMANDI K J, HEIDARYSAFA M, et al. Text Classification Algorithms:A Survey[J]. Information, 2019, 10(4):150-172.
[2]	DURAIRAJ D M, MOHAN B H K. A Convolutional Neural Network Based Approach to Financial Time Series Prediction[J]. Neural Computing and Applications, 2022, 34(16):13319-13337.
[3]	CAMASTRA F, CAPONE V, CIARAMELLA A, et al. Predictionof Environmental Missing Data Time Series by Support Vector Machine Regression and Correlation Dimension Estimation[J]. Environmental Modelling & Software, 2022, 150:1043-1053.
[4]	刘惠, 董锡耀, 杨志涵. 融合Stacking框架的BiGRU-LGB云负载预测模型[J]. 西安电子科技大学学报, 2023, 50(3):83-94.
	LIU Hui, DONG Xiyao, YANG Zhihan. Bigru-LGB Cloud Load Prediction Model Incorporating Stacking Framework[J]. Journal of Xidian University, 2023, 50(3):83-94.
[5]	张梦迪, 徐庆, 刘振鸿, 等. 基于动态滑动窗口BP神经网络的水质时间序列预测[J]. 环境工程技术学报, 2022, 12(3):809-815.
	ZHANG Mengdi, XU Qing, LIU Zhenhong, et al. Water Quality Time Series Prediction Based on Dynamic Sliding Window BP Neural Network[J]. Journal of Environmental Engineering Technology, 2022, 12(3):809-815.
[6]	MOHANTY M K, THAKURTA P K G, KAR S. Agricultural Commodity Price Prediction Model:A Machine Learning Framework[J]. Neural Computing and Applications, 2023, 35(20):15109-15128.
[7]	YULE G U. On A Method of Investigating Periodicities Disturbed Series,with Special Reference to Wolfer's Sunspot Numbers[J]. Philosophical Transactions of the Royal Society of London.Series A,Containing Papers of a Mathematical or Physical Character, 1927, 226(636-646):267-298.
[8]	WALKER G T. On Periodicity in Series of Related Terms[J]. Proceedings of the Royal Society of London.Series A,Containing Papers of a Mathematical and Physical Character, 1931, 131(818):518-532.
[9]	CHO C, KWON K, WU C. On Weather Data-Based Prediction of Gamma Exposure Rates Using Gradient Boosting Learning for Environmental Radiation Monitoring[J]. Sensors, 2022, 22(18):7062.
[10]	LIU S, FU B, WANG W, et al. Dynamic Sepsis Prediction for Intensive Care Unit Patients Using Xgboost-Based Model with Novel Time-Dependent Features[J]. IEEE Journal of Biomedical and Health Informatics, 2022, 26(8):4258-4269.
[11]	SIŁKA J, WIECZOREK M, WOŁNIAK M. Recurrent Neural Network Model for High-Speed Train Vibration Prediction from Time Series[J]. Neural Computing and Applications,2022,34(16):13305-13318.
[12]	HOCHREITER S, SCHMIDHUBER J. Long Short-Term Memory[J]. Neural Computation, 1997, 9(8):1735-1780. doi: 10.1162/neco.1997.9.8.1735 pmid: 9377276
[13]	HAFEZI L, REZAEIAN M. Neural Architecture for Persian Named Entity Recognition[C]//Proceedings of 4th Iranian Conference on Signal Processing and Intelligent Systems. Piscataway:IEEE, 2018:61-64.
[14]	KAUSHIK P, GUPTA A, ROY P P, et al. EEG-Based Age and Gender Prediction Using Deep BLSTM-LSTM Network Model[J]. IEEE Sensors Journal, 2019, 19(7):2634-2641. doi: 10.1109/JSEN.2018.2885582
[15]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention Is All You Need[J]. Advances in Neural Information Processing Systems, 2017, 30:2171-2184.
[16]	KITAEV N, KAISER Ł, LEVSKAYA A. Reformer:The Efficient Transformer[C]//Proceedings of 9th International Conference on Learning Representations. La Jolla: ICLR, 2020:1-12.
[17]	ZERVEAS G, JAYARAMAN S, PATEL D, et al. A Transformer-Based Framework for Multivariate Time Series Representation Learning[C]//Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. New York: ACM, 2021:2114-2124.
[18]	ZHOU H, ZHANG S, PENG J, et al. Informer:Beyond Efficient Transformer for Long Sequence Time-Series Forecasting[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto: AAAI, 2021:11106-11115.
[19]	LIU G, GUO J. Bidirectional LSTM with Attention Mechanism and Convolutional Layer for Text Classification[J]. Neurocomputing, 2019, 337:325-338.
[20]	PRADHAN T, KUMAR P, PAL S. CLAVER:An Integrated Framework of Convolutional Layer,Bidirectional LSTM with Attention Mechanism based Scholarly Venue Recommendation[J]. Information Sciences, 2021, 559:212-235.
[21]	AGARWAL N, BRUKHIM N, HAZAN E, et al. Boosting for Control of Dynamical Systems[C]//Proceedings of International Conference on Machine Learning. New York: PMLR, 2020:96-103.
[22]	GASPARIN A, LUKOVIC S, ALIPPI C. Deep Learning for Time Series Forecasting:The Electric Load Case[J]. CAAI Transactions on Intelligence Technology, 2022, 7(1):1-25.

数据集名称	类型	特征数	数据集长度
Badminton	国际羽毛球比赛数据	4	3 155 537
AirQuality^[21]	意大利空气质量数据	12	9 538
Power^[22]	法国家庭用电量数据	8	2 075 260

特征	值	取值数量
pos	中中场,中前场,中后场…	9
hand	反手,头顶,正手	3
tec	劈吊,勾,吊,抽,接杀挑…	19
path	中路,斜线,直线	3

网络模型	评价指标
网络模型	MAE↓	MSE↓	RMSE↓	PCC↑
LightGBM^[9]	0.438	0.558	0.491	0.704
LSTM^[12]	0.243	0.372	0.342	0.883
BLSTM^[13]	0.227	0.327	0.296	0.896
BLSTM-L^[14]	0.226	0.324	0.290	0.905
Transformer^[17]	0.232	0.387	0.305	0.887
Att-BLSTM^[19]	0.228	0.326	0.297	0.899
TA-LSTM(文中方法)	0.223	0.323	0.281	0.904

网络模型	AirQuality^[21]				Power^[21]
网络模型	MAE↓	MSE↓	RMSE↓	PCC↑	MAE↓	MSE↓	RMSE↓	PCC↑
LightGBM^[9]	0.649	0.727	0.693	0.803	0.450	0.534	0.478	0.716
LSTM^[12]	0.423	0.670	0.646	0.910	0.163	0.391	0.319	0.823
BLSTM^[13]	0.426	0.678	0.651	0.902	0.179	0.424	0.345	0.826
BLSTM-L^[14]	0.432	0.674	0.664	0.878	0.177	0.407	0.332	0.816
Transformer^[17]	0.461	0.809	0.708	0.901	0.184	0.390	0.350	0.824
Att-BLSTM^[19]	0.488	0.867	0.766	0.837	0.174	0.381	0.320	0.827
TA-LSTM(文中方法)	0.397	0.596	0.626	0.922	0.166	0.370	0.318	0.832

方法	Badminton
方法	MSE↓	MAE↓	RMSE↓	PCC↑
(w/o) Time Attention	0.236	0.365	0.301	0.890
(w/o) BLSTM	0.228	0.325	0.293	0.901
(w/o) LSTM	0.225	0.325	0.297	0.904
TA-LSTM	0.223	0.323	0.281	0.904