融合Stacking框架的BiGRU-LGB云负载预测模型

doi:10.19665/j.issn1001-2400.2023.03.008

摘要/Abstract

摘要：

随着云计算技术的飞速发展,越来越多的用户将应用部署在云平台上.。平台内集群资源的调度可以提高云平台数据中心的实际利用率,而高效的云平台负载预测是解决集群资源调度问题的关键技术,因此本文建立了一种融合Stacking框架、多层BiGRU网络和LightGBM算法的云负载预测模型。该模型的结构主要包括两种学习器:首先是初级学习器,使用时间编码层处理原始负载序列并利用BiGRU网络参数少、信息学习完整的特点减少模型训练时间和隐藏层数,学习负载序列中的时间维度信息;使用经过特征工程处理的原始负载序列来高效训练LightGBM算法,学习负载序列中的特征维度信息。然后是次级学习器,利用GRU网络整合时间和特征维度的负载信息,完成整个负载预测模型的训练。通过两层学习器的共同学习提高整体负载预测模型的预测精度。在华为云集群数据集上进行实验,结果表明,与传统的单一预测模型BiGRU、LightGBM等以及现有的组合预测模型GRU-LSTM相比,融合Stacking的BiGRU-LGB模型的预测精度提升约13%,训练时间开销得到一定程度的降低。

关键词: 云平台, 负载预测, 双向门控循环单元, LightGBM, Stacking集成框架

Abstract:

With the rapid development of cloud computing technology,more and more users deploy applications on cloud platforms.The scheduling of cluster resources within the platform can improve the actual utilization of the cloud platform data center,and efficient cloud platform load prediction is a key technology for solving the cluster resource scheduling problem,so this paper establishes a cloud load prediction model that incorporates the Stacking framework,multilayer bidirectional gated recurrent unit (BiGRU) network and light gradient boosting machine (LightGBM) algorithm.The structure of the model consists mainly of two kinds of learners:one is the primary learner,which uses a temporal encoding layer to process the original load sequence and reduces the training time and the number of hidden layers by taking advantage of the BiGRU network with fewer parameters and complete information learning,to learn the temporal dimension information in the load sequence with the original load sequence processed by feature engineering used to efficiently train the LightGBM algorithm to learn the load feature dimension information in the sequence.Then comes the other learner,which integrates the load information in temporal and feature dimensions using the GRU network to complete the training of the whole load prediction model.The prediction accuracy of the overall load prediction model is improved by joint learning of the two layers of learners.Experiments are conducted on Huawei Cloud Cluster dataset with the results showing that the prediction accuracy of the BiGRU-LGB model incorporating the Stacking is improved by about 13% and the training time overhead is reduced to some extent compared with the traditional single prediction models,such as BiGRU and LightGBM,and the existing combined prediction model GRU-LSTM.

Key words: cloud platform, load prediction, bidirectional gated cyclic unit, LightGBM, stacking integration framework

中图分类号:

TP183

刘惠,董锡耀,杨志涵. 融合Stacking框架的BiGRU-LGB云负载预测模型[J]. 西安电子科技大学学报, 2023, 50(3): 83-94.

LIU Hui,DONG Xiyao,YANG Zhihan. BiGRU-LGB cloud load prediction model incorporating stacking framework[J]. Journal of Xidian University, 2023, 50(3): 83-94.

图/表 10

表1

图1

图2

图3

图4

图5

图6

表2

图7

表3

参考文献 30

[1]	MEDEL V, TOLOSANA-CALASANZ R, BANARES J A, et al. Characterising Resource Management Performance in Kubernetes[J]. Computers and Electrical Engineering, 2018, 68:286-297. doi: 10.1016/j.compeleceng.2018.03.041
[2]	赖成喆, 陈瑶, 郭奇立, 等. 车联网中基于博弈论的合作下载激励方案[J]. 西安电子科技大学学报, 2021, 48(3):9-20.
	LAI Chengche, CHEN Yao, GUO Qili, et al. A Game Theory-Based Incentive Scheme for Cooperative Downloading in Vehicular Networking[J]. Journal of Xidian University, 2021, 48(3):9-20.
[3]	李华东, 张学亮, 王晓磊, 等. Kubernetes集群中多节点合作博弈负载均衡策略[J]. 西安电子科技大学学报, 2021, 48(6):16-22.
	LI Huadong, ZHANG Xueliang, WANG Xiaolei, et al. Multi-Node Cooperative Game Load Balancing Strategy in Kubernetes Clusters[J]. Journal of Xidian University, 2021, 48(6):16-22.
[4]	AN X, ZHOU X, LÜ X, et al. Sample Selected Extreme Learning Machine Based Intrusion Detection in Fog Computing and MEC[J]. Wireless Communications and Mobile Computing, 2018:1-10.
[5]	BI J, ZHANG L, YUAN H, et al. Hybrid Task Prediction Based on Wavelet Decomposition and ARIMA Model in Cloud Data Center[C]// 2018 IEEE 15th International Conference on Networking,Sensing and Control (ICNSC).Piscataway:IEEE, 2018:1-6.
[6]	ZHANG G P. Time Series Forecasting Using a Hybrid ARIMA and Neural Network Model[J]. Neurocomputing, 2003, 50:159-175. doi: 10.1016/S0925-2312(01)00702-0
[7]	LIU C, LIU C, SHANG Y, et al. An Adaptive Prediction Approach Based on Workload Pattern Discrimination in the Cloud[J]. Journal of Network and Computer Applications, 2017, 80:35-44. doi: 10.1016/j.jnca.2016.12.017
[8]	XU D, YANG S, LIU R. A Mixture of HMM,GA,and Elman Network for Load Prediction in Cloud-Oriented Data Centers[J]. Journal of Zhejiang University-Science C(Computers and Electronics), 2013, 14(11):845-858.
[9]	CETINSKI K, JURIC M B. AME-WPC:Advanced Model for Efficient Workload prediction in the Cloud[J]. Journal of Network and Computer Applications, 2015, 55:191-201. doi: 10.1016/j.jnca.2015.06.001
[10]	ZHANG W, LI B, ZHAO D, et al. Workload Prediction for Cloud Cluster Using a Recurrent Neural Network[C]//2016 International Conference on Identification,Information and Knowledge in the Internet of Things (IIKI). Piscataway:IEEE, 2016:104-109.
[11]	CHO K, VAN MERRI? NBOER B, GULCEHRE C, et al. Learning Phrase Representations Using RNN Encoder-Decoder for Statistical Machine Translation (2014)[J/OL].[2014-06-03]. https://arxiv.org/abs/1406.1078.
[12]	LIU C, LIU C, SHANG Y, et al. An Adaptive Prediction Approach Based on Workload Pattern Discrimination in the Cloud[J]. Journal of Network and Computer Applications, 2017, 80:35-44. doi: 10.1016/j.jnca.2016.12.017
[13]	SHI X, CHEN Z, WANG H, et al. Convolutional LSTM Network:A Machine Learning Approach for Precipitation Nowcasting[J]. Advances in Neural Information Processing Systems, 2015, 28:1-12.
[14]	蔡亮, 周泓岑, 白恒, 等. 基于多层BiLSTM和改进粒子群算法的应用负载预测方法[J]. 浙江大学学报(工学版), 2020, 54(12):2414-2422.
	CAI Liang, ZHOU Honcen, BAI Heng, et al. An Application Load Prediction Method Based on Multilayer BiLSTM and Improved Particle Swarm Algorithm[J]. Journal of Zhejiang University (Engineering Edition), 2020, 54(12):2414-2422.
[15]	SIAMI-NAMINI S, TAVAKOLI N, NAMIN A S. The Performance of LSTM and BiLSTM in Forecasting Time Series[C]//2019 IEEE International Conference on Big Data (Big Data). Piscataway:IEEE, 2019:3285-3292.
[16]	GUO J, WANG Z, CHEN H. On-Line Multi-Step Prediction of Short Term Traffic Flow Based on GRU Neural Network[C]// Proceedings of the 2nd International Conference on Intelligent Information Processing. New York: ACM, 2017:1-6.
[17]	DEY R, SALEM F M. Gate-Variants of Gated Recurrent Unit (GRU) Neural Networks[C]// 2017 IEEE 60th International Midwest Symposium on Circuits and Systems (MWSCAS).Piscataway:IEEE, 2017:1597-1600.
[18]	ZHOU H, ZHANG S, PENG J, et al. Informer:Beyond Efficient Transformer for Long Sequence Time-Series Forecasting[C]// Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto: AAAI, 2021, 35(12):11106-11115.
[19]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is All You Need[J]. Advances in Neural Information Processing Systems, 2017, 30:6000-6010.
[20]	WOLPERT D H. Stacked Generalization[J]. Neural Networks, 1992, 5(2):241-259. doi: 10.1016/S0893-6080(05)80023-1
[21]	KE G, MENG Q, FINLEY T, et al. Lightgbm:A Highly Efficient Gradient Boosting Decision Tree[J]. Advances in Neural Information Processing Systems, 2017, 30:3149-3157.
[22]	ZHU Q, ZHANG F, LIU S, et al. A Hybrid VMD-BiGRU Model for Rubber Futures Time Series Forecasting[J]. Applied Soft Computing, 2019, 84:105739. doi: 10.1016/j.asoc.2019.105739
[23]	LI S, LI W, COOK C, et al. Independently Recurrent Neural Network (IndRNN):Building a Longer and Deeper RNN[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2018:5457-5466.
[24]	FRIEDMAN J H. Greedy Function Approximation:A Gradient Boosting Machine[J]. The Annals of Statistics, 2001, 29(5):1189-1232. doi: 10.1214/aos/1013203450
[25]	王国胤, 于洪, 杨大春. 基于条件信息熵的决策表约简[J]. 计算机学报, 2002, 25(7):759-766.
	WANG Guoyin, YU Hong, YANG Dachun. Decision Table Simplification Based on Conditional Information Entropy[J]. Journal of Computer Science, 2002, 25(7):759-766.
[26]	BONCHI F, GIANNOTTI F, MAZZANTI A, et al. ExAMiner:Optimized Level-Wise Frequent Pattern Mining with Monotone Constraints[C]//Third IEEE International Conference on Data Mining. Piscataway:IEEE, 2003:11-18.
[27]	许阳, 任盛, 刘鑫. 基于皮尔逊相关系数法的台区窃电精准分析[J]. 大众用电, 2021, 36(8):14-15.
	XU Yang, REN Sheng, LIU Xin, et al. Precise Analysis of Power Theft in the Station Area Based on Pearson Correlation Coefficient Method[J]. Popular Electricity Consumption, 2021, 36(8):14-15.
[28]	RODRIGUEZ J D, PEREZ A, LOZANO J A. Sensitivity Analysis of K-Fold Cross Validation in Prediction Error Estimation[J]. IEEE transactions on Pattern Analysis and Machine Intelligence, 2009, 32(3):569-575. doi: 10.1109/TPAMI.2009.187
[29]	彭燕, 刘宇红, 张荣芬. 基于LSTM的股票价格预测建模与分析[J]. 计算机工程与应用, 2019, 55(11):209-212. doi: 10.3778/j.issn.1002-8331.1811-0239
	PENG Yan, LIU Yuhong, ZHANG Rongfen. Modeling and Analysis of Stock Price Prediction Based on LSTM[J]. Computer Engineering and Applications, 2019, 55(11):209-212. doi: 10.3778/j.issn.1002-8331.1811-0239
[30]	贺小伟, 徐靖杰, 王宾, 等. 基于GRU-LSTM组合模型的云计算资源负载预测研究[J]. 计算机工程, 2022, 48(5):11-17. doi: 10.19678/j.issn.1000-3428.0062452
	HE Xiaowei, XU Jingjie, WANG Bin, et al. Research on Cloud Computing Resource Load Prediction Based on Combined GRU-LSTM Model[J]. Computer Engineering, 2022, 48(5):11-17. doi: 10.19678/j.issn.1000-3428.0062452

字段	类型	说明
QUEUE_ID	int	队列标识,每个ID代表一个惟一的队列
CU	int	队列规格,不同规格的资源大小不一样。1CU为1核4 GB
STATUS	string	队列状态,当前队列的状态是否可用
QUEUE_TYPE	string	队列类型,不同类型适用于不同的任务,常见的有通用队列(general)和SQL队列
PLATFORM	string	队列平台,创建队列的机器平台
CPU_USAGE	int	CPU使用率,集群中各机器节点的CPU平均使用率
MEM_USAGE	int	内存使用率,集群中各机器节点的内存平均使用率
LAUNCHING_JOB_NUMS	int	提交中的作业数,即正在等待执行的作业
RUNNING_JOB_NUMS	int	运行中的作业数
SUCCEED_JOB_NUMS	int	已完成的作业数
CANCELLED_JOB_NUMS	int	已取消的作业数
FAILED_JOB_NUMS	int	已失败的作业数
DOTTING_TIME	bignt	采集时间,每5 min进行一次采集
RESOURCE_TYPE	string	资源类型,创建队列的机器类型
DISK_USAGE	int	磁盘使用

分组	KFOLD折数	BiGRU网络隐藏层数	数据映射维度	随机失活率	模型级联层数	M_MSE	M_MAE
Base	5	2	512	0.2	1	0.463	0.384
A	0					0.487	0.396
	10					0.503	0.402
B		1		0.4		0.472	0.391
		4		0.5		0.481	0.394
C			128			0.497	0.398
			1 024			0.523	0.412
D		1		0.0		0.475	0.392
		3		0.1		0.507	0.403
E					2	0.537	0.422
					3	0.586	0.437

预测模型	M_MSE	M_MAE	训练时间/s
BiGRU-LGB	0.463	0.384	53.526
GRU	1.108	0.832	23.241
LightGBM	0.615	0.505	31.241
BiGRU	0.702	0.596	46.874
Transformer	0.515	0.408	58.952
GRU-LSTM	0.583	0.436	65.427