基于Transformer编码器的金融文本情感分析方法

doi:10.16180/j.cnki.issn1007-7820.2020.09.002

摘要/Abstract

摘要：

目前针对文本情感分析的研究大多集中在商品评论和微博的情感分析领域,对金融文本的情感分析研究较少。针对该问题,文中提出一种基于Transformer编码器的金融文本情感分析方法。Transformer编码器是一种基于自注意力机制的特征抽取单元,在处理文本序列信息时可以把句中任意两个单词联系起来不受距离限制,克服了长程依赖问题。文中所提方法使用Transformer编码器构建情感分析网络。Transformer编码器采用多头注意力机制,对同一句子进行多次计算以捕获更多的隐含在上下文中的语义特征。文中在以金融新闻为基础构建的平衡语料数据集上进行实验,并与以卷积神经网络和循环神经网络为基础构建的模型进行对比。实验结果表明,文中提出的基于Transformer编码器的方法在金融文本情感分析领域效果最好。

关键词: 情感分析, 金融, 自注意力机制, Transformer编码器, 缩放点积注意力, 多头注意力

Abstract:

Sentiment analysis plays an important role in many fields, but most of these studies focus on the field of commodity reviews and microblogs, and lack of sentiment analysis of financial texts. To solve this problem, a financial text sentiment analysis method based on Transformer encoder is presented in this paper. Transformer encoder is a feature extraction unit based on the self-attention mechanism. When Transformer encoder processes text sequence information, it can link any two words in a sentence without distance restriction, which overcomes the problem of long-term dependencies. The multi-head attention mechanism is used to calculate the same sentence several times and capture more semantic features implied in the context. The experiment is car+ried out on a balanced corpus data set based on financial news. The experimental results show that the method based on Transformer encoder has best effect in the field of financial text sentiment analysis compared with the models based on convolution neural network and recurrent neural network.

Key words: sentiment analysis, finance, self-attention mechanism, transformer encoder, scaled dot-product attention, multi-head attention

中图分类号:

TP391

李福鹏,付东翔. 基于Transformer编码器的金融文本情感分析方法[J]. 电子科技, 2020, 33(9): 10-15.

LI Fupeng,FU Dongxiang. Sentiment Analysis Method of Financial Text Based on Transformer Encoder[J]. Electronic Science and Technology, 2020, 33(9): 10-15.

图/表 8

图1

图2

图3

图4

表1

表2

图5

表3

参考文献 17

[1]	付婧锐. 浅析行为金融学[J]. 中外企业家, 2019(2):51-52.
	Fu Jingrui. Analysis of behavioral finance[J]. Chinese& Foreign Entrepreneurs, 2019(2):51-52.
[2]	黄进. 金融领域中基于UGC的情感分析[D]. 上海:华东理工大学, 2014.
	Huang Jin. Sentiment analysis based on UGC in financial domain[D]. Shanghai:East China University of Science and Technology, 2014.
[3]	张道玲. 情感分析方法在金融语料库中的应用[D]. 广州:暨南大学, 2018.
	Zhang Daoling. The application of sentiment analysis in financial corpus[D]. Guangzhou:Jinan University, 2018.
[4]	Pak A, Paroubek P. Twitter as a corpus for sentiment analysis and opinion mining[C]. Valletta:Proceedings of the International Conference on Language Resources and Evaluation, 2010.
[5]	缪广寒. 基于Word2vec和SVM的微博情感挖掘与仿真分析[J]. 电子科技, 2018,31(5):81-83.
	Miu Guanghan. Emotion mining and simulation analysis of microblogging based on Word2vec and SVM[J]. Electronic Science and Technology, 2018,31(5):81-83.
[6]	孙建旺, 吕学强, 张雷瀚. 基于词典与机器学习的中文微博情感分析研究[J]. 计算机应用与软件, 2014,31(7):177-181.
	Sun Jianwang, Lü Xueqiang, Zhang Leihan. Sentiment analysis of Chinese microblogging based on lexicon and machine learning[J]. Computer Applications and Software, 2014,31(7):177-181.
[7]	胡航丽, 莫倩. 利用篇章结构改进股评观点分类的研究[J]. 小型微型计算机系统, 2009,30(5):899-902.
	Hu Hangli, Mo Qian. Research on opinion classification of stock recommendations based on discourse structure[J]. Journal of Chinese Computer Systems, 2009,30(5):899-902.
[8]	罗明, 黄海量. 一种基于语义标注特征的金融文本分类方法[J]. 计算机应用研究, 2018,35(8):47-50,54.
	Luo Ming, Huang Hailiang. New approach of financial text classification based on semantic annotation features[J]. Application Research of Computers, 2018,35(8):47-50,54.
[9]	Kim Y. Convolutional neural networks for sentence classification[C]. Doha:Conference on Empirical Methods in Natural Language Processing, 2014.
[10]	尹光花, 刘小明, 张露, 等. 基于LSTM特征模板的短文本情感要素分析与研究[J]. 电子科技, 2018,31(11):38-41,46.
	Yin Guanghua, Liu Xiaoming, Zhang Lu, et al. Sentiment elements of internet short texts for analysis and research based on LSTM network mode[J]. Electronic Science and Technology, 2018,31(11):38-41,46.
[11]	郑国伟, 吕学强, 夏红科, 等. 基于LSTM的金融新闻倾向性[J]. 计算机工程与设计, 2018,39(11):170-175.
	Zheng Guowei, Lü Xueqiang, Xia Hongke, et al. Financial news tendency based on LSTM[J]. Computer Engineering and Design, 2018,39(11):170-175.
[12]	王子牛, 吴建华, 高建瓴, 等. 基于深度神经网络和LSTM的文本情感分析[J]. 软件, 2018,39(12):19-22.
	Wang Ziniu, Wu Jianhua, Gao Jianling, et al. Textsenti-ment analysis based on deep neural network and LSTM[J]. Computer Engineering & Software, 2018,39(12):19-22.
[13]	李洋, 董红斌. 基于CNN和BiLSTM网络特征融合的文本情感分析[J]. 计算机应用, 2018,38(11):3075-3080.
	Li Yang, Dong Hongbin. Text sentiment analysis based on feature fusion of convolution neural network and bidir-ectional long short-term memory network[J]. Journal of Computer Applications, 2018,38(11):3075-3080.
[14]	Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[EB/OL]. (2017-06-12)[2019-05-11]. https://arxiv.org/abs/1706.03762.
[15]	杜慧, 俞晓明, 刘悦, 等. 融合词性和注意力的卷积神经网络对象级情感分类方法[J]. 模式识别与人工智能, 2018,31(12):62-68.
	Du Hui, Yu Xiaoming, Liu Yue, et al. CNN with part-of-speech and attention mechanism for targeted sentiment classification[J]. Pattern Recognition and Artificial Intell-igence, 2018,31(12):62-68.
[16]	Jimmy Lei Ba, Jamie Ryan Kiros. Layer normali-zation[EB/OL]. (2016-06-21)[2019-05-15]. https://arxiv.org/abs/1607.06450.
[17]	胡荣磊, 芮璐, 齐筱, 等. 基于循环神经网络和注意力模型的文本情感分析[J]. 计算机应用研究, 2019,36(11):3282-3285.
	Hu Ronglei, Rui Lu, Qi Xiao, et al. Text sentiment analysis based on recurrent neural networks and attention model[J]. Application Research of Computers, 2019,36(11):3282-3285.

标签	评价	内容
1	费用率影响净利润释放,长期估值仍具优势	中国天楹(000035)公司公告,15年前三季度实现收入4.6亿,同比增长61.31%,实现扣非净利润1.17亿,同比增长18.97%,并预告2015年全年增长26.85%~43.99%。投资要点:费用率持续攀升导致利润释放低于预期,明年一季度有望缓解。
0	宝钢净利创18年以来新低	宝钢净利创18年以来新低净利下滑超八成,公司董事、总经理戴志浩薪酬降幅达88.6%,巨额汇兑损失成业绩下滑主因之一新京报讯宝钢股份3月31日发布2015年年报显示,公司2015年1月~12月实现净利润10.13亿元,创下18年以来的新低,同比上一年度也大幅下降82.51%。

参数	数值
嵌入维度	200
学习率	0.001
多头注意力的Dropout	0.900
Sentence Length	400
Batch Size	200

模型	precision	recall	F₁
textCNN	0.968 63	0.925 83	0.946 75
CRNN	0.950 05	0.944 25	0.947 14
Bi-LSTM	0.975 67	0.863 48	0.916 16
Transformer	0.961 80	0.978 50	0.970 10