情感维度下的深度情感关联模型

doi:10.19665/j.issn1001-2400.2019.05.004

摘要/Abstract

摘要：

鉴于现有的情感模型只是从空间上对情感状态进行划分,忽略了情感之间的相互作用问题,建立了一种将多层限制玻尔兹曼机和情感关联认知网络相结合的深度情感关联模型。该模型将多层限制玻尔兹曼机训练得到的权值作为关联认知网络输入输出之间的权值,以三维情感模型中情感空间距离的倒数作为情感类别之间的关联度,通过训练关联认知网络得到最终的情感分类结果。选用TYUT1.0和CASIA情感语音库中的“高兴”“生气”和“中性”三种基本情感作为数据来源,分别采用深度信念网络和深度情感关联模型进行实验对比。实验结果显示,所构建的深度情感关联模型比深度信念网络的平均识别率最高高出6.06%,该模型得到了较好的识别结果。结果表明, 深度情感关联模型在语音情感识别上有较强的优越性和普适性,可以很好地反映情感之间的相互作用。

关键词: 限制玻尔兹曼机, 关联认知网络, 深度情感关联模型, 情感识别

Abstract:

In view of the fact that the existing emotion model only divides the emotional state from the space with the interaction between emotions neglected. Therefore, this paper proposes a deep emotion association model which combines the multi-layer Restricted Boltzmann Machine with the emotion interactive cognitive network. In this model, the weights trained by the multi-layer Restricted Boltzmann Machine are used as the weights between the inputs and outputs of the correlated cognitive network. The reciprocal of the emotional space distance in the three-dimensional PAD emotional model is used as the correlation degree between emotional categories. The result of emotion classification is obtained by training the interactive cognitive network. The three basic emotions of “happy”, “angry” and “neutral” in TYUT1.0 and CASIA emotional speech database are selected as data sources, and the Deep Belief Network and deep emotional association model are used for emotional recognition. Experimental results show that the average recognition rate of the deep emotional association model is 6.06% higher than that of the Deep Belief Network. It has a better emotional recognition performance. The results prove that the deep emotion association model has strong superiority and universality in speech emotion recognition, and can reflect the interaction between emotions well.

Key words: restricted Boltzmann machine, interactive cognitive network, deep affective interaction model, emotion recognition

中图分类号:

TP391.4

孙颖,吕慧芬,张雪英,马江河. 情感维度下的深度情感关联模型[J]. 西安电子科技大学学报, 2019, 46(5): 24-30.

SUN Ying,LV Huifen,ZHANG Xueying,MA Jianghe. Model of deep affective interaction in the emotional dimension[J]. Journal of Xidian University, 2019, 46(5): 24-30.

图/表 12

图1

图2

表1

图3

图4

表2

表3

图5

图6

表4

表5

图7

参考文献 20

[1]	KAMIńSKA D, TOMASZ S . Polish Emotional Speech Recognition Based on the Committee of Classifiers[J]. Przeglad Elektrotechniczny, 2017,2016(6):101-106.
[2]	陈月芬, 崔跃利, 王三秀 . 基于生理信号的情感识别技术综述[J]. 系统仿真技术, 2017,13(1):1-5.
	CHEN Yuefen, CUI Yueli, WANG Sanxiu . Review of Emotion Recognition Based on Physiological Signals[J]. System Simulation Technology, 2017,13(1):1-5.
[3]	LEKDIOUI K, MESSOUSSI R, RUICHEK Y , et al. Facial Decomposition for Expression Recognition Using Texture/Shape Descriptors and SVM Classifier[J]. Signal Processing: Image Communication, 2017,58:300-312.
[4]	ZHANG S, LIU G, LAI X . Classification of Evoked Emotions Using an Artificial Neural Network Based on Single, Short-Term Physiological Signals[J]. Journal of Advanced Computational Intelligence and Intelligent Informatics, 2015,19(1):118-126.
[5]	QUAN C, REN F . Weighted High-order Hidden Markov Models for Compound Emotions Recognition in Text[J]. Information Sciences, 2016,329:581-596.
[6]	LIU J, LIU T, CHEN J . Sequential Local-based Gaussian Mixture Model for Monitoring Multiphase Batch Processes[J]. Chemical Engineering Science, 2018,181:101-113.
[7]	ZHU L, CHEN L, ZHAO D , et al. Emotion Recognition from Chinese Speech for Smart Affective Services Using a Combination of SVM and DBN[J]. Sensors, 2017,17(7):1694.
[8]	RUIZ-GARCIA A, ELSHAW M, ALTAHHAN A , et al. A Hybrid Deep Learning Neural Approach for Emotion Recognition from Facial Expressions for Socially Assistive Robots[J]. Neural Computing and Applications, 2018,29(7):359-373.
[9]	ZHANG S, ZHANG S, HUANG T , et al. Learning Affective Features with a Hybrid Deep Model for Audio-Visual Emotion Recognition[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2018,28(10):3030-3043.
[10]	GUNES H, SCHULLER B, PANTIC M , et al. Emotion Representation, Analysis and Synconfproc in Continuous Space: A Survey[C]// Proceedings of the 2011 IEEE International Conference on Automatic Face and Gesture Recognition and Workshops. Washington: IEEE Computer Society, 2011: 827-834.
[11]	BANDARA D, VELIPASALAR S, BRATT S , et al. Building Predictive Models of Emotion with Functional Near-Infrared Spectroscopy[J]. International Journal of Human Computer Studies, 2018,110:75-85.
[12]	FENG S, CHEN C L P . A Fuzzy Restricted Boltzmann Machine: Novel Learning Algorithms Based on the Crisp Possibilistic Mean Value of Fuzzy Numbers[J]. IEEE Transactions on Fuzzy Systems, 2018,26(1):117-130.
[13]	ZHANG K, LIU J, CHAI Y , et al. An Optimized Dimensionality Reduction Model for High-dimensional Data Based on Restricted Boltzmann Machines[C]// Proceedings of the 2015 27th Chinese Control and Decision Conference. Piscataway: IEEE, 2015: 2939-2944.
[14]	宋静, 张雪英, 孙颖 , 等. 基于PAD情绪模型的情感语音识别[J]. 微电子学与计算机, 2016,33(9):128-131.
	SONG Jing, ZHANG Xueying, SUN Ying , et al. Emotional Speech Recognition Based on PAD Emotion Model[J]. Microelectronics & Computer, 2016,33(9):128-131.
[15]	LI X, ZHOU H, SONG S , et al. The Reliability and Validity of the Chinese Version of Abbreviated PAD Emotion Scales[C]// Lecture Notes in Computer Science: 3784. Heidelberg: Springer Verlag, 2005: 513-518.
[16]	孙颖, 张雪英 . 情感语音特征对语料库依赖性的统计分析[J]. 噪声与振动控制, 2011,31(4):132-136.
	SUN Ying, ZHANG Xueying . Statistical Analysis for Database Dependence in Classification of Emotional Speech by Using Different Features Extraction Approaches[J]. Noise and Vibration Control , 2011,31(4):132-136.
[17]	TAO J H, LIU F Z, ZHANG M , et al. Design of Speech Corpus for Mandarin Text to Speech[CA/OL]. [2018-12-20]..
[18]	孙颖, 宋春晓 . 相空间重构的情感语音特征提取及优化[J]. 西安电子科技大学学报, 2017,44(6):162-168.
	SUN Ying, SONG Chunxiao . Emotional Speech Feature Extraction and Optimization of Phase Space Reconstruction[J]. Journal of Xidian University, 2017,44(6):162-168.
[19]	SMITH L N . A Disciplined Approach to Neural Network Hyper-parameters: Part 1 -- Learning Rate, Batch Size, Momentum, and Weight Decay[J/OL]. [2018-12-20]..
[20]	高大文, 王鹏, 蔡臻超 . 人工神经网络中隐含层节点与训练次数的优化[J]. 哈尔滨工业大学学报, 2003,35(2):207-209.
	GAO Dawen, WNAG Peng, CAI Zhenchao . Optimization of Hidden Nodes and Training Times in Artificial Neural Network[J]. Journal of Harbin Institute of Technology, 2003,35(2):207-209.

编号	情感		均值
编号	情感	P	A	D
1	高兴	0.66	0.74	0.32
2	生气	-0.86	0.66	0.91
3	中性	0	-0.31	-0.06

特征类型	特征维数	统计量
韵律特征	1-38	语速、平均过零率、能量、基频、共振峰
MFCC特征	39-98	MFCC前12阶的偏度、峰度、均值、方差、中值
非线性几何	99-157	吸引子到圆心的距离、吸引子连续轨长度、吸引子连续轨角度、吸引子到标识线距离、吸引子轨迹之间的总长度
非线性属性	158-180	Hurst指数、关联维数、最小延迟时间、Kolmogorov熵、最大Lyapunov指数

情感语音库	样本	情感类型			总计
情感语音库	样本	高兴	生气	中性	总计
TYUT1.0	训练	130	160	240	530
	测试	57	69	104	230
	总计	187	229	344	760
CASIA	训练	130	140	130	400
	测试	56	54	55	165
	总计	186	194	185	565

训练次数	情感类型						平均
	高兴		生气		中性		平均
	方案1	方案2	方案1	方案2	方案1	方案2	方案1	方案2
200	68.42	77.19	60.87	69.57	94.23	93.27	77.83	82.17
250	73.68	71.93	66.67	72.46	93.27	93.27	80.43	81.74
300	68.42	73.68	63.77	72.46	94.23	93.27	78.70	82.17
350	78.95	70.18	65.22	72.46	88.46	93.27	79.13	81.30
400	75.44	75.44	63.77	71.01	94.23	93.27	80.43	82.17
450	71.93	70.18	63.77	73.91	94.23	93.27	79.57	81.74
500	80.87	71.93	60.87	73.91	89.42	93.27	78.70	82.17
550	78.95	68.42	60.87	75.36	92.31	93.27	79.57	81.74

训练次数	情感类型						平均
	高兴		生气		中性		平均
	方案1	方案2	方案1	方案2	方案1	方案2	方案1	方案2
200	66.07	78.57	90.74	88.89	76.36	81.82	77.58	83.03
250	75.00	78.57	90.74	85.19	72.73	80.00	79.39	81.21
300	73.21	76.79	90.74	92.59	70.91	81.82	78.18	83.64
350	66.07	76.79	90.74	90.74	74.55	81.82	76.97	83.03
400	73.21	75.00	90.74	83.33	72.73	83.64	78.79	80.61
450	67.86	82.14	88.89	85.19	74.55	81.82	76.97	83.03
500	76.79	75.00	88.89	88.89	72.73	78.18	79.39	80.61
550	82.14	82.14	85.19	85.19	74.55	80.00	80.61	82.42