一种新的直觉模糊最小二乘支持向量机

doi:10.19665/j.issn1001-2400.2022.05.015

摘要/Abstract

摘要：

最小二乘支持向量机只需求解一个线性方程组即可得到闭式解,训练速度快,因而广泛应用于分类问题。但最小二乘支持向量机模型容易受到离群点和噪声的影响,往往使得其分类精度下降,样本点的模糊加权是解决该问题的一种有效方法。直觉模糊集既包含样本点的隶属度信息又包含样本点的非隶属度信息,可以更加详细地刻画样本点的分布特征。为此,基于直觉模糊集,通过剔除离群点得到更加准确的类中心,再计算样本点到类中心的距离,得到样本点对其所在类的隶属度程度。同时,采用核k近邻的方法,查找样本点的k个近邻里另一类样本点的数量,进而得到样本点的非隶属度信息。最后,根据样本点的隶属度与非隶属度得到一种新的模糊值。进一步将提出的模糊值用于改进最小二乘支持向量机模型,通过赋予离群点和噪声低的模糊值,减少了它们对最小二乘支持向量机模型的影响,提高了最小二乘支持向量机模型的精度。实验结果表明,与已有算法对比,提出的算法可以有效降低离群点和噪声对最小二乘支持向量机模型的影响,提高模型的鲁棒性。

关键词: 最小二乘支持向量机, 模糊集, 离群点和噪声, k近邻

Abstract:

The least square support vector machine only needs to solve a linear system of equations to get a closed solution,so it is widely used in classification problems because of its fast training speed.However,the least squares support vector machine model is easily affected by outliers and noise,which often reduces the classification accuracy.Fuzzy weighting of sample points is an effective method to solve this problem.Intuitionistic fuzzy sets contain both membership information and non-membership information of sample points,which can describe the distribution characteristics of sample points in more detail.Therefore,based on the intuitionistic fuzzy set,this paper obtains a more accurate class center by eliminating outliers,and then calculates the distance between the sample point and the class center to obtain the membership degree of the sample point to its class.At the same time,the kernel k-nearest neighbor method is used to find the number of k neighboring sample points of another class,and then the non-membership information of sample points is obtained.Finally,a new fuzzy value is obtained according to the membership degree and non-membership degree of sample points.Furthermore,the proposed fuzzy values are used to improve the LSSVM model.By assigning outliers and fuzzy values with low noise,their influence on the LSSVM model is reduced and the accuracy of the LSSVM model is improved.Experimental results show that,compared with the existing algorithms,the proposed algorithm can reduce the influence of outliers and noise on the LSSVM model and improve the robustness of the model.

Key words: least square support vector machine, fuzzy sets, outliers and noise, k-nearest neighbor

中图分类号:

TP391

张丹,周水生,张文梦. 一种新的直觉模糊最小二乘支持向量机[J]. 西安电子科技大学学报, 2022, 49(5): 125-136.

ZHANG Dan,ZHOU Shuisheng,ZHANG Wenmeng. New intuitionistic fuzzy least squares support vector machine[J]. Journal of Xidian University, 2022, 49(5): 125-136.

图/表 8

图1

图2

图3

表1

表2

图4

表3

图5

参考文献 24

[1]	郭泽坤, 田隆, 韩宁, 等. 采用CNN-SSD的雷达HRRP小样本目标识别方法[J]. 西安电子科技大学学报, 2021, 48(2):7-14.
	GUO Zekun, TIAN Long, HAN Ning, et al. CNN-SSD Radar HRRP Small Sample Target Recognition Method is Adopted[J]. Journal of Xidian University, 2021, 48(2):7-14.
[2]	王俊祥, 黄霖, 张影, 等. 低复杂度的增强图像来源检测算法[J]. 西安电子科技大学学报, 2021, 48(1):96-106.
	WANG Junxiang, HUANG Lin, ZHANG Ying, et al. Low Complexity Enhanced Image Source Detection Algorithm[J]. Journal of Xidian University, 2021, 48(1):96-106.
[3]	MEHMOOD Z, ASGHAR S. Customizing SVM as A Base Learner with AdaBoost Ensemble to Learn from Multi-Class Problems:A Hybrid Approach AdaBoost-MSVM[J]. Knowledge-Based Systems, 2021, 217:106845. doi: 10.1016/j.knosys.2021.106845
[4]	MOODI M, GHAZVINI M, MOODI H. A Hybrid Intelligent Approach to Detect Android Botnet Using Smart Self-Adaptive Learning-Based PSO-SVM[J]. Knowledge-Based Systems, 2021, 222:106988. doi: 10.1016/j.knosys.2021.106988
[5]	SUYKENS J, VANDEWALLE J. Least Squares Support Vector Machine Classifiers[J]. Neural Processing Letters, 1999, 9(3):293-300. doi: 10.1023/A:1018628609742
[6]	YANG A, LI W, YANG X. Short-Term Electricity Load Forecasting Based on Feature Selection and Least Squares Support Vector Machine[J]. Knowledge-Based Systems, 2019, 163:159-173. doi: 10.1016/j.knosys.2018.08.027
[7]	HE C, WU T, LIU C, et al. A Novel Method of Composite Multiscale Weighted Permutation Entropy and Machine Learning for Fault Complex System Fault Diagnosis[J]. Measurement, 2020, 158:107748. doi: 10.1016/j.measurement.2020.107748
[8]	WANG J, SHAO W, JUNSEOK K. Combining MF-DFA and LSSVM for Retina Images Classification[J]. Biomedical Signal Processing and Control, 2020, 60:101943. doi: 10.1016/j.bspc.2020.101943
[9]	DONG L, LIAO J. Wavelet Kernel Function Based Multiscale LSSVM for Elliptic Boundary Value Problems[J]. Neurocomputing, 2019, 356:40-51. doi: 10.1016/j.neucom.2019.04.076
[10]	SUYKENS J A K, DE BRABANTER J, LUKAS L, et al. Weighted Least Squares Support Vector Machines:Robustness and Sparse Approximation Combining MF-DFA and LSSVM for Retina Images Classification[J]. Neurocomputing, 2002, 48(1-4):85-105. doi: 10.1016/S0925-2312(01)00644-0
[11]	YANG X, TAN L, HE L. A Robust Least Squares Support Vector Machine for Regression and Classification with Noise[J]. Neurocomputing, 2014, 140:41-52. doi: 10.1016/j.neucom.2014.03.037
[12]	MA J, ZHOU S, LI D. Robust Multiclass Least Squares Support Vector Classifier with Optimal Error Distribution[J]. Knowledge-Based Systems, 2021, 215:106652. doi: 10.1016/j.knosys.2020.106652
[13]	CHEN L, ZHOU S. Sparse Algorithm for Robust LSSVM in Primal Space[J]. Neurocomputing, 2018, 275:2880-2891. doi: 10.1016/j.neucom.2017.10.011
[14]	LIN C, WANG S. Fuzzy Support Vector Machines[J]. IEEE Transactions on Neural Networks. 2002, 13(2):464-471. doi: 10.1109/72.991432
[15]	TANG W. Fuzzy SVM with A New Fuzzy Membership Function to Solve the Two-Class Problems[J]. Neural Processing Letters, 2011, 34(3):209-219. doi: 10.1007/s11063-011-9192-y
[16]	BATUWITA R, PALADE V. FSVM-CIL:Fuzzy Support Vector Machines for Class Imbalance Learning[J]. IEEE Transactions on Fuzzy Systems. 2010, 18 (3):558-571. doi: 10.1109/TFUZZ.2010.2042721
[17]	LIU J. Fuzzy Support Vector Machine for Imbalanced Data with Borderline Noise[J]. Fuzzy Sets and Systems, 2021, 413:64-73. doi: 10.1016/j.fss.2020.07.018
[18]	TAO X, LI Q, REN C, et al. Affinity and Class Probability-Based Fuzzy Support Vector Machine for Imbalanced Data Sets[J]. Neural Networks, 2020, 122:289-307. doi: S0893-6080(19)30338-7 pmid: 31739268
[19]	FAN Q, WANG Z, LI D, et al. Entropy-Based Fuzzy Support Vector Machine for Imbalanced Datasets[J]. Knowledge-Based Systems, 2017, 115:87-99. doi: 10.1016/j.knosys.2016.09.032
[20]	ATANASSOV K. Intuitionistic Fuzzy Sets[J]. Fuzzy Sets and Systems, 1986, 20:87-96. doi: 10.1016/S0165-0114(86)80034-3
[21]	哈明虎, 黄澍, 王超, 等. 直觉模糊支持向量机[J]. 河北大学学报. 2011, 31(3):225-229.
	HA Minghu, HUANG Shu, WANG Chao, et al. Intuitionistic Fuzzy Support Vector Machine[J]. Journal of Hebei University, 2011, 31(3):225-229.
[22]	HA M, WANG C, CHEN J. The Support Vector Machine Based on Intuitionistic Fuzzy Number and Kernel Function[J]. Soft Computing, 2013, 17:635-641. doi: 10.1007/s00500-012-0937-y
[23]	REZVANI S, WANG X, POURPANAH F. Intuitionistic Fuzzy Twin Support Vector Machines[J]. IEEE Transactions on Fuzzy Systems, 2019, 27:2140-2151. doi: 10.1109/TFUZZ.2019.2893863
[24]	HAN J, KAMBER M. Data Mining:Concepts and Techniques[M]. Amsterdam: Elsevier, 2012.

数据集	样本数量	特征数量	正类数量	负类数量
Sonar	208	60	97	111
Breast Cancer	683	10	444	239
Haberman	306	3	225	81
Pima	768	8	268	500
Australian	690	14	307	383
German Number	1 000	24	300	700
Hepatitis	155	19	32	123
Ionosphere	351	34	225	126
Four Class	862	2	307	555
Tictactoe	958	9	626	332
Segment	2 310	19	990	1 320
Satimage	4 508	36	2 600	1 908

数据集	LSSVM	WLSSVM	SRLSSVM	RLSSVC	NIFLSSVM
Sonar	78.05(0.12)	84.15(0.38)	66.34(0.04)	89.27(0.03)	89.75(0.69)
Breast Cancer	96.40(0.00)	96.17(0.00)	96.54(0.00)	96.32(0.01)	96.89(0.00)
Haberman	79.34(0.21)	79.51(0.03)	78.85(0.00)	74.92(0.22)	78.68(0.19)
Pima	75.03(0.14)	77.17(0.00)	76.54(0.01)	77.58(0.00)	78.17(0.06)
Australian	80.58(0.01)	79.13(0.01)	79.13(0.00)	81.67(0.01)	81.88(0.02)
German Number	77.10(0.00)	77.25(0.00)	69.50(0.00)	78.00(0.06)	78.50(0.02)
Hepatitis	75.81(0.12)	78.71(0.09)	74.19(0.00)	77.42(0.02)	79.03(0.40)
Ionosphere	92.00(0.10)	91.57(0.80)	92.86(0.01)	92.86(0.02)	93.14(0.03)
Four Class	98.83(0.81)	98.84(0.01)	98.78(0.00)	99.76(0.00)	99.53(0.01)
Tictactoe	96.71(0.00)	98.43(0.00)	98.43(0.00)	98.95(0.00)	98.95(0.00)
Segment	96.73(0.00)	95.89(0.00)	93.07(0.00)	98.70(0.01)	99.46(0.00)
Satimage	83.60(6.89)	92.00(0.00)	98.71(0.00)	92.10(0.10)	96.71(0.00)
平均精度	83.35	87.40	85.17	88.13	89.22

数据集	FLSSVM		IFLSSVM		NIFLSSVM
数据集	训练准确率/%	训练时间/s	训练准确率/%	训练时间/s	训练准确率/%	训练时间/s
Sonar	89.75(0.07)	1.216 8	88.29(0.10)	0.109 2	89.75(0.69)	0.062 4
Breast Cancer	97.05(0.00)	12.636 1	95.74(0.00)	0.124 8	96.89(0.00)	0.124 8
Haberman	71.48(0.06)	2.308 8	72.13(0.06)	0.062 4	78.68(0.19)	0.015 6
Pima	77.45(0.03)	16.052 5	76.08(0.02)	0.187 2	78.17(0.06)	0.249 6
Australian	80.44(0.00)	12.979 3	81.16(0.02)	0.124 8	81.88(0.02)	0.171 6
German Number	75.60(0.00)	26.879 0	76.10(0.00)	0.265 2	78.50(0.02)	0.374 4
Hepatitis	78.06(0.02)	1.107 6	78.07(0.38)	0.0019	79.03(0.40)	0.062 4
Ionosphere	95.71(0.01)	3.244 8	96.86(0.00)	0.062 4	93.14(0.03)	0.062 4
Four Class	99.07(0.00)	19.531 3	99.19(0.00)	0.234 0	99.53(0.01)	0.280 8
Tictactoe	99.47(0.00)	25.007 0	98.53(0.00)	0.218 4	98.95(0.01)	0.436 8
Segment	95.46(0.00)	255.482 8	97.40(0.00)	4.565 6	99.46(0.00)	5.597 2
Satimage	91.55(0.00)	485.537 5	92.00(0.00)	5.288 4	91.70(0.00)	9.489 6