可实现双向自适应差分隐私的联邦学习方案

doi:10.19665/j.issn1001-2400.20230706

Abstract

Abstract:

With the explosive growth of personal data,the federated learning based on differential privacy can be used to solve the problem of data islands and preserve user data privacy.Participants share the parameters with noise to the central server for aggregation by training local data,and realize distributed machine learning training.However,there are two defects in this model:on the one hand,the data information in the process of parameters broadcasting by the central server is still compromised,with the risk of user privacy leakage;on the other hand,adding too much noise to parameters will reduce the quality of parameter aggregation and affect the model accuracy of federated learning.In order to solve the above problems,a bidirectional adaptive differential privacy federated learning scheme(Federated Learning Approach with Bidirectional Adaptive Differential Privacy,FedBADP) is proposed,which can adaptively add noise to the gradients transmitted by participants and central servers,and keep data security without affecting the model accuracy.Meanwhile,considering the performance limitations of the participants hardware devices,this model samples their gradients to reduce the communication overhead,and uses the RMSprop to accelerate the convergence of the model on the participants and central server to improve the accuracy of the model.Experiments show that our novel model can enhance the user privacy preserving while maintaining a good accuracy.

Key words: bidirectional adaptive noise, RMSprop, sampling, differential privacy, federated learning

CLC Number:

TP309

LI Yang, XU Jin, ZHU Jianming, WANG Youwei. Bidirectional adaptive differential privacy federated learning scheme[J].Journal of Xidian University, 2024, 51(3): 158-169.

Figures/Tables 7

References 25

[1]	MCMAHAN H B, MOORE E, RAMAGE D, et al. Communication-Efficient Learning of Deep Networks from Decentralized Data[C]//Proceedings of the 20th International Conference on Artificial Intelligence and Statistics. New York: PMLR, 2017:1273-1282.
[2]	JIANG J, KANTARCI B, OKTUG S F, et al. Federated Learning in Smart City Sensing:Challenges and Opportunities[J]. Sensors, 2020, 20(21):6230.
[3]	XU J, GLICKSBERG B S, SU C, et al. Federated Learning for Healthcare Informatics[J]. Journal of Healthcare Informatics Research 5, 2021, 5(1):1-19.
[4]	TU X, ZHU K, LUONG N C, et al. Incentive Mechanisms for Federated Learning:From Economic and Game Theoretic Perspective(2021)[J/OL].[2021-11-20]. https://arxiv.org/abs/2111.11850v1.
[5]	陈嘉翊, 孙晨雨, 周欣桐, 等. 基于联邦学习和同态加密的电力数据预测模型本地保护[J]. 信息安全研究, 2023, 9(3):228-234.
	CHEN Jiayi, SUN Chenyu, ZHOU Xintong, et al. Local Privacy Protection for Power Data Prediction Model Based on Federated Learning and Homomorphic Encryption[J]. Information Security Research, 2023, 9(3):228-234.
[6]	徐花, 田有亮. 差分隐私下的权重社交网络隐私保护[J]. 西安电子科技大学学报, 2022, 49(1):17-25.
	XU Hua, TIAN Youliang. Protection of Privacy of the Weighted Social Network under Differential Privacy[J]. Journal of Xidian University, 2022, 49(1):17-25.
[7]	WANG F, XIE M, TAN Z, et al. Preserving Differential Privacy in Deep Learning Based on Feature Relevance Region Segmentation[J]. IEEE Transactions on Emerging Topics in Computing, 2023, 12(1):307-315.
[8]	FU J, CHEN Z, HAN X. Adap DP-FL:Differentially Private Federated Learning with Adaptive Noise[C]//2022 IEEE International Conference on Trust,Security and Privacy in Computing and Communications(TrustCom).Piscataway:IEEE, 2022:656-663.
[9]	粟勇, 刘文龙, 刘圣龙, 等. 基于安全洗牌和差分隐私的联邦学习模型安全防护方法[J]. 信息安全研究, 2022, 8(3):270-276.
	SU Yong, LIU Wenlong, LIU Shenglong, et al. Secure Protection Method for Federated Learning Model Based on Secure Shuffling and Differential Privacy[J]. Information Security Research, 2022, 8(3):270-276.
[10]	晏燕, 董卓越, 徐飞, 等. 一种Hilbert编码的本地化位置隐私保护方法[J]. 西安电子科技大学学报, 2022, 50(2):147-160.
	YAN Yan, DONG Zhuofei, XU Fei, et al. Localized Location Privacy Protection Method Using the Hilbert Encoding[J]. Journal of Xidian University, 2022, 50(2):147-160.
[11]	KONEN J, MCMAHAN H B, YU F X, et al. Federated Learning:Strategies for Improving Communication Efficiency(2016)[J/OL].[2016-10-18].https://arxiv.org/abs/1610.05492.
[12]	LI T, SAHU A K, ZAHEER M, et al. Federated Optimization in Heterogeneous Networks(2018)[J/OL].[2018-12-14]. https://arxiv.org/abs/1812.06127v5.
[13]	LIU W, CHEN L, CHEN Y, et al. Accelerating Federated Learning via Momentum Gradient Descent[J]. IEEE Transactions on Parallel and Distributed Systems, 2020, 31(8):1754-66.
[14]	LIU X, LI Y, WANG Q, et al. Sparse Personalized Federated Learning via Maximizing Correlation(2021)[J/OL].[2021-07-12]. https://arxiv.org/abs/2107.05330v3.
[15]	MELIS L, SONG C, CRISTOFARO E D, et al. Inference Attacks Against Collaborative Learning(2018)[J/OL].[2018-05-10]. https://arxiv.org/abs/1805.04049v1.
[16]	ABADI M, CHU A, GOODFELLOW I, et al. Deep Learning with Differential Privacy[C]//Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. New York: ACM, 2016:308-318.
[17]	WEI K, LI J, DING M, et al. Federated Learning with Differential Privacy:Algorithms and Performance Analysis[J]. IEEE Transactions on Information Forensics and Security, 2020, 15:3454-3469.
[18]	WEI K, LI J, DING M, et al. User-Level Privacy-Preserving Federated Learning:Analysis and Performance Optimization[J]. IEEE Transactions on Mobile Computing, 2022, 21(9):3388-3401.
[19]	WU X, ZHANG Y, SHI M, et al. AnAdaptive Federated Learning Scheme with Differential Privacy Preserving[J]. Future Generation Computer Systems, 2022, 127:362-72.
[20]	XU Z, SHI S, LIU A X, et al. An Adaptive and Fast Convergent Approach to Differentially Private Deep Learning[C]//Proceedings of the IEEE INFOCOM 2020:International Conference on Computer Communications(INFOCOM). Piscataway:IEEE, 2020:1867-1876.
[21]	KUMAR G, PRIYA G, DILEEP M, et al. Image Deconvolution using Deep Learning-based Adam Optimizer[C]//2022 6th International Conference on Electronics,Communication and Aerospace Technology.Piscataway:IEEE, 2022:901-904.
[22]	XIANG L, YANG J, LI B. Differentially-Private Deep Learning from an Optimization Perspective[C]//Proceedings of the IEEE Conference on Computer Communications. Piscataway:IEEE, 2019:559-567.
[23]	DWORK C. Differential Privacy[C].Proceedings of the 33rd International Conference on Automata,Languages and Programming. Heidelberg:Springer, 2006:1-10.
[24]	LI N, LYU M, SU D, et al. Differential Privacy:From Theory to Practice[J]. Synthesis Lectures on Information Security,Privacy,and Trust, 2016, 8(4):1-138.
[25]	GEYER R C, KLEIN T, NABI M. Differentially Private Federated Learning:A Client Level Perspective(2017)[J/OL].[2017-12-20]. https://arxiv.org/abs/1712.07557v2.

文献	加噪方式
FedBADP	自适应双向加噪
ABADI等^[16]	客户端单向加噪
WEI K等^[17]	双向加噪
KUMAR G等^[21]	中心服务器加噪
XIANG L等^[22]	自适应加噪

算法	通信轮数	客户端采样率	样本采样率	准确率/%	差分隐私
DP-SGD^[16] FedBADP1	700 81	1.00 0.22	0.01 0.10	97.00 96.85	(8,1e-5)-DP (8,1e-288)-DP
CRD^[17] FedBADP2	30 32	0.60 0.50	1.00 0.10	90.00 90.91	(8,1e-5)-DP (8,1e-288)-DP
DPOPT^[22] FedBADP3	100 60	1.00 1.00	1.00 1.00	96.00 96.29	(5.48,1e-4)-DP (5.48,1e-135)-DP
CDP-FL^[25] FedBADP4	54 42	0.22 0.22	1.00 0.10	92.00 94.40	(8,1e-5)-DP (8,1e-288)-DP

Bidirectional adaptive differential privacy federated learning scheme

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 7

References 25

Related Articles 15

Metrics

Comments

Recommended 0

[1]	ZENG Yong, GUO Xiaoya, MA Baihe, LIU Zhihong, MA Jianfeng. Fine-grained defense methods in federated encrypted traffic classification [J]. Journal of Xidian University, 2024, 51(1): 157-164.
[2]	HUANG Heyuan, MU Caihong, FANG Yunfei, LIU Yi. Graph convolution neural network for recommendation using graph negative sampling [J]. Journal of Xidian University, 2024, 51(1): 86-99.
[3]	DING Hongfa,FU Peiwang,PENG Changgen,LONG Shigong,WU Ningbo. Histogram publishing algorithm for degree distribution via shuffled differential privacy [J]. Journal of Xidian University, 2023, 50(6): 219-236.
[4]	WANG Bo,LI Hongtao,WANG Jie,GUO Yina. Federated learning scheme for privacy-preserving of medical data [J]. Journal of Xidian University, 2023, 50(5): 166-177.
[5]	SONG Cheng,CHENG Daochen,PENG Weiping. Efficient federated learning privacy protection scheme [J]. Journal of Xidian University, 2023, 50(5): 178-187.
[6]	WANG Fangwei,XIE Meiyun,LI Qingru,WANG Changguang. Differentially private federated learning framework with adaptive clipping [J]. Journal of Xidian University, 2023, 50(4): 111-120.
[7]	LI Haiyang,GUO Jingjing,LIU Jiuzun,LIU Zhiquan. Privacy preserving byzantine robust federated learning algorithm [J]. Journal of Xidian University, 2023, 50(4): 121-131.
[8]	WANG Yuhua,GAO Sheng,ZHU Jianming,HUANG Chen. Efficient deep learning scheme with adaptive differential privacy [J]. Journal of Xidian University, 2023, 50(4): 54-64.
[9]	XU Mengfan,LI Xinghua. Privacy-preserving federated learning with non-transfer learning [J]. Journal of Xidian University, 2023, 50(4): 89-99.
[10]	YAN Yan,DONG Zhuoyue,XU Fei,FENG Tao. Localized location privacy protection method using the Hilbert encoding [J]. Journal of Xidian University, 2023, 50(2): 147-160.
[11]	QIN Rui,ZHANG Wei. Multi-scale fire detection algorithm with an anchor free structure [J]. Journal of Xidian University, 2022, 49(6): 111-119.
[12]	HE Wangpeng,CHEN Binqiang,LI Yang,CHEN Jing,GUO Baolong. Compressed sensing of subsampled dynamic signals during high-speed machining [J]. Journal of Xidian University, 2022, 49(4): 82-89.
[13]	XU Hua,TIAN Youliang. Protection of privacy of the weighted social network under differential privacy [J]. Journal of Xidian University, 2022, 49(1): 17-25.
[14]	HE Yunbin,LENG Xin,WAN Jing. Unbalanced data weighted boundary point integration undersampling method [J]. Journal of Xidian University, 2021, 48(4): 176-183.
[15]	SUN Zhengyang,DONG Mei,CHEN Baixiao. Interrupted sampling repeater jamming suppression based on time-frequency analysis and band-pass filtering [J]. Journal of Xidian University, 2021, 48(2): 139-146.