移动闪存的重复数据删除技术

doi:10.19665/j.issn1001-2400.2020.01.018

Abstract

Abstract:

The performance, capacity and reliability of the flash memory on devices such as smartphones and the Internet of Things are limited. Deduplication can resolve these restrictions by removing the duplicate I/O, but must be done under various resource constraints on the device. This paper proposes the M-Dedupe deduplication, which applies content-aware deduplication I/O requests on critical paths, and improves the performance and efficiency of mobile phones by improving the efficiency of flash garbage collection. The prototype system verification results show that the M-Dedupe reduces the write data by an average of 23.7%~42.5%, the average write response time by 21.2%~39.8%, and the average erased block by 16.8%~43.9%. Besides, it can achieve high-deciding duplication in the mobile flash system, improve the deduplication efficiency, and save the storage space.

Key words: mobile storage, deduplication, flash

CLC Number:

TP393.1

HE Qinlu,BIAN Genqing,SHAO Bilin,ZHANG Weiqi. Deduplication technology for the mobile flash[J].Journal of Xidian University, 2020, 47(1): 128-134.

Figures/Tables 10

References 23

[1]	CHEN R, WANG Y, HU J , et al. VFlash:Virtualized Flash for Optimizing the I/O Performance in Mobile Devices[J]. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2017,36(7):1203-1214.
[2]	LEE C, SIM D, HWANG J Y , et al. F2FS: a New File System for Flash Storage[C]//Proceedings of the 2015 13th USENIX Conference on File and Storage Technologies. Berkeley: USENIX Association, 2015: 273-286.
[3]	HAHN S S, LEE S, JI C , et al. Improving File System Performance of Mobile Storage Systems Using a Decoupled Defragmenter[C]//Proceedings of the 2017 USENIX Annual Technical Conference. Berkeley: USENIX Association, 2017: 759-771.
[4]	CHEN R, WANG Y, HU J , et al. Image-content-aware I/O Optimization for Mobile Virtualization[J]. ACM Transactions on Embedded Computing Systems, 2016,16(1):12.
[5]	WU S, CHEN X, MAO B . Exploiting the Data Redundancy Locality to Improve the Performance of Deduplication-based Storage Systems[C]//Proceedings of the 2016 International Conference on Parallel and Distributed Systems. Washington: IEEE Computer Society, 2016: 527-534.
[6]	UCHIGAITO H, MIURA S, NITO T . Efficient Data-allocation Scheme for Eliminating Garbage Collection during Analysis of Big Graphs Stored in NAND Flash Memory[J]. IEEE Transactions on Computers, 2018,67(5):646-657.
[7]	KIM Y, IMANI M, PATIL S , et al. CAUSE: Critical Application Usage-aware Memory System Using Non-volatile Memory for Mobile Devices[C]//Proceedings of the 2015 IEEE/ACM International Conference on Computer-Aided Design. Piscataway: IEEE, 2016: 690-696.
[8]	JIN Y T, AHN S, LEE S . Performance Analysis of NVMe SSD-based All-flash Array Systems[C]//Proceedings of the 2018 IEEE International Symposium on Performance Analysis of Systems and Software. Piscataway: IEEE, 2018: 12-21.
[9]	CAI Y, GHOSE S, HARATSCH E F , et al. Error Characterization, Mitigation, and Recovery in Flash-memory-based Solid-state Drives[J]. Proceedings of the IEEE, 2017,105(9):1666-1704.
[10]	MAO B, WU S Z, JIANG H , et al. Content-aware Trace Collection and I/O Deduplication for Smartphones[C]//Proceedings of the 33rd International Conference on Massive Storage Systems and Technology. Piscataway: IEEE, 2017: 514-522.
[11]	CHANG L P, SUNG P H, CHEN P T , et al. Eager Synching: A Selective Logging Strategy for Fast fsync on Flash-based Android Devices[J]. ACM Transactions on Embedded Computing Systems, 2016,16(2):34.
[12]	PARK J, LEE S, KIM J . DAC: Dedup-assisted Compression Scheme for Improving Lifetime of NAND Storage Systems[C]//Proceedings of the 2017 Design, Automation and Test in Europe. Piscataway: IEEE, 2017: 1249-1252.
[13]	KHAN A, LEE C G, HAMANDAWANA P , et al. A Robust Fault-tolerant and Scalable Cluster-wide Deduplication for Shared-nothing Storage Systems[C]//Proceedings of the 2018 26th IEEE International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems. Piscataway: IEEE, 2018: 87-93.
[14]	JI C, CHANG L P, SHI L , et al. Lightweight Data Compression for Mobile Flash Storage[J]. ACM Transactions on Embedded Computing Systems, 2017,16(5s):183.
[15]	REN J, LIANG M C J, WU Y , et al. Memory-centric Data Storage for Mobile Systems[C]//Proceedings of the 2015 USENIX Annual Technical Conference. Berkeley: USENIX Association, 2015: 599-611.
[16]	Adler-32算法 [EB/OL]. [2019-03-05].
[17]	WIDODO R N S, LIM H, ATIQUZZAMAN M . SDM: Smart Deduplication for Mobile Cloud Storage[J]. Future Generation Computer Systems, 2017,70:64-73.
[18]	CHEN R, WANG Y, HU J , et al. Image-content-aware I/O Optimization for Mobile Virtualization[J]. ACM Transactions on Embedded Computing Systems, 2016,16(1):12.
[19]	TAVAKKOL A, SADROSADATI M, GHOSE S , et al. FLIN: Enabling Fairness and Enhancing Performance in Modern NVMe Solid State Drives[C]//Proceedings of the 2018 International Symposium on Computer Architecture. Piscataway: IEEE, 2018: 397-410.
[20]	WU F, LU Z, ZHOU Y , et al. OSPADA: One-shot Programming Aware Data Allocation Policy to Improve 3D NAND Flash Read Performance[C]//Proceedings of the 2018 IEEE 36th International Conference on Computer Design. Piscataway: IEEE, 2018: 51-58.
[21]	TAVAKKOL A, GóMEZ-LUNA J, SADROSADATI M , et al. MQSim: a Framework for Enabling Realistic Studies of Modern Multi-queue SSD Devices[C]//Proceedings of the 2018 16th USENIX Conference on File and Storage Technologies. Berkeley: USENIX Association, 2018: 49-66.
[22]	YAN S, LI H, HAO M , et al. Tiny-tail Flash: Near-perfect Elimination of Garbage Collection Tail Latencies in NAND SSDs[C]//Proceedings of the 2017 15th USENIX Conference on File and Storage Technologies. Berkeley: USENIX Association, 2017: 15-28.
[23]	NGUYEN D T, ZHOU G, XING G , et al. Reducing Smartphone Application Delay through Read/Write Isolation[C]//Proceedings of the 2015 13th Annual International Conference on Mobile Systems, Applications, and Services. New York: ACM, 2015: 287-300.

参数	数值
总容量/GB	64
保留的自由数据块/%	10
垃圾回收策略	Greedy
每个盘片的块	1024
每个块上的页	64
页大小/kB	4

应用名称	重复率/%	IOPS	大小/MB	写比率/%
Wechat	26.6	4.6	345.5	90.41
QQ	26.6	1.5	84.4	96.40
Weibo	35.4	1.4	255.4	77.53
Tou tiao	23.8	2.0	64.9	86.48
Baidu Map	17.9	4.3	311.7	94.22
Chrome	27.0	3.0	191.5	90.41

Deduplication technology for the mobile flash

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 10

References 23

Related Articles 4

Metrics

Comments

Recommended 10

[1]	HE Yaping,HE Yucheng,ZHOU Lin. Permutation codesable to correct stable and unstable burst erasure errors [J]. Journal of Xidian University, 2021, 48(3): 49-55.
[2]	ZHANG Zhang;YU Wencheng;XIE Guangjun. 10bit 100MS/s Hybrid ADC [J]. Journal of Xidian University, 2018, 45(3): 80-85+116.
[3]	MU Jianjun;ZHAO Peng;JIAO Xiaopeng. Decoding of codes correcting a single translocation error for the cell's level of flash memory [J]. Journal of Xidian University, 2016, 43(6): 51-55+121.
[4]	ZHENG Xue-feng;HAO Yue;LIU Hong-xia;LI Pei-xian;LIU Dao-guang;HAN Xiao-liang. Study of the hot carrier reliability of the flash memory [J]. J4, 2004, 31(6): 821-824.