一种多尺度GAN的低剂量CT超分辨率重建方法

doi:10.19665/j.issn1001-2400.2022.02.026

Abstract

Abstract:

Low-dose CT can reduce X-ray radiation and damage to human body,but the imaging quality can also be significantly reduced.In order to obtain high quality images with fine structural details,a low dose CT image super-resolution reconstruction algorithm based on the multi-scale residual generation network (MSRGAN) is proposed to recover high resolution (HR) images from low resolution (LR) images with pathology unchanged.First,a residual network is introduced to prevent overfitting while realizing feature reuse.Second,the multi-scale network can make full use of image features of different sizes,enrich image details and improve the utilization rate of features in the reconstruction process.Finally,by combining the adversarial loss and content loss,the reconstructed image with a better perceived quality can be obtained when the generated feature is constrained.Experimental results show that compared with other algorithms,this method improves in SSIM,FSIM and PSNR indexes,that its GAN's IS,FID and SWD performance is better than that of the other two Gan-based algorithms,and that it has a better performance in edge contour detail,which fully proves the effectiveness of this algorithm.

Key words: image processing, super-resolution, generate antagonistic network, multi-scale features, low-dose CT

CLC Number:

TP391

XU Ying,LIU Shuai,SHAO Meng,YUE Guodong,AN Dong. Multi-scale generation antagonistic network for the low-dose CT images super-resolution reconstruction algorithm[J].Journal of Xidian University, 2022, 49(2): 228-236.

Figures/Tables 10

References 21

[1]	DONG C, LOY C C, HE K, et al. Image Super-Resolution Using Deep Convolutional Networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2):295-307. doi: 10.1109/TPAMI.2015.2439281
[2]	SHI W Z, CABALLERO J, HUSZÁR F, et al. Real-Time Single Image and Video Super Resolution Using an Efficient Sub-Pixel Convolutional Neural Network[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Piscataway:IEEE, 2016:1874-1883.
[3]	KIM J, LEE J K, LEE K M. Accurate Image Super-Resolution Using Very Deep Convolutional Networks[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Piscataway:IEEE, 2016:1646-1654.
[4]	MAO X, SHEN C, YANG Y, et al. Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections[C]// Proceedings of International Conference on Neural Information Processing Systems.Heidelberg:Springer, 2016:2810-2818.
[5]	LI J C, FANG F, MEI K F, et al. Multi-Scale Residual Network for Image Super-Resolution[C]// Proceedings of 2018 ECCV European Conference on Computer Vision.Heidelberg:Springer, 2018:527-542.
[6]	PARK S J, SON H, CHO S H. et al. SRFeat:Single Image Super-Resolution with Feature Discrimination[C]// Proceedings of 2018 ECCV European Conference on Computer Vision.Heidelberg:Springer, 2018:455-471.
[7]	BULAT A, YANG J, GEORGIOS T. To Learn Image Super-Resolution,Use a GAN to Learn How to do Image Degradation First[C]// Proceedings of 2018 ECCV European Conference on Computer Vision.Heidelberg:Springer, 2018:187-202.
[8]	LEDIG C, THEIS L, HUSZARF, et al. Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Piscataway:IEEE, 2017:105-114.
[9]	FRID-ADAR M, DIAMANT I, KLANG E, et al. GAN-Based Synthetic Medical Image Augmentation for Increased CNN Performance in Liver Lesion Classification[J]. Neurocomputing, 2018, 321:321-331. doi: 10.1016/j.neucom.2018.09.013
[10]	ISOLA P, ZHU J, ZHOU T, et al. Image-To-Image Translation with Conditional Adversarial Networks[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Piscataway:IEEE, 2017:5967-5976.
[11]	WU Z, SHEN C, DEN HENGELA V. Wider or Deeper:Revisiting the ResNet Model for Visual Recognition[J]. Pattern Recognition, 2019, 90:119-133. doi: 10.1016/j.patcog.2019.01.006
[12]	ODENA A, DUMOULIN V, OLAH C, et al. Deconvolution and Checkerboard Artifacts(2016)[EB/OL].[2016-10-17]https://distill.pub/2016/deconv-checkerboard/?utm_source=researcher_app&utm_medium=referral&utm_campaign=RESR_MRKT_Researcher_inbound.
[13]	SHI W, CABALLERO J, HUSZáR F, et al. Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Piscataway:IEEE, 2016:1874-1883.
[14]	WANG Z, JIANG K, YI P, et al. Ultra-Dense GAN for Satellite Imagery Super-Resolution[J]. Neurocomputing, 2020, 398:328-337. doi: 10.1016/j.neucom.2019.03.106
[15]	张杨忆, 林泓, 管钰华, 等. 改进残差块和对抗损失的GAN图像超分辨率重建[J]. 哈尔滨工业大学学报, 2019, 51(11):128-137.
	ZHANG Yangyi, LIN Hong, GUAN Yuhua, et al. GAN Image Super-Resolution Reconstruction Model with Improved Residual Block and Adversarial Loss[J]. Journal of Harbin Institute of Technology, 2019, 51(11):128-137.
[16]	ZHOU D, DUAN R, ZHAO L, et al. Single Image Super-Resolution Reconstruction Based on Multi-Scale Feature Mapping Adversarial Network[J]. Signal Processing, 2020, 166:107251. doi: 10.1016/j.sigpro.2019.107251
[17]	KASEM H M, HUNG K W, JIANG J. Spatial Transformer Generative Adversarial Network for Robust Image Super-Resolution[J]. IEEE Access, 2019, 7:182993-183009. doi: 10.1109/ACCESS.2019.2959940
[18]	MAHAPATRA D, BOZORGTABAR B, GARNAVI R, et al. Image Super-Resolution Using Progressive Generative Adversarial Networks for Medical Image Analysis[J]. Computerized Medical Imaging and Graphics, 2019, 71:30-39. doi: 10.1016/j.compmedimag.2018.10.005
[19]	ZHANG K, TAO D,GAOX, et al. Coarse-to-Fine Learning for Single-Image Super-Resolution[J]. IEEE Transactions on Neural Networks and Learning Systems, 2017, 28(5):1109-1122. doi: 10.1109/TNNLS.2015.2511069
[20]	DONG C, LOY CC, TANG X, et al. Accelerating the Super-Resolution Convolutional Neural Network[C]// European Conference on Computer Vision.Heidelberg:Springer, 2016:391-407.
[21]	YOU C, LI G, ZHANG Y, et al. CT Super-Resolution GAN Constrained by the Identical,Residual,and Cycle Learning Ensemble (GAN-CIRCLE)[J]. IEEE Transactions on Medical Imaging, 2020, 39(1):188-203. doi: 10.1109/TMI.2019.2922960

重建图像	A	B	C	D	E	F	G	H
CTF	3.82	3.90	3.81	3.97	3.87	3.91	3.93	3.85
FSRCNN	4.03	3.94	3.96	4.13	3.99	4.05	4.11	4.09
RDN	3.98	4.01	4.12	4.24	4.12	4.16	4.21	4.19
SRGAN	4.26	4.16	4.22	4.31	4.23	4.32	4.28	4.17
GAN-CIRCLE	4.37	4.35	4.41	4.39	4.43	4.38	4.41	4.36
MSRGAN	4.42	4.37	4.46	4.48	4.45	4.39	4.51	4.43

重建图像	SSIM	FSIM	PSNR	耗时控制/s
CTF	0.892	0.945	28.32	0.008 1
FSRCNN	0.903	0.952	29.15	1.213 6
RDN	0.843	0.960	30.48	1.312 5
SRGAN	0.836	0.971	29.86	1.423 7
GAN-CIRCLE	0.911	0.968	31.73	1.416 3
MSRGAN	0.924	0.982	31.87	1.409 1

模型	IS	FID	SWD(×103)
Real images	8.72	1.43	2.57
SRGAN	4.87	13.62	7.91
GAN-CIRCLE	5.93	7.04	6.86
MSRGAN	6.14	6.85	6.81

模型	SSIM	FSIM	PSNR
MSRGAN	0.924	0.982	31.87
MSRGAN-m	0.882	0.958	30.26
MSRGAN-v	0.876	0.943	29.98
MSRGAN-d	0.894	0.952	30.15

Multi-scale generation antagonistic network for the low-dose CT images super-resolution reconstruction algorithm

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 10

References 21

Related Articles 15

Metrics

Comments

Recommended 10

[1]	SUN Jingrong,XIE Linchang,DU Mengxin,LUO Liyan. Adaptive transmittance dehazing algorithm based on non-linear transformation [J]. Journal of Xidian University, 2022, 49(1): 208-215.
[2]	ZHANG Yuhao,CHENG Peitao,ZHANG Shuhao,WANG Xiumei. Lightweight image super-resolution with the adaptive weight learning network [J]. Journal of Xidian University, 2021, 48(5): 15-22.
[3]	HAN Yongsai,MA Shiping,HE Linyuan,LI Chenghao,ZHU Mingming,ZHANG Fei. Detection of the object in the fast remote sensing airport area on the improved YOLOv3 [J]. Journal of Xidian University, 2021, 48(5): 156-166.
[4]	ZHAN Keyu,SUN Yue,LI Ying. Video super-resolution based on multi-scale 3D convolution [J]. Journal of Xidian University, 2021, 48(5): 8-14.
[5]	SUN Zhengyang,DONG Mei,CHEN Baixiao. Interrupted sampling repeater jamming suppression based on time-frequency analysis and band-pass filtering [J]. Journal of Xidian University, 2021, 48(2): 139-146.
[6]	CHENG Tianyou,LIN Yanping,MA Xiaojun. Ultrasound probe guiding method using vision and force [J]. Journal of Xidian University, 2020, 47(1): 80-87.
[7]	WANG Ziyi,SU Yuting,LIU Yanyan,ZHANG Wei. Algorithm for segmentation of smoke using the improved DeeplabV3 network [J]. Journal of Xidian University, 2019, 46(6): 52-59.
[8]	LIU Shudong,WANG Xiaomin,ZHANG Yan. Symmetric residual convolution neural networks for the image super-resolution reconstruction [J]. Journal of Xidian University, 2019, 46(5): 15-23.
[9]	ZHANG Jintao,LEI Jie,WU Lingyun,HUANG Biying,LI Yunsong. Improved algorithm for hyperspectral endmember extraction and its FPGA implementation [J]. Journal of Xidian University, 2019, 46(4): 22-27.
[10]	ZHU Suya,DU Jianchao,LI Yunsong,WANG Xiaopeng. Method for bridge crack detection based on the U-Net convolutional networks [J]. Journal of Xidian University, 2019, 46(4): 35-42.
[11]	CHEN Xiaofan,SHEN Haijie,BIAN Qian,WANG Zhenduo,TIAN Xinzhi. Face image super-resolution with an attention mechanism [J]. Journal of Xidian University, 2019, 46(3): 148-153.
[12]	SHI Wenjun,GUO Congzhou,TONG Xiaochong,TIAN Yuan,CAO Wen. Super-resolution reconstruction of infrared remote sensing images with radiation fidelity [J]. Journal of Xidian University, 2019, 46(2): 107-113.
[13]	LAI Rui,GUAN Juntao,XU Kunran,XIONG Ai,YANG Yintang. Cascade residual learning method for infrared image nonuniformity correction [J]. Journal of Xidian University, 2019, 46(1): 14-19.
[14]	GUO Songge;LV Donghui;REN Yanli. Improved (k, n) tagged visual cryptography scheme [J]. Journal of Xidian University, 2018, 45(5): 170-176+183.
[15]	HUANG Yong;LIU Fang. Detecting water bridge in SAR images via a scene semantic algorithm [J]. Journal of Xidian University, 2018, 45(4): 40-44.