生成对抗网络用于视频去模糊

doi:10.19665/j.issn1001-2400.2019.06.016

Abstract

Abstract:

A video deblurring network based on the generative adversarial network and the Markovian discriminator is proposed to solve the video deblurring problem, which is caused by camera shaking or object movement. In this paper, we combine the pixel-space and feature-space loss, and design a discriminator based on the Markovian discriminator, which promotes the learning of image texture details and improves the quality of the generated image. The proposed method and the state-of-the-art methods are compared qualitatively and quantitatively on the test set and real video set, respectively. Experimental results indicate that the image deblurred by the proposed method has a higher peak signal-to-noise ratio and richer details.

Key words: video deblurring, generative adversarial network, Markovian discriminator

CLC Number:

TP391

SHEN Haijie,BIAN Qian,CHEN Xiaofan,WANG Zhenduo,TIAN Xinzhi. Video deblurring using the generative adversarial network[J].Journal of Xidian University, 2019, 46(6): 112-117.

Figures/Tables 5

References 16

[1]	SU S, DELBRACIO M, WANG J , et al. Deep Video Deblurring for Hand-held Cameras[C]//Proceedings of the 2017 30th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 237-246.
[2]	CHO S, WAND J, LEE S , et al. Video Deblurring for Hand-held Cameras Using Patch-based Synjournal[J]. ACM Transactions on Graphics, 2012,31(4):64.
[3]	DEBRACIO M, SAPIRO G . Hand-held Video Deblurring via Efficient Fourier Aggregation[J]. IEEE Transactions on Computational Imaging, 2015,1(4):270-283. doi: 10.1109/TCI.2015.2501245
[4]	TAO X, GAO H, SHEN X , et al. Scale-recurrent Network for Deep Image Deblurring[C]//Proceedings of the 2018 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2018: 8174-8182.
[5]	KUPYN O, BUDZAN V, MYKHAILCH M , et al. Deblurgan: Blind Motion Deblurring Using Conditional Adversarial Networks[C]//Proceedings of the 2018 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2018: 8183-8192.
[6]	WU M, LI W, GONG W . Two-frame Convolutional Neural Network for Blind Motion Image Deblurring[J]. Journal of Computer-Aided Design and Computer Graphics, 2018,30(12):2327-2334. doi: 10.3724/SP.J.1089.2018.17163
[7]	ZHANG K, LUO W, ZHONG Y , et al. Adversarial Spatio-temporal Learning for Video Deblurring[J]. IEEE Transactions on Image Processing, 2019,28(1):291-301. doi: 10.1109/TIP.2018.2867733 pmid: 30176588
[8]	GOODFELLOW I J, POUGET-ABADIE J, MIRZA M , et al. Generative Adversarial Nets[C]//Advances in Neural Information Processing Systems 27: 3. Vancouver: Neural Information Processing Systems Foundation, 2014: 2672-2680.
[9]	ISOLA P, ZHU J Y, ZHOU T , et al. Image-to-image Translation with Conditional Adversarial Networks[C]//Proceedings of the 2017 30th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 5967-5976.
[10]	LI C, WAND M . Precomputed Real-time Texture Synconfproc with Markovian Generative Adversarial Networks[C]//Lecture Notes in Computer Science: 9907. Heidelberg: Springer Verlag, 2016: 702-716.
[11]	ARJOVSKY M, CHINTALA S, BOTTOU L . Wasserstein GAN[J/OL]. [2019-5-16]. https://arxiv.org/abs/1701.07875.
[12]	PATHAK D, KRAHENBUHL P, DONAHUE J , et al. Context Encoders: Feature Learning by Inpainting[C]//Proceedings of the 2016 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2016: 2536-2544.
[13]	JOHNSON J, ALAHI A, FEI-FEI L . Perceptual Losses for Real-time Style Transfer and Super-resolution[C]//Lecture Notes in Computer Science: 9906. Heidelberg: Springer Verlag, 2016: 694-711.
[14]	TIELEMAN T, HINTON G . Lecture 6.5-rmsprop: Divide the Gradient by a Running Average of Its Recent Magnitude[J]. Coursera: Neural Networks for Machine Learning, 2012,4:26-30.
[15]	ZHANG R, ISOLA P, EFROS A A , et al. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric[C]//Proceedings of the 2018 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2018: 586-595.
[16]	MOORTHY A K, BOVIK A C . A Two-step Framework for Constructing Blind Image Quality Indices[J]. IEEE Signal Processing Letters, 2010,17(5):513-516. doi: 10.1109/LSP.2010.2043888

方法	文献[4]	文献[5]	文献[1]	文中方法
R_PSNR/dB	30.98	30.22	32.25	33.62
R_SSIM	0.928 5	0.897 6	0.942 3	0.957 5
感知距离^[15]	0.056 8	0.062 5	0.045 2	0.032 8

Video deblurring using the generative adversarial network

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 5

References 16

Related Articles 11

Metrics

Comments

Recommended 10

[1]	YU Haoyang,YIN Liang,LI Shufang,LV Shun. Recognition algorithm for the little sample radar modulation signal based on the generative adversarial network [J]. Journal of Xidian University, 2021, 48(6): 96-104.
[2]	SUN Yanjing,WEI Li,ZHANG Nianlong,YUN Xiao,DONG Kaiwen,GE Min,CHENG Xiaozhou,HOU Xiaofeng. Person re-identification method combining the DD-GAN and Global feature in a coal mine [J]. Journal of Xidian University, 2021, 48(5): 201-211.
[3]	WANG Junjun,SUN Yue,LI Ying. Cloud removal method for the remote sensing image based on the GAN [J]. Journal of Xidian University, 2021, 48(5): 23-29.
[4]	YANG Jingbo,ZHAO Qijun,LYU Zejun. Synthesis of the expression image and its application under the dimentional emotion model [J]. Journal of Xidian University, 2021, 48(5): 30-37.
[5]	WEI Ziyu,YANG Xi,WANG Nannan,YANG Dong,GAO Xinbo. Reciprocal bi-directional generative adversarial network for cross-modal pedestrian re-identification [J]. Journal of Xidian University, 2021, 48(2): 205-212.
[6]	JIANG Shaobin,DU Chun,CHEN Hao,LI Jun,WU Jiangjiang. Unsupervised adversarial learning method for hard disk failure prediction [J]. Journal of Xidian University, 2020, 47(2): 118-125.
[7]	ZHANG Zhiyuan,DIAO Yinghua. Pedestrian trajectory prediction model with social features and attention [J]. Journal of Xidian University, 2020, 47(1): 10-17.
[8]	YANG Xiaoli,LIN Suzhen. Method for multi-band image feature-level fusion based on the attention mechanism [J]. Journal of Xidian University, 2020, 47(1): 120-127.
[9]	JIA Yufeng,MA Li. Self-attention generative adversarial network with the conditional constraint [J]. Journal of Xidian University, 2019, 46(6): 163-171.
[10]	MO Jianwen,XU Kailiang,LIN Leping,OUYANG Ning. Text-to-image generation combined with mutual information maximization [J]. Journal of Xidian University, 2019, 46(5): 180-188.
[11]	CHEN Xiaofan,SHEN Haijie,BIAN Qian,WANG Zhenduo,TIAN Xinzhi. Face image super-resolution with an attention mechanism [J]. Journal of Xidian University, 2019, 46(3): 148-153.