Abstract:

Video super-resolution aims to restore high-resolution videos from low-resolution videos,which can effectively improve the display effect of videos.What is different from single image super-resolution is that how to exploit the information between contiguous video frames is important for video super-resolution.In order to improve the performance of video super-resolution and make full use of the spatio-temporal information on video frames,a video super-resolution model based on multi-scale 3D convolution is proposed,which takes continuous video frames as the input and outputs the reconstruction super-resolution result of the intermediate frame.This model consists of three modules:multi-scale feature extraction,feature fusion and high-resolution reconstruction.First,multi-scale 3D convolution is used for preliminary feature extraction.Then,3D convolution residual structure is adopted in feature fusion,and the feature maps are split,which can not only fuse the features of different scales,but also effectively reduce the number of network parameters.Finally,residual dense blocks and sub-pixel convolution are used for high-resolution reconstruction,and the reconstructed video frame is obtained by combining with the global residual connection.Experimental results of 3× and 4× super-resolution in Vid4 dataset show that compared with other methods,the proposed method can enhance the peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) performance effectively with a better visual effect.

Key words: video super-resolution, 3D convolution, residual network, spatio-temporal correlation