多路并行多尺度特征复用的遥感图像超分辨率

doi:10.16180/j.cnki.issn1007-7820.2024.06.008

摘要/Abstract

摘要：

遥感图像内部物体尺寸小、分布不均匀、耦合程度高,针对目前遥感图像超分辨率模型特征提取信息单一且利用不足的现状,文中提出一种多路并行多尺度特征复用网络模型以改进图像重建的性能。该模型使用局部特征级联和全局特征融合的结构融合多个网络残差块提取的特征信息,其中每个残差块由两个多尺度卷积单元串行连接。多尺度卷积单元通过对特征信息进行交叉融合,构建多路并行的分支提取图像特征。同时引入短跳跃连接加强不同分支之间的特征复用,通过长跳跃连接加强网络不同深度的特征融合。当放大因子为4时,在两个测试集上该模型的峰值信噪比分别为29.653 1 dB、29.037 4 dB,相对于其他模型的测试结果具有明显提升,因此所提模型在遥感图像超分辨率重建上具有较好的效果。

关键词: 遥感图像, 超分辨率, 多路径, 并行提取, 多尺度, 特征复用, 跳跃连接, 卷积神经网络

Abstract:

Objects in remote sensing images are small in size, unevenly distributed, and highly coupled. In view of the current situation that the feature extraction information of remote sensing image super-resolution models is single and underutilized, this study proposes a multi-path parallel multi-scale feature reuse network model to improve performance of image reconstruction. The model fuses feature information extracted from multiple network residual blocks using a structure of local feature cascade and global feature fusion, where each residual block is serially connected by two multi-scale convolutional units. The multi-scale convolution units construct multiple branches to extract image features in parallel through cross-fusion of feature information. At the same time, short skip connections are introduced to strengthen feature reuse between different branches, and long skip connections are introduced to strengthen feature fusion at different depths of the network. When the amplification factor is 4, the peak signal-to-noise ratios of the model on the two test sets are 29.653 1 dB and 29.037 4 dB respectively, and they are significantly improved compared with the test results of other models. Therefore, the proposed model has a good effect on super-resolution reconstruction of remote sensing images.

Key words: remote sensing image, super-resolution, multi-path, parallel extraction, multi-scale, feature reuse, skip connection, convolutional neural network

中图分类号:

TP391

赵旭, 胡德敏. 多路并行多尺度特征复用的遥感图像超分辨率[J]. 电子科技, 2024, 37(6): 61-68.

ZHAO Xu, HU Demin. Multi-Path Parallel Multi-Scale Feature Reuse for Remote Sensing Image Super-Resolution[J]. Electronic Science and Technology, 2024, 37(6): 61-68.

图/表 13

图1

图2

图3

图4

表1

表2

图5

图6

表3

表4

表5

表6

图7

参考文献 23

[1]	Dong C, Loy C C, He K, et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 38(2):295-307.
[2]	秦兴, 高晓琪, 陈滨. 基于压缩卷积神经网络的图像超分辨率算法[J]. 电子科技, 2020, 33(5):1-8.
	Qin Xing, Gao Xiaoqi, Chen Bin. Image super-resolution algorithm based on squeeze net convolution neural network[J]. Electronic Science and Technology, 2020, 33(5):1-8.
[3]	He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]. Las Vegas: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2016:770-778.
[4]	Lim B, Son S, Kim H, et al. Enhanced deep residual networks for single image super-resolution[C]. Honolulu: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops,2017:619-623.
[5]	Li Z, Yang J, Liu Z, et al. Feedback network for image super-resolution[C]. Long Beach: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2019:321-347.
[6]	Zhang Y, Li K, Li K, et al. Image super-resolution using very deep residual channel attention networks[C]. Munich: Proceedings of the European Conference on Computer Vision,2018:24-38.
[7]	Chen Y, Liu L, Phonevilay V, et al. Image super-resolution reconstruction based on feature map attention mechanism[J]. Applied Intelligence, 2021, 51(7):4367-4380.
[8]	Dai T, Cai J, Zhang Y, et al. Second-order attention net-work for single image super-resolution[C]. Long Beach: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2019:593-601.
[9]	Yan Y, Ren W, Hu X, et al. SRGAT:Single image super-resolution with graph attention network[J]. IEEE Transactions on Image Processing, 2021(30):4905-4918.
[10]	Lei S, Shi Z, Zou Z. Super-resolution for remote sensing images via local-global combined network[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(8):1243-1247.
[11]	Daihong J, Sai Z, Lei D, et al. Multi-scale generative adversarial network for image super-resolution[J]. Soft Computing, 2022, 26(8):3631-3641.
[12]	Ledig C, Theis L, Huszár F, et al. Photo-realistic single image super-resolution using a generative adversarial network[C]. Honolulu: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2017:262-268.
[13]	Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C]. Salt Lake City: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2018:1952-1958.
[14]	Yu Y, Li X, Liu F. E-DBPN:Enhanced deep back-project-ion networks for remote sensing scene image super-resolution[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(8):5503-5515.
[15]	Haris M, Shakhnarovich G, Ukita N. Deep back-projecti-on networks for super-resolution[C]. Salt Lake City: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2018:239-245.
[16]	Dong X, Xi Z, Sun X, et al. Transferred multi-perception attention networks for remote sensing image super-resolution[J]. Remote Sensing, 2019, 11(23):2857-2877.
[17]	雷鹏程, 刘丛, 唐坚刚, 等. 分层特征融合注意力网络图像超分辨率重建[J]. 中国图象图形学报, 2020, 25(9):1773-1786.
	Lei Pengcheng, Liu Cong, Tang Jiangang, et al. Hierarchical feature fusion attention network for image super-resolution reconstruction[J]. Journal of Image and Graphics, 2020, 25(9):1773-1786.
[18]	Szegedy C, Liu W, Jia Y, et al. Going deeper with convolutions[C]. Boston: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2015:705-712.
[19]	Shi W, Caballero J, Huszár F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]. Seattle: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2016:1101-1108.
[20]	Szegedy C, Vanhoucke V, Ioffe S, et al. Rethinking the inception architecture for computer vision[C]. Seattle: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2016:1321-1327.
[21]	Xie H, Jiang H, Liu X, et al. Super resolution for remote sensing images via improved residual network[C].Harbin:The Fifth International Conference on Mechanical, Control and Computer Engineering IEEE,2020:398-405.
[22]	Dong X, Sun X, Jia X, et al. Remote sensing image super-resolution using novel dense-sampling networks[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 59(2):1618-1633.
[23]	Kim J, Lee J K, Lee K M. Accurate image super-resolution using very deep convolutional networks[C]. Las Vegas: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2016:566-571.

方法	并行路径	卷积核大小	参数量	运算量FLOPs
MPSR	2	均3×3或5×5	287×10³	661×10⁶
本文	3	1×1、3×3、5×5	267×10³	615×10⁶

CRB模块数量m	单个CRB内 MMRU数量n	模型中总的 MMRU数量	PSNR/dB
2	3	6	37.327 8
2	4	8	37.400 6
3	3	9	37.449 1
2	5	10	37.455 0
3	4	12	37.497 7

模型	×2	×3	×4
模型	PSNR/SSIM	PSNR/SSIM	PSNR/SSIM
本文	37.280 1/0.953 8	32.223 6/0.880 2	29.653 1/0.805 6
Model-1	37.272 5/0.953 1	32.190 7/0.879 4	29.623 2/0.804 6
Model-2	37.113 6/0.952 3	32.122 2/0.877 5	29.523 8/0.802 6
Model-3	37.106 7/0.951 8	31.983 9/0.874 2	29.435 1/0.797 8
Model-4	37.033 3/0.951 5	31.928 6/0.874 0	29.339 2/0.794 9

模型	Test-UC		Test-RS
模型	PSNR/dB	SSIM	PSNR/dB	SSIM
SRCNN	35.734 8	0.941 3	33.541 5	0.907 4
LGCNET	36.313 8	0.946 3	33.804 8	0.911 7
IRN	37.077 1	0.951 6	34.055 4	0.915 0
RCAN	37.055 7	0.951 6	34.032 2	0.915 1
MPSR	37.278 2	0.953 5	34.098 8	0.915 6
VDSR	36.728 6	0.949 4	33.926 9	0.913 3
DSSR	37.089 2	0.951 9	34.049 8	0.915 2
本文	37.280 1	0.953 8	34.099 0	0.916 1

模型	Test-UC		Test-RS
模型	PSNR/dB	SSIM	PSNR/dB	SSIM
SRCNN	30.935 6	0.849 4	30.263 9	0.804 1
LGCNET	31.187 3	0.855 8	30.365 5	0.807 8
IRN	31.979 2	0.874 1	30.592 4	0.815 5
RCAN	31.962 3	0.875 0	30.601 6	0.816 2
MPSR	32.160 1	0.878 0	30.638 5	0.817 2
VDSR	31.497 4	0.863 1	30.462 5	0.811 6
DSSR	32.012 5	0.873 7	30.614 0	0.816 6
本文	32.223 6	0.880 2	30.636 0	0.818 2