融合片内语义和片间结构特征的自监督CT图像分类方法

doi:10.16180/j.cnki.issn1007-7820.2024.07.006

摘要/Abstract

摘要：

针对CT(Computed Tomography)图像分析存在人工标签稀缺、分类性能不佳等问题,文中提出一种融合片内语义和片间结构特征的自监督CT图像分类方法。该方法利用CT图像的层次结构特性和局部组成要素的语义特点,通过混淆切片生成算法对无标签的病灶部位图像进行处理,生成空间指数和混淆切片作为监督信息。在自监督辅助任务中利用ResNet50网络从混淆切片中同时提取与病灶部位相关的CT片内语义和片间结构特征,将学习到的特征迁移到后续医学分类任务中,使得最终模型从无标签数据中获得增益。实验结果表明,当被使用的有标签数据有限时,相比其他针对CT图像的二维模型和三维模型,所提方法的分类性能和标签利用效率更优。

关键词: 医学图像分类, 三维医学图像处理, CT图像, 自监督学习, 迁移学习, 小样本学习, 片内语义特征, 片间结构特征, ResNet50

Abstract:

In view of the scarcity of artificial labels and poor classification performance in CT(Computed Tomography) image analysis, a self-supervised CT image classification method combining in-slice semantic and interslice structural features is proposed in this study.In this method, the hierarchical structure of CT images and the semantic features of local components are utilized to process the unlabeled lesion images through the confusion section generation algorithm, and the spatial index and confusion section are generated as supervisory information.In the self-supervised auxiliary task, the ResNet50 network was used to extract both the intraslice semantic and interslice structural features related to the lesion site from the confused sections, and the learned features were transferred to the subsequent medical classification task, so that the final model gained from the unlabeled data.The experimental results show that compared with other 2D and 3D models for CT images, the proposed method can achieve better classification performance and label utilization efficiency when the used labeled data is limited.

Key words: medical image classification, 3D medical image processing, CT images, self-supervised learning, transfer learning, few shot learning, intra-slice semantic features, inter-slice structural features, ResNet50

中图分类号:

TP391

曹春萍, 许志华. 融合片内语义和片间结构特征的自监督CT图像分类方法[J]. 电子科技, 2024, 37(7): 43-52.

CAO Chunping, XU Zhihua. A Self-Supervised CT Image Classification Method Incorporating Intra-Slice Semantic and Inter-Slice Structural Features[J]. Electronic Science and Technology, 2024, 37(7): 43-52.

图/表 11

图1

图2

图3

图4

表1

图5

图6

图7

图8

表2

表3

参考文献 38

[1]	闫超, 孙占全, 田恩刚, 等. 基于深度学习的医学图像分割技术研究进展[J]. 电子科技, 2021, 34(2):7-11.
	Yan Chao, Sun Zhanquan, Tian Engang, et al. Research progress of medical image segmentation based on deep learning[J]. Electronic Science and Technology, 2021, 34(2):7-11.
[2]	郑光远, 刘峡壁, 韩光辉. 医学影像计算机辅助检测与诊断系统综述[J]. 软件学报, 2018, 29(5):1471-1514.
	Zheng Guangyuan, Liu Xiabi, Han Guanghui. Survey on medical image computer aided detection and diagnosis systems[J]. Journal of Software, 2018, 29(5):1471-1514.
[3]	陶笃纯. 迈向新世纪的医学影像技术[J]. 中国医学影像技术, 2000, 16(1):1-2.
	Tao Duchun. Medical imaging technology in the new century[J]. Chinese Journal of Medical Imaging Technology, 2000, 16(1):1-2.
[4]	Lecun Y, Bengio Y, Hinton G. Deep learning[J]. Nature, 2015, 521(7553):436-444.
[5]	Simonyan K, Zisserman A. Very deep convolutional networks for largescale image recognition[EB/OL].(2015-04-10) [2023-02-15] https://arxiv.org/abs/1409.1556.
[6]	Pan S J, Yang Q. A survey on transfer learning[J]. IE-EE Transactions on Knowledge and Data Engineering, 2009, 22(10):1345-1359.
[7]	Jing L, Tian Y. Selfsupervised visual feature learning with deep neural networks:A survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 43(11):4037-4058.
[8]	赵凯琳, 靳小龙, 王元卓. 小样本学习研究综述[J]. 软件学报, 2021, 32(2):349-369.
	Zhao Kailin, Jin Xiaolong, Wang Yuanzhuo. Survey on few-shot learning[J]. Journal of Software, 2021, 32(2):349-369.
[9]	庄福振, 罗平, 何清, 等. 迁移学习研究进展[J]. 软件学报, 2015, 26(1):26-39.
	Zhuang Fuzhen, Luo Ping, He Qing, et al. Suvery on transfer learning research[J]. Journal of Software, 2015, 26(1):26-39.
[10]	He K, Girshick R, Dollár P. Rethinking imagenet pretraining[C]. Seoul: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019:4918-4927.
[11]	Xu J. A review of self-supervised learning methods in the field of medical image analysis[J]. International Journal of Image,Graphics and Signal Processing, 2021, 13(4):33-46.
[12]	Shurrab S, Duwairi R. Self-supervised learning methods and applications in medical imaging analysis:A survey[J]. PeerJ Computer Science, 2022(8):1045-1052.
[13]	Chen L, Bentley P, Mori K, et al. Self-supervised learning for medical image analysis using image context restoration[J]. Medical Image Analysis, 2019, 58(5):101539-101543.
[14]	Flanders A E, Prevedello L M, Shih G, et al. Construction of a machine learning dataset through collaboration:The RSNA 2019 brain CT hemorrhage challenge[J]. Radiology:Artificial Intelligence, 2020, 2(3):190211-190220.
[15]	Jing L, Tian Y. Self-supervised visual feature learning with deep neural networks:A survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 43(11):4037-4058.
[16]	彭玉姣. 基于自监督学习的图像特征表示方法研究[D]. 合肥: 安徽大学, 2021:7-15.
	Peng Yujiao. Research on image feature representation based on self-supervised learning[D]. Hefei: Anhui University, 2021:7-15.
[17]	Xie Y, Xu Z, Zhang J, et al. Self-supervised learning of graph neural networks:A unified review[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 45(2):2412-2429.
[18]	Bai W, Chen C, Tarroni G, et al. Self-supervised learning for cardiac MR image segmentation by anatomical position prediction[C]. Shenzhen: International Conference on Medical Image Computing and Computer-Assisted Intervention, 2019:541-549.
[19]	Henaff O. Data-efficient image recognition with contrastive predictive coding[C]. Online: International Conference on Machine Learning, 2020:4182-4192.
[20]	Chen T, Kornblith S, Norouzi M, et al. A simple framework for contrastive learning of visual representations[C]. Online: International Conference on Machine Learning, 2020:1597-1607.
[21]	He K, Fan H, Wu Y, et al. Momentum contrast for unsupervised visual representation learning[C]. Online: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020:9729-9738.
[22]	Grill J B, Strub F, Altché F, et al. Bootstrap your own latent a new approach to self-supervised learning[J]. Advances in Neural Information Processing Systems, 2020, 33(3):21271-21284.
[23]	Chen X, He K. Exploring simple siamese representation learning[C]. Online: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021:15750-15758.
[24]	Zbontar J, Jing L, Misra I, et al. Barlow twins:Self-supe-rvised learning via redundancy reduction[C]. Online: International Conference on Machine Learning, 2021:12310-12320.
[25]	Ciga O, Xu T, Martel A L. Self supervised contrastive learning for digital histopathology[J]. Machine Learning with Applications, 2022(7):100198-100206.
[26]	Zhou Z, Sodha V, Rahman Siddiquee M M, et al. Models genesis:Generic autodidactic models for 3D medical image analysis[C]. Shenzhen: Medical Image Computing and Computer Assisted Intervention, 2019:384-393.
[27]	Taleb A, Lippert C, Klein T, et al. Multimodal self-supe-rvised learning for medical image analysis[C]. Online: International Conference on Information Processing in Medical Imaging, 2021:661-673.
[28]	Zheng H, Han J, Wang H, et al. Hierarchical self-supervised learning for medical image segmentation based on multi-domain data aggregation[C]. Strasbourg: International Conference on Medical Image Computing and Computer-Assisted Intervention, 2021:622-632.
[29]	Zhuang X, Li Y, Hu Y, et al. Self-supervised feature learning for 3D medical images by playing a rubik's cube[C]. Shenzhen: International Conference on Medical Image Computing and Computer-Assisted Intervention, 2019:420-428.
[30]	Noroozi M, Favaro P. Unsupervised learning of visual representations by solving jigsaw puzzles[C]. Amsterdam: Computer Vision-ECCV the Fourteenth European Conference, 2016:69-84.
[31]	Zhu J, Li Y, Hu Y, et al. Embedding task knowledge into 3D neural networks via self-supervised learning[EB/OL].(2020-06-10)[2023-02015]https://arxiv.org/abs/2006.05798.
[32]	Tang Y, Yang D, Li W, et al. Self-supervised pretraining of swin transformers for 3d medical image analysis[C]. New Orleans: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022:20730-20740.
[33]	Ahmed S, Iftekharuddin K M, Vossough A. Efficacy of texture,shape and intensity feature fusion for posterior-fossa tumor segmentation in MRI[J]. IEEE Transactions on Information Technology in Biomedicine, 2011, 15(2):206-213.
[34]	谢凯, 孙鸿飞, 林涛, 等. 影像组学中特征提取研究进展[J]. 中国医学影像技术, 2017, 33(12):1792-1796.
	Xie Kai, Sun Hongfei, Lin Tao, et al. Research progresses in feature extraction of radiomics[J]. Chinese Journal Medical Imaging Technology, 2017, 33(12):1792-1796.
[35]	Yu H, Winkler S. Image complexity and spatial information[C]. Klagenfurt: The Fifth International Workshop on Quality of Multimedia Experience, 2013:12-17.
[36]	郭小英, 李文书, 钱宇华, 等. 可计算图像复杂度评价方法综述[J]. 电子学报, 2020, 48(4):819-826. doi: 10.3969/j.issn.0372-2112.2020.04.024
	Guo Xiaoying, Li Wenshu, Qian Yuhua, et al. Computational evaluation methods of visual complexity perception for images[J]. Acta Electronica Sinica, 2020, 48(4): 819-826. doi: 10.3969/j.issn.0372-2112.2020.04.024
[37]	周兵, 刘玉霞, 杨欣欣, 等. 图像复杂度研究综述[J]. 计算机科学, 2018, 45(9):30-37. doi: 10.11896／j.issn.1002-137X.2018.09.004
	Zhou Bing, Liu Yuxia, Yang Xinxin, et al. Review of research on image complexity[J]. Computer Science, 2018, 45(9):30-37. doi: 10.11896／j.issn.1002-137X.2018.09.004
[38]	Buslaev A, Iglovikov V I, Khvedchenya E, et al. Albumentations:Fast and flexible image augmentations[J]. Information, 2020, 11(2):125-131.

预测类别	真实类别
预测类别	Positive	Negative
Positive	TP(True Positive)	FP(False Positive)
Negative	FN(False Negative)	TN(True Negative)

数据量	Context- Restore-2D	Rubik- Cube-3D	TCPC-3D	本文模型
20%	1.268	1.155	1.308	0.939
40%	0.112	0.739	0.991	0.689
60%	0.938	0.532	0.792	0.467
80%	0.892	0.396	0.524	0.201
100%	0.874	0.328	0.476	0.147
损失降幅	31.07%	71.60%	63.61%	84.35%

数据量	Context- Restore-2D	Rubik- Cube-3D	TCPC-3D	本文模型
20%	0.233	0.167	0.189	0.145
40%	0.191	0.138	0.167	0.109
60%	0.176	0.129	0.142	0.085
80%	0.161	0.110	0.124	0.072
100%	0.153	0.099	0.119	0.065
损失降幅	34.62%	40.7%	37.04%	55.17%