Research progress on medical image dataset expansion methods_Journal of Biomedical Engineering

Authors：

CHEN Ying ¹ ,  LIN Hongping ¹ , ZHANG Wei ¹ , FENG Longfeng ¹ , ZHENG Cheng ¹ , ZHOU Taohui ¹ , YI Zhen ² , LIU Lan ²

1. School of Software, Nanchang Hangkong University, Nanchang 330063, P. R. China;
2. Department of Medical Imaging, Jiangxi Cancer Hospital, Nanchang 330029, P. R. China;

Corresponding author：

LIN Hongping, Email: lin2472612203@163.com

Keywords：

Medical image expansion; Computer aided diagnosis system; Geometric transformation; Generative adversarial network

DOI：

10.7507/1001-5515.202206039

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Computer-aided diagnosis (CAD) systems play a very important role in modern medical diagnosis and treatment systems, but their performance is limited by training samples. However, the training samples are affected by factors such as imaging cost, labeling cost and involving patient privacy, resulting in insufficient diversity of training images and difficulty in data obtaining. Therefore, how to efficiently and cost-effectively augment existing medical image datasets has become a research hotspot. In this paper, the research progress on medical image dataset expansion methods is reviewed based on relevant literatures at home and abroad. First, the expansion methods based on geometric transformation and generative adversarial networks are compared and analyzed, and then improvement of the augmentation methods based on generative adversarial networks are emphasized. Finally, some urgent problems in the field of medical image dataset expansion are discussed and the future development trend is prospected.

Citation： CHEN Ying, LIN Hongping, ZHANG Wei, FENG Longfeng, ZHENG Cheng, ZHOU Taohui, YI Zhen, LIU Lan. Research progress on medical image dataset expansion methods. Journal of Biomedical Engineering, 2023, 40(1): 185-192. doi: 10.7507/1001-5515.202206039 Copy

1.	邹茂扬, 杨昊, 潘光晖, 等. 深度学习在医学图像配准上的研究进展与挑战. 生物医学工程学杂志, 2019, 36(4): 677-683.
2.	游森榕, 胡圣烨, 申妍燕, 等. 生成对抗网络在医学图像计算上的进展与展望. 计算机科学与应用, 2021, 11(7): 1949-1961.
3.	肖汉光, 冉智强, 黄金锋, 等. 基于电子计算机断层扫描图像的肺实质分割方法研究进展. 生物医学工程学杂志, 2021, 38(2): 379-386.
4.	Chlap P, Min H, Vandenberg N, et al. A review of medical image data augmentation techniques for deep learning applications. Journal of Medical Imaging and Radiation Oncology, 2021, 65(5): 545-563.
5.	陈英, 郑铖, 易珍, 等. 肝脏及肿瘤图像分割方法综述. 计算机应用研究, 2022, 39(3): 641-650.
6.	Thaha M M, Kumar K P M, Murugan B S, et al. Brain tumor segmentation using convolutional neural networks in MRI images. Journal of Medical Systems, 2019, 43(9): 294.
7.	Ding Y, Yu X, Yang Y. Modeling the probabilistic distribution of unlabeled data for one-shot medical image segmentation//Proceedings of the AAAI Conference on Artificial Intelligence. Vancouver: AAAI. 2021, 35(2): 1246-1254.
8.	Brahim I, Fourer D, Vigneron V, et al. Deep learning methods for MRI brain tumor segmentation: a comparative study//2019 Ninth International Conference on Image Processing Theory, Tools and Applications (IPTA). Istanbul: IEEE, 2019: 1-6.
9.	Afzal S, Maqsood M, Nazir F, et al. A data augmentation-based framework to handle class imbalance problem for Alzheimer’s stage detection. IEEE Access, 2019, 7: 115528-115539.
10.	Karani N, Erdil E, Chaitanya K, et al. Test-time adaptable neural networks for robust medical image segmentation. Medical Image Analysis, 2021, 68: 101907.
11.	Chen C, Hammernik K, Ouyang C, et al. Cooperative training and latent space data augmentation for robust medical image segmentation//International Conference on Medical Image Computing and Computer-Assisted Intervention. Strasbourg: Springer, 2021: 149-159.
12.	Novosad P, Fonov V, Collins D L, et al. Accurate and robust segmentation of neuroanatomy in T1‐weighted MRI by combining spatial priors with deep convolutional neural networks. Human Brain Mapping, 2020, 41(2): 309-327.
13.	Pan H, Feng Y, Chen Q, et al. Prostate segmentation from 3D MRI using a two-stage model and variable-input based uncertainty measure//2019 IEEE 16th International Symposium on Biomedical Imaging. Venice: IEEE, 2019: 468-471.
14.	Yang F, Liang F, Lu L, et al. Dual attention-guided and learnable spatial transformation data augmentation multi-modal unsupervised medical image segmentation. Biomedical Signal Processing and Control, 2022, 78: 103849.
15.	Chen Y, Yang X H, Wei Z, et al. Generative adversarial networks in medical image augmentation: a review. Computers in Biology and Medicine, 2022, 144: 105382.
16.	徐峰, 马小萍, 刘立波. 基于生成对抗网络的甲状腺超声图像文本跨模态检索方法. 生物医学工程学杂志, 2020, 37(4): 641-651.
17.	Frid-Adar M, Diamant I, Klang E, et al. GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification. Neurocomputing, 2018, 321: 321-331.
18.	Kwon G, Han C, Kim D. Generation of 3D brain MRI using auto-encoding generative adversarial networks//International Conference on Medical Image Computing and Computer-Assisted Intervention. Shenzhen: Springer, 2019: 118-126.
19.	Chuquicusma M J M, Hussein S, Burt J, et al. How to fool radiologists with generative adversarial networks? A visual turing test for lung cancer diagnosis//2018 IEEE 15th International Symposium on Biomedical Imaging. Washington: IEEE, 2018: 240-244.
20.	Calimeri F, Marzullo A, Stamile C, et al. Biomedical data augmentation using generative adversarial neural networks//International Conference on Artificial Neural Networks. Alghero: Springer, 2017: 626-634.
21.	Bermudez C, Plassard A J, Davis L T, et al. Learning implicit brain MRI manifolds with deep learning. Proceedings of SPIE-the International Society for Optical Engineering, 2018, 10574: 15704L.
22.	Wang Z, Lin Y, Cheng K T T, et al. Semi-supervised mp-MRI data synthesis with stitchlayer and auxiliary distance maximization. Medical Image Analysis, 2020, 59: 101565.
23.	Jiang Y, Chen H, Loew M, et al. COVID-19 CT image synthesis with a conditional generative adversarial network. IEEE Journal of Biomedical and Health Informatics, 2020, 25(2): 441-452.
24.	Wang Q, Zhang X, Zhang W, et al. Realistic lung nodule synthesis with multi-target co-guided adversarial mechanism. IEEE Transactions on Medical Imaging, 2021, 40(9): 2343-2353.
25.	Costa P, Galdran A, Meyer M I, et al. End-to-end adversarial retinal image synthesis. IEEE Transactions on Medical Imaging, 2017, 37(3): 781-791.
26.	Oliveira D A B. Controllable skin lesion synthesis using texture patches, bézier curves and conditional gans//2020 IEEE 17th International Symposium on Biomedical Imaging. Lowa: IEEE, 2020: 1798-1802.
27.	Zhang T, Cheng J, Fu H, et al. Noise adaptation generative adversarial network for medical image analysis. IEEE Transactions on Medical Imaging, 2019, 39(4): 1149-1159.
28.	储珺, 林文杰, 徐鹏. 目标检测中特征不匹配问题研究进展. 南昌航空大学学报:自然科学版, 2021, 35(3): 1-8.
29.	张颖麟, 胡衍, 东田理沙, 等. 生成对抗式网络及其医学影像应用研究综述. 中国图象图形学报, 2022, 27(3): 687-703.
30.	Kong L, Lian C, Huang D, et al. Breaking the dilemma of medical image-to-image translation. Advances in Neural Information Processing Systems, 2021, 34: 1964-1978.
31.	Luo Y, Nie D, Zhan B, et al. Edge-preserving MRI image synthesis via adversarial network with iterative multi-scale fusion. Neurocomputing, 2021, 452: 63-77.
32.	Xie G, Wang J, Huang Y, et al. FedMed-GAN: federated domain translation on unsupervised cross-modality brain image synthesis. arXiv preprint, 2022, arXiv: 2201.08953.
33.	Xie X, Chen J, Li Y, et al. MI2GAN: generative adversarial network for medical image domain adaptation using mutual information constraint//International Conference on Medical Image Computing and Computer-Assisted Intervention. Lima: Springer, 2020: 516-525.
34.	Yu B, Zhou L, Wang L, et al. Ea-GANs: edge-aware generative adversarial networks for cross-modality MR image synthesis. IEEE Transactions on Medical Imaging, 2019, 38(7): 1750-1762.
35.	Yang X, Lin Y, Wang Z, et al. Bi-modality medical image synthesis using semi-supervised sequential generative adversarial networks. IEEE Journal of Biomedical and Health Informatics, 2019, 24(3): 855-865.
36.	Gilbert A, Marciniak M, Rodero C, et al. Generating synthetic labeled data from existing anatomical models: an example with echocardiography segmentation. IEEE Transactions on Medical Imaging, 2021, 40(10): 2783-2794.
37.	Han C, Rundo L, Araki R, et al. Combining noise-to-image and image-to-image GANs: brain MR image augmentation for tumor detection. IEEE Access, 2019, 7: 156966-156977.
38.	Chen C, Qin C, Qiu H, et al. Realistic adversarial data augmentation for MR image segmentation// International Conference on Medical Image Computing and Computer-Assisted Intervention. Lima: Springer, 2020: 667-677.
39.	Chaitanya K, Karani N, Baumgartner C F, et al. Semi-supervised task-driven data augmentation for medical image segmentation. Medical Image Analysis, 2021, 68: 101934.
40.	Chaitanya K, Karani N, Baumgartner C F, et al. Semi-supervised and task-driven data augmentation// International Conference on Medical Image Computing and Computer-Assisted Intervention. Hongkong: Springer, 2020: 667-677.
41.	Cheng K, Iriondo C, Calivá F, et al. Adversarial policy gradient for deep learning image augmentation//International Conference on Medical Image Computing and Computer-Assisted Intervention. Shenzhen: Springer, 2019: 450-458.
42.	Gao Y, Tang Z, Zhou M, et al. Enabling data diversity: efficient automatic augmentation via regularized adversarial training//International Conference on Information Processing in Medical Imaging. Virtual Event: Springer, 2021: 85-97.

1. 邹茂扬, 杨昊, 潘光晖, 等. 深度学习在医学图像配准上的研究进展与挑战. 生物医学工程学杂志, 2019, 36(4): 677-683.
2. 游森榕, 胡圣烨, 申妍燕, 等. 生成对抗网络在医学图像计算上的进展与展望. 计算机科学与应用, 2021, 11(7): 1949-1961.
3. 肖汉光, 冉智强, 黄金锋, 等. 基于电子计算机断层扫描图像的肺实质分割方法研究进展. 生物医学工程学杂志, 2021, 38(2): 379-386.
4. Chlap P, Min H, Vandenberg N, et al. A review of medical image data augmentation techniques for deep learning applications. Journal of Medical Imaging and Radiation Oncology, 2021, 65(5): 545-563.
5. 陈英, 郑铖, 易珍, 等. 肝脏及肿瘤图像分割方法综述. 计算机应用研究, 2022, 39(3): 641-650.
6. Thaha M M, Kumar K P M, Murugan B S, et al. Brain tumor segmentation using convolutional neural networks in MRI images. Journal of Medical Systems, 2019, 43(9): 294.
7. Ding Y, Yu X, Yang Y. Modeling the probabilistic distribution of unlabeled data for one-shot medical image segmentation//Proceedings of the AAAI Conference on Artificial Intelligence. Vancouver: AAAI. 2021, 35(2): 1246-1254.
8. Brahim I, Fourer D, Vigneron V, et al. Deep learning methods for MRI brain tumor segmentation: a comparative study//2019 Ninth International Conference on Image Processing Theory, Tools and Applications (IPTA). Istanbul: IEEE, 2019: 1-6.
9. Afzal S, Maqsood M, Nazir F, et al. A data augmentation-based framework to handle class imbalance problem for Alzheimer’s stage detection. IEEE Access, 2019, 7: 115528-115539.
10. Karani N, Erdil E, Chaitanya K, et al. Test-time adaptable neural networks for robust medical image segmentation. Medical Image Analysis, 2021, 68: 101907.
11. Chen C, Hammernik K, Ouyang C, et al. Cooperative training and latent space data augmentation for robust medical image segmentation//International Conference on Medical Image Computing and Computer-Assisted Intervention. Strasbourg: Springer, 2021: 149-159.
12. Novosad P, Fonov V, Collins D L, et al. Accurate and robust segmentation of neuroanatomy in T1‐weighted MRI by combining spatial priors with deep convolutional neural networks. Human Brain Mapping, 2020, 41(2): 309-327.
13. Pan H, Feng Y, Chen Q, et al. Prostate segmentation from 3D MRI using a two-stage model and variable-input based uncertainty measure//2019 IEEE 16th International Symposium on Biomedical Imaging. Venice: IEEE, 2019: 468-471.
14. Yang F, Liang F, Lu L, et al. Dual attention-guided and learnable spatial transformation data augmentation multi-modal unsupervised medical image segmentation. Biomedical Signal Processing and Control, 2022, 78: 103849.
15. Chen Y, Yang X H, Wei Z, et al. Generative adversarial networks in medical image augmentation: a review. Computers in Biology and Medicine, 2022, 144: 105382.
16. 徐峰, 马小萍, 刘立波. 基于生成对抗网络的甲状腺超声图像文本跨模态检索方法. 生物医学工程学杂志, 2020, 37(4): 641-651.
17. Frid-Adar M, Diamant I, Klang E, et al. GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification. Neurocomputing, 2018, 321: 321-331.
18. Kwon G, Han C, Kim D. Generation of 3D brain MRI using auto-encoding generative adversarial networks//International Conference on Medical Image Computing and Computer-Assisted Intervention. Shenzhen: Springer, 2019: 118-126.
19. Chuquicusma M J M, Hussein S, Burt J, et al. How to fool radiologists with generative adversarial networks? A visual turing test for lung cancer diagnosis//2018 IEEE 15th International Symposium on Biomedical Imaging. Washington: IEEE, 2018: 240-244.
20. Calimeri F, Marzullo A, Stamile C, et al. Biomedical data augmentation using generative adversarial neural networks//International Conference on Artificial Neural Networks. Alghero: Springer, 2017: 626-634.
21. Bermudez C, Plassard A J, Davis L T, et al. Learning implicit brain MRI manifolds with deep learning. Proceedings of SPIE-the International Society for Optical Engineering, 2018, 10574: 15704L.
22. Wang Z, Lin Y, Cheng K T T, et al. Semi-supervised mp-MRI data synthesis with stitchlayer and auxiliary distance maximization. Medical Image Analysis, 2020, 59: 101565.
23. Jiang Y, Chen H, Loew M, et al. COVID-19 CT image synthesis with a conditional generative adversarial network. IEEE Journal of Biomedical and Health Informatics, 2020, 25(2): 441-452.
24. Wang Q, Zhang X, Zhang W, et al. Realistic lung nodule synthesis with multi-target co-guided adversarial mechanism. IEEE Transactions on Medical Imaging, 2021, 40(9): 2343-2353.
25. Costa P, Galdran A, Meyer M I, et al. End-to-end adversarial retinal image synthesis. IEEE Transactions on Medical Imaging, 2017, 37(3): 781-791.
26. Oliveira D A B. Controllable skin lesion synthesis using texture patches, bézier curves and conditional gans//2020 IEEE 17th International Symposium on Biomedical Imaging. Lowa: IEEE, 2020: 1798-1802.
27. Zhang T, Cheng J, Fu H, et al. Noise adaptation generative adversarial network for medical image analysis. IEEE Transactions on Medical Imaging, 2019, 39(4): 1149-1159.
28. 储珺, 林文杰, 徐鹏. 目标检测中特征不匹配问题研究进展. 南昌航空大学学报:自然科学版, 2021, 35(3): 1-8.
29. 张颖麟, 胡衍, 东田理沙, 等. 生成对抗式网络及其医学影像应用研究综述. 中国图象图形学报, 2022, 27(3): 687-703.
30. Kong L, Lian C, Huang D, et al. Breaking the dilemma of medical image-to-image translation. Advances in Neural Information Processing Systems, 2021, 34: 1964-1978.
31. Luo Y, Nie D, Zhan B, et al. Edge-preserving MRI image synthesis via adversarial network with iterative multi-scale fusion. Neurocomputing, 2021, 452: 63-77.
32. Xie G, Wang J, Huang Y, et al. FedMed-GAN: federated domain translation on unsupervised cross-modality brain image synthesis. arXiv preprint, 2022, arXiv: 2201.08953.
33. Xie X, Chen J, Li Y, et al. MI2GAN: generative adversarial network for medical image domain adaptation using mutual information constraint//International Conference on Medical Image Computing and Computer-Assisted Intervention. Lima: Springer, 2020: 516-525.
34. Yu B, Zhou L, Wang L, et al. Ea-GANs: edge-aware generative adversarial networks for cross-modality MR image synthesis. IEEE Transactions on Medical Imaging, 2019, 38(7): 1750-1762.
35. Yang X, Lin Y, Wang Z, et al. Bi-modality medical image synthesis using semi-supervised sequential generative adversarial networks. IEEE Journal of Biomedical and Health Informatics, 2019, 24(3): 855-865.
36. Gilbert A, Marciniak M, Rodero C, et al. Generating synthetic labeled data from existing anatomical models: an example with echocardiography segmentation. IEEE Transactions on Medical Imaging, 2021, 40(10): 2783-2794.
37. Han C, Rundo L, Araki R, et al. Combining noise-to-image and image-to-image GANs: brain MR image augmentation for tumor detection. IEEE Access, 2019, 7: 156966-156977.
38. Chen C, Qin C, Qiu H, et al. Realistic adversarial data augmentation for MR image segmentation// International Conference on Medical Image Computing and Computer-Assisted Intervention. Lima: Springer, 2020: 667-677.
39. Chaitanya K, Karani N, Baumgartner C F, et al. Semi-supervised task-driven data augmentation for medical image segmentation. Medical Image Analysis, 2021, 68: 101934.
40. Chaitanya K, Karani N, Baumgartner C F, et al. Semi-supervised and task-driven data augmentation// International Conference on Medical Image Computing and Computer-Assisted Intervention. Hongkong: Springer, 2020: 667-677.
41. Cheng K, Iriondo C, Calivá F, et al. Adversarial policy gradient for deep learning image augmentation//International Conference on Medical Image Computing and Computer-Assisted Intervention. Shenzhen: Springer, 2019: 450-458.
42. Gao Y, Tang Z, Zhou M, et al. Enabling data diversity: efficient automatic augmentation via regularized adversarial training//International Conference on Information Processing in Medical Imaging. Virtual Event: Springer, 2021: 85-97.

Journal of Biomedical Engineering

Research progress on medical image dataset expansion methods

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content