A cephalometric landmark detection method using dual-encoder on X-ray image_Journal of Biomedical Engineering

Authors：

 DAI Chao ¹ , HUANG Chaolin ² , XU Minpeng ¹ , WANG Yang ³

1. Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin 300354, P. R. China;
2. School of Electrical Engineering and Automation, Jiangxi University of Science and Technology, Nanchang 341099, P. R. China;
3. School of Mechanical Engineering, Anhui University of Technology, Maanshan, Anhui 243002, P. R. China;

Corresponding author：

DAI Chao, Email: daichao2021@tju.edu.cn

Keywords：

Dual-encoder; High-resolution; Cephalometric landmark; Transformer

DOI：

10.7507/1001-5515.202411034

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Accurate detection of cephalometric landmarks is crucial for orthodontic diagnosis and treatment planning. Current landmark detection methods are mainly divided into heatmap-based and regression-based approaches. However, these methods often rely on parallel computation of multiple models to improve accuracy, significantly increasing the complexity of training and deployment. This paper presented a novel regression method that can simultaneously detect all cephalometric landmarks in high-resolution X-ray images. By leveraging the encoder module of Transformer, a dual-encoder model was designed to achieve coarse-to-fine localization of cephalometric landmarks. The entire model consisted of three main components: a feature extraction module, a reference encoder module, and a fine-tuning encoder module, responsible for feature extraction and fusion of X-ray images, coarse localization of cephalometric landmarks, and fine localization of landmarks, respectively. The model was fully end-to-end differentiable and could learn the intercorrelation relationships between cephalometric landmarks. Experimental results showed that the successful detection rate (SDR) of our algorithm was superior to other existing methods. It attained the highest 2 mm SDR of 89.51% on Test Set 1 of the ISBI2015 dataset and 90.68% on the test set of the ISBI2023 dataset. Meanwhile, it reduces memory consumption and enhances the model’s popularity and applicability, providing more reliable technical support for orthodontic diagnosis and treatment plan formulation.

1.	Kamoen A, Dermaut S S S L, Verbeeck R. The clinical significance of error measurement in the interpretation of treatment results. Eur J Orthod, 2001, 23(5): 569-578.
2.	Silveira H L, Silveira H E. Reproducibility of cephalometric measurements made by three radiology clinics. Angle Orthod, 2006, 76(3): 394-399.
3.	Albarakati S F, Kula K S, Ghoneima A A. The reliability and reproducibility of cephalometric measurements: a comparison of conventional and digital methods. Dentomaxillofac Radiol, 2012, 41(1): 11-17.
4.	Chen Runnan, Ma Yuexin, Chen Nenglun, et al. Cephalometric landmark detection by attentive feature pyramid fusion and regression-voting// Medical Image Computing and Computer Assisted Intervention (MICCAI). Cham: Springer, 2019: 873-881.
5.	Devereux L, Moles D, Cunningham S J, et al. How important are lateral cephalometric radiographs in orthodontic treatment planning? Amer J of Orth and Dent Orth, 2011, 139(2): 175-181.
6.	Indermun S, Shaik S, Nyirenda C, et al. Human examination and artificial intelligence in cephalometric landmark detection-is AI ready to take over? Dentomaxillofac Radiol, 2023, 52(6): 20220362.
7.	Juneja M, Garg P, Kaur R, et al. A review on cephalometric landmark detection techniques. Biomed Signal Process Control, 2021, 66: 102486.
8.	Qian Jiahong, Luo Weizhi, Cheng Ming, et al. CephaNN: A multi-head attention network for cephalometric landmark detection. IEEE Access, 2020, 8: 112633-112641.
9.	Ao Y, Wu H. Feature aggregation and refinement network for 2d anatomical landmark detection. J of Dig Imag, 2023, 36(2): 547-561.
10.	Huang K, Feng F. An intelligent shooting reward learning network scheme for medical image landmark detection. Appl Sci, 2022, 12: 10190.
11.	Song Y, Qiao X, Iwamoto Y, et al. Automatic cephalometric landmark detection on X-ray images using a deep-learning method. Appl Sci, 2020, 10: 2547.
12.	Zhong Z, Li J, Zhan Z, et al. An attention-guided deep regression model for landmark detection in cephalograms// Medical Image Computing and Computer Assisted Intervention (MICCAI). Cham: Springer, 2019: 540-548.
13.	Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation// Medical Image Computing and Computer-Assisted Intervention (MICCAI). Cham: Springer, 2015, 9351: 5-16.
14.	Shaker A, Maaz M, Rasheed H, et al. UNETR++: delving into efficient and accurate 3d medical image segmentation. IEEE Trans Med Imaging, 2024, 43(9): 3377-3390.
15.	Ye Ziyang, Yu Haiyang, Li Bin. Uncertainty-aware u-net for medical landmark detection. Arxiv preprint, 2023: 2303.10349v1.
16.	Song Y, Qiao X, Iwamoto Y, et al. An efficient deep learning based coarse-to-fine cephalometric landmark detection method. IEICE Trans Info Sys, 2021, 104(8): 1359-1366.
17.	Zeng Minmin, Yan Zhenlei, Liu Shuai, et al. Cascaded convolutional networks for automatic cephalometric landmark detection. Med Ima Ana, 2021, 68: 101904.
18.	Zhang K, Zhang Z, Li Z, et al. Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Sig Proc Let, 2016, 23(10): 1499-1503.
19.	Gilmour L, Ray N. Locating cephalometric X-ray landmarks with foveated pyramid attention// International Conference on Medical Imaging with Deep Learning (MIDL). Montreal: PMLR, 2020: 262-276.
20.	Yao Jie, Zeng Wei, He Tao, et al. Automatic localization of cephalometric landmarks based on convolutional neural network. Amer J Orth Dent Orth, 2022, 161(3): 250-259.
21.	Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need// Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS). New York: Curran Associates Inc, 2017: 6000-6010.
22.	Yang S, Quan Z, Nie M, et al. Transpose: Keypoint localization via transformer// IEEE/CVF International Conference on Computer Vision (ICCV). Montreal: IEEE, 2021: 11782-11792.
23.	Yuan Yuhui, Fu Rao, Huang Lang, et al. Hrformer: High-resolution transformer for dense prediction// Proceedings of the 35th International Conference on Neural Information Processing Systems (NIPS). New York: Curran Associates Inc, 2021: 7281-7293.
24.	Mao Weian, Ge Yongtao, Shen Chunhua, et al. Poseur: Direct human pose regression with Transformers// Computer Vision-ECCV 2022: 17th European Conference. Cham: Springer, 2022, 6: 72-88.
25.	Li Hui, Guo Zidong, Rhee Seon-Min, et al. Towards accurate facial landmark detection via cascaded transformers// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE, 2022: 4176-4185.
26.	Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16x16 words: Transformers for image recognition at scale// International Conference on Learning Representations (ICLR). Vienna: ICLR, 2021: 1-21.
27.	He Kaiming, Zhang Xiangyu, Ren Shaoqing, et al. Deep residual learning for image recognition// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016: 770-778.
28.	Zhao Ting, Wu Xiangqian. Pyramid feature attention network for saliency detection// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach: IEEE/CVF, 2019: 3080-3089.
29.	Devlin J, Chang M W, Lee K, et al. BERT: Pre-training of deep bidirectional Transformers for language understanding// Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis: ACL Anthology, 2019, 1: 4171-4186.
30.	Zhang Hang, Zhang Jinwei, Li Chao, et al. All-net: Anatomical information lesion-wise loss function integrated into neural network for multiple sclerosis lesion segmentation. Neuroimage Clin, 2021, 32: 102854.
31.	S Muhammad A K, Kanwal Z, Ulfat B, et al. CEPHA29: Automatic cephalometric landmark detection challenge 2023. ArXiv preprint, 2022: 2212.04808.
32.	Jin Haibo, Che Haoxuan, Chen Hao. Unsupervised domain adaptation for anatomical landmark detection// International Conference on Medical Image Computing and ComputerAssisted Intervention (MICCAI). Cham: Springer, 2023, 14220: 695-705.
33.	Kingma D P, Ba J L. Adam: A method for stochastic optimization// International Conference on Learning Representations (ICLR). Ithaca: ArXiv, 2015: 13.
34.	Li Yanjie, Yang Sen, Liu Peidong, et al. Simcc: A simple coordinate classification perspective for human pose estimation// Computer Vision - ECCV 2022: 17th European Conference(ECCV). Cham: Springer, 2022, 13666: 89-106.
35.	King C H, Wang Y L, Lin W Y, et al. Automatic cephalometric landmark detection on x-ray images using object detection// IEEE 19th International Symposium on Biomedical Imaging (ISBI). Kolkata: IEEE, 2022: 1-4.
36.	Hong W, Kim S M, Choi J, et al. Deep reinforcement learning using a multi-scale agent with a normalized reward strategy for automatic cephalometric landmark detection// 2023 4th International Conference on Big Data Analytics and Practices (IBDAP). Bangkok: IEEE 2023: 1-6.
37.	Wang C W, Huang C T, Lee J H, et al. A benchmark for comparison of dental radiography analysis algorithms. Med Image Anal, 2016, 31: 63-76.

1. Kamoen A, Dermaut S S S L, Verbeeck R. The clinical significance of error measurement in the interpretation of treatment results. Eur J Orthod, 2001, 23(5): 569-578.
2. Silveira H L, Silveira H E. Reproducibility of cephalometric measurements made by three radiology clinics. Angle Orthod, 2006, 76(3): 394-399.
3. Albarakati S F, Kula K S, Ghoneima A A. The reliability and reproducibility of cephalometric measurements: a comparison of conventional and digital methods. Dentomaxillofac Radiol, 2012, 41(1): 11-17.
4. Chen Runnan, Ma Yuexin, Chen Nenglun, et al. Cephalometric landmark detection by attentive feature pyramid fusion and regression-voting// Medical Image Computing and Computer Assisted Intervention (MICCAI). Cham: Springer, 2019: 873-881.
5. Devereux L, Moles D, Cunningham S J, et al. How important are lateral cephalometric radiographs in orthodontic treatment planning? Amer J of Orth and Dent Orth, 2011, 139(2): 175-181.
6. Indermun S, Shaik S, Nyirenda C, et al. Human examination and artificial intelligence in cephalometric landmark detection-is AI ready to take over? Dentomaxillofac Radiol, 2023, 52(6): 20220362.
7. Juneja M, Garg P, Kaur R, et al. A review on cephalometric landmark detection techniques. Biomed Signal Process Control, 2021, 66: 102486.
8. Qian Jiahong, Luo Weizhi, Cheng Ming, et al. CephaNN: A multi-head attention network for cephalometric landmark detection. IEEE Access, 2020, 8: 112633-112641.
9. Ao Y, Wu H. Feature aggregation and refinement network for 2d anatomical landmark detection. J of Dig Imag, 2023, 36(2): 547-561.
10. Huang K, Feng F. An intelligent shooting reward learning network scheme for medical image landmark detection. Appl Sci, 2022, 12: 10190.
11. Song Y, Qiao X, Iwamoto Y, et al. Automatic cephalometric landmark detection on X-ray images using a deep-learning method. Appl Sci, 2020, 10: 2547.
12. Zhong Z, Li J, Zhan Z, et al. An attention-guided deep regression model for landmark detection in cephalograms// Medical Image Computing and Computer Assisted Intervention (MICCAI). Cham: Springer, 2019: 540-548.
13. Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation// Medical Image Computing and Computer-Assisted Intervention (MICCAI). Cham: Springer, 2015, 9351: 5-16.
14. Shaker A, Maaz M, Rasheed H, et al. UNETR++: delving into efficient and accurate 3d medical image segmentation. IEEE Trans Med Imaging, 2024, 43(9): 3377-3390.
15. Ye Ziyang, Yu Haiyang, Li Bin. Uncertainty-aware u-net for medical landmark detection. Arxiv preprint, 2023: 2303.10349v1.
16. Song Y, Qiao X, Iwamoto Y, et al. An efficient deep learning based coarse-to-fine cephalometric landmark detection method. IEICE Trans Info Sys, 2021, 104(8): 1359-1366.
17. Zeng Minmin, Yan Zhenlei, Liu Shuai, et al. Cascaded convolutional networks for automatic cephalometric landmark detection. Med Ima Ana, 2021, 68: 101904.
18. Zhang K, Zhang Z, Li Z, et al. Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Sig Proc Let, 2016, 23(10): 1499-1503.
19. Gilmour L, Ray N. Locating cephalometric X-ray landmarks with foveated pyramid attention// International Conference on Medical Imaging with Deep Learning (MIDL). Montreal: PMLR, 2020: 262-276.
20. Yao Jie, Zeng Wei, He Tao, et al. Automatic localization of cephalometric landmarks based on convolutional neural network. Amer J Orth Dent Orth, 2022, 161(3): 250-259.
21. Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need// Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS). New York: Curran Associates Inc, 2017: 6000-6010.
22. Yang S, Quan Z, Nie M, et al. Transpose: Keypoint localization via transformer// IEEE/CVF International Conference on Computer Vision (ICCV). Montreal: IEEE, 2021: 11782-11792.
23. Yuan Yuhui, Fu Rao, Huang Lang, et al. Hrformer: High-resolution transformer for dense prediction// Proceedings of the 35th International Conference on Neural Information Processing Systems (NIPS). New York: Curran Associates Inc, 2021: 7281-7293.
24. Mao Weian, Ge Yongtao, Shen Chunhua, et al. Poseur: Direct human pose regression with Transformers// Computer Vision-ECCV 2022: 17th European Conference. Cham: Springer, 2022, 6: 72-88.
25. Li Hui, Guo Zidong, Rhee Seon-Min, et al. Towards accurate facial landmark detection via cascaded transformers// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE, 2022: 4176-4185.
26. Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16x16 words: Transformers for image recognition at scale// International Conference on Learning Representations (ICLR). Vienna: ICLR, 2021: 1-21.
27. He Kaiming, Zhang Xiangyu, Ren Shaoqing, et al. Deep residual learning for image recognition// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016: 770-778.
28. Zhao Ting, Wu Xiangqian. Pyramid feature attention network for saliency detection// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach: IEEE/CVF, 2019: 3080-3089.
29. Devlin J, Chang M W, Lee K, et al. BERT: Pre-training of deep bidirectional Transformers for language understanding// Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis: ACL Anthology, 2019, 1: 4171-4186.
30. Zhang Hang, Zhang Jinwei, Li Chao, et al. All-net: Anatomical information lesion-wise loss function integrated into neural network for multiple sclerosis lesion segmentation. Neuroimage Clin, 2021, 32: 102854.
31. S Muhammad A K, Kanwal Z, Ulfat B, et al. CEPHA29: Automatic cephalometric landmark detection challenge 2023. ArXiv preprint, 2022: 2212.04808.
32. Jin Haibo, Che Haoxuan, Chen Hao. Unsupervised domain adaptation for anatomical landmark detection// International Conference on Medical Image Computing and ComputerAssisted Intervention (MICCAI). Cham: Springer, 2023, 14220: 695-705.
33. Kingma D P, Ba J L. Adam: A method for stochastic optimization// International Conference on Learning Representations (ICLR). Ithaca: ArXiv, 2015: 13.
34. Li Yanjie, Yang Sen, Liu Peidong, et al. Simcc: A simple coordinate classification perspective for human pose estimation// Computer Vision - ECCV 2022: 17th European Conference(ECCV). Cham: Springer, 2022, 13666: 89-106.
35. King C H, Wang Y L, Lin W Y, et al. Automatic cephalometric landmark detection on x-ray images using object detection// IEEE 19th International Symposium on Biomedical Imaging (ISBI). Kolkata: IEEE, 2022: 1-4.
36. Hong W, Kim S M, Choi J, et al. Deep reinforcement learning using a multi-scale agent with a normalized reward strategy for automatic cephalometric landmark detection// 2023 4th International Conference on Big Data Analytics and Practices (IBDAP). Bangkok: IEEE 2023: 1-6.
37. Wang C W, Huang C T, Lee J H, et al. A benchmark for comparison of dental radiography analysis algorithms. Med Image Anal, 2016, 31: 63-76.

Journal of Biomedical Engineering

Latest ArticlesA cephalometric landmark detection method using dual-encoder on X-ray image

Abstract Full text Figures/Tables Video References Cited by

Format

Content