Study on speech imagery electroencephalography decoding of Chinese words based on the CAM-Net model_Journal of Biomedical Engineering

Authors：

LIU Xiaolong ,  YANG Banghua , GAN An’an ,  ZHANG Jie

School of Mechanical and Electrical Engineering and Automation, Shanghai University, Shanghai 200444, P. R. China;

Corresponding author：

YANG Banghua, Email: yangbanghua@shu.edu.cn; ZHANG Jie, Email: jjy2001_cn@shu.edu.cn

Keywords：

Brain-computer interface; Speech imagery; Attention mechanism; Long short-term memory network

DOI：

10.7507/1001-5515.202503048

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Speech imagery is an emerging brain-computer interface (BCI) paradigm with potential to provide effective communication for individuals with speech impairments. This study designed a Chinese speech imagery paradigm using three clinically relevant words—“Help me”, “Sit up” and “Turn over”—and collected electroencephalography (EEG) data from 15 healthy subjects. Based on the data, a Channel Attention Multi-Scale Convolutional Neural Network (CAM-Net) decoding algorithm was proposed, which combined multi-scale temporal convolutions with asymmetric spatial convolutions to extract multidimensional EEG features, and incorporated a channel attention mechanism along with a bidirectional long short-term memory network to perform channel weighting and capture temporal dependencies. Experimental results showed that CAM-Net achieved a classification accuracy of 48.54% in the three-class task, outperforming baseline models such as EEGNet and Deep ConvNet, and reached a highest accuracy of 64.17% in the binary classification between “Sit up” and “Turn over”. This work provides a promising approach for future Chinese speech imagery BCI research and applications.

Citation： LIU Xiaolong, YANG Banghua, GAN An’an, ZHANG Jie. Study on speech imagery electroencephalography decoding of Chinese words based on the CAM-Net model. Journal of Biomedical Engineering, 2025, 42(3): 473-479. doi: 10.7507/1001-5515.202503048 Copy

1.	Vaughan T M, Heetderks W J, Trejo L J, et al. Brain-computer interface technology: a review of the Second International Meeting. IEEE Trans Neural Syst Rehabil Eng, 2003, 11(2): 94-109.
2.	伏云发, 郭衍龙, 张夏冰. 脑-机接口—革命性的人机交互. 北京: 国防工业出版社, 2020.
3.	刘艳鹏, 龚安民, 丁鹏, 等. 基于言语想象的脑机交互关键技术. 生物医学工程学杂志, 2022, 39(3): 596-611.
4.	Anumanchipalli G K, Chartier J, Chang E F. Speech synthesis from neural decoding of spoken sentences. Nature, 2019, 568(7753): 493-498.
5.	Riaz A, Akhtar S, Iftikhar S, et al. Inter comparison of classification techniques for vowel speech imagery using EEG sensors// The 2014 2nd International Conference on Systems and Informatics (ICSAI 2014). Shanghai: IEEE, 2014: 712-717.
6.	Zhang H, Guo Z, Chen F. The effects of different brain regions on fNIRS-based task-state detection in speech imagery// 2023 45th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). Sydney: IEEE, 2023: 1-4.
7.	Panachakel J T, Ramakrishnan A G. Decoding covert speech from EEG-a comprehensive review. Front Neurosci, 2021, 15: 642251.
8.	Van Gerven M, Farquhar J, Schaefer R, et al. The brain–computer interface cycle. J Neural Eng, 2009, 6(4): 041001.
9.	Lee S H, Lee Y E, Lee S W. Toward imagined speech based smart communication system: potential applications on metaverse conditions// 2022 10th International Winter Conference on Brain-Computer Interface (BCI). Gangwon-do: IEEE, 2022: 1-4.
10.	阮盛捷, 谢宏. 基于功能近红外信号的脑机接口(BCI)在肌萎缩性侧索硬化症(ALS)患者交流中的研究. 现代计算机(专业版), 2018(17): 3-7.
11.	Bisla M, Anand R S. Analysis of imagined speech characteristics using phase-based connectivity measures// 2023 IEEE 13th International Conference on Control System, Computing and Engineering (ICCSCE). Penang: IEEE, 2023: 68-73.
12.	郭苗苗, 齐志光, 王磊, 等. 语言脑机接口康复系统中的参数优化研究. 信号处理, 2018, 34(8): 974-983.
13.	Blankertz B, Tomioka R, Lemm S, et al. Optimizing spatial filters for robust EEG single-trial analysis. IEEE Signal Process Mag, 2007, 25(1): 41-56.
14.	Wang X, Lai Y H, Chen F. Intended speech classification with EEG signals based on a temporal attention mechanism: A study of mandarin vowels// 2024 46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). Orlando: IEEE, 2024: 1-4.
15.	Zhang Z, Ding X, Bao Y, et al. Chisco: An EEG-based BCI dataset for decoding of imagined speech. Sci Data, 2024, 11(1): 1265.
16.	Lawhern V J, Solon A J, Waytowich N R, et al. EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces. J Neural Eng, 2018, 15(5): 056013.
17.	Schirrmeister R T, Springenberg J T, Fiederer L D J, et al. Deep learning with convolutional neural networks for EEG decoding and visualization. Hum Brain Mapp, 2017, 38(11): 5391-5420.
18.	Hermosilla D M, Codorniú R T, Baracaldo R L, et al. Shallow convolutional network excel for classifying motor imagery EEG in BCI applications. IEEE Access, 2021, 9: 98275-98286.
19.	Ding Y, Robinson N, Zhang S, et al. TSception: Capturing temporal dynamics and spatial asymmetry from EEG for emotion recognition. IEEE Trans Affect Comput, 2022, 14(3): 2238-2250.
20.	Krishna G, Tran C, Han Y, et al. Speech synthesis using EEG// ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Barcelona: IEEE, 2020: 1235-1238.
21.	Lotte F, Brumberg J S, Brunner P, et al. Electrocorticographic representations of segmental features in continuous speech. Front Hum Neurosci, 2015, 9: 97.
22.	Bocquelet F, Hueber T, Girin L, et al. Key considerations in designing a speech brain-computer interface. J Physiol Paris, 2016, 110(4): 392-401.
23.	Ang K K, Chin Z Y, Zhang H, et al. Filter bank common spatial pattern (FBCSP) in brain-computer interface// 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence). Hong Kong: IEEE, 2008: 2390-2397.

1. Vaughan T M, Heetderks W J, Trejo L J, et al. Brain-computer interface technology: a review of the Second International Meeting. IEEE Trans Neural Syst Rehabil Eng, 2003, 11(2): 94-109.
2. 伏云发, 郭衍龙, 张夏冰. 脑-机接口—革命性的人机交互. 北京: 国防工业出版社, 2020.
3. 刘艳鹏, 龚安民, 丁鹏, 等. 基于言语想象的脑机交互关键技术. 生物医学工程学杂志, 2022, 39(3): 596-611.
4. Anumanchipalli G K, Chartier J, Chang E F. Speech synthesis from neural decoding of spoken sentences. Nature, 2019, 568(7753): 493-498.
5. Riaz A, Akhtar S, Iftikhar S, et al. Inter comparison of classification techniques for vowel speech imagery using EEG sensors// The 2014 2nd International Conference on Systems and Informatics (ICSAI 2014). Shanghai: IEEE, 2014: 712-717.
6. Zhang H, Guo Z, Chen F. The effects of different brain regions on fNIRS-based task-state detection in speech imagery// 2023 45th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). Sydney: IEEE, 2023: 1-4.
7. Panachakel J T, Ramakrishnan A G. Decoding covert speech from EEG-a comprehensive review. Front Neurosci, 2021, 15: 642251.
8. Van Gerven M, Farquhar J, Schaefer R, et al. The brain–computer interface cycle. J Neural Eng, 2009, 6(4): 041001.
9. Lee S H, Lee Y E, Lee S W. Toward imagined speech based smart communication system: potential applications on metaverse conditions// 2022 10th International Winter Conference on Brain-Computer Interface (BCI). Gangwon-do: IEEE, 2022: 1-4.
10. 阮盛捷, 谢宏. 基于功能近红外信号的脑机接口(BCI)在肌萎缩性侧索硬化症(ALS)患者交流中的研究. 现代计算机(专业版), 2018(17): 3-7.
11. Bisla M, Anand R S. Analysis of imagined speech characteristics using phase-based connectivity measures// 2023 IEEE 13th International Conference on Control System, Computing and Engineering (ICCSCE). Penang: IEEE, 2023: 68-73.
12. 郭苗苗, 齐志光, 王磊, 等. 语言脑机接口康复系统中的参数优化研究. 信号处理, 2018, 34(8): 974-983.
13. Blankertz B, Tomioka R, Lemm S, et al. Optimizing spatial filters for robust EEG single-trial analysis. IEEE Signal Process Mag, 2007, 25(1): 41-56.
14. Wang X, Lai Y H, Chen F. Intended speech classification with EEG signals based on a temporal attention mechanism: A study of mandarin vowels// 2024 46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). Orlando: IEEE, 2024: 1-4.
15. Zhang Z, Ding X, Bao Y, et al. Chisco: An EEG-based BCI dataset for decoding of imagined speech. Sci Data, 2024, 11(1): 1265.
16. Lawhern V J, Solon A J, Waytowich N R, et al. EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces. J Neural Eng, 2018, 15(5): 056013.
17. Schirrmeister R T, Springenberg J T, Fiederer L D J, et al. Deep learning with convolutional neural networks for EEG decoding and visualization. Hum Brain Mapp, 2017, 38(11): 5391-5420.
18. Hermosilla D M, Codorniú R T, Baracaldo R L, et al. Shallow convolutional network excel for classifying motor imagery EEG in BCI applications. IEEE Access, 2021, 9: 98275-98286.
19. Ding Y, Robinson N, Zhang S, et al. TSception: Capturing temporal dynamics and spatial asymmetry from EEG for emotion recognition. IEEE Trans Affect Comput, 2022, 14(3): 2238-2250.
20. Krishna G, Tran C, Han Y, et al. Speech synthesis using EEG// ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Barcelona: IEEE, 2020: 1235-1238.
21. Lotte F, Brumberg J S, Brunner P, et al. Electrocorticographic representations of segmental features in continuous speech. Front Hum Neurosci, 2015, 9: 97.
22. Bocquelet F, Hueber T, Girin L, et al. Key considerations in designing a speech brain-computer interface. J Physiol Paris, 2016, 110(4): 392-401.
23. Ang K K, Chin Z Y, Zhang H, et al. Filter bank common spatial pattern (FBCSP) in brain-computer interface// 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence). Hong Kong: IEEE, 2008: 2390-2397.

Journal of Biomedical Engineering

Study on speech imagery electroencephalography decoding of Chinese words based on the CAM-Net model

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content