Bilinear neural network with 3-D attention for brain decoding of motor imagery movements from the human EEG | JoVE Visualize

Bilinear neural network with 3-D attention for brain decoding of motor imagery movements from the human EEG

Chen-Chen Fan^1,2, Hongjun Yang¹, Zeng-Guang Hou^1,2,3

¹State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190 China.

Cognitive Neurodynamics|March 31, 2021

View abstract on PubMed

Summary

We developed QNet, a novel deep learning model for electroencephalography (EEG) signal analysis. QNet utilizes a 3D attention module (3D-AM) to automatically select relevant EEG channels and time points, improving classification accuracy.

Area of Science:

Neuroscience
Machine Learning
Signal Processing

Background:

Deep learning models have shown success in computer vision and natural language processing.
Convolutional networks have been previously applied to electroencephalography (EEG) signals, achieving competitive results.
Existing methods may lack efficient mechanisms for automatic selection of relevant EEG channels and time points.

Purpose of the Study:

To propose a novel deep learning network, QNet, for enhanced EEG signal classification.
To introduce a new attention module, 3D-AM, for learning attention weights across EEG channels, time points, and feature maps.
To evaluate the performance of QNet against state-of-the-art methods on the EEG Motor Movement/Imagery Dataset.

Main Methods:

Developed QNet, a novel network architecture incorporating a 3D attention module (3D-AM).
The 3D-AM module learns attention weights for EEG channels, time points, and feature maps, enabling automatic electrode and time selection.
Employed a dual-branch structure within QNet for fusing bilinear vectors for classification.
Conducted experiments on the EEG Motor Movement/Imagery Dataset for four, three, and two-class classification tasks.

Main Results:

QNet achieved average cross-validation accuracies of 65.82% (4-class), 74.75% (3-class), and 82.88% (2-class).
These results represent significant improvements over state-of-the-art methods, outperforming them by 7.24%, 4.93%, and 2.45%, respectively.
Visualization of attention weights demonstrated QNet's capability for effective electrode channel selection.

Conclusions:

QNet, with its 3D-AM module, offers an effective approach for EEG signal classification.
The attention mechanism in QNet facilitates automatic and data-driven selection of important EEG channels and time segments.
QNet demonstrates superior performance compared to existing methods, highlighting its potential for advancing EEG analysis.