Multimodal fusion of EEG-fNIRS: a mutual information-based hybrid classification framework | JoVE Visualize

Multimodal fusion of EEG-fNIRS: a mutual information-based hybrid classification framework

Roohollah Jafari Deligani¹, Seyyed Bahram Borgheai¹, John McLinden¹

¹Department of Electrical, Computer and Biomedical Engineering; University of Rhode Island, Kingston, RI 02881, USA.

Biomedical Optics Express|April 2, 2021

View abstract on PubMed

Summary

This study introduces a new framework for fusing electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) data. This multimodal approach significantly improves the classification of neurological conditions like ALS.

Area of Science:

Neuroimaging
Biomedical Engineering
Computational Neuroscience

Background:

Multimodal data fusion in neuroimaging aims to leverage complementary information from different modalities.
Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) offer complementary electro-hemodynamic insights into neural activity.
Existing multimodal studies often lack systematic approaches for merging EEG-fNIRS features effectively.

Purpose of the Study:

To propose a systematic framework for feature-level fusion of EEG and fNIRS data.
To optimize feature complementarity, redundancy, and relevance for improved classification accuracy.
To enhance the classification of pathological conditions versus healthy controls using fused neuroimaging data.

Main Methods:

Developed a framework for classifying fused EEG-fNIRS data at the feature level.
Employed a mutual information-based feature selection approach to maximize feature complementarity.
Utilized a cross-validation process to select an optimized feature subset for classification.
Recorded data from amyotrophic lateral sclerosis (ALS) patients and controls during a visuo-mental task.

Main Results:

The proposed framework achieved considerably improved hybrid classification performance compared to individual modalities.
Classification performance was superior to conventional methods without feature selection.
Demonstrated the efficacy of optimizing feature complementarity for multimodal neuroimaging analysis.

Conclusions:

The developed framework offers a systematic approach to fuse EEG-fNIRS data, enhancing classification accuracy.
Optimizing feature complementarity is crucial for maximizing the benefits of multimodal neuroimaging.
The proposed method shows potential for broader neuro-clinical applications in diagnosing and understanding neurological disorders.