TaijiGNN: A New Cycle-Consistent Generative Neural Network for High-Quality Bidirectional Transformation between RGB and Multispectral Domains

Area of Science:

• Computer Vision
• Image Processing
• Machine Learning

Background:

• Multispectral images (MSIs) and RGB images (RGBs) have disparate definitions and information entropies, posing challenges for spectrum transformation.
• Reconstructing MSIs from RGBs is particularly difficult due to information loss and domain differences.

Purpose of the Study:

• To propose a novel generative neural network, Taiji Generative Neural Network (TaijiGNN), for effective MSI reconstruction from RGB images.
• To address the challenges of spectral image translation and improve training efficiency.

Main Methods:

• Developed TaijiGNN with two generators (G_MSI, G_RGB) forming dual cyclic translations between RGB and MSI domains.
• Replaced traditional convolutional neural networks (CNNs) with multilayer perceptron (MLP) networks for generator implementation, enhancing simplicity and performance.
• Modified loss functions by removing identity losses and incorporating consistent paired image losses to optimize training.

Main Results:

• Achieved state-of-the-art results on CAVE and ICVL datasets using significantly less training data.
• Demonstrated high performance with a simplified MLP architecture, requiring only half the CAVE dataset and one-fifth of the ICVL dataset for comparable results.
• The dual-generator system, inspired by Taiji philosophy, reached a dynamic balance during training, ensuring convergence.

Conclusions:

• TaijiGNN offers a robust and efficient solution for multispectral image reconstruction from RGB data.
• The proposed MLP-based generators and modified loss functions significantly improve training effectiveness and reduce data requirements.
• This approach advances spectral image translation, enabling high-quality MSI reconstruction with minimal training resources.