CCFusion: an infrared and visible image fusion method based on cycle consistency and covariance compression disentanglement module

Wenfeng Song; Xiaoqing Luo; Zhancheng Zhang

doi:10.1117/12.3034152

13 June 2024 CCFusion: an infrared and visible image fusion method based on cycle consistency and covariance compression disentanglement module

Wenfeng Song, Xiaoqing Luo, Zhancheng Zhang

Author Affiliations +

Proceedings Volume 13180, International Conference on Image, Signal Processing, and Pattern Recognition (ISPP 2024); 131801C (2024) https://doi.org/10.1117/12.3034152
Event: International Conference on Image, Signal Processing, and Pattern Recognition (ISPP 2024), 2024, Guangzhou, China

Abstract

Thanks to the feature disentanglement capability of disentangled representation methods, deep learning-based de-entanglement methods have achieved strong performance in image fusion tasks. However, it should be noted that training a powerful performance of a disentangled representation method is a challenging problem due to the domain differences between image fusion tasks and mainstream disentanglement tasks. To overcome this problem, an infrared and visible image fusion method based on cycle consistency and the covariance compression disentanglement module proposed in this manuscript. Firstly, the proposed covariance compression disentanglement module statistically counts the global information by global covariance pool and adaptively enhances the desired features and suppresses others by the covariance compression module. Secondly, it makes the disentanglement module cycle-consistent by disentanglement the fusion image, and strengthens the disentanglement by the proposed consistency loss function and feature similarity loss function constraints. Specifically, in this paper, multi-source images are input into the disentanglement module, modal features and common features are disentangled, and fusion results are generated by fusing the modal features and common features of multi-source images. After generating the fusion result, the fusion result is input into the disentanglement module again, and the re-decomposed features are used to calculate the consistency loss with the multi-source image features. On the publicly available TNO dataset, subjective and objective evaluations show that the method proposed in this manuscript has better fusion performance than other state-of-the-art methods.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Wenfeng Song, Xiaoqing Luo, and Zhancheng Zhang "CCFusion: an infrared and visible image fusion method based on cycle consistency and covariance compression disentanglement module", Proc. SPIE 13180, International Conference on Image, Signal Processing, and Pattern Recognition (ISPP 2024), 131801C (13 June 2024); https://doi.org/10.1117/12.3034152

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
6 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Image fusion

Covariance

Infrared imaging

Feature fusion

Image compression

Infrared radiation

Visible radiation

Show All Keywords

Keywords/Phrases

Search In:

Publication Years