Unsupervised Misaligned Infrared and Visible Image Fusion via Cross-Modality Image Generation and Registration

Recent learning-based image fusion methods have marked numerous progress in pre-registered multi-modality data, but suffered serious ghosts dealing with misaligned multi-modality data, due to the spatial deformation and the difficulty narrowing cross-modality discrepancy. To overcome the obstacles, in this paper, we present a robust cross-modality generation-registration paradigm for unsupervised misaligned infrared and visible image fusion (IVIF). Specifically, we propose a Cross-modality Perceptual Style Transfer Network (CPSTN) to generate a pseudo infrared image taking a visible image as input. Benefiting from the favorable geometry preservation ability of the CPSTN, the generated pseudo infrared image…

• NN
  National Natural Science Foundation of China

  Awards: 61733002, 62027826, 61922019
• FR
  Fundamental Research Funds for the Central Universities