MATR: Multimodal Medical Image Fusion via Multiscale Adaptive Transformer

Owing to the limitations of imaging sensors, it is challenging to obtain a medical image that simultaneously contains functional metabolic information and structural tissue details. Multimodal medical image fusion, an effective way to merge the complementary information in different modalities, has become a significant technique to facilitate clinical diagnosis and surgical navigation. With powerful feature representation ability, deep learning (DL)-based methods have improved such fusion results but still have not achieved satisfactory performance. Specifically, existing DL-based methods generally depend on convolutional operations, which can well extract local patterns but have limited capability in…

• NN
  National Natural Science Foundation of China

  Awards: 62176081, 62072348
• NK
  National Key Research and Development Program of China

  Award: 2019YFC1509604