MB-TaylorFormer V2: Improved Multi-Branch Linear Transformer Expanded by Taylor Formula for Image Restoration

Recently, Transformer networks have demonstrated outstanding performance in the field of image restoration due to the global receptive field and adaptability to input. However, the quadratic computational complexity of Softmax-attention poses a significant limitation on its extensive application in image restoration tasks, particularly for high-resolution images. To tackle this challenge, we propose a novel variant of the Transformer. This variant leverages the Taylor expansion to approximate the Softmax-attention and utilizes the concept of norm-preserving mapping to approximate the remainder of the first-order Taylor expansion, resulting in a linear computational complexity. Moreover, we introduce a…

• NN
  National Natural Science Foundation of China

  Awards: U24A20251, 62372480
• SY
  Sun Yat-sen University

  Award: 24lgqb015
• FR
  Fundamental Research Funds for the Central Universities
• BA
  Basic and Applied Basic Research Foundation of Guangdong Province

  Award: 2025A1515011361