Learning A Sparse Transformer Network for Effective Image Deraining

Transformers-based methods have achieved significant performance in image deraining as they can model the non-local information which is vital for high-quality image reconstruction. In this paper, we find that most existing Transformers usually use all similarities of the tokens from the query-key pairs for the feature aggregation. However, if the tokens from the query are different from those of the key, the self-attention values estimated from these tokens also involve in feature aggregation, which accordingly interferes with the clear image restoration. To overcome this problem, we propose an effective DeRaining network, Sparse Transformer (DRSformer) that can adaptively keep the most useful self-attention…

• NN
  National Natural Science Foundation of China

  Award: U22B2049,U19B2040,61922043,61872421,62272230
• NK
  National Key Research and Development Program of China

  Award: 2018AAA0102001
• FR
  Fundamental Research Funds for the Central Universities

  Award: 30920041109