UCTransNet: Rethinking the Skip Connections in U-Net from a Channel-Wise Perspective with Transformer

Most recent semantic segmentation methods adopt a U-Net framework with an encoder-decoder architecture. It is still challenging for U-Net with a simple skip connection scheme to model the global multi-scale context: 1) Not each skip connection setting is effective due to the issue of incompatible feature sets of encoder and decoder stage, even some skip connection negatively influence the segmentation performance; 2) The original U-Net is worse than the one without any skip connection on some datasets. Based on our findings, we propose a new segmentation framework, named UCTransNet (with a proposed CTrans module in U-Net), from the channel perspective with attention mechanism. Specifically, the CTrans (Channel…

• NN
  National Natural Science Foundation of China

  Awards: 62076059, No.62076059
• FR
  Fundamental Research Funds for the Central Universities

  Award: N2016001