Multi-class Token Transformer for Weakly Supervised Semantic Segmentation

This paper proposes a new transformer-based framework to learn class-specific object localization maps as pseudo labels for weakly supervised semantic segmentation (WSSS). Inspired by the fact that the attended regions of the one-class token in the standard vision transformer can be leveraged to form a class-agnostic localization map, we investigate if the transformer model can also effectively capture class-specific attention for more discriminative object localization by learning multiple class tokens within the transformer. To this end, we propose a Multi-class Token Transformer, termed as MCTformer, which uses multiple class tokens to learn interactions between the class tokens and the patch tokens. The…

• CR
  Cooperative Research Centres, Australian Government Department of Industry
• HK
  Hong Kong University of Science and Technology

  Award: R9253
• SM
  SENSHIN Medical Research Foundation
• AR
  Australian Research Council

  Award: DP210101682,DP210102674,DP200103223