Dual Branch Masked Transformer for Hyperspectral Image Classification

Transformer has been widely used in hyperspectral image (HSI) classification tasks because of its ability to capture long-range dependencies. However, most Transformer-based classification methods lack the extraction of local information or do not combine spatial and spectral information well, resulting in insufficient extraction of features. To address these issues, in this study, a dual-branch masked Transformer (Dual-MTr) model is proposed. Masked Transformer (MTr) is used to pretrain vision transformer (ViT) by reconstruction of both masked spatial image and spectral spectrum, which embeds the local bias by the process of recovering from localized patches to the global original input. Different…

• NN
  National Natural Science Foundation of China

  Awards: Grant 62371169, Grant 61971164 and Grant U20B2041, 62371169, U20B2041, 61971164