Self-Training Multi-Sequence Learning with Transformer for Weakly Supervised Video Anomaly Detection

Weakly supervised Video Anomaly Detection (VAD) using Multi-Instance Learning (MIL) is usually based on the fact that the anomaly score of an abnormal snippet is higher than that of a normal snippet. In the beginning of training, due to the limited accuracy of the model, it is easy to select the wrong abnormal snippet. In order to reduce the probability of selection errors, we first propose a Multi-Sequence Learning (MSL) method and a hinge-based MSL ranking loss that uses a sequence composed of multiple snippets as an optimization unit. We then design a Transformer-based MSL network to learn both video-level anomaly probability and snippet-level anomaly scores. In the inference stage, we propose to use the…

• NN
  National Natural Science Foundation of China

  Awards: B07048, 61836009, IRT 15R53, No.62076192, 62076192
• HE
  Higher Education Discipline Innovation Project

  Award: B07048
• NK
  National Key Research and Development Program of China

  Award: B07048