VadCLIP: Adapting Vision-Language Models for Weakly Supervised Video Anomaly Detection

The recent contrastive language-image pre-training (CLIP) model has shown great success in a wide range of image-level tasks, revealing remarkable ability for learning powerful visual representations with rich semantics. An open and worthwhile problem is efficiently adapting such a strong model to the video domain and designing a robust video anomaly detector. In this work, we propose VadCLIP, a new paradigm for weakly supervised video anomaly detection (WSVAD) by leveraging the frozen CLIP model directly without any pre-training and fine-tuning process. Unlike current works that directly feed extracted features into the weakly supervised classifier for frame-level binary classification, VadCLIP makes full use…

• NN
  National Natural Science Foundation of China

  Awards: 62101453, U19B2037, 62301432, 62306240, U23B2013
• CP
  China Postdoctoral Science Foundation

  Award: 2023TQ0272
• FR
  Fundamental Research Funds for the Central Universities

  Award: D5000220431