Action bank: A high-level representation of activity in video

Activity recognition in video is dominated by low- and mid-level features, and while demonstrably capable, by nature, these features carry little semantic meaning. Inspired by the recent object bank approach to image representation, we present Action Bank, a new high-level representation of video. Action bank is comprised of many individual action detectors sampled broadly in semantic space as well as viewpoint space. Our representation is constructed to be semantically rich and even when paired with simple linear SVM classifiers is capable of highly discriminative performance. We have tested action bank on four major activity recognition benchmarks. In all cases, our performance is better than the state of…

• NS
  National Science Foundation
• UD
  U.S. Department of the Interior
• DA
  Defense Advanced Research Projects Agency
• IA
  Intelligence Advanced Research Projects Activity
• IB
  Interior Business Center