STAR-Transformer: A Spatio-temporal Cross Attention Transformer for Human Action Recognition
Keimyung University · Genome and Company (South Korea)
Abstract
In action recognition, although the combination of spatiotemporal videos and skeleton features can improve the recognition performance, a separate model and balancing feature representation for cross-modal data are required. To solve these problems, we propose Spatio-TemporAl cRoss (STAR)-transformer, which can effectively represent two cross-modal features as a recognizable vector. First, from the input video and skeleton sequence, video frames are output as global grid tokens and skeletons are output as joint map tokens, respectively. These tokens are then aggregated into multi-class tokens and input into STAR-transformer. The STAR-transformer encoder consists of a full spatio-temporal attention (FAttn)…
Citation impact
- FWCI
- 11.24
- Percentile
- 100%
- References
- 63
Authors
4Topics & keywords
- Computer science
- Transformer
- Encoder
- Artificial intelligence
- Pattern recognition (psychology)
- RGB color model
- Computer vision
- Feature extraction