articleIEEE Transactions on Pattern Analysis and Machine IntelligenceJan 31, 2023Closed access

Attention Spiking Neural Networks

MYMan YaoGZGuangshe ZhaoHZHengyu ZhangYHYifan HuLDLei Deng

Peng Cheng Laboratory · Xi'an Jiaotong University · +6 more institutions

PubMed
Indexed incrossrefpubmed

Abstract

Brain-inspired spiking neural networks (SNNs) are becoming a promising energy-efficient alternative to traditional artificial neural networks (ANNs). However, the performance gap between SNNs and ANNs has been a significant hindrance to deploying SNNs ubiquitously. To leverage the full potential of SNNs, in this paper we study the attention mechanisms, which can help human focus on important information. We present our idea of attention in SNNs with a multi-dimensional attention module, which infers attention weights along the temporal, channel, as well as spatial dimension separately or simultaneously. Based on the existing neuroscience theories, we exploit the attention weights to optimize membrane…

Citation impact

229
total citations
FWCI
28.52
Percentile
100%
References
78
Citations per year

Authors

8

Topics & keywords

Topics
Keywords
  • Spiking neural network
  • Computer science
  • Artificial intelligence
  • Leverage (statistics)
  • Block (permutation group theory)
  • Pattern recognition (psychology)
  • MNIST database
  • Artificial neural network
UN Sustainable Development Goals
  • Affordable and clean energy
No related works found for this paper.

Funding