Learning Important Features Through Propagating Activation Differences

The purported "black box" nature of neural networks is a barrier to adoption in applications where interpretability is essential. Here we present DeepLIFT (Deep Learning Important FeaTures), a method for decomposing the output prediction of a neural network on a specific input by backpropagating the contributions of all neurons in the network to every feature of the input. DeepLIFT compares the activation of each neuron to its 'reference activation' and assigns contribution scores according to the difference. By optionally giving separate consideration to positive and negative contributions, DeepLIFT can also reveal dependencies which are missed by other approaches. Scores can be computed efficiently in a…

• RF
  Russian Foundation for Basic Research

  Award: 18-34-20024
• NI
  National Institutes of Health

  Awards: R13 HG006650, U01 HG009380, R01 HG007348, HG006650, U41 HG007346