Evaluating the Visualization of What a Deep Neural Network Has Learned

Deep neural networks (DNNs) have demonstrated impressive performance in complex machine learning tasks such as image classification or speech recognition. However, due to their multilayer nonlinear structure, they are not transparent, i.e., it is hard to grasp what makes them arrive at a particular classification or recognition decision, given a new unseen data sample. Recently, several approaches have been proposed enabling one to understand and interpret the reasoning embodied in a DNN for a single test image. These methods quantify the “importance” of individual pixels with respect to the classification decision and allow a visualization in terms of a heatmap in pixel/input space. While the usefulness of…

• DF
  Deutsche Forschungsgemeinschaft

  Award: MU 987/17-1
• BF
  Bundesministerium für Bildung und Forschung

  Award: 01IS14013A
• NR
  National Research Foundation of Korea