Adversarial Examples Are Not Easily Detected

Neural networks are known to be vulnerable to adversarial examples: inputs that are close to natural inputs but classified incorrectly. In order to better understand the space of adversarial examples, we survey ten recent proposals that are designed for detection and compare their efficacy. We show that all can be defeated by constructing new loss functions. We conclude that adversarial examples are significantly harder to detect than previously appreciated, and the properties believed to be intrinsic to adversarial examples are in fact not. Finally, we propose several simple guidelines for evaluating future proposed defenses.

• WA
  William and Flora Hewlett Foundation
• CI
  Canadian Institute for Advanced Research
• MU
  Multidisciplinary University Research Initiative
• AF
  Air Force Office of Scientific Research