Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures

Machine-learning (ML) algorithms are increasingly utilized in privacy-sensitive applications such as predicting lifestyle choices, making medical diagnoses, and facial recognition. In a model inversion attack, recently introduced in a case study of linear classifiers in personalized medicine by Fredrikson et al., adversarial access to an ML model is abused to learn sensitive genomic information about individuals. Whether model inversion attacks apply to settings outside theirs, however, is unknown. We develop a new class of model inversion attack that exploits confidence values revealed along with predictions. Our new attacks are applicable in a variety of settings, and we explore two in depth: decision trees…

• NS
  National Science Foundation

  Award: CNS-1546033 CNS-1330308 CNS-1065134
• DA
  Defense Advanced Research Projects Agency

  Award: FA8650-10-C-7088