Class imbalance should not throw you off balance: Choosing the right classifiers and performance metrics for brain decoding with imbalanced data

Machine learning (ML) is increasingly used in cognitive, computational and clinical neuroscience. The reliable and efficient application of ML requires a sound understanding of its subtleties and limitations. Training ML models on datasets with imbalanced classes is a particularly common problem, and it can have severe consequences if not adequately addressed. With the neuroscience ML user in mind, this paper provides a didactic assessment of the class imbalance problem and illustrates its impact through systematic manipulation of data imbalance ratios in (i) simulated data and (ii) brain data recorded with electroencephalography (EEG), magnetoencephalography (MEG) and functional magnetic resonance imaging…

• MU
  McGill University
• CR
  Canada Research Chairs
• M
  Mitacs
• CF
  Canada First Research Excellence Fund
• ID
  Institut de Valorisation des Données
• CF
  Courtois Foundation
• CI
  Canadian Institutes of Health Research
• NS
  Natural Sciences and Engineering Research Council of Canada
• FD
  Fonds de recherche du Québec – Nature et technologies

  Awards: 265502, 2020-RS4-265502