Simple and Scalable Predictive Uncertainty Estimation using Deep\n Ensembles

Deep neural networks (NNs) are powerful black box predictors that have\nrecently achieved impressive performance on a wide spectrum of tasks.\nQuantifying predictive uncertainty in NNs is a challenging and yet unsolved\nproblem. Bayesian NNs, which learn a distribution over weights, are currently\nthe state-of-the-art for estimating predictive uncertainty; however these\nrequire significant modifications to the training procedure and are\ncomputationally expensive compared to standard (non-Bayesian) NNs. We propose\nan alternative to Bayesian NNs that is simple to implement, readily\nparallelizable, requires very little hyperparameter tuning, and yields high\nquality predictive uncertainty estimates. Through a…