Bayesian Model Comparison and Significance: Widespread Errors and How to Correct Them

Abstract Bayes factors have become a popular tool in exoplanet spectroscopy for testing atmosphere models against one another. We show that the commonly used method for converting these values into significance “sigmas” is invalid. The formula is neither justified nor recommended by its original paper, and overestimates the confidence of results. We use simple examples to demonstrate the invalidity and prior sensitivity of this approach. We review the standard Bayesian interpretation of the Bayes factor as an odds ratio and recommend its use in conjunction with the Akaike information criterion or Bayesian predictive information criterion simplified in future analyses (Python implementations are included). As a…