An Empirical Evaluation of Bayesian Inference Methods for Bayesian Neural Networks

Abstract

In this paper we perform an empirical evaluation of different inference methods for Bayesian neural networks (BNN). We compare two major approaches, including Hamiltonian Monte Carlo (HMC) and Variational Inference (VI). Both methods are implemented using Tensorflow Probability Toolbox. We first evaluate how different factors involved in HMC and VI affect the inference results, focusing on classification problem. We then show benefits of Bayesian inference in generalization over point estimation approach. Finally, we provide a way of quantifying uncertainty based on Bayesian inference.