Deep-learning-accelerated Bayesian inference for state-space models

Abstract

Bayesian inference is an important statistical tool for estimating uncertainties in model parameters from data. One very important method is the Metropolis-Hastings algorithm, which allows for parameter inference when analytical solutions are intractable. The only requirement is that the likelihood function can be evaluated. However, it is a computationally expensive algorithm, as it is usually run for several thousand iterations. This is especially true for inference in state-space models, where the likelihood is computed via Bayesian filtering, which is a costly operation in and of itself. We propose a new method for doing Bayesian inference via Metropolis- Hastings for state-space models by replacing the standard likelihood computation with a neural network. The network is trained on data generated by a much shorter earlier run of Metropolis-Hastings. We show, both qualitatively and quantitatively, that our method produces comparable results to the traditional method for several models. Moreover, our results indicate that the performance of our method is consistent as the dimensionality of the state-space model increases. Finally, we show that our method is much more computationally efficient than the traditional method for large runs. We investigate at what point our method becomes the preferable alternative and find that the threshold occurs at quite small runs, both in terms of computational time and desired output size.