Sequential Bayesian inference with intractable likelihoods: A sequential mixture model method for posterior and likelihood estimation

Abstract

In a Bayesian setting with realistic models it is not unusual that it is not possible to perform exact parameter inference. This is typically due to the fact that the likelihood function is not available in closed form or is intractable. Likelihood-free methods perform parameter inference in models where evaluating the likelihood is an intractable problem, but sampling data from a generative model is possible. With the expansion of machine learning, recent approaches learn the posterior distribution of the parameters by sequential updates of neural-network based density estimators. While these methods perform well, they require a network architecture to be specified and training the neural-network can be computationally demanding and time consuming. In this work we present a Bayesian inference method which, in place of neural networks, uses Gaussian mixtures sequentially learned through an expectation-maximization procedure. Posterior samples are then obtained via MCMC through an informative and self-tuned proposal sampler. Only the number of components in the Gaussian mixture needs to be specified to run the algorithm. We show the feasibility of this method and benchmark it against two neural-network based state-of-the-art Bayesian methods in 4 simulation studies. The results show that the proposed method is competitive and in some cases even outperforms the other methods in terms of simulation efficiency. Additionally, it is in most cases significantly faster to run.