Sequential Normalising Flow for Approximating Smoothing Distributions - Evaluating Sequential Normalising Flow as a Cost-Efficient Alternative for Smoothing Distribution Estimation

Abstract

This thesis addresses the smoothing problem, in which the goal is to estimate a time series of latent variables from a corresponding time series of noisy observations. Traditional methods such as the Kalman Smoother and the Unscented Kalman Smoother can solve this problem, but rely on Gaussian assumptions that introduce bias when applied to nonlinear systems, motivating the development of faster and more flexible alternatives. Hence, the Sequential Normalising Flow (SNF) is created, a model that allows sampling from an approximation of the smoothing distribution rather than computing it analytically. The SNF consists of an embedding model, implemented as a Gated Recurrent Unit (GRU), which encodes future observations into a fixed-dimensional representation, and a generative model implemented as a normalising flow. While training requires substantial computational resources, inference is significantly cheaper. The SNF is evaluated on two cases: a linear Gaussian state-space model and a nonlinear Gaussian state-space model. In both cases, traditional methods serve as a reference point. For the linear case, the SNF approximates the smoothing distribution well, with computational times comparable to those of the Kalman Smoother. In the nonlinear case, the SNF has more difficulty accurately estimating the smoothing distribution, which is believed to stem from the embedding model’s limited ability to represent the observations. The thesis concludes that the SNF shows promise for certain cases and identifies the key difficulties in scaling the approach to nonlinear and more complex problems.