Learning using Privileged Time Series

Abstract

In this thesis, we study the impact of having privileged information, in the form of intermediate time series, available during the training of models for long-term prediction. An algorithm which incorporates these time series is presented, and we prove that it is more efficient when the time series are drawn from a Gaussian Linear dynamical system in comparison to a linear baseline model without access to the privileged information. Notably, the main analysis tool which we use is Rao Blackwell’s theorem. Using synthetically generated data, we validate the theoretical results and characterize the algorithm’s behavior, but also test the limits of the algorithm by evaluating it on synthetic data where the assumptions of the theoretical analysis are violated. Furthermore, the applicability of the algorithm is investigated on real-world datasets for forecasting air quality in Chinese cities and predicting Alzheimer’s disease progression. We show that our approach is preferable to classical learning in most settings, especially when data is scarce. Furthermore, empirical results are used to discuss the trade-off between bias and variance when using the proposed algorithm. We conclude that learning with privileged information in time series data for long-term prediction is beneficial and a highly interesting subject for further research. Proposals are given for future work, such as extensions to non-linear models or investigating how privileged time series could be beneficial for latent variable systems.