Equivariant Inductive Biases for Weather Prediction with PEAR - Investigating the exploitation of rotational symmetries for accurate transformer-based weather forecasting over the HEALPix discretisation

Abstract

Weather forecasting is a complex challenge due to its intrinsically complex physical dynamics that define the evolution of the system. In recent years, deep learning weather prediction has emerged as a promising alternative to classical numerical weather prediction, matching or outperforming it on several benchmarks at a fraction of the inference time. This thesis contributes to this direction by analysing the symmetries of this system in relation to the group SO(2): the rotation of the Earth around its own axis. The study builds on Pangu Equal ARea (PEAR), a transformer-based model operating on the Hierarchical Equal Area isoLatitude Pixelization (HEALPix) discretisation of the sphere, and examines whether the symmetry awareness of this architecture can be increased from two complementary perspectives: the data on which the model is trained, and the architecture itself. In the first part, starting from ERA5, the reanalysis dataset that provides global estimates of the surface and atmospheric variables, we introduce a new 2-hourly sampling, which allows a comparison of PEAR’s equivariance behaviour across three configurations of dataset and forecast horizon. The analysis shows that the equivariance error is dominated by the architecture and the forecast horizon rather than by the sampling. The second part introduces two architectural modifications, an iso-latitude interspersed windowing scheme and a set of HEALPix-aware convolutions, designed to better align the model with the rotational symmetry of the sphere. These modifications successfully reduce the equivariance error at the surface level, but fail to improve it at the upper atmospheric levels, and do not translate into a forecasting advantage over the baseline. This outcome highlights the difficulty of embedding inductive biases in the case of domains that involve using high-dimensional samples, specifically in relation to window-based attention mechanisms.