Differentiable Monte Carlo Samplers with Piecewise Deterministic Markov Processes
Typ
Examensarbete för masterexamen
Master's Thesis
Master's Thesis
Program
Engineering mathematics and computational science (MPENM), MSc
Publicerad
2023
Författare
Seyer, Ruben
Modellbyggare
Tidskriftstitel
ISSN
Volymtitel
Utgivare
Sammanfattning
Gradient estimation by Monte Carlo methods, to e.g. find optimization directions, is an important component of many problems in statistics and machine learning. In one approach, related to the reparameterization trick, the
sampling method itself is differentiated pathwise to obtain a sampler for the
gradient. Unfortunately, the Hamiltonian Monte Carlo and other common
methods contain a non-differentiable rejection step, for which pathwise derivatives do not provide unbiased estimates and corrections are computationally expensive. Here we use recently developed rejection-free methods
based on piecewise deterministic Markov processes (PDMPs) to construct differentiable Monte Carlo methods. These handle unnormalized target densities as well as unbiased estimates of the target density. We find couplings (re-parameterizations) for two PDMP methods, the Bouncy Particle sampler and
the Zig-Zag sampler, which make them differentiable. The former is pathwise differentiable while the latter requires correction for large sample path
perturbations, made efficient by our coupling. We investigate the theoretical
properties of the resulting estimators, which only require a single sampler
run. This opens up a promising new approach to stochastic gradient estimation problems.
Beskrivning
Ämne/nyckelord
Piecewise deterministic Markov processes, Monte Carlo, gradient estimation, pathwise derivatives, reparameterization trick, probabilistic programming