Differentiable Monte Carlo Samplers with Piecewise Deterministic Markov Processes

Abstract

Gradient estimation by Monte Carlo methods, to e.g. find optimization directions, is an important component of many problems in statistics and machine learning. In one approach, related to the reparameterization trick, the sampling method itself is differentiated pathwise to obtain a sampler for the gradient. Unfortunately, the Hamiltonian Monte Carlo and other common methods contain a non-differentiable rejection step, for which pathwise derivatives do not provide unbiased estimates and corrections are computationally expensive. Here we use recently developed rejection-free methods based on piecewise deterministic Markov processes (PDMPs) to construct differentiable Monte Carlo methods. These handle unnormalized target densities as well as unbiased estimates of the target density. We find couplings (re-parameterizations) for two PDMP methods, the Bouncy Particle sampler and the Zig-Zag sampler, which make them differentiable. The former is pathwise differentiable while the latter requires correction for large sample path perturbations, made efficient by our coupling. We investigate the theoretical properties of the resulting estimators, which only require a single sampler run. This opens up a promising new approach to stochastic gradient estimation problems.