Interactive Reconstruction of Monte Carlo Sampled Images with Depth of Field

Abstract

The likelihood of deploying Monte Carlo path tracing as a real-time rendering technique for global illumination in production systems is ever-increasing. In recent years, developments in both software and hardware, have taken us much closer to a first version of such systems. Fast reconstruction techniques for approximating higher quality images from low sample count Monte Carlo renders, without adding additional samples, has been particularly influential. We develop a convolution neural network for reconstructing Monte Carlo rendered images with low sample counts at interactive speeds. In particular, we focus on extending an already developed neural network to support depth of field effects. Our network is a deep autoencoder that utilizes a set of auxiliary buffers, containing additional information about each pixel. We propose a novel auxiliary buffer based on the circle of confusion size in each pixel. We show that by allowing the network to access this buffer during reconstruction, it learns to distinguish between points in and out of focus. Our network reconstructs images at highly interactive frame rates but does not meet the reconstruction quality of many other approaches. We discuss potential reasons behind these performance limitations and suggest a few next steps to improve reconstruction.