Learning Continuous Video Representation from Event Cameras

Abstract

Event cameras are biologically inspired sensors that operate differently from conventional cameras. Rather than measuring pixel intensities at fixed intervals, event cameras detect per-pixel intensity changes, offering high dynamic range, low latency, high temporal resolution, minimal motion blur, and low power consumption. However, traditional computer vision algorithms cannot be applied to event data due to the radically different operating paradigm. One approach to bridge this gap is to reconstruct conventional images from event data. While this approach retains the high dynamic range and minimal motion blur, it does not fully capture the high temporal resolution of event cameras. In this thesis, we utilize Local Implicit Functions for spatiotemporal video reconstruction, aiming to preserve the high temporal resolution of event data as well as allow for the generation of videos with an arbitrary spatial resolution. We show that our method reaches reconstruction quality similar to comparable state-of-theart approaches, and significantly outperforms simple baselines for spatial upscaling up to 3x. Our analysis also suggests that our representation retains the high temporal resolution of event data. Additionally, our approach offers per-pixel uncertainty estimations, which have the potential to enhance the performance of downstream computer vision applications.