Outdoor global pose estimation from RGB and 3D data

Abstract

Accurate and robust localization is crucial for safe and reliable autonomous vehicles. In this thesis we present a machine learning approach for localization using RGB images and 3D structure information. More specifically we use two types of neural networks, one that predicts a pose in an given environment with a RGB-D image as input and one that uses a 3D LiDAR scan and a prebuilt 3D map of the environment to refine the prediction. The advantages of using neural networks for localization is the constant runtime and that it use natural navigation, i.e. without the need of infrastructure enhancement. We show that the RGB-D network gains increased accuracy from the multi-modal RGB-D data, compared to an uni-modal network trained on RGB or depth images when used in outdoor scenes. Additionally, we show the benefit of using 3D LiDAR data for pose refinement.