Uncertainty estimation in multi-modal 3D object detection

Abstract

Abstract Object detection is an important part of many autonomous driving systems, providing condensed information about the vehicle’s surroundings. For good performance in different environmental conditions, multi-modal object detection is often used, where information from different sensors are fused. Due to factors such as sensor noise, occlusion, and adverse weather conditions, there is an inherent uncertainty in the object detection task. Most state-of-the-art approaches for multi-modal 3D object detection do not model these uncertainties for regression. Explicitly modeling and estimating the uncertainties leads to higher interpretability, allows analysis of difficult situations, and can improve the performance of downstream tasks. In this work, we explore how uncertainty can be modeled and estimated in multimodal 3D object detection. We show that directly modeling the uncertainties of bounding box parameters can provide meaningful uncertainty estimates without sacrificing neither predictive performance nor computational efficiency. We compare modeling the uncertainties both separately per detection, using normally distributed random vectors, and jointly per frame, using Poisson multi-Bernoulli random finite sets. Our results show that separate modeling enhances predictive performance, while joint modeling yields more accurate uncertainty estimates. Additionally, we demonstrate that these predicted uncertainties can identify unlabeled data where the model performs poorly, underscoring their importance for more interpretable and safe autonomous driving systems.