4D Radar-Camera Fusion for Enhanced Point Cloud Clustering

Abstract

Abstract Detecting and tracking objects in the environment is a central task in radar perception systems. However, due to the radar’s relatively low resolution, limited semantic information, and sensor-specific artifacts such as multipath reflections, a perception framework relying solely on radar data is likely to deliver limited performance. By fusing the radar data with information from a complementary sensor, such as a camera, the additional semantic information can mitigate these issues. Recent contributions to the field of radar-camera fusion focus mainly on deep learning methods, which perform well but require large amounts of annotated data. Collecting and labeling such data can be infeasible because of time or budget constraints. This thesis instead explores an approach to 4D radar-camera fusion utilizing pretrained image models, with the goal of achieving better tracking performance. The suggested method projects a radar point cloud onto a corresponding image, associates radar points with objects detected in the image, and adds this information to the point cloud. The added information is used to distinguish points that are part of objects from those that are not. It is also used when grouping the point cloud, by suggesting which points likely belong to the same object. Evaluation indicates that the suggested method improves the tracking performance compared to using radar alone. Furthermore, the method shows potential for real-time deployment in runtime evaluations.