Sparsely Annotated Semantic Segmen tation of Weather-Related Road Surface Conditions

Abstract

This project aims to build upon a machine learning pipeline for semantic segmentation of road conditions, focusing on classifying weather-affected surfaces such as dry, wet, slush, snow, and ice. Accurate detection of road surface conditions is crucial for autonomous driving systems and advanced driver-assistance systems as it directly influences vehicle control strategies, safety measures, and overall driving experience. Unlike most research in semantic segmentation, which relies heavily on densely annotated datasets that require significant manual labor to generate, this project utilises sparsely annotated data. These sparse labels, though containing less information, substantially reduce the need for manual annotation. Additionally, the provided data uses soft labels, representing a probability distribution over class conditions, which differs from the commonly used hard labels representing a single class. Data collection involves vehicles equipped with a front-facing camera recording the road and two laser detectors that gathers information about the road surface conditions. Two pre-processing approaches were explored: one crops the original input image, and the other performs an image transformation to simulate a bird’s-eye view of the road. Multiple new machine learning models were implemented, but it was observed that the choice of model did not significantly affect performance, indicating possible limitations in the provided data. Consequently, various approaches for augmenting data and methods to extract further information from unlabeled pixels were explored, some of which marginally enhanced performance. The pipeline’s performance was evaluated using conventional metrics such as accuracy and mean intersection over union, as well as through visualisation of the resulting semantic segmentation.