Uncertainty-aware weather-related road surface condition classification

Abstract

Autonomous vehicles rely on collaborative systems for decision-making during operation, one of which is machine learning algorithms. Using cameras and sensors mounted on the car, these algorithms can analyse captured data to interpret surroundings, enabling it to make informed and safe driving decisions. The algorithms are trained on data gathered in the field. In real-time use, samples that do not belong to the training distribution, referred to as Out-Of-Distribution (OOD) samples, can occur, which the algorithm then confidently uses to make driving decisions. In a worst-case scenario, this can lead to injuring passengers or property. The primary objective of this thesis is to implement and analyse methods for detecting OOD samples in a road surface condition model. The machine learning algorithm was a convolutional neural network, trained to detect dry, wet and snowy road surface conditions. The methods used to detect samples were Maximum Softmax Probability, Energy-Based OOD detection, Outlier Exposure, Virtual Outlier Synthesis, Rectified Activations, Deep Nearest Neighbour, and Virtual Logit Matching. Evaluation of the methods involved three datasets - Cars, Slush and Glare - constructed from the same database as the training data, thus considered as near OOD datasets. Additionally, two publicly available datasets, CIFAR-10 and Texture were used as far OOD datasets. Deep Nearest Neighbour and Virtual Logit Matching performed the best, achieving near-perfect results, when evaluated on the far OOD datasets. When tested on the cars dataset, Virtual Logit Matching exhibited a notable deviation in performance compared to the other methods, although it still performed poorly. On the rest of the near OOD data, the detectors did not outperform the two baseline detectors, Energy-Based OOD detection and Maximum Softmax Probability.