Trajectory Prediction for Automotive Applications using Federated Learning

Abstract

Advanced Driver Assistance Systems (ADAS) and Autonomous Driving Systems (ADS) increasingly rely on Deep Learning (DL) models. While DL models achieve state-of-the-art performance for a variety of tasks, they are not robust across a wide range of traffic scenarios, require large quantities of continuously collected data, and must follow safety and privacy regulations. Federated learning (FL) enables the training of DL models to be done locally at each client (vehicle), sharing and aggregating the trained models while keeping the data local. This allows previously unreachable data to be harnessed for improved performance. Here, performance refers to the maximum reached accuracy. FL also enables models to be trained across regions, limiting sensitive data sharing while training on diverse datasets. This master thesis evaluates the performance of FL for trajectory prediction (a cen tral part of ADAS and ADS) compared to a centralized learning (CL) approach. An FL framework was implemented and validated through classification experiments on the MNIST dataset using a convolutional neural network (CNN). The performance of a CL setup (one client) was used as a benchmark, achieving a validation accu racy of approximately 99 %. Results show that FL with multiple clients requires more training to converge but eventually saturates at a similar level of performance. Training on independent and identically distributed (IID) data yielded the best performance, while non-IID data introduced more noise and overall lower perfor mance for the FL approach. Selecting a smaller fraction of clients each round (client fraction) corresponded to lower performance when evaluating non-IID data, while speeding up training due to processing fewer data samples each round. To test the effects of FL algorithms on trajectory prediction performance, the nuScenes dataset, a collection of data from vehicles driving in Boston and Singapore, was used. The data was transformed into 2D bird’s eye-view (BEV) images and fed to CoverNet, a trajectory prediction model based on the residual network ResNet-50 which applies classification over a set of trajectories. The experiments included varying federated optimization algorithms [FedAvg, FedAvgM, FedProx], number of clients, client frac tion, data distribution techniques to test IID/non-IID data, and a direct comparison of FL to CL. The federated optimization algorithms had no notable impact on the results. More clients resulted in a slower convergence rate but similar maximum per formance to the benchmark CL setup, close to the results of the original CoverNet publication. Reducing the client fraction resulted in faster training and, contrary to the MNIST results, no notable performance or convergence rate difference. FL performed similarly to CL on both IID and non-IID data. Simulating the case where FL unlocks data from both Boston and Singapore to be used showed substantially improved performance, compared to CL using local data from only one city. Using FL and half the data from each city also showed improved performance over CL, displaying the importance of the data diversity FL can enable.